UAPI Grok 140T/NPU Primary Stack Review 2026-06-22
Grok review of OpenVINO GenAI as the primary NUC 140T/NPU stack and benchmark blockers.
Promotion Metadata
- Source feedback id:
112 - Source feedback ids:
112 - Promotion request id:
43 - Feedback category:
beneficial - Feedback source:
[REDACTED:high-entropy] - Feedback created at:
2026-06-22T18:22:34.954146+00:00 - Target slug:
[REDACTED:high-entropy] - Review state:
review - Reviewer note: AI lane auto-promoted redacted feedback into internal catalog context.
Source Feedback Body
Feedback 112
- Product:
uapi - Category:
beneficial - Related slug:
[REDACTED:high-entropy] - Source:
[REDACTED:high-entropy] - Created at:
2026-06-22T18:22:34.954146+00:00
Grok 140T/NPU Primary Stack Review (2026-06-22)
ALIVE: true
Lane: Grok / UAPI hardware autorouter
Sources: local diary/2026-06-22-*.md ([REDACTED:high-entropy].md, [REDACTED:high-entropy].md, grok-benchmark-preflight-command-plan.md, [REDACTED:high-entropy].md), [REDACTED:high-entropy].md; public primary: openvinotoolkit/openvino.genai, Intel developer articles, docs.openvino.ai.
Primary Stack: OpenVINO GenAI for Arc 140T + NPU
Recommended primary: OpenVINO GenAI (openvino-genai) + OpenVINO Runtime for both NPU ("AI Boost") and Arc 140T iGPU on NUC15CRSU9 (Core Ultra 9 285H).
Source URLs (primary only)
- GitHub: https://github.com/openvinotoolkit/openvino.genai (Apache-2.0)
- Docs: https://openvinotoolkit.github.io/openvino.genai/ ; https://docs.openvino.[REDACTED:high-entropy].html
- Install:
pip install openvino-genai - Usage:
import openvino_genai as ov_genai; pipe = ov_genai.LLMPipeline(model_path, "NPU")or"GPU" - Model prep:
optimum-cli export openvino --model ... --weight-format int4 ... - Intel validation on exact HW: Core Ultra 9 285H + Arc 140T GPU (driver ~32.0.101.x) with openvino-genai (2025.2 dev). Throughput examples published (e.g. Qwen3-30B-A3B MoE ~34 t/s on Arc; NPU support Day-0 for select models). https://www.intel.[REDACTED:high-entropy].html
Confirmed (from sources + inventory match)
- OpenVINO Runtime explicitly supports CPU + GPU + NPU.
- GenAI LLMPipeline accepts device="NPU" or "GPU".
- HW match: NUC has Arc 140T (drv 32.0.101.8331), NPU Intel AI Boost (drivers 32.0.100.x), ~64 GiB RAM. Matches Intel test configs.
- No external deps for core tokenization/runtime.
- Python probe in preflight: openvino_genai ABSENT today.
- License clean for review (Apache-2.0).
Assumptions only (not confirmed)
- Full Gemma 2B/3B-class support on this NUC's NPU/140T without custom export or limits (model-specific; small SLMs favored for NPU).
- Performance targets (first_token <3000ms, >15 t/s) achievable on NPU vs 140T (inventory shows shared mem behavior TBD; reported adapterRAM ~2 GiB vs "32GB" label).
- NUC Windows driver stack (no WSL) will match Intel reference configs exactly.
- Optimum-Intel export path will be stable for target Gemma variants.
Benchmark blockers (explicit)
- No packages in default Python 3.13.12 (torch, openvino, openvino_genai, optimum, nncf all ABSENT).
- Operator gate required for any pip install + weight download + small Gemma/SLM smoke (1-3 prompts, max_new_tokens~32-64).
- Metrics gate: capture first_token_ms, tokens_per_sec, load_time_s, peak_mem_mb, device, stability. Do not claim
local.nuc.gemma.npuor.140tuntil evidence. - NPU known issues from upstream (prompt len limits ~2k tokens in some reports, cache dir, garbled long outputs).
- C: free ~704 GiB OK; idle confirmed; no concurrent Conductor load.
- Model IR export + quant step must be validated pre-bench (not just HF direct).
- No current route impl; candidates only.
What must be benchmarked before any route claim
Small Gemma-class on NPU path and on 140T path. Record device fallback, thermals/power if exposed, VRAM sharing. Re-bootstrap after install.
Blockers: install+bench gate (planning/preflight only per sources), Codex PlacementDecision contracts, no weights yet, SN850X separate.
Sources used: listed local + primary GitHub/Intel docs only. No broad search.
End. Constraints met.