All runbooks
runbookuapiredacted view

UAPI Grok OpenVINO Gemma Model Candidate Notes 2026-06-22

Grok candidate model notes and constraints for OpenVINO GenAI on NUC NPU and Arc 140T.

Updated 2026-06-22·Freshness: Reference·Sensitive tokens or credentials were hidden before display.

UAPI Grok OpenVINO Gemma Model Candidate Notes 2026-06-22

Grok candidate model notes and constraints for OpenVINO GenAI on NUC NPU and Arc 140T.

Promotion Metadata

  • Source feedback id: 130
  • Source feedback ids: 130
  • Promotion request id: 61
  • Feedback category: beneficial
  • Feedback source: [REDACTED:high-entropy]
  • Feedback created at: 2026-06-22T18:32:58.693203+00:00
  • Target slug: [REDACTED:high-entropy]
  • Review state: review
  • Reviewer note: AI lane auto-promoted redacted feedback into internal catalog context.

Source Feedback Body

Feedback 130

  • Product: uapi
  • Category: beneficial
  • Related slug: [REDACTED:high-entropy]
  • Source: [REDACTED:high-entropy]
  • Created at: 2026-06-22T18:32:58.693203+00:00

Grok OpenVINO Gemma Model Candidate Notes (2026-06-22)

Primary: OpenVINO GenAI first (openvino-genai + LLMPipeline + optimum export) for NUC Gemma/SLM on 140T + NPU.

Authority: Research notes. Local only. Public docs cited. No installs/downloads/inference.

HW (NUC-IPEX)

Core Ultra 9 285H (16c), Arc 140T (drv 32.0.101.8331), NPU Intel AI Boost (13 TOPS Int8), ~64 GiB, Win11 10.0.26200, Python 3.13.12 (no AI pkgs), boot SN8100 (no SN850X). WSL absent. Refs: local preflight evidence + https://www.intel.[REDACTED:high-entropy].html

Primary Commands

pip install openvino-genai pipe = ov_genai.LLMPipeline(path, "NPU") # or "GPU" Export: optimum-cli export openvino --model --weight-format int4 ... GitHub: https://github.com/openvinotoolkit/openvino.genai Site: https://openvinotoolkit.github.io/openvino.genai/ NPU guide: https://docs.openvino.[REDACTED:high-entropy].html Supported (Gemma* listed): https://openvinotoolkit.github.io/openvino.genai/docs/supported-models/

Candidates (small Gemma/SLM; NPU first)

  • google/gemma-2-2b-it (Gemma2, 2B)
  • google/gemma-2b-it / gemma-1.1-2b-it
  • google/gemma-3-1b-it (Gemma3; NPU notes in releases)
  • google/gemma-3-4b-it (borderline; verify export) Gemma 4 E2B/E4B: NPU via LiteRT/OpenVINO backend (Intel collab; refs on Lunar Lake). Preview for direct GenAI. Not first smoke. Sources: supported-models; https://zenn.dev/jkudo/articles/ae85d7d099e672 ; Intel Gemma 4 blog.

NPU Constraints

Export: --weight-format int4 --sym --ratio 1.0 --group-size 128 (small) or -1 (CW). NF4 limited to Series 2+ Lunar Lake; use INT4 sym here (Arrow Lake-H 13 TOPS). Static: MAX_PROMPT_LEN=1024 default, MIN_RESPONSE_LEN=128. Tune pipeline_config (DYNAMIC prefill). Short prompts for smoke. Small models favored (<4-5B). Cache with CACHE_DIR. Pins + latest NPU driver recommended. Pre-converted HF options exist. Limits: compile time, prompt size, quality vs other devices.

140T (GPU) vs NPU

"GPU" device: more headroom/throughput. Shared mem TBD (~2 GiB reported vs 32GB label). Same IR works; switch device at load. Validate CPU fallback. NPU: low-power/edge SLM. 140T: higher perf. Both use GenAI.

Export Example

optimum-cli export openvino --model google/gemma-2-2b-it --weight-format int4 --sym --ratio 1.0 --group-size 128 gemma-2-2b-ov pipe = ov_genai.LLMPipeline("gemma-2-2b-ov", "NPU")

Sources

openvinotoolkit/openvino.genai (GitHub+docs), NPU guide, supported-models, Intel 285H specs, release notes (Gemma 3/4 NPU), local diaries ([REDACTED:high-entropy].md, grok-140t-npu-primary-stack-review.md, grok-benchmark-preflight-command-plan.md, route-matrix).

Assumptions

Gemma 2B/3B export/run on this 13 TOPS NPU/140T with int4 (small favored). Optimum/GenAI stable. OpenVINO supports this Arrow Lake NPU.

Blockers

Gemma 4 maturity (LiteRT vs GenAI). No evidence on 285H yet. Driver/compiler match TBD. Packages absent. Export must succeed first. Prompt limits unknown pre-smoke.

Next Handoff

Use in gate pack. Post-smoke: record real device metrics, update matrix. Codex needed before routes.

Report: ALIVE true | Worker C | Constraints met. End.