Cross-platform on-device models → react-native-executorch (0.10+); Apple-only → @react-native-ai/apple (Apple's own models, nothing to ship); local RAG → react-native-rag 0.10 on executorch 0.10.4+ (packaged), or hand-roll it with @react-native-ai/llama + nitro-sqlite/sqlite-vec. Fast-moving (confidence: low): validate on-device perf before committing.
private/offline LLM over local docs → @react-native-ai/llama + nitro-sqlite/sqlite-vec (budget RAM for generator + embedder TOGETHER; see reading)
packaged RAG on react-native-executorch 0.10 → react-native-rag 0.10 (@react-native-rag/executorch peers executorch ^0.10.4; still on executorch 0.9 → react-native-rag 0.9.0)
Android with Gemini (on-device Nano or cloud) via ADK → @react-native-ai/adk
iOS 26+ only and want MLX with TTS/STT in one Nitro module → react-native-nitro-mlx (early; pin)
Low confidence — fast-moving or lightly-vetted domain: treat the pick as a vetted lead and prototype before committing.
Options & tradeoffs
the field considered — and why each one isn’t the default here
option
tradeoff
evidence
react-native-executorch
cross-platform on-device models (Meta ExecuTorch) with per-silicon delegates and worklet-callable JSI; tool-calling/embeddings since 0.4, OCR since 0.3. 0.10.5 (2026-10-06, verified vs npm 2026-10-08); 0.10 was a GROUND-UP REWRITE, and it still pins react-native-worklets <0.13, which rules out Reanimated 4.7 — see note: EXECUTORCH 0.10.
17k/wk
@react-native-ai/apple
Apple on-device LLM + SpeechAnalyzer (iOS 26); zero cloud, no model to ship; RN 0.80+ (preview). 0.12.0 (2026-01-28, verified vs npm 2026-09-24): eight months without a release, so the preview label is doing real work.
@react-native-ai/adk
Google Gemini on ANDROID via the Agent Development Kit: on-device Gemini Nano + cloud models behind one Vercel-AI-SDK provider; tool calling + streaming; same react-native-ai family as the Apple provider. 0.12.1 (2026-07-07, verified vs npm 2026-09-24).
react-native-nitro-mlx
Apple MLX on-device LLM + TTS/STT via Nitro Modules, with tool calling (ChatSession API); iOS 26+ ONLY; v0.7 (2026-08-22, verified vs npm 2026-09-24), ~100★ — niche/early.
34/wk
react-native-rag
on-device RAG over ExecuTorchLLM with a local vector store. 0.10.0 (2026-10-05, verified vs npm 2026-10-08) SHIPPED the executorch 0.10 migration: @react-native-rag/executorch 0.10.0 peers react-native-executorch ^0.10.4, @react-native-rag/op-sqlite 0.10.0 alongside; 0.9.0 remains the line for executorch 0.9. See note: RAG VERSION PINCER.
195/wk
@react-native-ai/llama
llama.cpp behind the Vercel AI SDK provider interface: runs GGUF chat AND embedding models on one engine, which makes a hand-rolled local RAG pipeline practical; pair with react-native-nitro-sqlite + sqlite-vec (see reading: Fitting RAG in Your Pocket). 0.12.0 (2026-01-28, verified vs npm 2026-09-24).
speech-to-text without network; whisper.rn (0.7.2, ~800★, verified vs npm + repo 2026-07-28) is the concrete binding — whisper.cpp under the hood, and it also carries NVIDIA Parakeet ASR alongside Whisper.
Choose a runtime, not a model: the runtime (loading, tensors, hardware backend, JS bindings) changes slowly, while the model leaderboard churns monthly, so this entry selects runtimes and leaves model choice to your own on-device benchmarks. react-native-executorch holds the cross-platform lane because it is the only option that treats both platforms and multiple accelerators as first-class, and since 0.10 its orchestration is TypeScript you can edit. Apple's own stack wins the Apple-only lane by shipping no model at all. Local RAG has a packaged path again since react-native-rag 0.10 followed the executorch rewrite (2026-10-05), with one engine plus a vector store as the hand-rolled alternative. The area outpaces any settled recommendation (confidence: low) → validate on real devices.
EXECUTORCH 0.10 REWRITE (2026-09-08, verified vs the GitHub release + the 0.10.1 patch; npm latest 0.10.2, 2026-09-11): the library was rebuilt on a TWO-LAYER architecture — native now exposes only the ExecuTorch runtime, tensors and fast ops, while task orchestration moved into transparent TypeScript pipelines you can read, edit, chain, or replace with a custom model, where 0.9 shipped opaque C++ task modules with rigid interfaces. (1) HARDWARE ACCELERATION: 0.9 was mostly CPU; 0.10 wires per-silicon delegates across 130+ pre-exported model variants (Core ML / Apple Neural Engine, 70+ variants · MLX over Metal for LLM generation, ~40 exports · Vulkan on Android, 25+ exports), with the multi-threaded XNNPACK CPU path as the fallback; the vendor frames the win as inference speed, thermal throttling and battery, not just latency. (2) WORKLETS: every native JSI function carries "worklet"; by default, so model calls run off the main JS thread or inside UI worklets, which is what makes live-camera inference with VisionCamera straightforward. (3) BINARY SIZE: backend binaries are fetched on demand at install (zero-config by default), and a `react-native-executorch` block in package.json narrows the download to the backends/tasks you use. Migration: 100% functional parity is claimed and a `react-native-executorch/legacy` entry point preserves the pre-0.10 API for screen-by-screen migration, but it is marked DEPRECATED and slated for removal → a stepping stone, not a resting place. Take 0.10.1 (2026-09-09) or later, not 0.10.0: 0.10.1 fixes iOS `use_frameworks!` duplicate-header build failures and silent setup failures, and pins react-native-worklets >=0.10.0 <0.13.0 (unchanged in 0.10.2 and in 0.10.5, 2026-10-06, verified vs npm 2026-10-08). Reanimated 4.7 requires Worklets 0.13.x (RB-E-ANIMATION), so an app on executorch 0.10.x stays on Reanimated 4.6 (Worklets 0.12.x, RN 0.83–0.87) until executorch widens that range. The repo also publishes vX.Y.Z-libs GitHub tags (native library builds, v0.10.5-libs on 2026-09-22); those are not npm versions of the package.
RAG VERSION PINCER (2026-09-24, verified vs npm + the GitHub repo): react-native-rag is still 0.9.0 (last publish 2026-05-26) and built against executorch's pre-0.10 API, while executorch's legacy entry point for that API is deprecated. A migration is now in flight: PR #23 "feat(executorch)!: migrate to react-native-executorch 0.10" has been open since 2026-09-17 with commits through 2026-09-22 — breaking, unmerged and unreleased. So the risk moved from "may not follow" to "is following, not shipped" → roll the pipeline yourself on executorch 0.10 today, and re-check the package when a release carrying PR #23 appears. RESOLVED 2026-10-08 (verified vs npm): react-native-rag 0.10.0 published 2026-10-05 with @react-native-rag/executorch 0.10.0 (peers react-native-executorch ^0.10.4, react-native-worklets >=0.10.0 <0.13.0) and @react-native-rag/op-sqlite 0.10.0 → the packaged path is back on the current executorch; the hand-rolled llama path stays the option when you want one engine for embedder and generator.
RB-E-AI-UI owns cloud-LLM app UI; RB-E-MEDIA owns camera pipelines; RB-E-ANIMATION owns the worklets runtime; RB-E-NATIVE owns JSI memory and threading; RB-E-STORAGE owns the vector store.
HISTORY: 2026-10-08 REOPENED: react-native-rag 0.10.0 shipped the executorch 0.10 migration — packaged RAG is back on the current executorch; default lists both paths. 2026-10-08 CORRECTED: executorch npm latest is 0.10.5 (2026-10-06); 0.10.4 (2026-09-28) became a real npm release after the 09-24 correction. Its Worklets <0.13 pin keeps executorch apps on Reanimated 4.6. 2026-09-24 CORRECTED: executorch npm latest is 0.10.2 — the "0.10.4" cited 2026-09-21 is a v0.10.4-libs GitHub tag; @react-native-ai/apple 0.12.0 was published 2026-01-28 (2026-02-11 is the registry's metadata-modified date). 2026-09-24 REOPENED: react-native-rag PR #23 migrates to executorch 0.10 (open, breaking, unreleased); the default local-RAG pick moved to the @react-native-ai/llama path, matching the existing when-clause. 2026-09-21 CHALLENGED → WEAKENED: re-check — rag still 0.9.0 against executorch 0.10.x. 2026-09-10 CHALLENGED → WEAKENED: react-native-rag predates the executorch rewrite; @react-native-ai/apple months quiet. 2026-09-10 GAP CLOSED: executorch v0.10 ground-up rewrite (TS pipelines, Core ML/MLX/Vulkan delegates, worklets, on-demand binaries). 2026-08-24 GAP CLOSED: nitro-mlx pin v0.5 → v0.7. 2026-08-18 GAP CLOSED: local-RAG build reading (llama.cpp + sqlite-vec) + @react-native-ai/llama row. 2026-08-04 GAP CLOSED: Margelo on-device face-recognition reading. 2026-07-28 GAP CLOSED: whisper.rn named as the offline-ASR binding (+ Parakeet).
Canonical reading
Editorial annotations on why each piece matters — the articles themselves are the originals; read them there.
The build behind this entry's react-native-rag / local-RAG option rows, done by hand so every constraint is visible: a WhatsApp export chunked, embedded and searched entirely on a $100 Android phone in airplane mode. The stack is one engine and one store — @react-native-ai/llama (llama.cpp) runs BOTH the embedder and the generator, and react-native-nitro-sqlite statically links sqlite-vec so KNN is a vec0 virtual table queried with `WHERE embedding MATCH ? AND k = ?`, no second ML runtime and nothing fetched at runtime. What makes it durable is the failure list. MEMORY IS THE CEILING, not accuracy: ~900MB usable RAM holds Qwen2.5-0.5B-Instruct Q4 (469MB) beside the 119MB embedder, and Qwen2.5-1.5B (1.07GB) hard-crashed the process with an uncatchable native OOM that JS cannot trap. The context window is paid for in KV-cache RAM, so it is capped at 4,096 tokens and a 21,000-token export is REJECTED outright ('Context is full'), not truncated — which is the whole reason retrieval exists here. Query and documents must be embedded by the same model with the same normalization, or search returns noise (the author calls this the number-one RAG pitfall). The multilingual embedder (paraphrase-multilingual-MiniLM-L12-v2, 384 dims, ~119MB at Q4) is worth its size the moment chats aren't pure English — all-MiniLM-L6 collapsed non-English messages to nearly identical vectors. Chunk boundaries are part of the data model: strip the speaker or timestamp and the model hallucinates them back. And a small model needs an explicit 'say you don't know' escape plus replayed recent turns, or follow-up questions retrieve the wrong thing entirely.
A reasoned cost/latency/privacy case for on-device TTS (Kokoro via react-native-executorch), with the economics and the tradeoffs (voice/language coverage, a custom C++ phonemizer). A durable framing for the on-device-vs-cloud decision.
Survey evidence that on-device AI is production-proven, not experimental: Apple ships multimodal-embedding photo search + hybrid Siri; Meta runs ExecuTorch in WhatsApp (network prediction), Instagram (SqueezeSAM cutouts) and Messenger (transcription); Google ships scam detection + Gemini Nano; Snapchat runs 10MB AR models. Ends in RN ExecuTorch pointers (Gemma/Qwen, ~30MB quantized Whisper Tiny STT) — vendor self-promotion, they author the lib. The durable framing: 'fine-tuning is what makes a small model good enough.'
The end-to-end build this entry's option rows only imply: a fully on-device recognition pipeline — detection → tracking → alignment → embedding → matching → liveness — on VisionCamera 5 frame processors, with ONNX Runtime (CPU/XNNPACK) for inference and Reanimated shared values bridging worklet and JS. Two things make it durable knowledge rather than a tutorial. First, the architectural reason it can work at all: the frame processor gets each YUV frame synchronously on a dedicated native thread as a worklet, with no bridge serialization and no pixel copy into JS. Second, real numbers on a Galaxy S21 Ultra — YuNet detection (~230KB model) 3–6ms/frame at 128×128, SFace embedding (37MB, 19MB int8) ~29ms, ~39ms end-to-end per detection cycle, gallery matching in microseconds for 512 entries — plus the budget trick of running detection every 10th frame and letting tracking bridge the gaps. Uses ONNX Runtime rather than this entry's ExecuTorch default, which is itself a useful data point.
The reviewed long-form essay behind this entry — the why, not a how-to. Also on GitHub.
About on-device AI and ML in React Native
The distinction that organises everything: a runtime is not a model
The single most common confusion in this area is treating "which on-device AI library"
and "which model" as one decision. They are not, and conflating them produces advice with
a shelf life measured in weeks.
A runtime is what this entry selects: the thing that loads a model, moves tensors,
picks a hardware backend, and gives JavaScript a way to call it. Runtimes change slowly
and their differences are architectural — which platforms, which accelerators, what the
threading story is, how big the binary gets.
A model is the weights you feed that runtime, and the leaderboard churns monthly.
This corpus deliberately does not rank models: a recommendation that names this quarter's
best small LLM is wrong by the next harvest, and a reader who needs that is better served
by a benchmark than by an encyclopedia. What the corpus can usefully say is which runtime
gives you the freedom to swap models when the leaderboard moves.
Keep the layers separate and the rest of the entry reads cleanly.
The default, and why
Cross-platform on-device models → react-native-executorch (0.10+). Apple-only →
@react-native-ai/apple. Local RAG → @react-native-ai/llama plus a local vector store, until
react-native-rag releases support for the executorch rewrite (below).
executorch takes the cross-platform lane because it is the only option here that treats
both platforms and multiple accelerators as first-class, and because its v0.10
rewrite changed what kind of library it is. Before that release, tasks were opaque native
C++ modules: you got the pipeline its authors imagined, and custom model execution ran
into rigid interfaces. After it, the native layer exposes the runtime, tensors and fast
operators, while orchestration lives in TypeScript pipelines you can read, edit, chain, or
replace. That is the difference between a library that ships features and a library you
can build on — and it is why the swap-the-model freedom above is real here rather than
aspirational.
The same release wired per-silicon acceleration across a large set of pre-exported model
variants — Core ML reaching the Apple Neural Engine, MLX over Metal for LLM generation on
Apple silicon, Vulkan for Android GPUs — with a multi-threaded XNNPACK CPU path as the
fallback for devices that have no accelerator. The vendor frames that win as inference
speed, thermal throttling and battery, and the framing is the right one: on a phone,
sustained workloads fail on heat and power long before they fail on raw latency.
Two more properties of that release matter more than they sound. Every native JSI function
carries the worklet directive by default, so model calls can run off the main JavaScript
thread or inside UI worklets — which is what makes live-camera inference tractable rather
than janky. And backend binaries are fetched on demand at install, with a package.json
block to narrow the download to the backends you actually use, because on-device AI
runtimes are heavy and shipping every accelerator you do not call is pure app-size cost.
@react-native-ai/apple wins the Apple-only lane by not competing at all: it exposes
Apple's own on-device LLM and SpeechAnalyzer, so there is no model to ship and nothing to
accelerate. That is the cheapest possible on-device story when the constraint permits it.
Its preview label is doing real work — its latest release, 0.12.0, dates from 2026-01-28
— so treat it as a platform bet rather than a maintained dependency.
The failure mode this entry now names: a version pincer
The RAG leg carries a caveat that a 2026-09-10 challenge surfaced, and it is the kind of
problem that only appears when you look at two packages together.
react-native-rag is built against executorch's pre-rewrite API. executorch has since
shipped the rewrite, kept the old surface alive behind a legacy entry point, and marked
that entry point deprecated and slated for removal. The RAG library's last release is
still 0.9.0 (2026-05-26). So a team adopting local RAG today faces a pincer rather than a
choice: pin executorch to the old API and inherit a deprecation with a stated expiry, or
take the new architecture before the RAG layer has followed.
The library is following: as of 2026-09-24 a breaking pull request migrating it to
executorch 0.10 (#23) is open and active, but unmerged and unreleased. Until a release
carries it, the advice holds: on the rewrite, assemble the pipeline yourself from a
generation engine plus a local vector store, and verify the pairing on a spike before
committing a feature to it. Re-check the package when that release lands.
The landscape, and when each piece earns its place
Speech is the most mature on-device task and the one most likely to justify itself
commercially, because transcription is expensive in the cloud and privacy-sensitive by
nature. expo-speech-recognition covers the managed path; whisper.rn is the concrete
binding when you want whisper.cpp models, and it now carries NVIDIA Parakeet alongside
Whisper — which is itself a useful signal that the runtime-versus-model split is real:
the binding outlived the model it was named after.
Platform-specific providers. On Android, @react-native-ai/adk puts Google's Gemini —
on-device Gemini Nano and cloud models — behind one Vercel-AI-SDK provider. On iOS 26+,
react-native-nitro-mlx runs Apple MLX LLMs plus TTS/STT through Nitro Modules; it is early
and niche, so pin it.
Rolling your own RAG is the default lane today, not a fallback. A single engine that
runs both chat and embedding models means one dependency for the two halves of retrieval,
which is precisely what makes a hand-rolled pipeline practical instead of a research
project. Pair it with a local vector store.
The reading attached to this entry carries the constraint that catches teams out: budget
RAM for the generator and the embedder together. Two models resident at once is the
memory profile you actually ship, not the one you measured while testing them separately.
Tradeoffs and failure modes to name out loud
Ranking models in a durable document. The leaderboard churns monthly. Select a
runtime; benchmark models yourself, on your target devices.
Measuring latency and forgetting thermals. A phone that hits its power budget
throttles, and sustained inference is exactly the workload that gets there.
Ignoring binary size until submission. On-device runtimes and accelerator libraries
are heavy; narrow the backends you link.
Assuming two libraries that mention each other are actually in step. The RAG pincer
above is the concrete instance. Check publish dates on both sides of an integration.
Treating a preview label as decoration. Where a package is labelled preview and has
been quiet, the label is the more reliable of the two signals.
Committing before a device spike. This entry is confidence: low on purpose. Validate
on-device performance on real hardware before designing a feature around it.
How it interacts with the rest of the stack
This entry is on-device inference; cloud-LLM application UI belongs to RB-E-AI-UI, and
the two are frequently mixed in one product. Camera-driven inference sits against
RB-E-MEDIA, and the worklet threading that makes it smooth is RB-E-ANIMATION's
worklets runtime doing the work. The custom-native questions underneath — how a JSI module
should own memory and threads — belong to RB-E-NATIVE. A local vector store is a
storage decision (RB-E-STORAGE).
In one paragraph
Choose a runtime, not a model. On-device AI in React Native has a clear cross-platform
answer in executorch, whose rewrite turned it into a library you can build pipelines on
rather than one you accept pipelines from, with real per-silicon acceleration and an
install that no longer drags every backend along. Take Apple's own stack when the product
is Apple-only and you are content with a platform bet. For local RAG, know that the
packaged option predates the rewrite it depends on (its migration is an open pull request,
not a release) — assemble the pipeline yourself on the new architecture, budget RAM for generator and embedder together,
and spike on real devices before you design around any of it.
react-brain