Android NNAPI LLM Deployment: How to Use Hardware Acceleration

Android Neural Networks API (NNAPI) is Android’s hardware abstraction layer for accelerated ML inference. When you run a language model on Android using TensorFlow Lite, ONNX Runtime, or MediaPipe, NNAPI is what routes the computation to the appropriate hardware accelerator — Qualcomm’s Hexagon DSP, MediaTek’s APU, Samsung’s NPU, or Mali GPU — depending on which chip is in the device. Understanding how NNAPI works, how to verify it is actually delegating to hardware, and how to configure it correctly is essential for Android LLM deployment that performs well across device tiers.

Important update: Google deprecated NNAPI in Android 15. Its official migration guide now points developers to TensorFlow Lite (LiteRT) in Google Play services, optionally with the GPU delegate, and to AICore for Gemini Nano. NNAPI still works on many devices, so this guide remains useful for understanding existing code and supporting older devices, but do not start new projects that depend on it. Treat the NNAPI sections below as legacy guidance and plan a migration.

What NNAPI Does

NNAPI is not an inference runtime itself — it is a delegation layer that sits between ML frameworks and device-specific hardware drivers. When you enable the NNAPI delegate in TensorFlow Lite or ONNX Runtime, the framework passes the model’s computation graph to NNAPI, which then queries the available hardware accelerators on that specific device and determines which operations each accelerator supports. Operations supported in hardware run on the NPU, DSP, or GPU; unsupported operations fall back to CPU. The set of supported operations varies by device and Android version — a Snapdragon 8 Gen 2 supports far more NNAPI operations than a mid-range MediaTek device from 2021.

NNAPI vs Direct Hardware Frameworks

The alternative to NNAPI is using device-manufacturer-specific frameworks directly: Qualcomm’s QNN (Qualcomm Neural Networks), MediaTek’s NeuroPilot, or Samsung’s One API. These vendor SDKs provide tighter hardware integration and often better performance than NNAPI on their respective chips. The tradeoff is portability — code written against QNN only runs on Qualcomm devices. NNAPI provides a single API that works across all Android devices, with each manufacturer providing NNAPI drivers that map to their hardware. For production apps targeting the full Android device market, NNAPI is the right abstraction. For performance-critical tools targeting a specific chip (edge devices with known hardware), the vendor SDK may be worth the portability cost.

Figure 1 — NNAPI delegation: how operations route to hardware

Framework NNAPI path Hardware target TFLiteNnApiDelegate()Qualcomm Hexagon / MediaTek APU / Mali GPU ONNX RuntimeNnapiExecutionProviderSame hardware via NNAPI drivers MediaPipeGPU delegate (OpenCL)GPU only, bypasses NNAPI for LLM

Enabling NNAPI in TensorFlow Lite

In TensorFlow Lite for Android, add the NnApiDelegate to the interpreter options. The delegate automatically queries available hardware and partitions the model graph: operations supported in hardware run accelerated, operations not supported fall back to CPU. Set the acceleratorName to a specific processor if you want to target a known hardware unit explicitly (for example, “qti-default” for Qualcomm devices), or leave it unset to let NNAPI choose the best available accelerator automatically. Enable the allowFp16PrecisionForFp32 option for models stored in float32 — this allows NNAPI to compute in float16, which is faster on most NPUs at a small precision cost.

Verifying NNAPI Delegation

A critical and often skipped step: verifying that your operations are actually running on hardware, not silently falling back to CPU. There are two approaches. First, use adb logcat and filter for NNAPI-related messages — a successful delegation shows which operations were accepted by the hardware accelerator and which fell back to CPU. Second, use Android Profiler in Android Studio to observe CPU utilisation during inference — if CPU is near 100% during generation, NNAPI is not delegating effectively. Third, time your inference with NNAPI enabled vs disabled and compare — if NNAPI is helping, enabled inference should be significantly faster (2–5x on supported hardware). If the times are similar, the hardware accelerator is not being used for most operations.

NNAPI and LLM Operations: The Coverage Gap

The honest assessment of NNAPI for LLM inference in 2026: operation coverage for transformer architectures is incomplete on many devices. The attention mechanism operations, particularly fused multi-head attention, are not supported in NNAPI on most Android devices. This means that for many LLM operations, NNAPI falls back to CPU, negating the acceleration benefit. The layers most likely to benefit from NNAPI acceleration are the linear projections (matrix multiplications) in the feed-forward and projection layers. The attention computation typically runs on CPU regardless of NNAPI settings on devices without dedicated transformer hardware. Qualcomm Snapdragon 8 Gen 2 and Gen 3 have the best NNAPI LLM coverage; budget and mid-range devices have minimal hardware acceleration for transformer inference.

ONNX Runtime Mobile with NNAPI

ONNX Runtime Mobile is an alternative to TFLite for Android LLM deployment. Export your model to ONNX format (using optimum or transformers.onnx), then load it with ONNX Runtime Mobile on Android with the NNAPI execution provider enabled. ONNX Runtime’s NNAPI provider follows the same delegation pattern as TFLite — supported ops go to hardware, others fall back to CPU. ONNX Runtime has broader operator support than TFLite for transformer models, which means more operations may successfully delegate to NNAPI hardware. For models you are already working with in ONNX format, ONNX Runtime Mobile is worth benchmarking against TFLite on your target device.

LiteRT: The Current Android ML Standard

In 2025, Google unified TensorFlow Lite and related mobile ML tools under the LiteRT brand. LiteRT is the current recommended framework for Android on-device ML, including LLM inference. It retains NNAPI delegation support and adds improved transformer operation coverage compared to the original TFLite. For new Android projects starting in 2026, LiteRT is the appropriate starting point rather than the original TFLite, though the migration from TFLite to LiteRT is straightforward and the API is largely compatible.

Frequently Asked Questions

Does NNAPI work on all Android devices?
NNAPI is available on all Android devices running Android 8.1 (API level 27) or higher. However, hardware acceleration quality varies dramatically by device — a flagship Snapdragon device has much better NNAPI coverage than a budget MediaTek device. Always test on your minimum supported device tier.
Is NNAPI the same as the NPU?
No. NNAPI is the software API; the NPU (Neural Processing Unit) is the hardware. NNAPI is what your code calls; the NNAPI drivers route computation to the NPU (or GPU or DSP) depending on what is available and what operations are supported.
What Android API level is required for good NNAPI LLM support?
Android 10 (API 29) improved NNAPI significantly with version 1.2. Android 11 (API 30) added further improvements. For the best LLM operation coverage, target Android 11+ devices. For production apps, test on Android 10 as your minimum and document which operations fall back to CPU.

Last updated: October 2026

Optimising NNAPI Performance in Practice

Several practical optimisations improve NNAPI LLM performance on Android. Quantise your model to int8 before deployment — NNAPI hardware accelerators support int8 operations far more broadly than float16 or float32, meaning more operations successfully delegate to hardware rather than falling back to CPU. Use dynamic range quantisation via TFLite’s post-training quantisation tools, which converts float32 model weights to int8 with minimal accuracy loss. Enable the cache directory option in NnApiDelegate so that NNAPI’s compilation result is cached to disk — this eliminates the 1–3 second compilation overhead on subsequent launches of the app. Set the execution preference to SUSTAINED_SPEED for inference tasks that run continuously (like streaming token generation) rather than FAST_SINGLE_ANSWER, which is optimised for single one-shot inference. Profile your specific model on representative devices using Android Studio’s profiler before committing to a deployment configuration, as NNAPI behaviour is highly device-specific and benchmarks from one chipset do not predict performance on another.

Practical Recommendation for 2026

For Android LLM deployment in 2026, start with LiteRT in Google Play services and its GPU delegate as the primary path, with CPU as the automatic fallback. Because NNAPI has been deprecated since Android 15, use it only to support existing code or older devices. Test NNAPI delegation on your three target device tiers — flagship (Snapdragon 8 Gen 2+), mid-range (Snapdragon 6 Gen 1 or MediaTek Dimensity 900), and entry-level (MediaTek Helio G series) — and document which operations actually delegate to hardware on each. For models where NNAPI provides minimal benefit on your target devices, consider MLC-LLM as an alternative — it compiles models specifically for each device’s instruction set and often achieves better performance than NNAPI delegation for transformer architectures.

NNAPI represents Android’s effort to provide a unified hardware acceleration API across a deeply fragmented device ecosystem. The quality of that abstraction varies enormously by device and chipset generation. Testing thoroughly on representative devices, verifying delegation is actually occurring, and using quantisation to maximise the number of operations that successfully route to hardware are the three practices that separate a fast Android LLM deployment from one that silently runs on CPU despite NNAPI being enabled.

Designing a Safe Fallback Strategy

Because hardware acceleration varies so much between devices, never assume it works. Try to initialise the accelerated path inside a try-catch block, and fall back to the CPU if initialisation fails or the result looks wrong. Remember the outcome per device so you do not repeat a failing attempt on every launch, and expose a hidden setting that lets support staff force CPU mode when diagnosing a problem. Report which path was used in your analytics so you can see what share of your users actually benefit.

Leave a Comment