Apple’s Core ML framework is the recommended path for deploying language models in iOS, macOS, watchOS, and tvOS apps. Where Android developers use MediaPipe or MLC-LLM, Apple developers use Core ML — it is the native ML inference framework that automatically uses whichever hardware accelerator is available on the Apple device: Neural Engine, GPU, or CPU. For LLM inference specifically, Core ML is most practical for smaller models (1B–3B parameters) on-device, with larger models delegating to cloud APIs or running on the Mac’s unified memory. This guide covers converting models to Core ML format, integrating them into an app, and performance expectations across Apple Silicon generations.
Core ML and the Apple Neural Engine
Apple Neural Engine (ANE) is the dedicated ML accelerator present in all Apple Silicon chips (A11 Bionic and later on iPhone/iPad, M1 and later on Mac). It is optimised for the matrix operations that dominate neural network inference and delivers significantly better power efficiency than GPU for supported workloads. Core ML automatically dispatches model operations to ANE when the model architecture and data types are compatible. For LLMs, this means models converted to Core ML format with ANE-optimised quantisation (int4 or int8) can achieve better battery life per token than GPU-only alternatives — an important consideration for mobile apps where battery impact directly affects user retention.
Converting a Model to Core ML Format
Apple provides the coremltools Python library for converting models to the .mlpackage or .mlmodel format Core ML uses. For Hugging Face transformer models, the workflow is: load the model in PyTorch, convert to Core ML using ct.convert() with a torch.jit.trace() of the model, and specify the compute precision (float16 for GPU, int4 or int8 for ANE). Apple’s open-source swift-transformers library and the apple/coremltools repository on GitHub provide additional tooling and example conversion scripts for popular architectures including Llama, Mistral, and OpenELM. The python_coreml_stable_diffusion repository provides conversion patterns that generalise to language models.
Figure 1 — Core ML compute units: hardware dispatch
Apple’s MLX Framework
For macOS applications on Apple Silicon, MLX is a more practical alternative to Core ML for LLM inference. MLX is Apple’s own array framework for ML research and inference, optimised for the unified memory architecture of M-series chips. The mlx-lm library runs Llama, Mistral, Qwen, Phi, and most popular open-weight LLMs directly from Hugging Face weights without conversion. It achieves the best tokens-per-second performance on Apple Silicon of any available framework — typically 20–40% faster than Ollama’s llama.cpp backend on the same hardware. For macOS app integration, embed mlx-lm via Python.framework or a Swift process bridge; for iOS, Core ML remains the primary path because MLX does not currently have an iOS deployment target. Core ML is the right choice for shipping to the App Store; MLX is the right choice for a macOS power tool or developer utility.
Integrating Core ML into a Swift App
Xcode generates a Swift wrapper class from any .mlpackage file you add to the project. Call the generated prediction method with your tokenised input, receive the output logits, apply your sampling strategy (argmax for greedy, temperature sampling for more natural text), and decode the output token IDs back to text. The Core ML model runs asynchronously — use MLModel.load(contentsOf:) with async/await and call prediction() on a background thread to avoid blocking the main UI thread. For autoregressive generation, loop prediction calls, feeding each new token back as input context, until you hit a stop token or maximum length. Use a Swift AsyncStream to publish each generated token to the UI in real time.
Model Size and App Store Distribution
The App Store has a 4GB on-demand resource limit per app download. A 1B model in int4 quantisation is approximately 500MB–700MB, which comfortably fits. A 3B model in int4 is approximately 1.5–2GB. For models larger than 2GB, use on-demand resources (App Store hosted) or download the model from your own server on first launch with a progress indicator. Never bundle large model files directly in the app binary — this bloats the initial download and causes App Review friction. Apple’s Core ML Model Deployment documentation covers the recommended patterns for managing model file lifecycle in iOS apps.
Frequently Asked Questions
- Which is better for iOS LLM inference: Core ML or MediaPipe?
- Core ML is the native Apple framework and has better Neural Engine utilisation, tighter Xcode integration, and App Store optimised distribution. MediaPipe is cross-platform (Android + iOS) with a unified API. For iOS-only apps, Core ML is the better choice. For cross-platform mobile apps, MediaPipe reduces the amount of platform-specific code.
- What is the largest model I can run on an iPhone with Core ML?
- On iPhone 15 Pro with 8GB RAM, a 3B model in int4 (approximately 1.5GB) runs reliably. Larger models risk memory pressure and app crashes. For iPad Pro M4 with 16GB RAM, 7B models in int4 become feasible. Always test memory usage with Instruments on your target device range.
- Does Core ML work on older iPhones without Neural Engine?
- Core ML falls back to GPU (Metal) on A10 Fusion and earlier devices, and to CPU if needed. Neural Engine acceleration requires A11 Bionic (iPhone X/8) or later. Older devices will work but will be slower and draw more power per inference.
Last updated: October 2026
Using Core ML with LangChain and Python Backends
Core ML is primarily an on-device framework for native iOS and macOS apps, but it also integrates into Python workflows via the coremltools library for model evaluation and conversion testing. For macOS apps that have a Python backend (for instance, a developer tool built with Python and a native Swift UI layer), you can call Core ML models from Python using coremltools.models.MLModel.predict(). This is useful for rapid prototyping and testing model quality before integrating the same model into the native Swift inference loop. The python-to-CoreML bridge is slower than native Swift inference due to data marshalling overhead, so always use native Swift for production inference in shipped apps. For prototyping and comparison testing, the Python path is convenient and requires no additional setup beyond coremltools.
Core ML vs MLX: Choosing the Right Apple Framework
The distinction matters more as Apple Silicon Macs become the primary development machine for many developers. Core ML is the right framework when: you are shipping to the App Store, targeting iOS or iPadOS, need hardware-optimised dispatch across CPU/GPU/ANE automatically, or want Xcode’s native model integration tools. MLX is the right framework when: you are building a macOS-only application, want the fastest possible inference on Apple Silicon, need flexibility to experiment with quantisation and model configurations, or are building tools for developers and researchers rather than end users. Many production AI applications on Apple platforms use both: MLX during development and benchmarking on the developer’s Mac, Core ML for the shipped iOS and macOS app.
Recommended Models for Core ML Deployment
Not every open-weight LLM converts cleanly to Core ML. The safest starting points are models with existing conversion scripts and community validation. Apple’s own OpenELM family (270M, 450M, 1.1B, 3B) is specifically designed for Apple Silicon and ships with Core ML conversion scripts in the official repository. Phi-3 Mini (3.8B) from Microsoft has strong community Core ML conversions available on Hugging Face. Llama 3.2 1B and 3B are the most widely deployed models on Apple devices in 2026, with Core ML versions maintained by Apple’s ML team. For chat-oriented models, Mistral 7B in int4 Core ML quantisation runs on iPad Pro M4 at usable speeds. Start with one of these validated models rather than attempting a custom conversion for your first Core ML integration — once the integration is working end to end, swapping to a custom model is straightforward.
Testing on Real Devices and Measuring Performance
Simulators are not representative for ML workloads, so always profile on physical hardware. Xcode includes a Core ML performance report that shows which compute units ran each part of your model and how long each layer took, which is the fastest way to find out whether the Neural Engine is really being used or whether layers are falling back to the GPU or CPU. Instruments adds memory and energy profiling, which matters because a model that is fast but drains the battery or triggers memory warnings will not survive in production.
Test on the oldest and lowest-memory device you intend to support, not just your own phone. Run long generation sessions as well as single prompts, because sustained inference causes thermal throttling and tokens per second can drop noticeably after a few minutes. Measure time to first token separately from throughput, since users judge responsiveness by how quickly text starts appearing. Record these numbers for each model and quantisation level so you can make an informed trade-off between quality, speed, and size. Re-run the same benchmarks after every operating system update, as Core ML optimisations change between releases and can shift your results in either direction.