How to Build llama.cpp from Source and Enable Metal GPU on Mac

llama.cpp is the C++ inference engine that powers Ollama, MLX-LM (partially), LM Studio, and most other local LLM tools. Running it directly — rather than through a wrapper — gives you access to the latest models and quantizations before they appear in higher-level tools, full control over compilation flags, and the ability to benchmark raw inference performance. On Mac, building with Metal support unlocks GPU acceleration using Apple’s own compute framework. This guide covers the complete build process and getting up and running with your first model.

Why Build from Source?

Most users do not need to build llama.cpp from source — Ollama wraps it and handles everything automatically. The reasons to build directly: you want the absolute latest code before Ollama ships an update, you need specific compile-time flags for unusual hardware or quantization methods, you are contributing to llama.cpp development, or you want to benchmark raw inference speed without wrapper overhead. For day-to-day local AI use, Ollama is the better experience. For developers and researchers who want the lowest level of access to the inference engine, building from source is worthwhile.

Prerequisites

# macOS (Apple Silicon or Intel)
# Install Xcode Command Line Tools (includes Metal SDK and clang)
xcode-select --install

# Install cmake via Homebrew
brew install cmake

# Verify
cmake --version  # should show 3.24+
clang --version  # should show Apple clang

Xcode Command Line Tools include the Metal framework headers needed for GPU acceleration. The full Xcode IDE is not required — just the command line tools. If you already have Xcode installed, the command line tools are included.

Building llama.cpp on Mac with Metal

# Clone the repository
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Build with Metal GPU support (Apple Silicon and Intel Macs with AMD GPU)
cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j$(sysctl -n hw.logicalcpu)

# The key binaries are now in build/bin/:
# llama-cli       — interactive chat CLI
# llama-server    — OpenAI-compatible HTTP server
# llama-bench     — benchmarking tool
# llama-quantize  — quantize models to different formats
# llama-gguf-split — split/merge large GGUF files

The -j$(sysctl -n hw.logicalcpu) flag uses all available CPU cores for parallel compilation, reducing build time from ~5 minutes to ~1-2 minutes on a modern Mac. The build produces release-optimised binaries in build/bin.

Downloading a Model

llama.cpp uses GGUF format models. Download from Hugging Face — the most popular models are available in GGUF format from Bartowski, LM Studio, and the model authors themselves:

# Install huggingface_hub for easy downloads
pip install huggingface_hub

# Download Llama 3.1 8B Q4_K_M
huggingface-cli download \
  bartowski/Meta-Llama-3.1-8B-Instruct-GGUF \
  Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  --local-dir ./models

# Or use wget directly (get the URL from the model page)
wget -P ./models \
  "https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf"

Running Your First Inference

# Interactive chat mode
./build/bin/llama-cli \
  -m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -c 4096 \
  -ngl 99 \
  --chat-template llama3 \
  -i

# Single prompt mode (non-interactive)
./build/bin/llama-cli \
  -m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -c 2048 \
  -ngl 99 \
  -p "What is Apple Silicon?" \
  -n 200

Key flags: -ngl 99 puts all layers on the GPU (Metal on Mac). -c 4096 sets the context window. --chat-template llama3 applies the correct chat format. After the first run, you will see a line like ggml_metal: loaded kernel confirming Metal GPU acceleration is active, followed by the model loading and a tokens-per-second rate during generation.

Using llama-server: OpenAI-Compatible API

The llama-server binary provides an OpenAI-compatible HTTP server — the same interface Ollama exposes, but running llama.cpp directly:

./build/bin/llama-server \
  -m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99 \
  --host 127.0.0.1 \
  --port 8080 \
  --chat-template llama3

# Test with curl
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama","messages":[{"role":"user","content":"Hello!"}]}'

Any tool pointing at an OpenAI-compatible API can now use your locally built llama.cpp server. The advantage over Ollama: you have control over exactly which version of llama.cpp is running and can use models or quantizations that have not yet been packaged into Ollama releases. The disadvantage: no model management — you manage GGUF files manually.

Benchmarking with llama-bench

# Full benchmark: prompt processing and token generation
./build/bin/llama-bench \
  -m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -ngl 99 \
  -p 512 -n 128  # process 512 tokens, generate 128

# Compare GPU vs CPU
./build/bin/llama-bench -m ./models/model.gguf -ngl 99  # GPU
./build/bin/llama-bench -m ./models/model.gguf -ngl 0   # CPU only

llama-bench outputs prompt processing speed (PP tokens/s) and generation speed (TG tokens/s) — the two key metrics. Generation speed is what you experience interactively; prompt processing speed matters for long context or batch workloads. Compare results between GPU and CPU to confirm Metal acceleration is working correctly — GPU should be 10-30x faster on Apple Silicon.

Quantizing Models with llama-quantize

If you have a full-precision GGUF model (F16 or F32), you can quantize it to smaller formats without re-downloading:

# List available quantization types
./build/bin/llama-quantize --help

# Quantize from F16 to Q4_K_M (recommended balance)
./build/bin/llama-quantize \
  ./models/model-f16.gguf \
  ./models/model-q4km.gguf \
  Q4_K_M

# Or to Q5_K_M for higher quality
./build/bin/llama-quantize \
  ./models/model-f16.gguf \
  ./models/model-q5km.gguf \
  Q5_K_M

This is particularly useful when you have downloaded a fine-tuned model in F16 format (common for community fine-tunes) and want to reduce it to a more practical size for inference. The quantize binary runs on CPU but is fast — quantizing an 8B model from F16 to Q4_K_M takes 1-3 minutes.

Keeping Up with llama.cpp Updates

llama.cpp updates very frequently — sometimes multiple times per day. New model architectures, quantization formats, and performance improvements ship continuously. To stay current:

cd llama.cpp
git pull
cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j$(sysctl -n hw.logicalcpu)

A rebuild after a git pull takes 1-2 minutes rather than the initial 5 minutes since cmake caches the build configuration. Consider pinning to specific release tags if you need stable behaviour for a long-running deployment — the main branch is always the latest and occasionally breaks compatibility with existing workflows. For production use, Ollama’s versioned releases are a more stable choice than tracking llama.cpp main directly.

When Direct llama.cpp Makes Sense

The most compelling use cases for running llama.cpp directly rather than through Ollama: you want to test a new model format or quantization type immediately after it merges into llama.cpp main, before Ollama supports it. You need specific compilation flags for unusual hardware setups. You want to contribute to llama.cpp development and need a fast edit-compile-test cycle. You want to create a custom server with specific inference settings baked in rather than passed as API parameters. For all other use cases, Ollama abstracts llama.cpp cleanly and is the better user experience — but knowing how to build and run llama.cpp directly is valuable knowledge for anyone doing serious local AI work on Mac.

Linux Build: CUDA and HIPBLAS

For Linux users with NVIDIA or AMD GPUs, the build flags differ. CUDA for NVIDIA requires the CUDA toolkit installed (nvidia-cuda-toolkit on Debian/Ubuntu) and adds the DGGML_CUDA=ON flag. ROCm/HIP for AMD requires AMD’s ROCm toolkit and uses DGGML_HIPBLAS=ON. On Linux with an NVIDIA GPU, llama.cpp with CUDA typically produces faster generation than on Mac with Metal for equivalent GPU memory bandwidth, because CUDA’s more mature compute stack has had longer to optimise transformer inference kernels. On Linux, also consider building with -DGGML_FLASH_ATTN=ON to enable Flash Attention at the llama.cpp level, which provides memory and speed benefits at long context lengths similar to the OLLAMA_FLASH_ATTENTION flag in Ollama. Ollama passes this through automatically when OLLAMA_FLASH_ATTENTION=1 is set, but building direct llama.cpp without the flag disables it by default.

Useful llama.cpp CLI Reference

A quick reference for the most useful llama-cli flags: -m MODEL path to your GGUF file; -ngl N GPU layers (99 for all on GPU, 0 for CPU only); -c N context size in tokens; -n N max tokens to generate; --temp F temperature (0.0-2.0); --top-p F top-p sampling; --repeat-penalty F repeat penalty; -t N CPU threads (for partial CPU inference); --chat-template NAME apply the correct chat format (llama3, llama2, chatml, mistral, etc.); -i interactive mode; -p TEXT initial prompt. For the full flag list, run ./build/bin/llama-cli --help. The documentation embedded in the help output is comprehensive and more current than any tutorial, since it updates with each llama.cpp release.

Troubleshooting Common Build Issues

A few issues come up regularly when building llama.cpp from source. If cmake fails with Metal-related errors, verify Xcode Command Line Tools are installed with xcode-select -p — it should print a path. If it does not, run xcode-select --install and try again. If the build succeeds but Metal is not used during inference, check the output for lines mentioning Metal — if you see only CPU backend messages, the Metal build may not have compiled correctly; try a clean build by removing the build directory and rerunning cmake. If inference is slow even with Metal, confirm -ngl 99 is in your command — without this flag, llama.cpp defaults to CPU inference even when built with Metal support. If you see memory errors with large models, reduce -ngl to a smaller number to partially offload to CPU, or use a more aggressively quantized model. For CUDA builds on Linux, ensure the CUDA toolkit version matches your driver version — version mismatches are the most common source of CUDA build failures.

The llama.cpp Ecosystem

Beyond the core binaries, llama.cpp has spawned a rich ecosystem. llama-cpp-python is the Python binding library. Ollama wraps llama.cpp’s inference behind a user-friendly server. LM Studio uses llama.cpp as one of its backends. Jan AI uses llama.cpp. text-generation-webui can use llama.cpp as its llama backend. The GGUF format developed by and for llama.cpp has become the de facto standard for distributing quantized open-weights models — virtually every model you encounter in the local AI community is distributed in GGUF format with llama.cpp compatibility as the baseline requirement. Understanding llama.cpp is therefore understanding the core infrastructure that makes the entire local AI ecosystem possible, even if you personally interact with it only through wrappers like Ollama most of the time.

Getting the Best Performance: Key Settings

After building, a few settings consistently make a meaningful difference to generation speed. Always use -ngl 99 to put all layers on GPU — even partial CPU offloading slows inference significantly. Match your context size to what you actually need: -c 2048 uses less KV cache memory than -c 32768, leaving more headroom for the model weights and producing faster generation. For Apple Silicon, Flash Attention (--flash-attn flag in recent llama.cpp versions) reduces memory use at long context and speeds generation at 8K+ context by 10-25%. Thread count for CPU operations: -t $(sysctl -n hw.perflevel0.physicalcpu) uses only the performance cores rather than efficiency cores, which is often faster for inference on Apple Silicon since the efficiency cores are much slower and add latency rather than throughput when included. These four adjustments — full GPU offload, right-sized context, Flash Attention, and performance-core threading — produce the best possible inference speed from a given hardware setup without any model changes.