mljourney

llama.cpp and Ollama are not alternatives to each other — Ollama wraps llama.cpp. But developers often face a choice between using Ollama’s API and using llama.cpp directly through its Python bindings. This guide covers the real differences in speed, flexibility, and developer experience, and when the llama-cpp-python library is the better choice over the Ollama … Read more

llama.cpp is the C++ inference engine that powers Ollama, MLX-LM (partially), LM Studio, and most other local LLM tools. Running it directly — rather than through a wrapper — gives you access to the latest models and quantizations before they appear in higher-level tools, full control over compilation flags, and the ability to benchmark raw … Read more

Local AI finds genuinely useful applications in some less-discussed professional and personal contexts. This guide covers three specific use cases — education for children, research assistance for journalists, and note-taking support for therapists — where the privacy, cost, and customisability of local LLMs make them particularly well-suited, and where cloud AI alternatives raise concerns worth … Read more

This guide covers the more advanced aspects of Apple MLX: how to quantize your own models, where to find the best community models, how MLX compares to llama.cpp in real-world performance, and practical Python workflows for building MLX-based applications. If you are new to MLX, the getting started guide earlier in this series covers the … Read more

One of MLX’s most compelling features is the ability to fine-tune LLMs directly on Apple Silicon — something Ollama does not support. LoRA (Low-Rank Adaptation) fine-tuning lets you adapt a pre-trained model to your specific data, domain, or style using a fraction of the memory and compute required for full fine-tuning. On an M4 Pro … Read more

Apple MLX-LM makes it straightforward to run popular open-weights models directly on Apple Silicon Macs — Llama, Mistral, Qwen, Gemma, Phi, and many others. This guide covers the specific setup for the most commonly used models, performance expectations on different Mac configurations, and how to get the best results from each model family. Finding MLX … Read more

MLX is Apple’s open-source machine learning framework built specifically for Apple Silicon. Where Ollama is a general-purpose local LLM server, MLX is a lower-level framework — closer to PyTorch in concept — that lets you run and manipulate models with full access to the computation graph, while taking full advantage of Apple Silicon’s unified memory … Read more

Choosing between a Mac and a Windows PC for local AI work in 2026 is a genuinely interesting hardware decision — not a simple “NVIDIA is always better” answer. Apple Silicon’s unified memory architecture gives it meaningful advantages for large model inference, while NVIDIA maintains leads in raw throughput, fine-tuning, and ecosystem breadth. This guide … Read more

OpenAI’s Whisper is an open-source speech recognition model that runs locally and transcribes audio with impressive accuracy across dozens of languages. It handles podcasts, video interviews, meetings, lectures, voice memos, and phone call recordings — any audio content you want converted to text. This guide covers setting up local Whisper transcription, the practical workflow for … Read more

Multimodal local LLMs — models that accept both images and text — can read text from images with surprising accuracy. While they are not replacements for dedicated OCR engines on all tasks, they excel at extracting structured information from complex documents, understanding context around text, and handling layouts that traditional OCR struggles with. This guide … Read more