Small language models and large language models are not competing products — they are different tools suited to different jobs, and choosing between them requires understanding which properties of a model actually matter for the task at hand. The question is not “which is better” but “which fits the constraints and requirements of this specific … Read more
NVIDIA’s Jetson Nano is an embedded computing platform built specifically for AI inference at the edge. With its 4GB RAM, 128-core Maxwell GPU, and power consumption under 10 watts, it sits at an interesting intersection: capable enough for meaningful LLM inference but constrained enough that model choice and optimisation matter significantly. Running an LLM on … Read more
llm.js (also referred to as web-llm or browser-llm in some contexts) refers to a family of JavaScript libraries that enable language model inference directly in the browser or in Node.js environments without a Python backend. The space is fragmented — several projects use similar names — but the practical goal is the same: bring LLM … Read more
WebLLM is an open-source project from the MLC (Machine Learning Compilation) team at CMU that runs large language models directly inside the web browser using WebGPU. No server, no API key, no backend — the model weights download to the user’s machine and inference runs entirely in the browser tab using the GPU through the … Read more
On-device LLM inference means running the full model inference pipeline on the hardware where the application runs — no cloud API, no server, no network dependency. For developers building desktop applications, mobile apps, embedded systems, and edge deployments, understanding the full stack of on-device inference — model formats, runtime engines, hardware acceleration, and integration patterns … Read more
Running a language model directly on a smartphone — no server, no API, no internet connection required — was a niche experiment in 2023 and is a practical deployment option in 2026. The hardware caught up: modern high-end phones have 8–12GB of RAM, powerful NPUs, and the inference frameworks to use them efficiently. This guide … Read more
Choosing the best small language model for edge deployment in 2026 means matching model capabilities to the specific constraints of your target hardware — RAM, compute, power budget, connectivity, and whether the deployment is a microcontroller, a smartphone, a Raspberry Pi, or an edge server. There is no single best model; there is a best … Read more
Alibaba’s Qwen 2.5 series offers models at every size from 0.5B to 72B, but the three smallest — 0.5B, 1.5B, and 3B — deserve particular attention because they cover the range where deployment constraints are tightest and the choice between them has the most practical consequence. All three are genuinely capable for their size, support … Read more
SmolLM2 is Hugging Face’s family of compact language models designed for efficient on-device and edge deployment. Released in late 2024, the family spans 135M, 360M, and 1.7B parameters — sizes that run on smartphones, Raspberry Pis, microcontrollers, and CPU-only environments where larger models are impractical. Despite their small size, SmolLM2 models punch above their weight … Read more
Phi-4’s benchmark results surprised many observers when Microsoft released them in late 2024: a 14B parameter model outperforming models of 30B, 70B, and in some cases even larger on reasoning-heavy tasks. Understanding these results — what the benchmarks actually measure, why Phi-4 does well on them, where it falls short, and what the numbers mean … Read more
