mljourney

A practical guide to PyTorch CUDA memory management: how the caching allocator works, reading memory_stats and memory_summary, finding memory leaks with allocation snapshots, what empty_cache actually does, fixing fragmentation with expandable_segments, zero_grad set_to_none, and a systematic approach to diagnosing OOM errors at training time.

A practical guide to Mistral Nemo 12B for local use: what makes it distinctive including a native 128K context window, strong multilingual support, and the efficient Tekken tokeniser, hardware requirements at each quantisation level, running it with Ollama and configuring a 32K context Modelfile, benchmarking it against Llama 3.2 8B for your specific tasks, the four scenarios where its VRAM premium is justified, and how it compares to Mistral 7B and Mixtral 8x7B.

A practical guide to fine-tuning embedding models with contrastive learning: the Multiple Negatives Ranking Loss objective, building training datasets with synthetic query generation, hard negative mining, Matryoshka Representation Learning for flexible dimensions, evaluation with InformationRetrievalEvaluator, and when domain adaptation is actually worth the engineering cost.

A practical guide to connecting local LLMs to Obsidian for AI-powered note-taking: setting up Smart Connections with nomic-embed-text embeddings to search and chat across your entire vault, using the Ollama plugin for in-note summarisation and writing assistance, the BMO Chatbot for a persistent sidebar assistant, practical workflows for daily note processing, research synthesis, idea generation, and meeting note cleanup, and context window configuration for long notes.

A practical guide to using torch.profiler for GPU bottleneck analysis: profiler setup and schedule configuration, reading TensorBoard traces, identifying DataLoader gaps and CPU-GPU sync stalls, memory profiling, CUDA kernel occupancy, distributed training profiling, and a step-by-step workflow for translating profiler output into concrete training optimizations.

A practical guide to running vision models locally with Ollama: available models including LLaVA, moondream2, Qwen2.5-VL and Gemma 3, using vision models from the CLI with image flags, calling them via the Ollama Python library and the OpenAI-compatible API with base64-encoded images, batch processing a folder of images, and practical use cases including screenshot OCR, chart analysis, document data extraction, and inventory cataloguing.

A practical comparison of Pinecone, Weaviate, and Qdrant for production RAG systems: architecture, filtered search performance, hybrid BM25 + vector search, cost at scale, ecosystem integrations, and a decision framework for choosing between fully managed and self-hosted vector databases.

A complete guide to running Ollama in Docker: CPU-only and NVIDIA GPU passthrough with the NVIDIA Container Toolkit, AMD ROCm setup, pulling models into the running container, a full Docker Compose stack combining Ollama with Open WebUI, environment variables for parallel requests and model keep-alive, persisting models across redeployments with named volumes, and when to use Docker versus native Ollama installation.

A practical comparison of sparse retrieval (BM25), dense retrieval (embeddings), and hybrid search for RAG pipelines: how each method works, where each fails, reciprocal rank fusion, learned sparse models like SPLADE and ColBERT, reranking, and how to choose the right configuration for your corpus and query distribution.

A complete guide to Msty, the local LLM desktop app built for multi-model comparison: connecting to existing Ollama models, running the same prompt against multiple models simultaneously, attaching document Knowledge collections to conversations, conversation branching for exploring different approaches, the prompts library for reusable system prompts, integrating cloud providers like OpenAI and Anthropic alongside local models, and how Msty compares to Jan, LM Studio, and Open WebUI.