Peter Song

Peter Song is a machine learning, data engineer, and AI writer with a focus on practical ML workflows. He shares insights on machine learning, data tools, and MLOps.

How to Orchestrate Databricks DLT Pipelines with Airflow

by

Orchestrating Delta Live Tables pipelines within a broader data ecosystem requires integrating DLT’s declarative framework with external workflow management systems. Apache Airflow has emerged as the de facto standard for complex data orchestration, providing sophisticated scheduling, dependency management, and monitoring capabilities that complement DLT’s pipeline execution strengths. While DLT excels at managing internal pipeline dependencies … Read more

Databricks DLT Pipeline Monitoring and Debugging Guide

by

Delta Live Tables pipelines running in production require constant vigilance to maintain reliability and performance. Unlike traditional batch jobs that fail loudly and obviously, streaming pipelines can degrade silently—processing slows, data quality declines, or costs spiral without immediately apparent failures. Effective monitoring catches these issues before they impact downstream consumers, while skilled debugging resolves problems … Read more

How to Build a DLT Pipeline in Databricks Step by Step

by

Delta Live Tables (DLT) represents Databricks’ declarative framework for building reliable, maintainable data pipelines. Unlike traditional ETL approaches that require extensive boilerplate code and manual orchestration, DLT allows you to focus on transformation logic while the framework handles dependencies, error handling, data quality, and infrastructure management automatically. This paradigm shift from imperative to declarative pipeline … Read more

Top 15 Best Open Source Large Language Models

by

The open source large language model landscape has undergone a dramatic transformation, evolving from a handful of experimental models to a thriving ecosystem that rivals proprietary alternatives. What began as Meta’s bold move to release LLaMA has sparked a revolution, with tech giants, research labs, and startups contributing powerful models that anyone can use, modify, … Read more

Deep Learning with Keras: Building Neural Networks from Scratch

by

Building neural networks from scratch might sound daunting, but Keras has democratized deep learning by providing an elegant, intuitive framework that makes creating sophisticated models remarkably straightforward. Whether you’re a beginner taking your first steps into deep learning or an experienced practitioner prototyping new architectures, Keras offers the perfect balance of simplicity and power. This … Read more

OCR and Deep Learning: Building Smarter Document Processing Systems

by

Every organization drowns in documents—invoices, contracts, medical records, forms, receipts, and reports that contain critical information trapped in paper or digital images. Traditional optical character recognition systems could extract text from clean, well-formatted documents, but they struggled with real-world challenges: poor image quality, varied layouts, multiple languages, handwriting, and complex formatting. Deep learning has fundamentally … Read more

SAP Meets AI: Exploring Machine Learning in Enterprise Systems

by

Enterprise resource planning systems have long been the backbone of modern business operations, orchestrating everything from supply chains to financial reporting. SAP, the global leader in enterprise software, is undergoing a profound transformation as machine learning becomes deeply embedded into its ecosystem. This convergence of traditional ERP capabilities with artificial intelligence is not merely an … Read more

How Deep Learning Is Transforming Healthcare?

by

The healthcare industry stands at the threshold of a revolutionary change, driven by one of the most powerful technologies of our time: deep learning. This subset of artificial intelligence, inspired by the human brain’s neural networks, is fundamentally reshaping how we diagnose diseases, develop treatments, and deliver patient care. From detecting cancer with unprecedented accuracy … Read more

How to Quantize LLM Models

by

Large language models have become incredibly powerful, but their size presents a significant challenge. A model like Llama 2 70B requires approximately 140GB of memory in its full precision format, making it inaccessible to most individual developers and small organizations. Quantization offers a solution, compressing these models to a fraction of their original size while … Read more

Long-Term Memory in LLMs

by

Language models have become incredibly sophisticated, yet they’ve historically faced a critical limitation: they forget. Every conversation starts from scratch, every interaction lacks context from previous exchanges, and users must repeatedly provide the same information. Long-term memory in large language models (LLMs) represents a paradigm shift that’s transforming how AI assistants interact with users, creating … Read more