Peter Song

Peter Song is a machine learning, data engineer, and AI writer with a focus on practical ML workflows. He shares insights on machine learning, data tools, and MLOps.

How to Preprocess Categorical Data in Python

by

Categorical data—variables representing discrete categories like product types, customer segments, or geographic regions—permeates real-world datasets, yet most machine learning algorithms expect numerical inputs, creating a fundamental preprocessing challenge. Unlike numerical features where values naturally exist on a scale, categorical variables encode qualitative distinctions that require thoughtful transformation into numerical representations that preserve semantic meaning while … Read more

Kernel PCA vs Linear PCA: Strengths and Limits

by

Principal Component Analysis (PCA) is one of the most widely used dimensionality reduction techniques in machine learning and data analysis. Its ability to compress high-dimensional data into fewer dimensions while retaining maximum variance makes it invaluable for visualization, noise reduction, and preprocessing. However, standard linear PCA has a fundamental limitation: it can only capture linear … Read more

Optimizing Parquet Schemas for ML Training Performance

by

Machine learning training on large datasets has become the bottleneck in modern ML workflows. While practitioners obsess over model architecture and hyperparameters, they often overlook a fundamental performance constraint: how quickly training data can be read from disk and fed into GPUs or CPUs. When training models on terabytes of data stored in Parquet files, … Read more

L1 vs L2 Regularization Impact on Sparse Feature Models

by

Regularization is a cornerstone of machine learning model training, preventing overfitting by penalizing model complexity. While most practitioners understand that L1 and L2 regularization serve this goal, the profound differences in how they shape model behavior—especially with sparse feature sets—are often underappreciated. These differences aren’t subtle theoretical curiosities but practical distinctions that determine whether your … Read more

Best Practices for AWS DMS Monitoring and Logging

by

AWS Database Migration Service (DMS) has become the go-to solution for migrating databases to AWS, enabling everything from simple lifts-and-shifts to complex heterogeneous migrations and ongoing replication for hybrid architectures. Yet the power of DMS comes with operational complexity—replication tasks can lag, fail silently during full loads, encounter data type conversion errors, or experience network … Read more

Best Learning Rate Schedules for Training Deep Neural Networks from Scratch

by

The learning rate stands as the single most influential hyperparameter in training deep neural networks, yet maintaining a fixed learning rate throughout training represents a fundamentally suboptimal strategy. When training from scratch—without transfer learning or pretrained weights—the optimization landscape changes dramatically as training progresses: early epochs require aggressive exploration with large learning rates to escape … Read more

Feature Store Design Patterns for Small Data Teams

by

Feature stores have emerged as critical infrastructure in production machine learning, promising to solve the twin challenges of training-serving skew and feature reusability across projects. Yet the canonical implementations—Feast, Tecton, or custom systems built at Uber and Airbnb—assume resources that small data teams simply don’t have: dedicated MLOps engineers, managed Kubernetes clusters, real-time streaming infrastructure, … Read more

How to Tune CatBoost Models for Structured E-commerce Data

by

CatBoost has emerged as the gradient boosting algorithm of choice for e-commerce practitioners working with structured data, and for good reason. Its native handling of categorical features eliminates the preprocessing headaches that plague other algorithms, its ordered boosting reduces overfitting on the noisy conversion signals typical in retail, and its GPU acceleration makes it practical … Read more

Nonlinear Dimensionality Reduction for High-Noise Datasets

by

High-dimensional data presents a fundamental challenge in machine learning and data science: when datasets contain hundreds or thousands of features, visualization becomes impossible, computation becomes expensive, and the curse of dimensionality causes many algorithms to fail. Dimensionality reduction techniques offer a solution by projecting data into lower dimensions while preserving important structure. However, when your … Read more

Gradient Noise Scale and Batch Size Relationship

by

When training neural networks, practitioners face a fundamental question that significantly impacts both model quality and training efficiency: what batch size should I use? The answer isn’t simply “as large as your GPU memory allows” or “stick with the default.” The relationship between batch size and gradient noise scale reveals deep insights into the optimization … Read more