● Blog

Notes from real projects

Streaming, lakehouses, AI and forecasting: what worked, with the code to show it.

1 December 2024 · 8 min

Building real-time data pipelines with Kafka and Spark

How I design and run streaming pipelines with Apache Kafka and Spark Structured Streaming for production workloads.

KafkaSparkStreaming
→
15 November 2024 · 6 min

RAG pipelines for business LLM applications

How to build retrieval-augmented generation that answers from your own data, and shows where each answer came from.

LLMRAGAI
→
28 October 2024 · 7 min

Delta Lake best practices for lakehouses

Patterns for building reliable, fast lakehouses with Delta Lake on Databricks, from partitioning to maintenance.

Delta LakeDatabricksLakehouse
→
10 October 2024 · 5 min

Predictive maintenance with machine learning

Building models that spot equipment failures before they happen, so repairs can be planned instead of rushed.

Machine learningIoTForecasting
→