DataTwinLabs
● Expertise
Data, AI and software. One person, one project.
Most projects need all three. Here's how the work usually moves between them, from the first conversation to going live.
Data engineering
AI & ML
Software engineering
Week 1–2
01 Explore
Figure out which decisions this should help with, and what data you already have.
Lead:Data engineering
Data engineering
leads
Check the sources — APIs, sensors, files, registries — for quality, gaps and access.
AI & ML
Judge whether prediction or search is realistic with this data, or overkill.
Software engineering
Map the users, their routine and where the result should live.
In practice → Den Bosch: traffic, weather and air-quality feeds scoped per building.
Week 2–4
02 Collect & organise
Bring the scattered data together in one place you can trust.
Lead:Data engineering
Data engineering
leads
Pipelines that run on their own, clean the data and store it in one shape.
AI & ML
Decide which signals a model will need later, and prepare them now.
Software engineering
Clear agreements on the data, so every screen shows the same numbers.
In practice → Traffic, noise and air quality streams combined into one city data model.
Week 4–7
03 Predict
Add AI only where it clearly helps someone make a better call.
Lead:AI & ML
Data engineering
Training and test data kept reproducible, so results can be checked.
AI & ML
leads
Forecasts, spotting unusual patterns, and answering questions from documents.
Software engineering
Models packaged as services that stay fast and fail gracefully.
In practice → 24-hour energy and traffic forecasts; TwinQuery AI answers policy questions with sources.
Week 6–9
04 Build the tool
Make it easy to use for people who don't care how it works.
Lead:Software engineering
Data engineering
Prepare the data so screens load in under a second.
AI & ML
Let people ask questions in plain words and get plain answers.
Software engineering
leads
3D viewer, dashboards, alerts and live updates in the browser.
In practice → Click a building and see its details, live traffic, weather and air quality in one panel.
Week 9+
05 Go live
Put it live and keep it reliable as the data and the needs change.
Lead:Software engineering
Data engineering
Alerts when data arrives late, drops off or changes shape.
AI & ML
Regular checks of predictions against reality, and retraining when needed.
Software engineering
leads
Cloud hosting, automatic updates, documentation and handover.
In practice → Forecast charts reviewed with the team before anyone relies on them.
Where the effort shifts
A rough idea of where the time goes at each stage. One area leads, but the other two never stop.
Explore
Collect & organise
Predict
Build the tool
Go live
● What teams hire for
The most requested data skills. I work in every one.
Bars show how often each skill comes up in data job demand, relative to the top one.
● Senior & architecture level
The engineering underneath the results.
Data engineering
Data platforms that stay correct as they grow
Architecture
Layered warehouse and lakehouse designs (raw, cleaned, business-ready) on Google Cloud and BigQuery.
Pipelines
Batch and streaming ETL/ELT with Airbyte, dbt, Airflow, Kafka and Spark, built to be rerun safely.
Modelling
Dimensional models and clear data contracts, so every number means the same thing everywhere.
Quality & trust
Automated tests, freshness checks, lineage and alerts before bad data reaches a report.
Cost & performance
Partitioning, clustering and incremental loads to keep cloud bills and query times down.
Software engineering
Systems built to last, not just to launch
System design
Service-oriented and microservice architectures with clean boundaries and REST APIs.
Code that others can extend
Object-oriented design, design patterns and dependency injection, reviewed and tested.
Security
Experience from banking: authentication, audit trails and careful handling of sensitive data.
Delivery
Version control, CI/CD, containers and monitoring, so releases are routine rather than risky.
Technical leadership
Leading developers, making the design calls and writing them down so the team can follow them.
● Architecture
From raw data to models in production, on one platform.
A typical end-to-end setup I design and build on Azure: ingestion, a Databricks lakehouse, machine learning tracked in MLflow, dashboards, and the DevOps and MLOps that keep it all running.
Why one person for all three
Nothing lost in handovers
The person who sets up the data also builds the model and the screen, so nothing gets lost between teams.
AI only where it pays off
I do the data work first, so I can tell you honestly when a simple rule works better than a model.
Live, not a demo
Every project ends up live, monitored and documented for your team.