Engineer, architect, advisor · London & remote

David Farrington.

I’ve been writing software for 25+ years. The last few have been about production AI systems, ML platforms, and the architecture that keeps them running — in retail, supply chain, and IoT.

CV (docx) · GitHub · LinkedIn · Stack Overflow · projects@shipit.ltd

About

I work hands-on across the stack and prefer being close to the code. I’ve led teams and architectures, but my best work happens when I’m still in the editor — shaping a system, fixing what’s slow, and shipping something useful at the end of the week.

Over 25 years across freelance, permanent and contract — from start-ups and scale-ups through to large enterprise. Mostly Python, mostly on backend, data, ML and infrastructure problems. Recently the work is increasingly AI-shaped: LLM applications, retrieval over real-world knowledge, agents that call tools, and the eval and ops discipline that keeps any of it running. Alongside that, ML platform engineering and forecasting at scale.

Available for contract through my limited company, shipit.ltd.

Currently working on

Two pillars at the moment: production LLM systems — assistants, chatbots, multi-step agents, document/workflow automation — and ML platforms / forecasting at scale. The model itself is rarely the hardest part; most of the engineering sits around it: retrieval, tool integration, evals, observability, cost and latency, and integration with the rest of the team’s stack.

Model APIs
Anthropic Claude, OpenAI, Gemini
Retrieval
pgvector, Pinecone, Chroma, hybrid search
Agents
Tool use, structured outputs, MCP
ML platforms
Forecasting, demand modelling, MLOps, feature stores
Eval & ops
Golden sets, regression suites, tracing, prompt caching
Frameworks
Anthropic SDK, OpenAI SDK, LangChain, LlamaIndex, Pydantic AI
Surface
FastAPI, Slack, web widgets, REST, async workers
Domains
Retail, supply chain, IoT, payments

Areas of work

Backend

Python services and APIs. Tested, observable, replaceable.

Python FastAPI Django DRF Flask Celery SQS / SNS

Cloud & infra

Infrastructure as code, CI/CD, the platform engineering to keep things shipping.

AWS GCP Terraform Lambda ECS / Fargate Kubernetes Docker GitHub Actions

Data

Pipelines, warehouses, and the modelling that makes raw data useful.

Postgres BigQuery Redshift Airflow DBT Spark Pandas Athena

Architecture & leadership

System design, technical hiring, mentoring. Helping teams get unstuck.

System design DDD Microservices 12-factor Hiring Mentoring

Process

Agile that ships. CI/CD, code review, pair programming, tests that pay for themselves.

Scrum XP Kanban TDD BDD CI/CD

ML & forecasting

Production ML systems for demand, promotional uplift, coupon redemption, short-term in-store forecasting.

Forecasting Demand MLOps Feature stores BigQuery ML Vertex AI SageMaker

Event-driven & IoT

Sensor and event streams, async architectures, message-driven backends.

Event-driven IoT Sensor data Lidar Kafka EventBridge Pub/Sub

Currently exploring

Agent eval and observability, voice and multimodal interfaces, model context protocol in production.

Agent eval MCP Voice Multimodal

Experience

Twenty years across freelance, permanent, and contract work. Selected highlights below; full history on LinkedIn or in the CV.

Oct 2022 – present
3+ years

ML Platform Engineer & System Architect HEB

Forecasting and platform work on HEB’s ML team out of Texas. Mid- and long-term demand forecasting, promotional uplift, coupon redemption, and short-term demand for made-in-store products. System architecture across the platform, plus the engineering that keeps models reproducible, observable, and shipping.

ML platform Forecasting GCP BigQuery Python Vertex AI Architecture

Oct 2021 – Oct 2022
1 year

Lead Engineer BKWAI

Leading the rebuild of BKWAI’s SaaS offering and laying the foundations for their data platform — working with IoT, sensor and lidar data. Supported the rollout of their ML pipelines, ran the security review, and helped shape the broader engineering direction.

Lead engineer SaaS rebuild Data platform IoT Lidar ML pipelines Security

Apr–Oct 2021
6 months

Software & Data Engineer Zilch, Risk & Data

New microservice for credit & risk decisioning, plus the data platform and grass-roots ingestion / modelling capabilities. ETL with Python lambdas and PySpark on Glue, replication with DMS, ELT with DBT and kleene.ai, reporting in Looker. FastAPI on ECS Fargate, all provisioned with Terraform.

FastAPI Terraform AWS Glue DBT ECS Fargate

Nov 2020 – Apr 2021
6 months

Software & Data Engineer GFK, NEWRON

Working with engineers and data scientists to deliver products and insights through GFK’s market-intelligence platform. Deploying ML models on Kubernetes (GCP), GraphQL APIs, MLFlow, ELT inside Postgres.

GCP Kubernetes MLFlow GraphQL

Aug – Oct 2020
3 months

Sole Data Engineer BCG Digital Ventures, Replicant

Data-engineering support for a validation sprint in the renewable-energy space. Airflow for orchestration, AWS for infrastructure (Fargate, Lambdas, S3 data lake, Athena), terraform-managed.

Airflow AWS Athena Terraform

Apr – Jul 2020
4 months

Sole Backend Engineer ErnieApp

Took over all backend engineering — supporting live iOS/Android apps, building data capabilities, technical hiring, BI platform, integration and load testing, localisation, and helping the team move to Scrum.

Backend lead BI Load testing

Aug 2019 – Feb 2020
7 months

Senior Software / Data Engineer Heycar UK

Helping the scale-up move rapidly through commercialisation. Recruitment of engineers, scrum master and CTO. Drove data collection and best practices, built out a data lake and initial pipelines while maintaining product delivery pace.

Data lake Recruitment Scale-up

Jun – Aug 2019
3 months

Senior Python Engineer BCG Digital Ventures, Heycar

Assisting with the multi-million-pound launch of heycar.co.uk — ongoing product development, load testing, backend engineer recruitment, and taking over backend work as incubation staff rolled off.

Launch Load testing Hiring

Apr – Jun 2019
3 months

Senior Software / Data Engineer BCG Digital Ventures, Prophesea

Final phase of a market-predictions product for shipping. Cross-functional incubation team bringing data-science insight into deliverable software products.

Cross-functional Data science

Dec 2018 – Mar 2019
4 months

Lead Engineer J Sainsbury’s, Data Science

Cross-training data scientists in engineering, shaping process, implementing agile frameworks, individual tutoring, and technical leadership across the department.

Tech leadership Mentoring

Jul – Nov 2018
5 months

Senior Software Engineer, Replenishment J Sainsbury’s, Data Science

Working directly with data scientists and delivery managers to productionise POCs and ETL pipelines — from warehouse / in-store stock replenishment to checkout optimisation.

ETL Production ML

Jun 2018
1 month

Senior Software Engineer, Apps J Sainsbury’s, Android

Building, releasing and supporting the Sainsbury’s Groceries Android app — CI pipeline for continual automated release, refactoring the legacy codebase.

Android CI

Feb – May 2018
4 months

Senior Software Engineer, Alexa J Sainsbury’s

Designed and built Sainsbury’s voice-shopping Alexa skill — from initial development through systems integration to internal and closed-group customer trials. Direct work with Amazon.

Alexa Voice UI

Jan 2018
1 month

Senior Software Engineer Gamesys, Games Services

Integrating two core services for the latest in-game events feature, plus rebuilding the legacy application that serves and controls the company’s Facebook applications.

Python Integration

Aug 2016 – Dec 2018
2.5 years

Team Lead, Games Services Gamesys Ltd.

Leading a team of Python developers building and maintaining systems on Google AppEngine, Container Engine, Compute Engine, and Flexible Environment. Reporting and analytics with BigQuery and DataDog. Backlog management, hiring, training, technical roadmap.

Team lead GCP BigQuery DataDog

May – Aug 2016
4 months

Senior Python Developer, Games Services Gamesys Ltd.

Maintaining and enhancing a suite of products supporting games at scale on Facebook, Apple App Store, Google Play and Amazon App Store.

Python Cross-platform

Jun 2013 – May 2016
3 years

Lead Backend Developer & DevOps Engineer Scoota Ltd.

Product development on a suite of ad creation/serving tools. Initially front-end (JavaScript SPA), moved into back-end (Django micro-service APIs) and DevOps (Ansible, Terraform on AWS). EC2, ELB, S3, CloudFront, Redshift, Elastic Transcoder, Kinesis, RDS.

Django Microservices Ansible Terraform AWS

Dec 2011 – Jun 2013
2 years

Full Stack Developer Scoota Ltd.

Production work, tight deadlines, microsites and campaign work in Python, PHP, ASPX, HTML, CSS, JavaScript.

Full stack Multi-language

Jan 2009 – Dec 2011
2 years

IT & Communications British Design Innovation

University job — member services, IT support, web/mail/file servers, information architecture, internal business processes.

Jan 2002 – present
20+ years

Director / Full Stack Developer shipit Ltd. (Freelance)

Freelance work going back further than the formal contracts above — graphic design, web and app development, infrastructure and hosting for individuals and small companies. shipit Ltd incorporated 01/01/2018, self-employed prior.

Director, CTO

Ouiteo

Co-running an e-commerce startup that uses data and clever algorithms to automate retail and supply-chain operations for small-to-large businesses — inventory management, sales campaigns, business analytics, and forecasting.

E-commerce Forecasting Inventory Analytics Director

Support

Ground Control

Supporting and upgrading their landscaping work-scheduling and management platform. Cloud infrastructure, backend and data with event-driven patterns. Building and integrating AI-first tools for automation, insights and code quality. Also assisting with hiring.

Cloud Backend Data Event-driven AI tooling Hiring

Support

The Payments Expert

Building and integrating automation and AI tools into their workflow for more effective sales campaigns, and to make general business operations run smoother and more autonomously.

Automation AI Workflows Operations

2007–2010

University of Brighton — BSc (hons) Digital Media Development, 2:1

Project Planning & Control, Programming, Database Design, Information Systems & Architectures, 3D Graphics & Animation, Video Production, Business Information Systems, Games Development.

2005–2007

Oaklands College — National Diploma in ICT, Triple Distinction

Intermediate Computing, Business Calculations, Business Information Systems, Software Development, Visual Basic, C++, Computer Systems, Systems Analysis & Design, Web Design, Database Management, Networking.

2002–2003

Oaklands College — GNVQ in ICT

Basic computing skills, database design, project management, design & presentation.

1994–2000

Kings Langley School — GCSEs, including intermediate maths

Get in touch

Email is the easiest: projects@shipit.ltd. You can also drop a note below.

Protected by reCAPTCHA.

GitHub · LinkedIn · Stack Overflow