Entry 01 · Case Study

Migrating IoT Telemetry Ingestion to a New Schema with Zero Event Loss

Client — Tenderd (YC S18)Year — 2025Kafka · CDC · Telemetry · Reliability

200+live devices, one ping per minute — never paused

A third-party device server that streams telemetry from the fleet was moving to a new cloud version. The change repointed writes from the existing location/alarm tables to new v2 tables. We ingest that data via CDC → Kafka, so the cutover risked dropped or duplicated pings across every device on the platform.

§ ConstraintsThe terms of the problem

No lost events — telemetry is the source of truth for utilization and safety
Live devices (~200+, each sending a ping per minute) can't be paused
Pre-existing reliability bugs in the ingestor had to be fixed as part of the work

“The cutover would have amplified the latent bugs. Fixing them first is what made zero-loss achievable.”— from the migration notes

§ ArchitectureThe approach

Δ = 0parity between old and new streams, verified before cutover

I updated the ingestor to consume from the new location_v2 / alarm_v2 CDC streams and map the new shape onto the existing internal model, cutting over without gaps. Along the way I fixed a class of Kafka reliability bugs that predated the work: offsets that weren't being committed correctly, and failure handling that let a single bad message stall or replay the whole batch — replacing it with proper per-message handling and commit semantics.

FIG. 1— Dual-run reconciliation. Both schemas consumed in parallel; the parity counter converges to Δ = 0 before the cutover line is crossed.

§ Trade-offsWhat it cost, and why it was worth it

Map new schema onto the existing internal modelDownstream consumers stayed unchanged, so the migration stayed contained to ingestion.careful mapping and validation to guarantee parity between old and new shapes

Fix the offset/poison-message bugs during the migrationThe cutover would have amplified these latent bugs; fixing them first made zero-loss achievable.larger scope than a straight repoint

§ OutcomeThe record

Cut over to the v2 telemetry schema with zero event loss
Removed a class of Kafka reliability bugs (uncommitted offsets, poison-message stalls)
More reliable ingestion underpinning utilization and safety metrics

Commissions · Enquiries

Facing a cutover, a deadline, or a pipeline you can’t pause?

This is the kind of work I take on commission — audits, ingestion builds, and enterprise module sprints, scoped in writing before work begins.

Book a call View the rate card

Next entry: the Aramco booking module →