Context
Maintaining business continuity at any cost
The situation included several high-risk factors:
-
A small-sized client (10 people) with no internal IT team.
-
The imminent departure (within a month) of the only person managing industrial IT. No handover planned, and too little time to hire.
-
Critical infrastructure running on known—but poorly understood—technologies, with a mountain of hidden technical debt.
-
And the cherry on top, a dangerously incompetent cloud host, who, without approval, permanently wiped the company’s production server during a “maintenance” task.
Our approach
Technical consulting, hands-on support and rapid response
We presented an offer in under a week to take over the infrastructure from the outgoing employee. Within two days, we got up to speed and started reinforcing the system’s deployment.
There were no backups in place—despite the system holding five years of essential business data.
And then… disaster struck:
On the very first day, just before our official engagement began, the client’s cloud provider wiped the server disk. Completely. Without warning or consent.
Faced with this catastrophe, we split responsibilities. The client dug up partial data backups and we rebuilt the server from scratch with an identical setup (CentOS 7—already end-of-life—which complicated things but helped avoid additional incompatibilities in the heat of the moment).
We then restored what we could: Grafana and InfluxDB API keys, dashboards and time-series data.
Thanks to our technical mastery, core services were restored quickly.
Once the client confirmed the system was up and running again with the available data, we offered—free of charge—to set up a decentralized backup, buying time until key decision-makers returned from vacation (and giving ourselves some peace of mind too).
Results
A documented system and a clearer view of technical de
By the end of our intervention, the client’s infrastructure was back in working order and we had full operational knowledge to maintain it.
There’s still room for improvement:
The client remains in control of their data—and knows we’ll be there, when needed, with advice and support.