River monitoring with Datadog

water level (m) over time

Last month, Ard├Ęche experienced very heavy precipitations in the span of couple of hours. As a result, the dam located upriver from me opened the floodgates (literally), which caused the Chassezac level to raise by about 6.5m in about 1.5h. My basement was completely flooded, and the water level stabilized just about a 1m from the house ground floor. We had just enough time to move our belongings to the first floor. The riverside was unrecognizable, to the point where we found fish in the trees.

3 weeks later, the same thing happened, but this time the dam managers did their job. They only let enough water to make the dam wasn't overrun, while keeping everyone safe downriver.

What really bothered me though, is that at no point were we alerted of anything by EDF (the company managing the grid). No text, to alert, nothing.

Datadog to the rescue.

Using custom scripts, I now measure the river level at the station before and after my house. I also keep tabs on the amount of rain measured at these stations, as well as the general alert level.

By "chance", the first flood stopped right before the house level, and the second one stopped right before the basement. By extrapolating just a bit, I'm now able to have a good idea of the impact of a flood by looking at the river level at the station upriver.

I thus created Datadog monitors over the river level and the alert level, and I hooked them to a personal Pagerduty account, using their free tier.

I made sure to enable Critical Alerts for High Urgency in the app settings, which enables Pagerduty to override my phone volume preference, to wake me up even if is is in silent mode.

Now, if the dam managers decide to open the gates during the night (it has happened), I'll know.

pagerduty

Comments