Fettling my oil tank readings into influxdb.
A couple of years ago I set up rtl_433 on a pi to listen to the level sender and log the data. The readings I have are intermittent and were spread over various places. Initially they got logged to csv on the pi. Later I set up influx & told rtl_433 to send the readings there. That wasn't satisfactory as it gives you no control over the database schema. So mosquitto and rode-red were recently added to the mix and once I'd figured them out and designed my schema readings started going to a new measurement (equiv of an sql table). Useful metadata is added and the data goes in as appropriate data types instead of all floats.
Now comes the hard part, getting all the historic data in. Some was in the old measurement, some was still on the pi in csv form. I got there eventually but I hit upon a dirty little secret of node-red. The influx output node doesn't use the time in the mqtt message it receives, the timestamp sent to influx is the node-red inject time. Yep, node-red corrupts your data.
That, to me as a control system engineer, is effing nasty. Recording the
wrong timestamp on your data ffs! I'd get fired if I did that at work and rightly so.
There is a solution, don't use the influx inject node, use the influx batch inject node. That's designed (obviously) for sending multiple data points in one go but it works fine with just one. Another day of head scratching and I had data flowing via batch inject and the timestamps in influx were correct. Pity about all the corrupt data....
Luckily I'm a cautious engineer & when I got rtl_433 logging to influx and then later to mqtt I didn't stop the local csv logging
This meant I could identify, delete and re-inject the readings node-red had cocked up.
Next I need to fix the nodes I have recording temperature and humidity data. I should have correct backups for that too.
/Toddles off to see if I can submit a bug report...
