I am currently working on an application (Qt) which needs to deal with a (for me) huge data stream. I have 3 sensors which each provide 3 measurement values (+ a timestamp) at about 25 kHz. Im pretty sure, that there will be more sensors in the future. The application should run 24/7.
The task is now to
- collect the data from the sensors
- convert the measurement values
- save them to a file
- and visualize the converted values.
Part 1 and 2 are working. For part 3 I do have a simple ofstream doing its task well.
For part 4 I am currently unsure how to deal with the high storage amount the whole application will need. I need to visualize different parts of the measurements, sometimes the whole period (with low resolution), but sometimes also only a short period with nearly full resolution. I´m actually storing all the values in a QVector and drawing the requested period in my custom QQuickItem which will then be shown via qml.
This concept does have 2 major problems:
- the performance
- the limited QVector size
Are there any other qt containers/concepts which will perform better?
Or do I need to use something like a time-series databases? If so, any recommendations for a free one working (offline) under Windows (with Qt)?