How to save file in Raspberry pi before stream to Influxdb and visualize in Grafana?

user_19767717 0
Viewed 42

I have created a python code for send my GPS data to influxdb by using influxdb.clients. Is that can anyone help me to give any ways to save the data in the raspberry pi if there is no internet connection, then after getting the connection send straightway all the data to the Influxdb which i have created already?

from datetime import datetime
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS
import serial

# Setup database
token = "<mytoken>"
org = "<myorg>"
bucket = "<mybucket>"

with InfluxDBClient(url="<influxurl>", token=token, org=org) as client:
    write_api = client.write_api(write_options=SYNCHRONOUS)
    # Setup dataload
    json_dataload = []

    ser = serial.Serial("/dev/ttyS0")
    gpgga_info = "$GPGGA,"

    GPGGA_buffer = 0
    NMEA_buff = 0


    def convert_to_degrees(raw_value):
        decimal_value = raw_value / 100.00
        degrees = int(decimal_value)
        mm_mmmm = (decimal_value - int(decimal_value)) / 0.6
        position = degrees + mm_mmmm
        position = "%.4f" % position
        return position


    while True:
        received_data = str(ser.readline())  # read NMEA string received
        GPGGA_data_available = received_data.find(gpgga_info)  # check for NMEA>
        if (GPGGA_data_available > 0):
            GPGGA_buffer = received_data.split("$GPGGA,", 1)[1]  # store data com>
            NMEA_buff = (GPGGA_buffer.split(','))
            nmea_latitude = []
            nmea_longitude = []

            extract_latitude = NMEA_buff[1]  # extract latitude from >
            extract_longitude = NMEA_buff[3]  # extract longitude from>
            lat = float(extract_latitude)
            lat = convert_to_degrees(lat)
            longi = float(extract_longitude)
            longi = convert_to_degrees(longi)

            point = Point("latest GPS") \
            .field("latitude", lat) \
            .field("longitude", longi) \
            .time(datetime.utcnow(), WritePrecision.NS)

            json_dataload.append(point)

            # Send our payload

            write_api.write(bucket, org,json_dataload)
raspberry-pi4 influxdb gps python
1 Answers

Just setup Telegraf service on your Raspberry PI and write all data to it instead of InfluxDB. Telegraf has a buffer which stores not written data and perform a flush when InfluxDB is available again. What is more, you will be able to batch your data within Telegraf and collect some additional metrics such as CPU/MEM usage of your Raspberry PI.

To do so, install Telegraf Telegraf downloads (remember to choose appropiate CPU arch and OS) and edit config: /etc/telegraf/telegraf.conf In this file you should config http input for InfluxDB data [[inputs.http_listener]] or [[inputs.http_listener_v2]]. Then config [[outputs.influxdb_v2]] for InfluxDB v2.x data output. After that, restart telegraf service and check its status. If it is "running", you should be able to write data to Telegraf.

Telegraf has many options/plugins which may allow you to collect, buffer and send data to InfluxDB.

Check Telegraf repository for plugin documentation and some configuration examples: Telegraf plugins repository

Ad1:

In Telegraf plugins there is also [[inputs.tail]], using this plugin, you can configure data input from file/directory. This option may be good if you need some persistence during no internet connection period (in case of power outages). If power outages are not a problem, you should use http listener plugin.

user_166399130
Related