Is there a way for me to extract request information from headers/body using telegraf

user_13247682 0
Viewed 44

So I have telegraf installed on one of our ec2 instances. And on the instance we have a spring boot application installed servicing POST requests and forwarding them on.

I was wondering if it is possible to extract information from requests coming into this service such as header information or key value pairs from a json body and then forward this information onto telegraf?

Telegraf would be then push this information to influxdb.

The idea is that we will extract this information and display using grafana so we can visualise how many requests are coming from where.

I know there are some plugins like http_listener but I can't tell from the readme if this could possibly work or if there is a better way to get this working?

Thanks for any information you can provide in advance!

tig telegraf influxdb monitoring rest
1 Answers

You could try work with this sample. The sample code could be:

[[inputs.http]]
# URL for your data in JSON format
urls = ["https://yourURL/sample.json"]

# Overwrite measurement name from default `http` to `someOtherMeasurement`
name_override = "someOtherMeasurement"

# Data from HTTP in JSON format
data_format = "json_v2"

      # Add a subtable to use the `json_v2` parser
      [[inputs.http.json_v2]]

          # Add an object subtable for to parse a JSON object
          [[inputs.http.json_v2.object]]

              # Parse data in `data.stations` path only
              path = "data.stations"

              #Set station metadata as tags
              tags = ["yourTags"]

              # Latest station information reported at `last_reported`
              timestamp_key = "theTimeStamp"

              # Time is reported in unix timestamp format
              timestamp_format = "unix"
              
              # all other json key-value pairs will be turned into fields

An excerpt of original HTTP JSON response is:

{
   "data":{
      "stations":[
         {
            "last_reported":1655171050,
            "is_renting":1,
            "num_bikes_available":21,
            "is_installed":1,
            "legacy_id":"72",
            "station_status":"active",
            "num_ebikes_available":2,
            "is_returning":1,
            "eightd_has_available_keys":false,
            "num_docks_disabled":0,
            "station_id":"72",
            "num_bikes_disabled":1,
            "num_docks_available":32
         },
         "..."
      ]
   },
   "last_updated":1655171085,
   "ttl":5
}

Above Telegraf script will turn the JSON into following line protocol:

someOtherMeasurement,station_id=72 eightd_has_available_keys=false,is_installed=1,is_renting=1,is_returning=1,legacy_id="72",num_bikes_available=21,num_bikes_disabled=1,num_docks_available=32,num_docks_disabled=1,num_ebikes_available=2,station_status="active" 1655171050000000000

You could play with the sample JSON with Telegraf to update your configurations. Happy coding.

user_184889550
Related