Routing IoTHub device telemetry messages to storage account and Power BI

Question

I am attempting to route device telemetry data for a device connected to Azure IoTHub.

I have defined the custom endpoint in message routing to a storage account with the Encoding format set to JSON and routing query set to true.

This has successfully sent the data to the storage account but the telemetry data in the message body is in base 64 shown below

{"EnqueuedTimeUtc":"2022-07-13T13:03:28.4770000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"SensorTile","connectionAuthMethod":"{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}","connectionDeviceGenerationId":"6*********971","enqueuedTime":"2022-07-13T13:03:28.4770000Z"},"Body":"eyJBY2NlbGVyb21ldGVyIjp7IlkiOi0xNSwiWCI6MTAsIloiOjEwMzZ9LCJ0cyI6IjIwMjItMDctMTNUMTU6MDM6MjguNDAwKzAyMDAiLCJpZCI6IlNlbnNvclRpbGUifQ=="}

Reading the documentation "When using JSON encoding, you must set the contentType to application/json and contentEncoding to UTF-8 in the message system properties. Both of these values are case-insensitive. If the content encoding is not set, then IoT Hub will write the messages in base 64 encoded format."

I understand it is possible to translate the data to a UTF-8 format by setting the systemProperties contentType to application/json and contentEncoding to UTF-8 but I am unsure where and how to actually do this or can I use another service such as Stream Analytics/Fuctions/EventHub to achive this?

Also is it possible to filter messages via route query so that only telemetry data is routed ignoring the rest?

Any help is greatly appreciated

user_8800237 · Answer

The content type and content encoding need to be set when sending the message. That means the device is in charge of setting the properties. Here's a few examples in different languages.

C#

using var eventMessage = new Message(Encoding.UTF8.GetBytes(messagePayload))
{
    ContentEncoding = Encoding.UTF8.ToString(),
    ContentType = "application/json",
};

C

IOTHUB_MESSAGE_HANDLE message_handle = IoTHubMessage_CreateFromString(message);
(void)IoTHubMessage_SetContentTypeSystemProperty(message_handle, "application%2fjson");
(void)IoTHubMessage_SetContentEncodingSystemProperty(message_handle, "utf-8");

Java

Message msg = new Message(msgStr);
msg.setContentType("application/json");
msg.setProperty("temperatureAlert", temperature > 28 ? "true" : "false");

user_5238158 · Answer

Setting content type and content encoding is the responsibility of the IoT device that is sending messages to the IoT hub. Setting these values depends on the language-specific device SDK used by the IoT device.

Without these settings, the message routing queries based on the message body won't work as mentioned in this link. Also to filter messages based on the telemetry data, you don't need filtering queries. You can create a route with 'Data source' - 'Device Telemetry Messages' so that only device telemetry data will be routed. Please find the attached screenshot for reference:

Routing IoTHub device telemetry messages to storage account and Power BI

2 Answers