Opentelemetry: How to add logs to a span

Viewed 1288

I am using OpenTelemetry java auto instrumentation in my spring boot app. Is there a way to make the application logs part of the spans that are created?

My autoconfig settings are as below:

-Dotel.traces.exporter=jaeger
-Dotel.metrics.exporter=none
-Dotel.exporter.jaeger.endpoint=http://localhost:14250
-Dotel.resource.attributes=service.name=myService
-javaagent:C:/path/to/opentelemetry-javaagent-1.0.1-all.jar
2 Answers

OpenTelemetry ships logs separately to the telemetry data obtained from auto instrumentation, and does not interleave log data I'm afraid. We ship our logs via the use of FluentBit (https://medium.com/opentelemetry/introducing-the-fluentbit-exporter-for-opentelemetry-574ec133b4b4).

You may wish to use manual instrumentation and add spans, span attributes and/or events to pertinent code blocks, to add log like context to the metadata utilised downstream.

As you are using Spring Boot, it would be advisable to use one of the starter dependencies, such as opentelemetry-otlp-exporter-starter (https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/spring/starters/otlp-exporter-starter), which should get you most of the way there. You want to use the @WithSpan annotation to decorate your methods, which will enable you to obtain the current span easily. See https://opentelemetry.lightstep.com/java/.

The official docs have a few examples, that may help, but be aware that the API and SDK are changing rapidly, so examples don't always work - https://opentelemetry.io/docs/java/manual_instrumentation/.

Detailed information regarding OpenTelemetry and logging: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/overview.md

Adding logs to spans depends a bit on the backend you are using for collecting the traces/spans and visualizing them.

I used Jaegar, which interprets OTEL events as logs in the UI, and so I wrote a custom log appender which put app logs into an event, which was picked up subsequently in the UI.

More details here:

https://stackoverflow.com/a/68739794/2715083

Related