Export opentelemetry metrics to open search

user_17148159 0
Viewed 134

I am trying to export opentelemetry metrics to open search.

My configurations are as mentioned below

 metrics-pipeline:
  source:
    otel_metrics_source::
  processor:
    - otel_metrics_raw_processor:
  sink:
    - opensearch:
        hosts: ["https://<domain-name>:443"]
        insecure: true
        username: "username"
        password: "password

I was going through one of the data prepper issue and came to know that metrics support is included recently.

https://github.com/opensearch-project/data-prepper/issues/242

I am not able to find proper documentation on this.

In the data prepper pod, getting the below exception

    com.amazon.dataprepper.model.plugin.NoPluginFoundException: Unable to find a plugin named 'otel_metrics_source:'. Please ensure that plugin is annotated with appropriate values.
    at com.amazon.dataprepper.plugin.DefaultPluginFactory.lambda$getPluginClass$2(DefaultPluginFactory.java:111) ~[data-prepper.jar:1.5.1]
    at java.util.Optional.orElseThrow(Optional.java:401) ~[?:?]
    at com.amazon.dataprepper.plugin.DefaultPluginFactory.getPluginClass(DefaultPluginFactory.java:111) ~[data-prepper.jar:1.5.1]
    at com.amazon.dataprepper.plugin.DefaultPluginFactory.loadPlugin(DefaultPluginFactory.java:62) ~[data-prepper.jar:1.5.1]

Appreciate any inputs on this.

opensearch-dashboards amazon-opensearch open-telemetry opensearch
2 Answers
user_140145370

Just remove the extra colon at the end of otel_metrics_source , set ssl flag to false and add the index in opensearch section.

Thank you @Karsten Schnitter for your help.

Updated configurations

    metrics-pipeline:
         source:
            otel_metrics_source:
              ssl: false
         processor:
            - otel_metrics_raw_processor:
         sink:
            - opensearch:
                 hosts: ["https://<domain-name>:443"]
                 insecure: true
                 username: "username"
                 password: "password
                 index: metrics-otel-v1-%{yyyy.MM.dd}
user_171481590
Related