OpenTelemetry Support

Runtime (Server-Wide) OpenTelemetry

The runtime SDK is managed by the Monitoring Service and emits server-level signals such as JVM runtime metrics and the HTTP request-duration histogram. When the runtime SDK is enabled it also processes application telemetry — spans and metrics emitted by deployed applications are handled by the same SDK instance. Using the runtime SDK is therefore generally advised.

Enable it by either:

Per-Application OpenTelemetry

A per-application SDK can be activated independently of the runtime SDK by setting the MicroProfile Config property otel.sdk.disabled=false for the application (or the equivalent system property / environment variable OTEL_SDK_DISABLED=false). Note that the per-application SDK is more limited than the runtime SDK — for example, JVM runtime metrics are only published by the runtime SDK and are not available from a per-application SDK.

Default Settings

The per-application SDK uses OpenTelemetry autoconfiguration. With no other configuration properties specified, the per-application OpenTelemetry components will:

The runtime (server-wide) SDK uses explicit configuration. Unless configured, all exporters — traces, metrics, and logs — default to none and no signals are exported automatically.

To easily receive traces in default configuration run the following Jaeger container and then access its UI at port 16686:

 
docker run --name jaeger \
   -e COLLECTOR_OTLP_ENABLED=true \
   -p 16686:16686 \
   -p 4317:4317 \
   -p 4318:4318 \
   jaegertracing/all-in-one:latest

Span Convention Settings

In order to improve consistency among products our implementation offers several strategies of naming and tagging spans of Jakarta REST client and server calls.

These are configured by specifying the MicroProfile Config property payara.telemetry.span-convention for your application.

The following values are accepted:

Suppressing Export Warning Message

When the export endpoint is not available in default setting the following error message is logged by logger io.opentelemetry.exported.internal.grpc.OkHttpGrpcExporter:

 
Failed to export spans. The request could not be executed. Full error message: Failed to connect to `localhost/127.0.0.1:4317`

If you don’t intend to run an OTEL endpoint on localhost or at all you need to configure the exporter by means of MP Config Properties, system properties, or environment variables:

Dependencies

The appropriate versions of APIs are provided by Payara BOM artifact matching the version of the Azul Payara distribution you are using.

Azul Payara provides the following APIs, and it is advised to mark them as provided dependencies in your application and don’t package them with it:

Using Opentelemetry Manually

 
import io.opentelemetry.api.trace.Tracer;

@RequestScoped
public class TracedComponent {

    @Inject
    Tracer tracer;

    public void tracedMethod() {
        var span = tracer.spanBuilder("tracedTask").startSpan();

        try (var scope = span.makeCurrent()) {
            // do work, add information to span
            if (span.isRecording()) {
                // compute expensive events or tags
                // only if the span is being sampled
            }
        } finally {
            span.setStatus(StatusCode.OK);
            span.end();
        }
    }
}

Provider Components in Library Directory

Autoconfiguration can pick up components, which are placed in directory domain_dir/lib for example by means of the asadmin add-library command.

Provider Components in Application Code

Applications can declare and use their own OpenTelemetry components such as exporters or samplers by writing against autoconfiguration SPI and placing appropriate Service Loader resource in the application

 
fish.payara.example.LogExporter.Provider

 
public class LogExporter implements SpanExporter {

    private static final Logger LOGGER = Logger.getLogger(LogExporter.class.getName());

    @Override
    public CompletableResultCode export(Collection<SpanData> spans) {
        spans.forEach(s -> LOGGER.info(s.toString()));
        return CompletableResultCode.ofSuccess();
    }

    @Override
    public CompletableResultCode flush() {
        return CompletableResultCode.ofSuccess();
    }

    @Override
    public CompletableResultCode shutdown() {
        return CompletableResultCode.ofSuccess();
    }

    // This is registered as SPI and creates configured exporter
    public static class Provider implements ConfigurableSpanExporterProvider {

        @Override
        public SpanExporter createExporter(ConfigProperties configProperties) {
            return new LogExporter();
        }

        @Override
        public String getName() {
            return "logs";
        }
    }
}