Server-Emitted OpenTelemetry Signals
When OpenTelemetry is enabled, the server automatically emits telemetry signals without requiring any application changes. Two instrumentation scopes are used, corresponding to two OpenTelemetry SDK instances:
-
payara-runtime— the runtime (server-wide) SDK. Emits JVM runtime metrics and, when enabled, processes all telemetry including application signals. Using the runtime SDK is generally advised; see enabling OpenTelemetry. -
fish.payara.telemetry— the per-application SDK. Scoped to a single deployed application; emits spans and metrics for application-level subsystems (HTTP, JAX-RS, IIOP, Jakarta Concurrency, JAX-WS, MicroProfile Fault Tolerance). More limited than the runtime SDK — for example, it does not publish JVM runtime metrics.
For setup and configuration, see OpenTelemetry configuration.
Resource Attributes
Every SDK instance (both the runtime SDK and per-application SDKs) is populated with the following resource attributes by default.
Attributes that are already provided by the OpenTelemetry auto-configuration (for example, through OTEL_RESOURCE_ATTRIBUTES) are not overwritten.
| Attribute | Type | Description |
|---|---|---|
|
string |
Name of the Payara domain. |
|
string |
Name of the server instance within the domain. |
|
string |
Runtime type of the instance, lower-cased (e.g. |
|
string |
Full Payara server version string. |
|
string |
A random UUID generated each time the SDK is initialised. Uniquely identifies this instance startup. |
|
string |
JVM implementation name as reported by |
|
string |
JVM vendor string as reported by |
|
string |
JVM version string as reported by |
|
string |
Defaults to |
The payara.subsystem Attribute
Spans emitted by Jakarta EE subsystem instrumentation carry a payara.subsystem string attribute that identifies the subsystem that created the span.
This attribute provides a stable, low-cardinality way to filter or group spans by subsystem in a monitoring backend.
| Value | Subsystem |
|---|---|
|
JAX-RS client outbound calls and JAX-RS server endpoint spans. |
|
Jakarta Concurrency waiting spans (queue latency before a task starts). |
|
JAX-WS (SOAP) client outbound call spans. |
Traces (Spans)
The following spans are created automatically.
All spans belong to the fish.payara.telemetry instrumentation scope unless noted otherwise.
HTTP Server Span
| Property | Value |
|---|---|
Span name |
HTTP method at span start (e.g. |
SpanKind |
|
Trigger |
Every inbound HTTP request processed by |
| Attribute | Always present | Description |
|---|---|---|
|
yes |
Normalised HTTP method. Unknown methods are mapped to |
|
yes |
Request scheme ( |
|
yes |
Full request URI path. |
|
yes |
Server host name from the request. |
|
yes |
Server port number. |
|
if present |
Raw query string, omitted when there is no query. |
|
if present |
|
|
if present |
Remote IP address of the client. |
|
if resolved |
Low-cardinality route template (e.g. |
|
yes |
HTTP response status code. |
|
yes |
Fixed value |
|
status >= 500 only |
String representation of the status code (e.g. |
|
Note
|
The error.type attribute is only recorded in the http.server.request.duration histogram, not on the span itself; the span status is set to ERROR instead.
|
SOAP Fault Enrichment
When a JAX-WS response contains a SOAP fault, the active SERVER span (started by the HTTP layer) is enriched with the following additional attributes. No separate span is created.
| Attribute | Description |
|---|---|
|
Qualified SOAP fault code (e.g. |
|
Human-readable fault description from the endpoint (also used as the span status description). |
The span status is set to ERROR with soap.fault.string as the description.
JAX-RS Client Span
| Property | Value |
|---|---|
Span name |
HTTP method of the outbound request (e.g. |
SpanKind |
|
Trigger |
Every outbound JAX-RS client call when OpenTelemetry is enabled for the application. |
| Attribute | Always present | Description |
|---|---|---|
|
yes |
Full URI of the outbound request. |
|
yes |
HTTP method. |
|
yes |
Target host. |
|
if not -1 |
Target port. Omitted when the URI does not specify a port. |
|
yes |
Fixed value |
|
yes |
HTTP response status code received. |
Context is propagated to the downstream service via the configured OTel propagator (W3C Trace Context by default).
IIOP/CORBA Client Span
| Property | Value |
|---|---|
Span name |
CORBA operation name (e.g. |
SpanKind |
|
Trigger |
Every outbound IIOP call from the ORB client-side interceptor, via |
| Attribute | Always present | Description |
|---|---|---|
|
yes |
Fixed value |
|
yes |
CORBA operation name, identical to the span name. |
The active OTel context is propagated to the server side as a serialised W3C carrier inside an IIOP service context.
IIOP/CORBA Server Span
| Property | Value |
|---|---|
Span name |
CORBA operation name received from the client. |
SpanKind |
|
Trigger |
Remote inbound IIOP calls processed by |
| Attribute | Always present | Description |
|---|---|---|
|
yes |
Fixed value |
|
yes |
CORBA operation name. |
Jakarta Concurrency Waiting Span
| Property | Value |
|---|---|
Span name |
|
SpanKind |
|
Trigger |
Each time a task is picked up from the executor queue, immediately before the task body runs. The span starts at task submission time and ends when the worker thread picks up the task, recording queue latency. |
The waiting span ends before the parent context is made current on the worker thread, so user-created spans remain direct children of the original parent span rather than of the waiting span.
| Attribute | Always present | Description |
|---|---|---|
|
yes |
Fixed value |
|
yes |
ID of the worker thread that picked up the task. |
|
yes |
Name of the worker thread. |
|
yes |
Queue latency in milliseconds (fractional), measured from task submission to pickup. |
|
Note
|
When a contextual proxy is passivated and reactivated (for example when held by a @Stateful EJB), the transient timing fields are lost and the waiting span is silently skipped; only the OTel context is restored.
|
@WithSpan Span
| Property | Value |
|---|---|
Span name |
The |
SpanKind |
Value of the |
Trigger |
CDI bean methods annotated with |
Method parameters annotated with @io.opentelemetry.instrumentation.annotations.SpanAttribute are captured as span attributes.
To disable @WithSpan tracing for a specific method via MicroProfile Config, set:
com.example.MyBean/doWork/WithSpan/enabled=false
The config key format is [fully.qualified.ClassName]/[methodName]/WithSpan/enabled.
Disabling by method name applies to all overloads sharing that name.
Metrics
HTTP Server Metric
Instrumentation scope: fish.payara.telemetry
| Property | Value |
|---|---|
Name |
|
Instrument type |
Histogram (double) |
Unit |
|
Description |
Duration of HTTP server request processing, measured from the outermost server entry point to response completion. Async requests are measured until the async context completes. |
Explicit histogram bucket boundaries (seconds):
0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0
| Attribute | Always present | Description |
|---|---|---|
|
yes |
Normalised HTTP method. |
|
yes |
Request scheme ( |
|
yes |
HTTP response status code. |
|
if resolved |
Low-cardinality route template. Omitted when routing could not determine a template. |
|
status >= 500 only |
String representation of the status code (e.g. |
JVM Runtime Metrics
Instrumentation scope: payara-runtime
These metrics are collected at server startup using JVM MXBeans and are independent of any deployed application.
| Metric name | Instrument type | Unit | Attributes |
|---|---|---|---|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
Counter (async) |
|
— |
|
Counter (async) |
|
— |
|
UpDownCounter (async) |
|
— |
|
Counter, double (async) |
|
— |
|
UpDownCounter (async) |
|
— |
|
Gauge, double (async) |
|
— |
|
UpDownCounter (async) |
|
|
|
UpDownCounter (async) |
|
|
|
Histogram (double) |
|
|
|
Note
|
jvm.cpu.time is sourced from OperatingSystemMXBean.getProcessCpuTime() (nanoseconds) and converted to seconds.
jvm.gc.duration is sourced from GC notification events (milliseconds) and converted to seconds.
Both units are mandated by the MicroProfile Telemetry specification and TCK; they are not UCUM s in the conventional sense.
|
MicroProfile Fault Tolerance Metrics
Instrumentation scope: the application’s active meter (same scope as other application signals).
When OpenTelemetry is enabled, MicroProfile Fault Tolerance metrics are emitted as OpenTelemetry instruments. The metric names and semantics are defined by the MicroProfile Fault Tolerance — Integration with Metrics and Telemetry specification.
The following Payara-specific details apply:
-
Each metric carries the
mp.scopeattribute (the MicroProfile Metrics registry scope name, e.g.base) in addition to its standard MicroProfile tags. -
Duration histograms (
ft.timeout.executionDuration,ft.bulkhead.runningDuration,ft.bulkhead.waitingDuration) are recorded inseconds, as mandated by the MicroProfile Fault Tolerance specification. The unit stringsecondsdiffers from the UCUMsused by other OTel metrics. -
ft.circuitbreaker.state.totalis emitted as a monotonic counter with unitnanoseconds, recording cumulative nanoseconds spent in each circuit-breaker state (open,halfOpen,closed).