LMC Exporter

Troubleshooting

Use the items below to identify common LMC Exporter issues, understand likely causes, and apply the recommended resolution.

Core Configuration

Startup fails before LMC discovery

Cause

In local-file mode: missing or invalid license_file, invalid signature, invalid fingerprint, or expired license outside the grace period.

Resolution

Configure a valid license_file. For normal deployment, use the embedded public keys compiled into the binary.

Expired license warning appears

Cause

The license is expired but still inside the 36-hour grace period.

Resolution

Replace or renew the license before the grace period ends.

Gateway mode fails before LMC discovery

Cause

Gateway configuration, credentials, installation state, or the requested workload is invalid, or Gateway did not grant every requested component.

Resolution

Check the startup error, verify the endpoint and authentication profile, confirm gateway_state_directory ownership and mode 0700, and give every target a valid component_instance_id. Gateway mode requires a full grant before startup continues.

Gateway installation identity is locked or rejected

Cause

Another exporter is using the state directory, or installation-id has unsafe permissions, ownership, type, or content.

Resolution

Give each exporter installation its own persistent local state directory. Do not share or copy installation IDs, and do not delete an invalid identity to bypass the error; correct the deployment or restore the expected state.

Exporter exits after a Gateway renewal failure

Cause

The current lease expired, or Gateway returned an authoritative entitlement loss. Temporary failures permit work only while the existing lease remains valid.

Resolution

Restore Gateway connectivity or entitlement, then restart the exporter. A terminal lease closure cannot be reopened by a late response.

Startup fails with an unsupported TOML key error

Cause

The TOML file contains a key that the parser does not support.

Resolution

Remove or rename the unsupported key. Use only documented top-level, [[targets]], and [[metric_scopes]] keys.

Startup fails with `missing required target app`

Cause

A [[targets]] entry does not define app.

Resolution

Add app = "<app-name>" to every [[targets]] table.

Startup fails with `missing required target server_key`

Cause

A [[targets]] entry does not define server_key.

Resolution

Add server_key = "<key>" to every [[targets]] table.

HTTP scrape returns `503`

Cause

The requested target subtree is not fully subscribed and in sync.

Resolution

Wait for readiness, verify the configured server_key, and rerun with debug = true or --debug for diagnostics.

Multi-target scrape cannot select a target

Cause

The request does not include enough routing information.

Resolution

Include server_key, or include app and instance. If multiple targets share the same app and instance, include server_key too.

Scoped endpoint reports that target routing is required

Cause

A metric scope was selected, but multiple targets are configured.

Resolution

Add the normal target-routing query parameters, such as ?server_key=82.

No periodic snapshots appear on stdout

Cause

HTTP mode is enabled.

Resolution

This is expected. When the HTTP listener is enabled, periodic Prometheus snapshots are no longer written to stdout.

Too much diagnostic output is needed for investigation

Cause

Debug logging is disabled by default.

Resolution

Set debug = true in TOML or pass --debug.

Quick Start

Installed binary cannot start or service cannot find it

Cause

The archive does not match the host platform, a runtime library is missing, or the active-install link or service path is incorrect.

Resolution

Use the supplied EL8 or EL9 archive for your host. Verify the active-install link and platform-specific binary name from Installation. Check missing-library errors with your system administrator. Ensure the service account has shared-memory access and uses an absolute license path.

Startup fails before LMC discovery begins

Cause

In local-file mode, license validation failed or license_file is missing. For Gateway mode, use the Gateway troubleshooting items above.

Resolution

Provide a valid customer license JSON file with --license-file or license_file.

No snapshot output appears

Cause

The selected subtree is not fully subscribed and in sync yet, or no metric mappings match the selected app and paths.

Resolution

Re-run with --debug, confirm the server key and app value, and verify that the selected root is discovered.

HTTP request returns `503`

Cause

The requested target subtree is not ready, or the selected root, target object, or shared-memory server was lost.

Resolution

Wait for initial sync, confirm the query routing parameters, and check diagnostics with --debug.

HTTP scrape returns the wrong target or cannot disambiguate

Cause

Multiple configured targets share the same app and instance.

Resolution

Include server_key in the scrape query, such as /metrics?server_key=82.

Metrics are not written to `stdout` in HTTP mode

Cause

This is expected behavior.

Resolution

Query the HTTP endpoint instead. When HTTP mode is enabled, periodic snapshots are not written to stdout.

CLI command is rejected

Cause

A required value is missing or an option value is invalid.

Resolution

Check the configured targets or --server-key values, --metrics-interval is greater than 0, and --metrics-port is between 1 and 65535.

Exporter Targets

HTTP scrape returns `503`

Cause

The requested target subtree is not ready yet.

Resolution

Wait for the initial sync to complete. If it persists, verify server_key, app, instance, and shared-memory visibility.

Scrape cannot identify a target

Cause

Multiple targets are configured and the query does not include enough routing information.

Resolution

Add server_key, or use a unique app and instance combination.

Scrape by `app` and `instance` is ambiguous

Cause

More than one target has the same app and instance.

Resolution

Include server_key in the query.

TOML `[[targets]]` entries are ignored

Cause

CLI flags such as --server-key or --instance cleared the target tables and selected global target settings. Multiple global server keys still create multiple targets.

Resolution

Remove the overriding CLI flags or move the target selection into TOML.

No metrics are returned for a configured target

Cause

The selected app mapping, classes, seed path groups, or seed paths may not match the subscribed data.

Resolution

Confirm the app value and review any classes, seed_path_groups, or seed_paths filters.

Startup fails before discovery

Cause

In local-file mode, license validation failed or license_file is missing. For Gateway mode, use the Gateway troubleshooting items above.

Resolution

Configure a valid license_file. The exporter validates the license before opening the shared-memory pool.

Legacy ADH MQoS groups do not work

Cause

Legacy m_qo_s_* aliases are no longer valid.

Resolution

Use consumer_mqos_service_service, mqos_service_attributes, and mqos_service_datastreams.

ADH and RRCP Trans Metrics

New ADH metrics are missing after upgrade

Cause

Existing filters exclude the new classes or paths, or those objects are absent from the deployed ADH.

Resolution

Review the new class/group selectors in Metric Configuration, target readiness, and scoped metric patterns. Verify the expected objects exist in the source.

Timestamp metric is absent or shows the wrong event age

Cause

The source value is blank or unsupported, the process timezone differs from the source, or a yearless value is interpreted in the current year.

Resolution

Check the source value and scrape diagnostic comments. Match the exporter timezone to the application and validate populated dates, especially across year boundaries. Do not replace missing timestamp samples with zero.

ADH queries or joins change after upgrade

Cause

The new RRCP driver and ThreadBindingStats CPU labels change existing series identities.

Resolution

Review label matching, recording rules, joins, and cardinality using the ADH label changes. Allow for the transition from old to new series.

Gateway rejects a workload after adding RRCP Trans

Cause

The Gateway deployment does not meet the Keystone Gateway v0.2.1 requirement, lacks the rrcp_trans entitlement, or the target lacks a valid stable component identity.

Resolution

Use Keystone Gateway v0.2.1 for RRCP Trans licensing, provision the rrcp_trans entitlement, and set component_instance_id for every target. The complete workload must be granted before any target starts collecting.

Confirm