# Troubleshooting

Use the items below to identify common LMC Exporter issues, understand likely causes, and apply the recommended resolution.

## Core Configuration

## Startup fails before LMC discovery

### Cause

In local-file mode: missing or invalid `license_file`, invalid signature, invalid fingerprint, or expired license outside the grace period.

### Resolution

Configure a valid `license_file`. For normal deployment, use the embedded public keys compiled into the binary.

## Expired license warning appears

### Cause

The license is expired but still inside the 36-hour grace period.

### Resolution

Replace or renew the license before the grace period ends.

## Gateway mode fails before LMC discovery

### Cause

Gateway configuration, credentials, installation state, or the requested workload is invalid, or Gateway did not grant every requested component.

### Resolution

Check the startup error, verify the endpoint and authentication profile, confirm `gateway_state_directory` ownership and mode `0700`, and give every target a valid `component_instance_id`. Gateway mode requires a full grant before startup continues.

## Gateway installation identity is locked or rejected

### Cause

Another exporter is using the state directory, or `installation-id` has unsafe permissions, ownership, type, or content.

### Resolution

Give each exporter installation its own persistent local state directory. Do not share or copy installation IDs, and do not delete an invalid identity to bypass the error; correct the deployment or restore the expected state.

## Exporter exits after a Gateway renewal failure

### Cause

The current lease expired, or Gateway returned an authoritative entitlement loss. Temporary failures permit work only while the existing lease remains valid.

### Resolution

Restore Gateway connectivity or entitlement, then restart the exporter. A terminal lease closure cannot be reopened by a late response.

## Startup fails with an unsupported TOML key error

### Cause

The TOML file contains a key that the parser does not support.

### Resolution

Remove or rename the unsupported key. Use only documented top-level, `[[targets]]`, and `[[metric_scopes]]` keys.

## Startup fails with \`missing required target app\`

### Cause

A `[[targets]]` entry does not define `app`.

### Resolution

Add `app = "<app-name>"` to every `[[targets]]` table.

## Startup fails with \`missing required target server\_key\`

### Cause

A `[[targets]]` entry does not define `server_key`.

### Resolution

Add `server_key = "<key>"` to every `[[targets]]` table.

## HTTP scrape returns \`503\`

### Cause

The requested target subtree is not fully subscribed and in sync.

### Resolution

Wait for readiness, verify the configured `server_key`, and rerun with `debug = true` or `--debug` for diagnostics.

## Multi-target scrape cannot select a target

### Cause

The request does not include enough routing information.

### Resolution

Include `server_key`, or include `app` and `instance`. If multiple targets share the same `app` and `instance`, include `server_key` too.

## Scoped endpoint reports that target routing is required

### Cause

A metric scope was selected, but multiple targets are configured.

### Resolution

Add the normal target-routing query parameters, such as `?server_key=82`.

## No periodic snapshots appear on stdout

### Cause

HTTP mode is enabled.

### Resolution

This is expected. When the HTTP listener is enabled, periodic Prometheus snapshots are no longer written to `stdout`.

## Too much diagnostic output is needed for investigation

### Cause

Debug logging is disabled by default.

### Resolution

Set `debug = true` in TOML or pass `--debug`.

## Quick Start

## Installed binary cannot start or service cannot find it

### Cause

The archive does not match the host platform, a runtime library is missing, or the active-install link or service path is incorrect.

### Resolution

Use the supplied EL8 or EL9 archive for your host. Verify the active-install link and platform-specific binary name from [Installation](https://portal.cjcit.io/knowledge-base/lmc_exporter/0.19/getting-started/installation/). Check missing-library errors with your system administrator. Ensure the service account has shared-memory access and uses an absolute license path.

## Startup fails before LMC discovery begins

### Cause

In local-file mode, license validation failed or `license_file` is missing. For Gateway mode, use the Gateway troubleshooting items above.

### Resolution

Provide a valid customer license JSON file with `--license-file` or `license_file`.

## No snapshot output appears

### Cause

The selected subtree is not fully subscribed and in sync yet, or no metric mappings match the selected app and paths.

### Resolution

Re-run with `--debug`, confirm the server key and app value, and verify that the selected root is discovered.

## HTTP request returns \`503\`

### Cause

The requested target subtree is not ready, or the selected root, target object, or shared-memory server was lost.

### Resolution

Wait for initial sync, confirm the query routing parameters, and check diagnostics with `--debug`.

## HTTP scrape returns the wrong target or cannot disambiguate

### Cause

Multiple configured targets share the same app and instance.

### Resolution

Include `server_key` in the scrape query, such as `/metrics?server_key=82`.

## Metrics are not written to \`stdout\` in HTTP mode

### Cause

This is expected behavior.

### Resolution

Query the HTTP endpoint instead. When HTTP mode is enabled, periodic snapshots are not written to `stdout`.

## CLI command is rejected

### Cause

A required value is missing or an option value is invalid.

### Resolution

Check the configured targets or `--server-key` values, `--metrics-interval` is greater than `0`, and `--metrics-port` is between `1` and `65535`.

## Exporter Targets

## HTTP scrape returns \`503\`

### Cause

The requested target subtree is not ready yet.

### Resolution

Wait for the initial sync to complete. If it persists, verify `server_key`, `app`, `instance`, and shared-memory visibility.

## Scrape cannot identify a target

### Cause

Multiple targets are configured and the query does not include enough routing information.

### Resolution

Add `server_key`, or use a unique `app` and `instance` combination.

## Scrape by \`app\` and \`instance\` is ambiguous

### Cause

More than one target has the same `app` and `instance`.

### Resolution

Include `server_key` in the query.

## TOML \`\[\[targets\]\]\` entries are ignored

### Cause

CLI flags such as `--server-key` or `--instance` cleared the target tables and selected global target settings. Multiple global server keys still create multiple targets.

### Resolution

Remove the overriding CLI flags or move the target selection into TOML.

## No metrics are returned for a configured target

### Cause

The selected app mapping, classes, seed path groups, or seed paths may not match the subscribed data.

### Resolution

Confirm the `app` value and review any `classes`, `seed_path_groups`, or `seed_paths` filters.

## Startup fails before discovery

### Cause

In local-file mode, license validation failed or `license_file` is missing. For Gateway mode, use the Gateway troubleshooting items above.

### Resolution

Configure a valid `license_file`. The exporter validates the license before opening the shared-memory pool.

## Legacy ADH MQoS groups do not work

### Cause

Legacy `m_qo_s_*` aliases are no longer valid.

### Resolution

Use `consumer_mqos_service_service`, `mqos_service_attributes`, and `mqos_service_datastreams`.

## ADH and RRCP Trans Metrics

## New ADH metrics are missing after upgrade

### Cause

Existing filters exclude the new classes or paths, or those objects are absent from the deployed ADH.

### Resolution

Review the new class/group selectors in [Metric Configuration](https://portal.cjcit.io/knowledge-base/lmc_exporter/0.19/configuration/metric-configuration/#adh-coverage), target readiness, and scoped metric patterns. Verify the expected objects exist in the source.

## Timestamp metric is absent or shows the wrong event age

### Cause

The source value is blank or unsupported, the process timezone differs from the source, or a yearless value is interpreted in the current year.

### Resolution

Check the source value and scrape diagnostic comments. Match the exporter timezone to the application and validate populated dates, especially across year boundaries. Do not replace missing timestamp samples with zero.

## ADH queries or joins change after upgrade

### Cause

The new RRCP driver and ThreadBindingStats CPU labels change existing series identities.

### Resolution

Review label matching, recording rules, joins, and cardinality using the [ADH label changes](https://portal.cjcit.io/knowledge-base/lmc_exporter/0.19/configuration/metric-configuration/#adh-coverage). Allow for the transition from old to new series.

## Gateway rejects a workload after adding RRCP Trans

### Cause

The Gateway deployment does not meet the Keystone Gateway v0.2.1 requirement, lacks the `rrcp_trans` entitlement, or the target lacks a valid stable component identity.

### Resolution

Use Keystone Gateway v0.2.1 for RRCP Trans licensing, provision the `rrcp_trans` entitlement, and set `component_instance_id` for every target. The complete workload must be granted before any target starts collecting.