Skip to main content
Isthmus has built-in OpenTelemetry (OTel) support for distributed tracing and metrics. When enabled, every SQL query and MCP tool call emits traces and metrics to your OTel collector — Jaeger, Grafana Tempo, Datadog, Honeycomb, or any OTLP-compatible backend.

Enabling OpenTelemetry

Set the --otel CLI flag or OTEL_ENABLED environment variable:
isthmus --otel
OTEL_ENABLED=true isthmus
At startup, Isthmus logs that OTel is active:
{"level":"INFO","msg":"opentelemetry enabled"}

Configuring the exporter

Isthmus uses OTLP gRPC exporters for both traces and metrics. The OTel SDK reads standard environment variables to configure the exporter endpoint:
Env varDefaultDescription
OTEL_EXPORTER_OTLP_ENDPOINTlocalhost:4317OTLP gRPC endpoint for traces and metrics
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT(falls back to above)Override endpoint for traces only
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT(falls back to above)Override endpoint for metrics only
These are standard OTel SDK environment variables — not Isthmus-specific. See the OTel SDK docs for the full list.

Example: local Jaeger

# Start Jaeger (all-in-one, with OTLP gRPC on :4317)
docker run -d --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  jaegertracing/all-in-one:latest

# Start Isthmus with OTel
OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317 \
DATABASE_URL=postgres://user:pass@localhost:5432/mydb \
isthmus --otel
Then open http://localhost:16686 to see traces.

Example: Grafana Tempo + Prometheus

OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317 \
DATABASE_URL=postgres://user:pass@localhost:5432/mydb \
isthmus --otel

Example: MCP client config

{
  "mcpServers": {
    "isthmus": {
      "command": "isthmus",
      "args": ["--otel"],
      "env": {
        "DATABASE_URL": "postgres://user:pass@localhost:5432/mydb",
        "OTEL_EXPORTER_OTLP_ENDPOINT": "localhost:4317"
      }
    }
  }
}

Traces

Isthmus emits two types of spans:

Query spans

Every SQL query executed through the query tool creates a span:
AttributeTypeDescription
db.systemstringAlways "postgresql"
db.operation.namestring"query"
db.statementstringThe SQL statement
db.response.rowsintNumber of rows returned (on success)
If the query fails (validation error, execution error), the error is recorded on the span.

Tool call spans

Every MCP tool call (discover, describe_table, query) creates a span:
AttributeTypeDescription
mcp.toolstringTool name (e.g. "query", "describe_table")
errorbooltrue if the tool call failed
Tool call spans are parents of query spans — so a query tool call shows the full lifecycle: MCP dispatch, SQL validation, query execution, and masking.

Metrics

Isthmus exposes four metrics:
MetricTypeUnitDescription
isthmus.query.countCounterTotal number of SQL queries executed
isthmus.query.durationHistogrammsSQL query execution time
isthmus.query.errorsCounterTotal number of failed queries (validation + execution)
isthmus.tool.durationHistogrammsMCP tool call duration (end-to-end)

Useful queries

If your backend supports PromQL or a similar query language:
# Query throughput (queries per second)
rate(isthmus_query_count_total[5m])

# Error rate
rate(isthmus_query_errors_total[5m]) / rate(isthmus_query_count_total[5m])

# p95 query latency
histogram_quantile(0.95, rate(isthmus_query_duration_bucket[5m]))

# p99 tool call latency
histogram_quantile(0.99, rate(isthmus_tool_duration_bucket[5m]))

Service resource

Isthmus registers itself with the following OTel resource attributes:
AttributeValue
service.nameisthmus
service.versionBuild version (e.g. v0.5.0)
These appear in your tracing UI and can be used to filter traces.

Graceful shutdown

When Isthmus receives SIGTERM or SIGINT, it flushes all pending traces and metrics before exiting (with a 5-second timeout). This ensures no data is lost during graceful shutdown.

When OTel is disabled

When --otel is not set (the default), all tracing and metrics use no-op implementations. There is zero overhead — no goroutines, no allocations, no network calls. You can safely leave OTel disabled in development.

Semantic conventions

Isthmus follows the OpenTelemetry Semantic Conventions for database spans:
  • db.system = "postgresql" (database semconv)
  • db.operation.name for the operation type
  • db.statement for the SQL query
  • db.response.rows for the result size