# OpenTelemetry signals

> Traces, metrics, logs, errors, and experimental guest CPU profiles in Elide: what Elide records for each, the settings that control it, and how your code adds its own.

| Signal | Elide records | Your code adds | Settings |
| --- | --- | --- | --- |
| Traces | HTTP server and client spans, build and test spans | Spans through the tracer API | `traces`, `runtime.traces` |
| Metrics | HTTP, process, and error metrics | Counters, histograms, gauges, observable instruments | `metrics`, `runtime.metrics` |
| Logs | Its own warnings and errors | Logs API records, `console.*`, Python `logging` | `logs`, `runtime.logs` |
| Errors | Uncaught exceptions, rejections, failing handlers, crashes | `span.recordException(…)` | follows logs, traces, and metrics |
| Profiles (experimental) | Guest CPU samples | — | `profiles` |

User settings (`traces`, `metrics`, `logs`) govern telemetry from your code; `runtime` settings govern Elide's own.
Turning a signal off with `OTEL_TRACES_EXPORTER=none` (or the metrics and logs equivalents) turns off both.

## Traces

Spans start synchronously: the sampling decision and span context are available as soon as `startSpan` returns.
Context follows `await`, timers, and callbacks in JavaScript, and `contextvars` in Python.

| Setting | Effect | Default |
| --- | --- | --- |
| `traces.enabled` | Record spans from your code. | on |
| `traces.sampler`, `traces.ratio` | Head sampler for your spans. | `parentbased_always_on` |
| `runtime.traces` | Record Elide's HTTP and toolchain spans. | on |

With `parentbased_*` samplers, a sampled caller's `traceparent` keeps the request sampled, and your spans inside it
follow. Turning `traces.enabled` off keeps your spans' context flowing without recording them, so outgoing requests
still continue the caller's trace.

Recording never waits on export. When an export queue is full, spans are dropped and counted; read the counts with
`status()` from [`elide:telemetry`](https://elide.help/docs/opentelemetry/javascript#elidetelemetry) or
[`elide.telemetry`](https://elide.help/docs/opentelemetry/python#elidetelemetry).

## Metrics

| Setting | Effect | Default |
| --- | --- | --- |
| `metrics.enabled` | Record instruments from your code. | on |
| `metrics.interval` | Collect and export every interval. | `60.s` |
| `metrics.temporality` | `cumulative` or `delta`. | `cumulative`; `delta` for Datadog |
| `runtime.metrics` | Record Elide's HTTP, process, and error metrics. | on |

Your code can create counters, up-down counters, histograms (with `advice.explicitBucketBoundaries` in JavaScript or
`explicit_bucket_boundaries_advisory` in Python), gauges, and observable counters, up-down counters, and gauges.
Observable callbacks run at each collection. Elide's metric names are listed in
[Emitted telemetry](https://elide.help/docs/opentelemetry/reference#metrics).

Pending measurements are exported at exit, so a short script's metrics are not lost.

## Logs

| Setting | Effect | Default |
| --- | --- | --- |
| `logs.enabled` | Export records from your code. | on |
| `logs.level` | Minimum severity exported. | `info` |
| `logs.console` | Also export `console.*` and Python `logging` output. | on |
| `runtime.logs` | Minimum severity of Elide's own records. | `warn` |

Capturing output does not change what the program prints. `console.*` output is exported in the `console` scope;
Python `logging` records keep their logger's name as the scope, and only records some handler would print are
exported. Log API records keep structured bodies (objects, lists, bytes).

## Errors

Errors are exported as OTLP records, so every destination receives them through whichever of traces, logs, and
metrics it accepts:

| Source | `elide.error.origin` | Span status |
| --- | --- | --- |
| Uncaught exception | `uncaught` | — |
| Unhandled promise rejection | `rejection` | — |
| Request handler throws or rejects | `handler` | server span `ERROR` |
| `span.recordException(error)` | `guest_api` | unchanged |
| Earlier process crashed | `crash` | — |

Each produces an `exception` log record, an `exception` span event, and an `elide.errors` increment; see
[Errors](https://elide.help/docs/opentelemetry/correlation#errors). `elide.errors` is a runtime metric and follows `runtime.metrics`.

## Profiles

Guest CPU profiles are experimental and off by default. Elide samples the stacks of running JavaScript and Python code
and exports OTLP profiles, with each sample linked to the span active on its thread.

```pkl
opentelemetry {
  profiles { enabled = true; frequency = 99 }
}
```

```bash
ELIDE_OTEL_PROFILES=true elide serve app.ts   # 100 samples per second
ELIDE_OTEL_PROFILES=49 elide serve app.ts     # 49 samples per second
```

| Detail | Value |
| --- | --- |
| Destinations | `otlp` only; the receiver must accept the OTLP profiles signal |
| Export | Every 30 seconds and at exit, to `/v1development/profiles` over HTTP |
| Frequency | 1 to 10,000 samples per second; default 100 |
| Coverage | JavaScript and Python code; not Elide's own code |
| Conflicts | Off while `--profiler` is in use |

## Shutdown

At exit Elide hands what is still pending to a background exporter, so the program exits without waiting on the
network; `--otel-shutdown=inline` exports it from the exiting process instead, for up to `shutdownTimeout` (5 seconds
by default). See [At exit](https://elide.help/docs/opentelemetry/configuration#at-exit). `flush()` in
[JavaScript](https://elide.help/docs/opentelemetry/javascript#elidetelemetry) and [Python](https://elide.help/docs/opentelemetry/python#elidetelemetry) exports
on demand.

## Samples

- [Custom span in TypeScript](https://elide.help/docs/code-samples#custom-span-in-typescript)
- [Metrics instruments](https://elide.help/docs/code-samples#metrics-instruments)
- [Python span and logging](https://elide.help/docs/code-samples#python-span-and-logging)