# Python

> Structured logs from your Python services, with a logging handler. Standard library only.

Source: https://docs.lyrelog.com/sdks/python

<SdkHeader name="Python" />

The `lyrelog` package needs Python 3.11 or later and has no dependencies.

<Callout type="info">
Internal preview: this package isn't published yet. Until it is, `uv add lyrelog` fails: there
is no package on PyPI.
</Callout>

## Install [#install]

```bash
uv add lyrelog
```

## Start [#start]

Once, at startup. `lyrelog.init` starts the default client (flushing and replacing any earlier
one) and returns it; the module-level functions use it.

```python
import lyrelog

lyrelog.init(
    api_key="lyr_srv_...",   # a server key: it can append events, never read them
    release="2.0.1",         # goes to resource.app_version
    app_id="orders-api",
)
```

Use a server key, `lyr_srv_...`, from the project's **API keys** page (**Send events from a
server**). It is a secret: keep it in your server's environment, never in an app or a web page.
Its events are [verified](/concepts/verification), and one address has no daily share of
the month. An ingest key, `lyr_ing_...`, works too, but its events aren't verified.

Events go to `https://ingest.lyrelog.com`. Pass `endpoint` only to send them somewhere else. It
is the base URL (a pasted `.../v1/events:batch` path is stripped) and must be `https://`, or
`http://` for localhost.

An empty `api_key` gives a client that accepts events and sends nothing, so a
missing environment variable in development doesn't crash the app. A missing key or a bad
endpoint is reported once as a warning on the `lyrelog` logger.

`lyrelog.Client(endpoint, api_key, ...)` is the same without the module-level default, for a
second project or for tests: every function below is a method on it. `lyrelog.get_client()`
returns the default client, or `None` before `init`.

## Log events [#log-events]

One function per level: `trace`, `debug`, `info`, `notice`, `warning`, `error`, `critical`, or
`lyrelog.log(level, message, ...)`.

```python
lyrelog.info("checkout_done", attrs={"items": 3, "latency_ms": 412})
lyrelog.error("payment_declined", category="checkout", attrs={"code": 51}, payload={"response": raw})
```

A log call never blocks and never raises: anything odd you pass (objects, NaN, cycles) is turned
into something valid. After `shutdown()`, log calls do nothing.

## Loggers for each area [#loggers-for-each-area]

```python
billing = lyrelog.logger("billing")
billing.warning("refund_denied", attrs={"order_id": order.id}, user_id=order.user_id)
```

A logger follows the default client at call time, so you can make it at import, before `init`.

## Users [#users]

```python
lyrelog.set_user("u_123")   # attached to every following event, process-wide
lyrelog.info("checkout_done", user_id=request_user)   # for this event only
```

A server handles many people at once, so prefer `user_id=` on each call to `set_user`.

## The logging handler [#the-logging-handler]

The quickest way is `install_logging`, which adds one `LoggingHandler` to the root logger (and,
with `uvicorn=True`, the default, to `uvicorn` and `uvicorn.error`) and lowers the root logger to
`level` if it is above it, so `INFO` records get through. Calling it again replaces the earlier
handler, and `uninstall_logging()` removes it; `uvicorn.access` is left out (record requests in a
middleware instead).

```python
import logging
import lyrelog

lyrelog.install_logging(logging.INFO)
```

To add the handler yourself, remember that the root logger stops at `WARNING` by default: a
handler set to `INFO` gets nothing below `WARNING` until the root logger lets it through.

```python
import logging
import lyrelog

logging.basicConfig(level=logging.INFO)   # or logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(lyrelog.LoggingHandler(level=logging.INFO))

log = logging.getLogger("app.orders")
log.info("order %s shipped", order_id, extra={"items": 3, "user_id": uid})
log.exception("email_failed")   # inside an except: the traceback goes in the payload
```

| `logging` | Lyrelog |
|---|---|
| Logger name (`app.matches`, `uvicorn.error`) | `category` |
| `DEBUG` ... `CRITICAL` | The same; below `DEBUG` is `trace`, between `INFO` and `WARNING` is `notice` |
| The formatted message | `message`; with arguments, the template goes to `attrs.msg_template` to group by |
| `extra=` fields | `attrs`, except `user_id` (the event's user) and `payload` (kept as `payload.data`) |
| `exc_info` | `payload.exception` and `attrs.error_type`, `attrs.error_message` |
| File, line, function | `payload.source` |

The handler never forwards the SDK's own logger, so it can't loop. That logger is named
`lyrelog`: to see what the SDK itself reports (a refused key, a wrong endpoint, dropped events),
give it a handler, for example `logging.getLogger("lyrelog").setLevel(logging.DEBUG)` with
`basicConfig`. For events you want to chart,
prefer direct calls with a stable name: a formatted message is prose.

## FastAPI [#fastapi]

```python
import logging
import os
import time

import lyrelog
from fastapi import FastAPI, Request

lyrelog.init(
    api_key="lyr_srv_...",   # a server key, or read it from your own settings
    platform="api",          # instead of "python", to tell it apart from a worker
    release=os.environ.get("GIT_SHA"),
    app_id="orders-api",
)
# uvicorn.error carries the tracebacks of unhandled exceptions. Uvicorn's loggers don't
# propagate to the root logger (propagate=False), so a root handler never sees them:
# uvicorn=True attaches the handler to them too. Uvicorn replaces its loggers' handlers when
# it applies its config, so call this after that: here, at import of the app module that
# `uvicorn app:app` runs, or in the app's lifespan startup.
lyrelog.install_logging(logging.INFO, uvicorn=True)

app = FastAPI()
http = lyrelog.logger("http")

@app.middleware("http")
async def record_request(request: Request, call_next):
    start = time.perf_counter()
    status = 500
    try:
        response = await call_next(request)
        status = response.status_code
        return response
    finally:
        route = request.scope.get("route")   # set by the router once it matched
        http.log(
            "error" if status >= 500 else "warning" if status >= 400 else "info",
            "http_request_served",
            attrs={
                "method": request.method,
                "route": getattr(route, "path", "unmatched"),   # /matches/{match_id}, not the id
                "status": status,
                "duration_ms": round((time.perf_counter() - start) * 1000, 1),
            },
        )
```

The route template keeps ids out of the attribute, so the dashboard can count by route. The
middleware doesn't capture unhandled exceptions itself: uvicorn logs each one with its traceback
on `uvicorn.error`, and `install_logging` forwards that record with `error_type`,
`error_message` and the traceback, so capturing it here too would record it twice. With gunicorn
or `uvicorn --workers`, each worker process gets its own sender.

## Django [#django]

Start the client once, in `settings.py` or your `AppConfig.ready()`, and send `logging` records
through Django's `LOGGING` setting:

```python
# settings.py
import lyrelog

lyrelog.init(api_key="lyr_srv_...", platform="api", app_id="shop")

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {"lyrelog": {"class": "lyrelog.LoggingHandler", "level": "INFO"}},
    "root": {"handlers": ["lyrelog"], "level": "INFO"},
    "loggers": {
        # Django's request errors carry the traceback of a 500.
        "django.request": {"handlers": ["lyrelog"], "level": "WARNING", "propagate": False},
    },
}
```

## A worker or a script [#a-worker-or-a-script]

A job that runs for many people sets the user on each event, and stops the client before it
ends:

```python
import os
import time

import lyrelog

lyrelog.init(api_key=os.environ.get("LYRELOG_KEY", ""), platform="worker", app_id="digest")
jobs = lyrelog.logger("jobs")

def send_digest(user):
    started = time.perf_counter()
    try:
        deliver(user)
    except Exception:
        jobs.capture_exception(attrs={"job": "digest"}, user_id=user.id)
        return
    jobs.info("digest_sent", attrs={"duration_ms": round((time.perf_counter() - started) * 1000)},
              user_id=user.id)

if __name__ == "__main__":
    for user in due_users():
        send_digest(user)
    lyrelog.shutdown()   # sends what's left, up to 5 s
```

## Serverless and short scripts [#serverless-and-short-scripts]

The worker sends every 5 s and once more at exit (`atexit`). A function that is frozen between
requests (AWS Lambda, Cloud Functions) may never reach either: call `lyrelog.flush()` before
returning, and check what it returns if you need to know.

```python
def handler(event, context):
    try:
        return process(event)
    finally:
        lyrelog.flush(timeout=2)
```

`atexit` doesn't run after `os._exit()`, or when the process gets `SIGTERM` without a handler.
Uvicorn and gunicorn handle `SIGTERM` and exit normally; in your own long-running process, stop
the client in a handler:

```python
import signal
import sys

import lyrelog

def on_sigterm(signum, frame):
    lyrelog.shutdown(timeout=2)
    sys.exit(0)

signal.signal(signal.SIGTERM, on_sigterm)
```

## Uncaught exceptions [#uncaught-exceptions]

The SDK doesn't install a hook for uncaught exceptions. To record them, set one yourself, for
the main thread and for other threads:

```python
import sys
import threading

import lyrelog

def report(exc_type, exc, tb):
    lyrelog.capture_exception(exc, level="critical")
    lyrelog.flush(timeout=2)
    sys.__excepthook__(exc_type, exc, tb)

def report_thread(args):
    if args.exc_value is not None and not issubclass(args.exc_type, SystemExit):
        name = args.thread.name if args.thread else ""
        lyrelog.capture_exception(args.exc_value, attrs={"thread": name}, level="critical")
    threading.__excepthook__(args)

sys.excepthook = report
threading.excepthook = report_thread
```

Frameworks catch exceptions inside requests themselves: for those, use the middleware above or
the framework's logger.

## Exceptions [#exceptions]

```python
try:
    sync()
except Exception:
    lyrelog.capture_exception(category="jobs")
```

With no argument it captures the exception being handled; pass one to capture another. It logs
`exception` at `error` with `attrs.error_type` and `attrs.error_message` (the same in every SDK),
and the type, message and traceback in the payload. `message=` and `level=` change the event's
name and level, and `user_id=` sets its user. A logger's `capture_exception` uses its category.

## Environments and platform [#environments-and-platform]

`environment` comes from the `LYRELOG_ENVIRONMENT` environment variable, else `production`.
`platform` is `python`. Set either in `init`:

```python
lyrelog.init(api_key=..., environment="staging", platform="worker")
```

Any name works: lowercase letters, digits, `-` and `_`, up to 32. Other text is normalized for you.

## Consent [#consent]

Consent is for apps that run on someone's device. A server SDK sends what it is given; there is
no consent option.

## Redacting and filtering [#redacting-and-filtering]

A value becomes `"[redacted]"`, in attributes and the payload, when its key name, lowercased and
without `-`, `_`, `.` or spaces, is or ends with one of `authorization`, `cookie`, `password`,
`passwd`, `secret`, `token`, `apikey`, `privatekey`, `credential`, `credentials`, `accesskey` or
`signature`. So `X-Api-Key`, `csrf_token`, `Set-Cookie` and `user_token` are redacted;
`token_count` and `max_tokens` aren't.

`redact_keys=[...]` replaces that list with yours (matched the same way), and `redact_keys=[]`
turns redaction off. To keep the defaults and add one, pass
`redact_keys=[*lyrelog.DEFAULT_REDACT_KEYS, "card_number"]`.

To change or drop events yourself, pass `before_send` to `init`: a function that gets each event
and returns it, changed or not, or `None` to drop it.

```python
def before_send(event):
    if event["message"] == "health_check":
        return None
    return event

lyrelog.init(api_key="lyr_srv_...", before_send=before_send)
```

## Flush and stop [#flush-and-stop]

```python
sent = lyrelog.flush()   # rarely needed: it sends every 5 s and at exit
lyrelog.shutdown()       # also runs at exit: one last attempt, up to 5 s
```

`flush` waits until everything logged before the call has been sent, up to its `timeout` (5 s
by default), and returns `True` if the queue drained, `False` if events are still waiting (the
server asked to wait with `Retry-After`, or it couldn't be reached) or were lost. Each request to
the server times out after 15 s.

`client.stats()` (on the client `init` returns) reports `queued`, `dropped` (queue full),
`delivered`, `rejected` (lines the server refused), `discarded` (batches dropped after a 4xx or
3xx), `filtered` (dropped by `before_send`), `failed` (events that couldn't be shaped, or whose
`before_send` raised), `reentrant` (logged from inside the SDK, for example a signal handler) and
`disabled` (a refused key, no key or a bad endpoint).

## In development and tests [#in-development-and-tests]

Leave `api_key` empty: the client accepts events, sends nothing and warns once on the `lyrelog`
logger.

```python
lyrelog.init(api_key=os.environ.get("LYRELOG_KEY", ""))
```

Without `init`, the module-level functions and loggers do nothing.

## Verified events [#verified-events]

A verified event is one that can't have been sent by someone who copied the public ingest key
out of an app or a web page. What that proves, and what it doesn't, is on
[Verified events](/concepts/verification).

Send with a server key, `lyr_srv_...`. Every event it sends is verified (`verified_by` is
`server`) and doesn't count toward the
[daily share of one address](/concepts/verification#the-daily-share-of-one-address). An owner
makes it on the project's **API keys** page, **Send events from a server**, after a recent sign-in
(within 10 minutes), and it is shown once. It is a secret: keep it out of the repository and read
it from the server's environment or secret store.

```python
import os

lyrelog.init(api_key=os.environ["LYRELOG_KEY"])  # lyr_srv_..., set in the server's environment
```

To check, open the [explorer](/dashboard/explorer): in **Filter**, pick **Verified by** and
**Server key** (the other values are **App Attest** and **Not verified**). An opened event says it under
**Context**, in its **Sender** line: **Verified, server key**, or **Not verified, public key**.

Events sent with an ingest key, `lyr_ing_...`, or a full-access key, `lyr_adm_...`, arrive
unverified: anyone holding a public key could have sent the same, and they count toward the daily
share. A verified event proves who sent it, not that what it says is true.

See [Verified events](/concepts/verification) for who verifies what and the daily share.

## Configuration [#configuration]

| Option | Default | Meaning |
|---|---|---|
| `endpoint` | `https://ingest.lyrelog.com` | The base URL; a pasted `/v1/...` path is stripped |
| `api_key` | required | The project's server key (`lyr_srv_...`), or an ingest key. Empty: nothing is sent |
| `environment` | `LYRELOG_ENVIRONMENT`, else `production` | Any valid name |
| `platform` | `python` | `api`, `worker`... |
| `release` | none | `resource.app_version` |
| `app_id` | none | `resource.app_id`: the service name |
| `minimum_level` | `"debug"` | Events below it are dropped before queueing |
| `flush_interval` | `5.0` s | How often the worker sends |
| `max_queue` | `10_000` | Oldest dropped beyond it |
| `max_queue_bytes` | 16 MiB | The same cap, in bytes |
| `redact_keys` | `DEFAULT_REDACT_KEYS` | The key names to redact; replaces the defaults |
| `before_send` | none | Change an event, or return `None` to drop it |
| `session_timeout` | 30 min | Idle time that starts a new session. Only on `lyrelog.Client`, not `init` |

## Delivery and limits [#delivery-and-limits]

| | |
|---|---|
| Batches | Up to 500 events or 512 KiB, gzip. A full batch goes out at once, otherwise every `flush_interval`. An event over 1 MiB is dropped and counted. |
| Retries | 1 s doubling to 5 min with equal jitter, honoring `Retry-After` (at most an hour); the same bytes are resent. Any other 4xx drops the batch; a 3xx drops it with a warning (redirects aren't followed). A refused key (`401`, `403`) stops sending until the process restarts and keeps the events in memory. |
| Monthly quota | When a Free organization has used its month's events, the server answers `429`. The events stay in memory and go out once the server takes them again, up to `max_queue`. The SDK logs one warning on the `lyrelog` logger with the server's message, and warns again only after a batch has been accepted. |
| Caps | Past `max_queue` the oldest go, and a `lyrelog_events_dropped` warning with the count goes out with the next batch. |
| Threads and forks | Thread-safe. After `os.fork` the child gets a fresh queue, worker, install and session. |
| Context | `os_name` (`platform.system()`), `os_version` (`platform.release()`), `device_model` (`platform.machine()`), an install id per process, and a session id per process that changes after 30 minutes without an event. |
| Limits | Fitted before queueing: nested objects past 4 levels become a JSON string, dots in attribute names become `_`, a payload over 64 KiB becomes `{"lyrelog_truncated": true, "bytes": N, "preview": "..."}`, tracebacks keep their last 48 KiB. |