Early accessBy invitation, and the SDKs aren't published yet.Ask for an invitation
Skip to Content
Docs

Python

Structured logs from your Python services, with a logging handler. Standard library only.

Servers, workers, scripts

The lyrelog package needs Python 3.11 or later and has no dependencies.

Internal preview: this package isn't published yet. Until it is, uv add lyrelog fails: there is no package on PyPI.

Install

uv add lyrelog

Start

Once, at startup. lyrelog.init starts the default client (flushing and replacing any earlier one) and returns it; the module-level functions use it.

import lyrelog

lyrelog.init(
    api_key="lyr_srv_...",   # a server key: it can append events, never read them
    release="2.0.1",         # goes to resource.app_version
    app_id="orders-api",
)

Use a server key, lyr_srv_..., from the project's API keys page (Send events from a server). It is a secret: keep it in your server's environment, never in an app or a web page. Its events are verified, and one address has no daily share of the month. An ingest key, lyr_ing_..., works too, but its events aren't verified.

Events go to https://ingest.lyrelog.com. Pass endpoint only to send them somewhere else. It is the base URL (a pasted .../v1/events:batch path is stripped) and must be https://, or http:// for localhost.

An empty api_key gives a client that accepts events and sends nothing, so a missing environment variable in development doesn't crash the app. A missing key or a bad endpoint is reported once as a warning on the lyrelog logger.

lyrelog.Client(endpoint, api_key, ...) is the same without the module-level default, for a second project or for tests: every function below is a method on it. lyrelog.get_client() returns the default client, or None before init.

Log events

One function per level: trace, debug, info, notice, warning, error, critical, or lyrelog.log(level, message, ...).

lyrelog.info("checkout_done", attrs={"items": 3, "latency_ms": 412})
lyrelog.error("payment_declined", category="checkout", attrs={"code": 51}, payload={"response": raw})

A log call never blocks and never raises: anything odd you pass (objects, NaN, cycles) is turned into something valid. After shutdown(), log calls do nothing.

Loggers for each area

billing = lyrelog.logger("billing")
billing.warning("refund_denied", attrs={"order_id": order.id}, user_id=order.user_id)

A logger follows the default client at call time, so you can make it at import, before init.

Users

lyrelog.set_user("u_123")   # attached to every following event, process-wide
lyrelog.info("checkout_done", user_id=request_user)   # for this event only

A server handles many people at once, so prefer user_id= on each call to set_user.

The logging handler

The quickest way is install_logging, which adds one LoggingHandler to the root logger (and, with uvicorn=True, the default, to uvicorn and uvicorn.error) and lowers the root logger to level if it is above it, so INFO records get through. Calling it again replaces the earlier handler, and uninstall_logging() removes it; uvicorn.access is left out (record requests in a middleware instead).

import logging
import lyrelog

lyrelog.install_logging(logging.INFO)

To add the handler yourself, remember that the root logger stops at WARNING by default: a handler set to INFO gets nothing below WARNING until the root logger lets it through.

import logging
import lyrelog

logging.basicConfig(level=logging.INFO)   # or logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(lyrelog.LoggingHandler(level=logging.INFO))

log = logging.getLogger("app.orders")
log.info("order %s shipped", order_id, extra={"items": 3, "user_id": uid})
log.exception("email_failed")   # inside an except: the traceback goes in the payload
loggingLyrelog
Logger name (app.matches, uvicorn.error)category
DEBUG ... CRITICALThe same; below DEBUG is trace, between INFO and WARNING is notice
The formatted messagemessage; with arguments, the template goes to attrs.msg_template to group by
extra= fieldsattrs, except user_id (the event's user) and payload (kept as payload.data)
exc_infopayload.exception and attrs.error_type, attrs.error_message
File, line, functionpayload.source

The handler never forwards the SDK's own logger, so it can't loop. That logger is named lyrelog: to see what the SDK itself reports (a refused key, a wrong endpoint, dropped events), give it a handler, for example logging.getLogger("lyrelog").setLevel(logging.DEBUG) with basicConfig. For events you want to chart, prefer direct calls with a stable name: a formatted message is prose.

FastAPI

import logging
import os
import time

import lyrelog
from fastapi import FastAPI, Request

lyrelog.init(
    api_key="lyr_srv_...",   # a server key, or read it from your own settings
    platform="api",          # instead of "python", to tell it apart from a worker
    release=os.environ.get("GIT_SHA"),
    app_id="orders-api",
)
# uvicorn.error carries the tracebacks of unhandled exceptions. Uvicorn's loggers don't
# propagate to the root logger (propagate=False), so a root handler never sees them:
# uvicorn=True attaches the handler to them too. Uvicorn replaces its loggers' handlers when
# it applies its config, so call this after that: here, at import of the app module that
# `uvicorn app:app` runs, or in the app's lifespan startup.
lyrelog.install_logging(logging.INFO, uvicorn=True)

app = FastAPI()
http = lyrelog.logger("http")


@app.middleware("http")
async def record_request(request: Request, call_next):
    start = time.perf_counter()
    status = 500
    try:
        response = await call_next(request)
        status = response.status_code
        return response
    finally:
        route = request.scope.get("route")   # set by the router once it matched
        http.log(
            "error" if status >= 500 else "warning" if status >= 400 else "info",
            "http_request_served",
            attrs={
                "method": request.method,
                "route": getattr(route, "path", "unmatched"),   # /matches/{match_id}, not the id
                "status": status,
                "duration_ms": round((time.perf_counter() - start) * 1000, 1),
            },
        )

The route template keeps ids out of the attribute, so the dashboard can count by route. The middleware doesn't capture unhandled exceptions itself: uvicorn logs each one with its traceback on uvicorn.error, and install_logging forwards that record with error_type, error_message and the traceback, so capturing it here too would record it twice. With gunicorn or uvicorn --workers, each worker process gets its own sender.

Django

Start the client once, in settings.py or your AppConfig.ready(), and send logging records through Django's LOGGING setting:

# settings.py
import lyrelog

lyrelog.init(api_key="lyr_srv_...", platform="api", app_id="shop")

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {"lyrelog": {"class": "lyrelog.LoggingHandler", "level": "INFO"}},
    "root": {"handlers": ["lyrelog"], "level": "INFO"},
    "loggers": {
        # Django's request errors carry the traceback of a 500.
        "django.request": {"handlers": ["lyrelog"], "level": "WARNING", "propagate": False},
    },
}

A worker or a script

A job that runs for many people sets the user on each event, and stops the client before it ends:

import os
import time

import lyrelog

lyrelog.init(api_key=os.environ.get("LYRELOG_KEY", ""), platform="worker", app_id="digest")
jobs = lyrelog.logger("jobs")


def send_digest(user):
    started = time.perf_counter()
    try:
        deliver(user)
    except Exception:
        jobs.capture_exception(attrs={"job": "digest"}, user_id=user.id)
        return
    jobs.info("digest_sent", attrs={"duration_ms": round((time.perf_counter() - started) * 1000)},
              user_id=user.id)


if __name__ == "__main__":
    for user in due_users():
        send_digest(user)
    lyrelog.shutdown()   # sends what's left, up to 5 s

Serverless and short scripts

The worker sends every 5 s and once more at exit (atexit). A function that is frozen between requests (AWS Lambda, Cloud Functions) may never reach either: call lyrelog.flush() before returning, and check what it returns if you need to know.

def handler(event, context):
    try:
        return process(event)
    finally:
        lyrelog.flush(timeout=2)

atexit doesn't run after os._exit(), or when the process gets SIGTERM without a handler. Uvicorn and gunicorn handle SIGTERM and exit normally; in your own long-running process, stop the client in a handler:

import signal
import sys

import lyrelog


def on_sigterm(signum, frame):
    lyrelog.shutdown(timeout=2)
    sys.exit(0)


signal.signal(signal.SIGTERM, on_sigterm)

Uncaught exceptions

The SDK doesn't install a hook for uncaught exceptions. To record them, set one yourself, for the main thread and for other threads:

import sys
import threading

import lyrelog


def report(exc_type, exc, tb):
    lyrelog.capture_exception(exc, level="critical")
    lyrelog.flush(timeout=2)
    sys.__excepthook__(exc_type, exc, tb)


def report_thread(args):
    if args.exc_value is not None and not issubclass(args.exc_type, SystemExit):
        name = args.thread.name if args.thread else ""
        lyrelog.capture_exception(args.exc_value, attrs={"thread": name}, level="critical")
    threading.__excepthook__(args)


sys.excepthook = report
threading.excepthook = report_thread

Frameworks catch exceptions inside requests themselves: for those, use the middleware above or the framework's logger.

Exceptions

try:
    sync()
except Exception:
    lyrelog.capture_exception(category="jobs")

With no argument it captures the exception being handled; pass one to capture another. It logs exception at error with attrs.error_type and attrs.error_message (the same in every SDK), and the type, message and traceback in the payload. message= and level= change the event's name and level, and user_id= sets its user. A logger's capture_exception uses its category.

Environments and platform

environment comes from the LYRELOG_ENVIRONMENT environment variable, else production. platform is python. Set either in init:

lyrelog.init(api_key=..., environment="staging", platform="worker")

Any name works: lowercase letters, digits, - and _, up to 32. Other text is normalized for you.

Consent is for apps that run on someone's device. A server SDK sends what it is given; there is no consent option.

Redacting and filtering

A value becomes "[redacted]", in attributes and the payload, when its key name, lowercased and without -, _, . or spaces, is or ends with one of authorization, cookie, password, passwd, secret, token, apikey, privatekey, credential, credentials, accesskey or signature. So X-Api-Key, csrf_token, Set-Cookie and user_token are redacted; token_count and max_tokens aren't.

redact_keys=[...] replaces that list with yours (matched the same way), and redact_keys=[] turns redaction off. To keep the defaults and add one, pass redact_keys=[*lyrelog.DEFAULT_REDACT_KEYS, "card_number"].

To change or drop events yourself, pass before_send to init: a function that gets each event and returns it, changed or not, or None to drop it.

def before_send(event):
    if event["message"] == "health_check":
        return None
    return event


lyrelog.init(api_key="lyr_srv_...", before_send=before_send)

Flush and stop

sent = lyrelog.flush()   # rarely needed: it sends every 5 s and at exit
lyrelog.shutdown()       # also runs at exit: one last attempt, up to 5 s

flush waits until everything logged before the call has been sent, up to its timeout (5 s by default), and returns True if the queue drained, False if events are still waiting (the server asked to wait with Retry-After, or it couldn't be reached) or were lost. Each request to the server times out after 15 s.

client.stats() (on the client init returns) reports queued, dropped (queue full), delivered, rejected (lines the server refused), discarded (batches dropped after a 4xx or 3xx), filtered (dropped by before_send), failed (events that couldn't be shaped, or whose before_send raised), reentrant (logged from inside the SDK, for example a signal handler) and disabled (a refused key, no key or a bad endpoint).

In development and tests

Leave api_key empty: the client accepts events, sends nothing and warns once on the lyrelog logger.

lyrelog.init(api_key=os.environ.get("LYRELOG_KEY", ""))

Without init, the module-level functions and loggers do nothing.

Verified events

A verified event is one that can't have been sent by someone who copied the public ingest key out of an app or a web page. What that proves, and what it doesn't, is on Verified events.

Send with a server key, lyr_srv_.... Every event it sends is verified (verified_by is server) and doesn't count toward the daily share of one address. An owner makes it on the project's API keys page, Send events from a server, after a recent sign-in (within 10 minutes), and it is shown once. It is a secret: keep it out of the repository and read it from the server's environment or secret store.

import os

lyrelog.init(api_key=os.environ["LYRELOG_KEY"])  # lyr_srv_..., set in the server's environment

To check, open the explorer: in Filter, pick Verified by and Server key (the other values are App Attest and Not verified). An opened event says it under Context, in its Sender line: Verified, server key, or Not verified, public key.

Events sent with an ingest key, lyr_ing_..., or a full-access key, lyr_adm_..., arrive unverified: anyone holding a public key could have sent the same, and they count toward the daily share. A verified event proves who sent it, not that what it says is true.

See Verified events for who verifies what and the daily share.

Configuration

OptionDefaultMeaning
endpointhttps://ingest.lyrelog.comThe base URL; a pasted /v1/... path is stripped
api_keyrequiredThe project's server key (lyr_srv_...), or an ingest key. Empty: nothing is sent
environmentLYRELOG_ENVIRONMENT, else productionAny valid name
platformpythonapi, worker...
releasenoneresource.app_version
app_idnoneresource.app_id: the service name
minimum_level"debug"Events below it are dropped before queueing
flush_interval5.0 sHow often the worker sends
max_queue10_000Oldest dropped beyond it
max_queue_bytes16 MiBThe same cap, in bytes
redact_keysDEFAULT_REDACT_KEYSThe key names to redact; replaces the defaults
before_sendnoneChange an event, or return None to drop it
session_timeout30 minIdle time that starts a new session. Only on lyrelog.Client, not init

Delivery and limits

BatchesUp to 500 events or 512 KiB, gzip. A full batch goes out at once, otherwise every flush_interval. An event over 1 MiB is dropped and counted.
Retries1 s doubling to 5 min with equal jitter, honoring Retry-After (at most an hour); the same bytes are resent. Any other 4xx drops the batch; a 3xx drops it with a warning (redirects aren't followed). A refused key (401, 403) stops sending until the process restarts and keeps the events in memory.
Monthly quotaWhen a Free organization has used its month's events, the server answers 429. The events stay in memory and go out once the server takes them again, up to max_queue. The SDK logs one warning on the lyrelog logger with the server's message, and warns again only after a batch has been accepted.
CapsPast max_queue the oldest go, and a lyrelog_events_dropped warning with the count goes out with the next batch.
Threads and forksThread-safe. After os.fork the child gets a fresh queue, worker, install and session.
Contextos_name (platform.system()), os_version (platform.release()), device_model (platform.machine()), an install id per process, and a session id per process that changes after 30 minutes without an event.
LimitsFitted before queueing: nested objects past 4 levels become a JSON string, dots in attribute names become _, a payload over 64 KiB becomes {"lyrelog_truncated": true, "bytes": N, "preview": "..."}, tracebacks keep their last 48 KiB.