Early accessBy invitation, and the SDKs aren't published yet.Ask for an invitation
Skip to Content
Docs

Java and Kotlin (JVM)

Structured logs from your Java and Kotlin services (Spring, Ktor, plain JVM), with a Logback appender. No dependencies beyond the Kotlin standard library.

Java and Kotlin servers, with a Logback appender

The JVM SDK needs Java 11 or later. It is written in Kotlin and every function can be called from Java. It runs the same engine as the Android SDK (lyrelog-core) and has no other dependency beyond the Kotlin standard library. SLF4J and Logback records can flow in through a separate, optional appender.

Internal preview: this package isn't published yet. Until it is, Gradle and Maven can't fetch com.lyrelog:lyrelog-jvm: it isn't on Maven Central, and the repository is private and has no release yet.

Install

From Maven Central. With Gradle, in build.gradle.kts:

dependencies {
    implementation("com.lyrelog:lyrelog-jvm:0.1.0")
    // Optional: SLF4J and Logback records (Spring Boot's default logging) to Lyrelog.
    implementation("com.lyrelog:lyrelog-jvm-logback:0.1.0")
}

With Maven, in pom.xml:

<dependencies>
  <dependency>
    <groupId>com.lyrelog</groupId>
    <artifactId>lyrelog-jvm</artifactId>
    <version>0.1.0</version>
  </dependency>
  <!-- Optional: SLF4J and Logback records to Lyrelog. -->
  <dependency>
    <groupId>com.lyrelog</groupId>
    <artifactId>lyrelog-jvm-logback</artifactId>
    <version>0.1.0</version>
  </dependency>
</dependencies>

The appender doesn't bring Logback: it uses the one your application already has (1.3 or later).

Start

Once, as early as possible: in main, before the framework starts.

import com.lyrelog.Configuration
import com.lyrelog.Lyrelog

fun main(args: Array<String>) {
    Lyrelog.start(Configuration(
        apiKey = System.getenv("LYRELOG_KEY"), // a server key, lyr_srv_...
        platform = "api",
    ))
    runApplication<OrdersApplication>(*args) // or embeddedServer(Netty, ...) in Ktor
}

In Java:

import com.lyrelog.Configuration;
import com.lyrelog.Lyrelog;

public static void main(String[] args) {
    Lyrelog.start(Configuration.builder(System.getenv("LYRELOG_KEY")).platform("api").build());
    SpringApplication.run(OrdersApplication.class, args);
}

Use a server key, lyr_srv_..., from the project's API keys page (Send events from a server). It is a secret: keep it in your server's environment, never in an app or a web page. Its events are verified, and one address has no daily share of the month. An ingest key, lyr_ing_..., works too, but its events aren't verified.

start returns true when it started Lyrelog. A second call while it runs is ignored and returns false. A null or blank key starts nothing and returns false, and every other call then does nothing, so a missing environment variable in development or tests doesn't stop the service. Events logged before start are ignored. start never throws.

Events go to https://ingest.lyrelog.com. Pass endpoint only to send them somewhere else. It is the base URL (a pasted .../v1/events:batch path is stripped) and must be https://, or http:// for localhost, 127.0.0.1 or ::1. Any other endpoint is refused: nothing is sent, and the SDK says so once on its diagnostics.

Log events

One function per level: trace, debug, info, notice, warning, error, critical, or Lyrelog.log(level, message, ...).

Lyrelog.info("checkout_done", attrs = mapOf("plan" to "pro", "latency_ms" to 412))
Lyrelog.error("payment_failed",
    category = "billing",
    attrs = mapOf("code" to "card_declined"),
    payload = mapOf("response" to body),
)

Attributes can be strings, numbers, booleans, null, lists (sent as strings) and nested maps up to 4 levels. The payload is any map, list or value. A log call never blocks and never throws: it copies the event and returns. After close, log calls do nothing.

Loggers for each area

val billing = Lyrelog.logger("billing")
billing.warning("invoice_retry", mapOf("attempt" to 2))

A logger is cheap to make and keep (a field, a companion object). It logs into whatever is running when it is called, so it can be made before start.

Users

A server handles many people at once. Set the user on the event, or for the work of one request:

Lyrelog.info("checkout_done", user = customerId)        // for this event only

Lyrelog.userScope(customerId).use {                     // for everything logged on this thread
    handle(request)                                     // until the scope is closed
}

Lyrelog.setUser("u_123")                                // every following event, every thread
Lyrelog.setUser(null)                                   // clear it

The event's own user wins over the scope's, and the scope's over setUser. A scope is thread-local: in Java, open it with try-with-resources in a servlet filter (as in the request filter below). Coroutines and reactive code (Ktor, WebFlux) hop between threads, so pass user on each event there instead. Use your own ids, not email addresses.

The Logback appender

With lyrelog-jvm-logback, every record your loggers write through SLF4J or Logback can also go to Lyrelog. Add the appender in logback.xml (in Spring Boot, logback-spring.xml):

<configuration>
  <appender name="LYRELOG" class="com.lyrelog.logback.LyrelogAppender" />

  <root level="INFO">
    <appender-ref ref="LYRELOG" />
  </root>
</configuration>

The appender forwards to the running Lyrelog, started in code as above. Or let it start Lyrelog itself, so the service needs no code at all; it then closes Lyrelog when Logback stops:

<appender name="LYRELOG" class="com.lyrelog.logback.LyrelogAppender">
  <apiKey>${LYRELOG_KEY}</apiKey>
  <platform>api</platform>
  <!-- <environment>staging</environment>, <endpoint>...</endpoint> -->
</appender>

Records logged before Lyrelog starts are ignored, so start it before your framework logs what you want to keep. To keep printing to the console as well, keep your console appender beside this one.

private static final org.slf4j.Logger log = LoggerFactory.getLogger(Invoices.class);

MDC.put("user_id", customerId);                 // the event's user
log.info("Order {} shipped", orderId);
log.atWarn().addKeyValue("attempt", 2).log("invoice_retry");
log.error("charge_failed", e);
SLF4J and LogbackLyrelog
Logger name (com.example.billing.Invoices)category, unless the appender has a <category> or the record a category key
TRACE, DEBUG, INFO, WARN, ERRORtrace, debug, info, warning, error
The formatted messagemessage; with arguments, the pattern goes to attrs.msg_template to group by
MDC entries, and SLF4J 2 key-value pairs (addKeyValue)attrs, except user_id (the event's user) and category. <includeMdc>false</includeMdc> leaves the MDC out
The exceptionattrs.error_type, attrs.error_message and payload.exception (frames, causes)
The record's timeThe event's time

Lyrelog's own minimum level still applies: trace records are dropped unless minimumLevel is Level.TRACE. The appender ignores records from com.lyrelog loggers and from the SDK's own threads, so it can never loop. There is no appender for Log4j 2 or java.util.logging. For events you want to chart, prefer direct calls with a stable name: a formatted message is prose.

A Spring request filter

There is no built-in filter. This one records every request as http_request_served in the http area, at info, warning for a 4xx and error for a 5xx, with the route pattern rather than the path, so ids don't explode the cardinality. It also sets the request's user for everything logged while it is handled:

import com.lyrelog.Level;
import com.lyrelog.Lyrelog;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Map;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import org.springframework.web.servlet.HandlerMapping;

@Component
public class LyrelogRequests extends OncePerRequestFilter {
    private final com.lyrelog.Logger http = Lyrelog.logger("http");

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain)
            throws ServletException, IOException {
        try (Lyrelog.UserScope user = Lyrelog.userScope(currentUserId(request))) { // your own id, or null
            long start = System.nanoTime();
            int status = 500; // when the handler throws, Spring writes the error page later
            try {
                chain.doFilter(request, response);
                status = response.getStatus();
            } finally {
                Object route = request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE);
                Level level = status >= 500 ? Level.ERROR : status >= 400 ? Level.WARNING : Level.INFO;
                http.log(level, "http_request_served", Map.of(
                    "method", request.getMethod(),
                    "route", route != null ? route.toString() : "unmatched", // /orders/{id}, not the id
                    "status", status,
                    "duration_ms", (System.nanoTime() - start) / 1_000_000));
            }
        }
    }
}

Spring logs an exception a handler throws on its own loggers, with the stack, so with the Logback appender it already arrives with error_type and error_message; capturing it in the filter too would record it twice.

Errors

try {
    charge(order)
} catch (e: PaymentException) {
    Lyrelog.captureError(e, category = "billing", attrs = mapOf("order_id" to order.id))
}

The event is exception, level error, with error_type (the class name) and error_message in the attributes and, in the payload, the description, up to 60 frames, the causes (up to 8, 20 frames each) and the suppressed exceptions. Your attributes are added; those two win. level changes the level and user sets the event's user. logger.captureError(e) uses the logger's category.

Uncaught exceptions

Off by default. With captureUncaught = true, an exception that no code caught, on any thread, is recorded as exception with handled: false, mechanism: uncaught_exception and the thread's name in thread. Then the handler that was there before runs (or the JVM's usual printout). When that thread was the last one, the shutdown hook sends the event before the JVM exits.

Lyrelog.start(Configuration(apiKey = System.getenv("LYRELOG_KEY"), captureUncaught = true))

Frameworks catch the exceptions of a request themselves and log them: those arrive through the Logback appender. An exception in a task given to an ExecutorService with submit goes to its Future, not to this handler.

Environments and platform

environment comes from the LYRELOG_ENVIRONMENT environment variable, else production. platform is jvm. Set either in the configuration:

Lyrelog.start(Configuration(apiKey = key, environment = "staging", platform = "worker"))

Any name works: lowercase letters, digits, - and _, up to 32. Other text in the configuration is cleaned for you (Staging EU is staging-eu); an invalid $LYRELOG_ENVIRONMENT is reported once and replaced by production, so a typo in a deploy never stops a service.

release (resource.app_version) defaults to the Implementation-Version of the jar the JVM was started with (java -jar orders.jar), and appId (resource.app_id) to its Implementation-Title, else the jar's file name, else the main class. Set them when that isn't what you want.

Consent is for apps that run on someone's device. A server SDK sends what it is given; there is no consent option.

Flush and stop

Lyrelog.flush()               // send what's queued now, waiting up to 5 s; true when it finished
Lyrelog.flushInBackground()   // the same, without waiting
Lyrelog.close()               // send what's left (up to closeTimeoutMillis), then stop

You rarely need any of them: the SDK sends every 5 s, and a JVM shutdown hook closes it when the JVM exits normally (the end of main, System.exit, SIGTERM and Ctrl-C). kill -9 skips the hook. Turn the hook off with shutdownHook = false and call close yourself.

flush respects a Retry-After the server asked for: it doesn't send sooner. After close, start can be called again.

In a serverless function (AWS Lambda), call Lyrelog.flush() before the handler returns: the runtime may freeze the process before the next send.

Where events wait

Events are written to a queue directory, one file per batch, and a batch leaves it only once the server has taken it.

  • By default, a new directory under java.io.tmpdir for this process, deleted on close. What wasn't sent by then is lost, as with an in-memory queue.
  • With storageDirectory, a directory of your own: what wasn't sent (the server was unreachable, or asked to wait) stays there and goes out after the next start. One process at a time: the directory is locked, and a second process that finds it locked queues in a temporary directory instead, and says so once.

Both are capped at 20 MB and 7 days, oldest first (maxDiskBytes, maxEventAgeMillis).

In development and tests

Leave the key out: Lyrelog.start(Configuration(apiKey = System.getenv("LYRELOG_KEY"))) starts nothing when the variable isn't set, and every call does nothing, so tests and a laptop without a key run the same code. The Logback appender does nothing either while Lyrelog isn't running.

To see what a test logs, point endpoint at a local HTTP server (plain http:// is allowed for loopback addresses) and read the batches it receives: gzipped NDJSON, one event per line, as on the HTTP API page.

From Java

Every function on Lyrelog is static from Java, with overloads for the optional arguments, and Configuration has a builder:

Lyrelog.start(Configuration.builder(System.getenv("LYRELOG_KEY"))
    .platform("api")
    .environment("staging")
    .captureUncaught(true)
    .build());

Lyrelog.info("checkout_done");
Lyrelog.info("checkout_done", "billing", Map.of("plan", "pro"));
Lyrelog.captureError(e, "billing");

try (Lyrelog.UserScope scope = Lyrelog.userScope(customerId)) {
    Lyrelog.logger("billing").warning("invoice_retry", Map.of("attempt", 2));
}

new Configuration(key) works too, and a logger's functions have overloads, so trailing arguments can be left out.

Verified events

A verified event is one that can't have been sent by someone who copied the public ingest key out of an app or a web page. What that proves, and what it doesn't, is on Verified events.

Send with a server key, lyr_srv_.... Every event it sends is verified (verified_by is server) and doesn't count toward the daily share of one address. An owner makes it on the project's API keys page, Send events from a server, after a recent sign-in (within 10 minutes), and it is shown once. It is a secret: keep it out of the repository and read it from the server's environment or secret store.

Lyrelog.start(Configuration(apiKey = System.getenv("LYRELOG_KEY"))) // lyr_srv_..., set in the server's environment

To check, open the explorer: in Filter, pick Verified by and Server key (the other values are App Attest and Not verified). An opened event says it under Context, in its Sender line: Verified, server key, or Not verified, public key.

Events sent with an ingest key, lyr_ing_..., or a full-access key, lyr_adm_..., arrive unverified: anyone holding a public key could have sent the same, and they count toward the daily share. A verified event proves who sent it, not that what it says is true.

See Verified events for who verifies what and the daily share.

Configuration

OptionDefaultMeaning
apiKeyrequiredThe project's server key (lyr_srv_...), or an ingest key. null or blank: nothing starts
endpointhttps://ingest.lyrelog.comThe base URL; a pasted /v1/... path is stripped. https only, http only for localhost
environment$LYRELOG_ENVIRONMENT, else productionAny valid name
platformjvmapi, worker...
releasethe started jar's Implementation-VersionSent as resource.app_version
appIdthe started jar's Implementation-Title or file name, else the main classSent as resource.app_id: the service name
minimumLevelLevel.DEBUGEvents below it are dropped before queueing
flushIntervalMillis5 sHow often the background thread sends
maxBatchBytes512 KiBA batch closes at whichever limit comes first (kept within 4 MiB)
maxBatchEvents500At most 2,000
maxQueue10,000Events waiting to be written; past it the oldest is dropped and counted
maxDiskBytes20 MBCap on the queue directory, oldest first
maxEventAgeMillis7 daysOlder queued events are dropped
redactedKeysnull: DEFAULT_REDACTED_KEYSKey names to redact; your list replaces the defaults, an empty list turns it off
storageDirectorya temporary directory per processWhere the queue lives; see Where events wait
captureUncaughtfalseRecord uncaught exceptions
shutdownHooktrueClose when the JVM shuts down
closeTimeoutMillis5 sHow long closing waits for the last send
diagnosticsstandard errorReceives the SDK's own messages (a Consumer<String>). Don't point it at a logger that feeds Lyrelog

Delivery and limits

Never blocksLogging copies the event and returns; one background writer puts it in the queue directory and never waits on the network. One background thread sends. Safe for concurrent use.
BatchesUp to 500 events or 512 KiB per request, gzipped, every flushIntervalMillis. An event over 1 MiB once encoded is dropped and counted.
Retries1 s doubling to 5 min, with full jitter, honoring Retry-After; the same bytes are resent. Any other 4xx drops the batch, a 3xx (a redirect, never followed) drops it, and a refused key (401, 403) stops sending until the process restarts and keeps the events. A batch the server keeps failing while others go through goes to the back, then is dropped.
Monthly quotaWhen a Free organization has used its month's events, the server answers 429. Events stay in the queue and go out once the server takes them again, within the 20 MB and 7 day caps. The SDK prints one warning on its diagnostics, and again only after a batch has been accepted in between.
Drops are visibleEvents lost to the caps become one lyrelog_events_dropped warning (area lyrelog) with the count.
Its own problemsA wrong endpoint, a 404, a refused key, rejected lines or an endpoint that doesn't answer are said once per process on diagnostics (standard error by default), each line starting with lyrelog: .
Contextos_name and os_version (the JVM's os.name and os.version), the app id, release, environment and platform, an install id per process and one session id per process.
SecretsValues under keys whose name is or ends with authorization, cookie, password, token, secret, apikey and the rest of the shared list become "[redacted]", at any depth, before anything is cut.
LimitsLong strings cut on a character boundary, extra attributes counted in lyrelog_dropped_attrs, payloads over 64 KiB summarized as {"lyrelog_truncated": true, "bytes": N, "preview": "..."}.
The keyPrinting a Configuration hides it.

Change the redaction list with redactedKeys: your list replaces the defaults, and an empty list turns redaction off. To add to it:

Lyrelog.start(Configuration(
    apiKey = System.getenv("LYRELOG_KEY"),
    redactedKeys = Configuration.DEFAULT_REDACTED_KEYS + "card_number",
))