Java and Kotlin (JVM)
Structured logs from your Java and Kotlin services (Spring, Ktor, plain JVM), with a Logback appender. No dependencies beyond the Kotlin standard library.
The JVM SDK needs Java 11 or later. It is written in Kotlin and every function can be called
from Java. It runs the same engine as the Android SDK (lyrelog-core) and
has no other dependency beyond the Kotlin standard library. SLF4J and Logback records can flow in
through a separate, optional appender.
Internal preview: this package isn't published yet. Until it is, Gradle and Maven can't fetch
com.lyrelog:lyrelog-jvm: it isn't on Maven Central, and the repository is private and has no
release yet.
Install
From Maven Central. With Gradle, in build.gradle.kts:
dependencies {
implementation("com.lyrelog:lyrelog-jvm:0.1.0")
// Optional: SLF4J and Logback records (Spring Boot's default logging) to Lyrelog.
implementation("com.lyrelog:lyrelog-jvm-logback:0.1.0")
}With Maven, in pom.xml:
<dependencies>
<dependency>
<groupId>com.lyrelog</groupId>
<artifactId>lyrelog-jvm</artifactId>
<version>0.1.0</version>
</dependency>
<!-- Optional: SLF4J and Logback records to Lyrelog. -->
<dependency>
<groupId>com.lyrelog</groupId>
<artifactId>lyrelog-jvm-logback</artifactId>
<version>0.1.0</version>
</dependency>
</dependencies>The appender doesn't bring Logback: it uses the one your application already has (1.3 or later).
Start
Once, as early as possible: in main, before the framework starts.
import com.lyrelog.Configuration
import com.lyrelog.Lyrelog
fun main(args: Array<String>) {
Lyrelog.start(Configuration(
apiKey = System.getenv("LYRELOG_KEY"), // a server key, lyr_srv_...
platform = "api",
))
runApplication<OrdersApplication>(*args) // or embeddedServer(Netty, ...) in Ktor
}In Java:
import com.lyrelog.Configuration;
import com.lyrelog.Lyrelog;
public static void main(String[] args) {
Lyrelog.start(Configuration.builder(System.getenv("LYRELOG_KEY")).platform("api").build());
SpringApplication.run(OrdersApplication.class, args);
}Use a server key, lyr_srv_..., from the project's API keys page (Send events from a
server). It is a secret: keep it in your server's environment, never in an app or a web page.
Its events are verified, and one address has no daily share of
the month. An ingest key, lyr_ing_..., works too, but its events aren't verified.
start returns true when it started Lyrelog. A second call while it runs is ignored and
returns false. A null or blank key starts nothing and returns false, and every other call
then does nothing, so a missing environment variable in development or tests doesn't stop the
service. Events logged before start are ignored. start never throws.
Events go to https://ingest.lyrelog.com. Pass endpoint only to send them somewhere else. It
is the base URL (a pasted .../v1/events:batch path is stripped) and must be https://, or
http:// for localhost, 127.0.0.1 or ::1. Any other endpoint is refused: nothing is sent,
and the SDK says so once on its diagnostics.
Log events
One function per level: trace, debug, info, notice, warning, error, critical, or
Lyrelog.log(level, message, ...).
Lyrelog.info("checkout_done", attrs = mapOf("plan" to "pro", "latency_ms" to 412))
Lyrelog.error("payment_failed",
category = "billing",
attrs = mapOf("code" to "card_declined"),
payload = mapOf("response" to body),
)Attributes can be strings, numbers, booleans, null, lists (sent as strings) and nested maps up
to 4 levels. The payload is any map, list or value. A log call never blocks and never throws: it
copies the event and returns. After close, log calls do nothing.
Loggers for each area
val billing = Lyrelog.logger("billing")
billing.warning("invoice_retry", mapOf("attempt" to 2))A logger is cheap to make and keep (a field, a companion object). It logs into whatever is
running when it is called, so it can be made before start.
Users
A server handles many people at once. Set the user on the event, or for the work of one request:
Lyrelog.info("checkout_done", user = customerId) // for this event only
Lyrelog.userScope(customerId).use { // for everything logged on this thread
handle(request) // until the scope is closed
}
Lyrelog.setUser("u_123") // every following event, every thread
Lyrelog.setUser(null) // clear itThe event's own user wins over the scope's, and the scope's over setUser. A scope is
thread-local: in Java, open it with try-with-resources in a servlet filter (as in
the request filter below). Coroutines and reactive code (Ktor,
WebFlux) hop between threads, so pass user on each event there instead. Use your own ids, not
email addresses.
The Logback appender
With lyrelog-jvm-logback, every record your loggers write through SLF4J or Logback can also go
to Lyrelog. Add the appender in logback.xml (in Spring Boot, logback-spring.xml):
<configuration>
<appender name="LYRELOG" class="com.lyrelog.logback.LyrelogAppender" />
<root level="INFO">
<appender-ref ref="LYRELOG" />
</root>
</configuration>The appender forwards to the running Lyrelog, started in code as above. Or let it start
Lyrelog itself, so the service needs no code at all; it then closes Lyrelog when Logback stops:
<appender name="LYRELOG" class="com.lyrelog.logback.LyrelogAppender">
<apiKey>${LYRELOG_KEY}</apiKey>
<platform>api</platform>
<!-- <environment>staging</environment>, <endpoint>...</endpoint> -->
</appender>Records logged before Lyrelog starts are ignored, so start it before your framework logs what you want to keep. To keep printing to the console as well, keep your console appender beside this one.
private static final org.slf4j.Logger log = LoggerFactory.getLogger(Invoices.class);
MDC.put("user_id", customerId); // the event's user
log.info("Order {} shipped", orderId);
log.atWarn().addKeyValue("attempt", 2).log("invoice_retry");
log.error("charge_failed", e);| SLF4J and Logback | Lyrelog |
|---|---|
Logger name (com.example.billing.Invoices) | category, unless the appender has a <category> or the record a category key |
TRACE, DEBUG, INFO, WARN, ERROR | trace, debug, info, warning, error |
| The formatted message | message; with arguments, the pattern goes to attrs.msg_template to group by |
MDC entries, and SLF4J 2 key-value pairs (addKeyValue) | attrs, except user_id (the event's user) and category. <includeMdc>false</includeMdc> leaves the MDC out |
| The exception | attrs.error_type, attrs.error_message and payload.exception (frames, causes) |
| The record's time | The event's time |
Lyrelog's own minimum level still applies: trace records are dropped unless minimumLevel is
Level.TRACE. The appender ignores records from com.lyrelog loggers and from the SDK's own
threads, so it can never loop. There is no appender for Log4j 2 or java.util.logging. For
events you want to chart, prefer direct calls with a stable name: a formatted message is prose.
A Spring request filter
There is no built-in filter. This one records every request as http_request_served in the
http area, at info, warning for a 4xx and error for a 5xx, with the route pattern rather than
the path, so ids don't explode the cardinality. It also sets the request's user for everything
logged while it is handled:
import com.lyrelog.Level;
import com.lyrelog.Lyrelog;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Map;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import org.springframework.web.servlet.HandlerMapping;
@Component
public class LyrelogRequests extends OncePerRequestFilter {
private final com.lyrelog.Logger http = Lyrelog.logger("http");
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
try (Lyrelog.UserScope user = Lyrelog.userScope(currentUserId(request))) { // your own id, or null
long start = System.nanoTime();
int status = 500; // when the handler throws, Spring writes the error page later
try {
chain.doFilter(request, response);
status = response.getStatus();
} finally {
Object route = request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE);
Level level = status >= 500 ? Level.ERROR : status >= 400 ? Level.WARNING : Level.INFO;
http.log(level, "http_request_served", Map.of(
"method", request.getMethod(),
"route", route != null ? route.toString() : "unmatched", // /orders/{id}, not the id
"status", status,
"duration_ms", (System.nanoTime() - start) / 1_000_000));
}
}
}
}Spring logs an exception a handler throws on its own loggers, with the stack, so with the
Logback appender it already arrives with error_type and error_message; capturing it in the
filter too would record it twice.
Errors
try {
charge(order)
} catch (e: PaymentException) {
Lyrelog.captureError(e, category = "billing", attrs = mapOf("order_id" to order.id))
}The event is exception, level error, with error_type (the class name) and error_message
in the attributes and, in the payload, the description, up to 60 frames, the causes (up to 8, 20
frames each) and the suppressed exceptions. Your attributes are added; those two win. level
changes the level and user sets the event's user. logger.captureError(e) uses the logger's
category.
Uncaught exceptions
Off by default. With captureUncaught = true, an exception that no code caught, on any thread,
is recorded as exception with handled: false, mechanism: uncaught_exception and the
thread's name in thread. Then the handler that was there before runs (or the JVM's usual
printout). When that thread was the last one, the shutdown hook sends the event before the JVM
exits.
Lyrelog.start(Configuration(apiKey = System.getenv("LYRELOG_KEY"), captureUncaught = true))Frameworks catch the exceptions of a request themselves and log them: those arrive through the
Logback appender. An exception in a task given to an ExecutorService with submit goes to its
Future, not to this handler.
Environments and platform
environment comes from the LYRELOG_ENVIRONMENT environment variable, else production.
platform is jvm. Set either in the configuration:
Lyrelog.start(Configuration(apiKey = key, environment = "staging", platform = "worker"))Any name works: lowercase letters, digits, - and _, up to 32. Other text in the configuration
is cleaned for you (Staging EU is staging-eu); an invalid $LYRELOG_ENVIRONMENT is reported
once and replaced by production, so a typo in a deploy never stops a service.
release (resource.app_version) defaults to the Implementation-Version of the jar the JVM
was started with (java -jar orders.jar), and appId
(resource.app_id) to its Implementation-Title, else the jar's file name, else the main class.
Set them when that isn't what you want.
Consent
Consent is for apps that run on someone's device. A server SDK sends what it is given; there is no consent option.
Flush and stop
Lyrelog.flush() // send what's queued now, waiting up to 5 s; true when it finished
Lyrelog.flushInBackground() // the same, without waiting
Lyrelog.close() // send what's left (up to closeTimeoutMillis), then stopYou rarely need any of them: the SDK sends every 5 s, and a JVM shutdown hook closes it when the
JVM exits normally (the end of main, System.exit, SIGTERM and Ctrl-C). kill -9 skips the
hook. Turn the hook off with shutdownHook = false and call close yourself.
flush respects a Retry-After the server asked for: it doesn't send sooner. After close,
start can be called again.
In a serverless function (AWS Lambda), call Lyrelog.flush() before the handler returns: the
runtime may freeze the process before the next send.
Where events wait
Events are written to a queue directory, one file per batch, and a batch leaves it only once the server has taken it.
- By default, a new directory under
java.io.tmpdirfor this process, deleted onclose. What wasn't sent by then is lost, as with an in-memory queue. - With
storageDirectory, a directory of your own: what wasn't sent (the server was unreachable, or asked to wait) stays there and goes out after the nextstart. One process at a time: the directory is locked, and a second process that finds it locked queues in a temporary directory instead, and says so once.
Both are capped at 20 MB and 7 days, oldest first (maxDiskBytes, maxEventAgeMillis).
In development and tests
Leave the key out: Lyrelog.start(Configuration(apiKey = System.getenv("LYRELOG_KEY"))) starts
nothing when the variable isn't set, and every call does nothing, so tests and a laptop without a
key run the same code. The Logback appender does nothing either while Lyrelog isn't running.
To see what a test logs, point endpoint at a local HTTP server (plain http:// is allowed for
loopback addresses) and read the batches it receives: gzipped NDJSON, one event per line, as on
the HTTP API page.
From Java
Every function on Lyrelog is static from Java, with overloads for the optional arguments, and
Configuration has a builder:
Lyrelog.start(Configuration.builder(System.getenv("LYRELOG_KEY"))
.platform("api")
.environment("staging")
.captureUncaught(true)
.build());
Lyrelog.info("checkout_done");
Lyrelog.info("checkout_done", "billing", Map.of("plan", "pro"));
Lyrelog.captureError(e, "billing");
try (Lyrelog.UserScope scope = Lyrelog.userScope(customerId)) {
Lyrelog.logger("billing").warning("invoice_retry", Map.of("attempt", 2));
}new Configuration(key) works too, and a logger's functions have overloads, so trailing
arguments can be left out.
Verified events
A verified event is one that can't have been sent by someone who copied the public ingest key out of an app or a web page. What that proves, and what it doesn't, is on Verified events.
Send with a server key, lyr_srv_.... Every event it sends is verified (verified_by is
server) and doesn't count toward the
daily share of one address. An owner
makes it on the project's API keys page, Send events from a server, after a recent sign-in
(within 10 minutes), and it is shown once. It is a secret: keep it out of the repository and read
it from the server's environment or secret store.
Lyrelog.start(Configuration(apiKey = System.getenv("LYRELOG_KEY"))) // lyr_srv_..., set in the server's environmentTo check, open the explorer: in Filter, pick Verified by and Server key (the other values are App Attest and Not verified). An opened event says it under Context, in its Sender line: Verified, server key, or Not verified, public key.
Events sent with an ingest key, lyr_ing_..., or a full-access key, lyr_adm_..., arrive
unverified: anyone holding a public key could have sent the same, and they count toward the daily
share. A verified event proves who sent it, not that what it says is true.
See Verified events for who verifies what and the daily share.
Configuration
| Option | Default | Meaning |
|---|---|---|
apiKey | required | The project's server key (lyr_srv_...), or an ingest key. null or blank: nothing starts |
endpoint | https://ingest.lyrelog.com | The base URL; a pasted /v1/... path is stripped. https only, http only for localhost |
environment | $LYRELOG_ENVIRONMENT, else production | Any valid name |
platform | jvm | api, worker... |
release | the started jar's Implementation-Version | Sent as resource.app_version |
appId | the started jar's Implementation-Title or file name, else the main class | Sent as resource.app_id: the service name |
minimumLevel | Level.DEBUG | Events below it are dropped before queueing |
flushIntervalMillis | 5 s | How often the background thread sends |
maxBatchBytes | 512 KiB | A batch closes at whichever limit comes first (kept within 4 MiB) |
maxBatchEvents | 500 | At most 2,000 |
maxQueue | 10,000 | Events waiting to be written; past it the oldest is dropped and counted |
maxDiskBytes | 20 MB | Cap on the queue directory, oldest first |
maxEventAgeMillis | 7 days | Older queued events are dropped |
redactedKeys | null: DEFAULT_REDACTED_KEYS | Key names to redact; your list replaces the defaults, an empty list turns it off |
storageDirectory | a temporary directory per process | Where the queue lives; see Where events wait |
captureUncaught | false | Record uncaught exceptions |
shutdownHook | true | Close when the JVM shuts down |
closeTimeoutMillis | 5 s | How long closing waits for the last send |
diagnostics | standard error | Receives the SDK's own messages (a Consumer<String>). Don't point it at a logger that feeds Lyrelog |
Delivery and limits
| Never blocks | Logging copies the event and returns; one background writer puts it in the queue directory and never waits on the network. One background thread sends. Safe for concurrent use. |
| Batches | Up to 500 events or 512 KiB per request, gzipped, every flushIntervalMillis. An event over 1 MiB once encoded is dropped and counted. |
| Retries | 1 s doubling to 5 min, with full jitter, honoring Retry-After; the same bytes are resent. Any other 4xx drops the batch, a 3xx (a redirect, never followed) drops it, and a refused key (401, 403) stops sending until the process restarts and keeps the events. A batch the server keeps failing while others go through goes to the back, then is dropped. |
| Monthly quota | When a Free organization has used its month's events, the server answers 429. Events stay in the queue and go out once the server takes them again, within the 20 MB and 7 day caps. The SDK prints one warning on its diagnostics, and again only after a batch has been accepted in between. |
| Drops are visible | Events lost to the caps become one lyrelog_events_dropped warning (area lyrelog) with the count. |
| Its own problems | A wrong endpoint, a 404, a refused key, rejected lines or an endpoint that doesn't answer are said once per process on diagnostics (standard error by default), each line starting with lyrelog: . |
| Context | os_name and os_version (the JVM's os.name and os.version), the app id, release, environment and platform, an install id per process and one session id per process. |
| Secrets | Values under keys whose name is or ends with authorization, cookie, password, token, secret, apikey and the rest of the shared list become "[redacted]", at any depth, before anything is cut. |
| Limits | Long strings cut on a character boundary, extra attributes counted in lyrelog_dropped_attrs, payloads over 64 KiB summarized as {"lyrelog_truncated": true, "bytes": N, "preview": "..."}. |
| The key | Printing a Configuration hides it. |
Change the redaction list with redactedKeys: your list replaces the defaults, and an empty list
turns redaction off. To add to it:
Lyrelog.start(Configuration(
apiKey = System.getenv("LYRELOG_KEY"),
redactedKeys = Configuration.DEFAULT_REDACTED_KEYS + "card_number",
))