Journald Can Drop Messages During a Log Burst

A voice transcription worker can fail in a tight loop: each rejected audio chunk produces an error, and a single request generates hundreds of lines. An operator opens the journal and sees only the beginning of the failure. A gap in the journal cannot establish that the worker stopped emitting lines; journald may have dropped them.

The journald configuration manual defines RateLimitIntervalSec= as the window and RateLimitBurst= as the number of messages a service may log within it. Once that service exceeds the effective burst, journald drops further messages until the interval ends and generates a message reporting the number dropped. The limit is applied per service, so an unrelated API process does not consume the voice worker’s allowance. A unit’s LogRateLimitIntervalSec= or LogRateLimitBurst= settings can override the journal-wide values for that service.

There is an easy trap in treating RateLimitBurst= as a fixed ceiling. The manual says journald multiplies the effective limit by a factor derived from free space available to the journal. Two hosts with the same configuration can therefore admit different bursts. Setting either rate-limit value to zero disables this rate limiting, but that change alone cannot make logging lossless. Storage mode and capacity still matter: Storage=volatile keeps the journal under /run/log/journal, while Storage=none drops stored entries. The SystemMaxUse= and RuntimeMaxUse= limits constrain persistent and volatile journal space respectively; journald removes archived files when reclaiming space, leaving less history to inspect.

For an incident review, look for journald’s dropped-message notice before interpreting a quiet interval as an application recovery. Record the worker’s configured unit overrides, journal storage mode and available space alongside the timestamps. If every rejected audio chunk must be accounted for, use an application-level count or durable event path with its own delivery guarantees, then reconcile that count with the journal. Treat the journal as operational evidence whose completeness has conditions, especially during the very burst that prompted the investigation.