Skip to content

Log store capacity and performance

These figures come from a synthetic load test. They give orders of magnitude: they depend on your hardware and your traffic. To measure your own installation, a read-only kit is provided (see below).

Reference results

Measurements on a development machine (WSL2, 8 cores, 16 GB), TimescaleDB, 2 million events over 14 days:

Measure Result
Ingestion about 38,000 events per second (batches of 2,000, p95 of 254 ms, no rejection)
Target traffic a few dozen events per second: a margin of about ×1000
Disk before compression about 430 bytes per event, indexes included
Disk after compression about 100 bytes per event (×10 compression)
Memory log store about 11 MB, PostgreSQL about 1 GB (cache) after the load

Typical response times (median):

Query Duration
Event search a few milliseconds
Tops, security over 24 h, campaigns over 24 h 60 to 250 ms
IP sheet 60 to 380 ms
Visit statistics over 7 days 3.6 to 4.6 s
Visit statistics over 14 days 8 to 10.5 s
Per-IP statistics over 7 days 5.3 to 5.8 s

Long statistics

Visit and per-IP statistics are the most expensive queries: they count distinct values on the raw data. The maximum query time is 15 seconds (LOGSTORE_QUERY_TIMEOUT_SECONDS); on a very large volume, prefer short periods.

Compression does not change the duration of slow queries.

Estimate disk space

With about 100 bytes per event after compression, 50 events per second represent about 4.3 million events per day, i.e. about 1.9 GB per day before compression and 430 MB after. Raw logs are kept 7 to 30 days, rollups much longer but for a much smaller volume. This calculation is an estimate to confirm with a measurement on your traffic.

Measure your installation

The log store's bench/ folder contains two independent tools:

Tool Where to run it Role
run.sh Development machine (Docker, Node 18 or later) Disposable stack, synthetic ingestion, API query latency
measure-prod.sh Production host Read-only measurement: volume per day, throughput, sizes, compression, resources
./measure-prod.sh <db-container> [<logstore-container> [<shipper-container>]] > measure-$(date +%Y%m%d).txt

The production kit writes nothing, uses no token and does not access the network. The file it produces contains no IP or path, only volumes, durations and sizes.