Features built to
explain every byte.
Explore the six core engineering breakthroughs behind KOLMOS — from autonomous explanation ladders and cryptographic WASM covenants to multi-wire drop-in database compatibility.
Standard Protocols & Direct Connectors
Six core pillars of
intelligent storage.
Click any capability below to explore how KOLMOS rethinks compression, query processing, sandboxed covenants, and cloud data architecture.
Explanation Ladder
Instead of storing raw byte blocks and hoping compression handles it, KOLMOS asks: 'what is the smallest program that regenerates this data, byte-for-byte?' Three rungs compete on an MDL cost formula; the engine continuously chooses the cheapest explanation.
$ kolmos --root ./store explain-report --json
{
"table": "telemetry_events",
"total_segments": 12,
"rung_distribution": {
"rung0_literal": 3,
"rung1_formula": 6, // e.g. adjusted_price = base_price * 1.18 + 5.0
"rung2_prototype": 3 // k-medoids cluster prototypes + member deltas
},
"raw_size_bytes": 104857600,
"stored_size_bytes": 18907660,
"compression_vs_parquet": "2.119x smaller"
}Three explanation rungs.
The cheapest program wins.
Every segment begins on Rung 0 for instantaneous write availability. Background workers continuously evaluate Rung 1 and Rung 2 alternatives, executing zero-downtime atomic swaps whenever a cheaper representation is discovered.
Rung 0 — Literal
zstd compression paired with per-corpus dictionary training. Every segment lands here first on ingest to ensure instant, predictable write latency.
Rung 1 — Formula
Autonomous formula mining detects linear combinations, sequences, string templates, and date offsets. Replaces gigabytes of numbers with a concise AST and small exception delta.
Rung 2 — Prototype
Bounded k-medoids clustering discovers representative prototypes across segments. Stores members as prototype references plus field-level delta exceptions.
Three phases.
Zero compromise.
KOLMOS unifies ingestion, autonomous background optimization, and standard DataFusion SQL querying into an append-only, crash-safe data pipeline.
Fast Append & Batched Ingestion
Data enters through batch connectors (PG/MySQL/Mongo/Parquet) or row-level DML into the local WAL write-buffer; batches accumulate to ~200K rows for explanation.
Background Explanation Ladder
Dedicated background workers (running strictly on a dedicated ComputePool, never on Tokio workers) evaluate cold segments against the MDL cost model. If Rung 1 or 2 wins, it executes a zero-downtime swap.
DataFusion SQL & WASM Decode
SQL queries hit Apache DataFusion. Header min/max statistics prune non-matching segments before fetch. Matching segments decode through the sandboxed WASM covenant into Arrow memory buffers.
Connect existing apps with
zero code changes.
KOLMOS speaks the native wire protocols of the databases you already run. Your ORMs, analytics dashboards, and application drivers connect transparently.
PostgreSQL Wire Protocol
Connect seamlessly with psql, pgAdmin, Prisma, SQLAlchemy, Metabase, Grafana, and any PostgreSQL client driver. Speaks pgwire natively with full ANSI SQL support.
MySQL Wire Protocol
Minimal high-performance server supporting handshake, auth, and COM_QUERY routing into DataFusion and WAL DML. Direct drop-in for PHP/Laravel, WordPress, and Node mysql2.
MongoDB Wire Protocol
Wire protocol supporting OP_MSG, SCRAM authentication, find, insert, update, and delete commands. Reuses native BSON mappings for zero-overhead document workflows.
How KOLMOS stacks up.
An honest architectural comparison across storage efficiency, self-documenting data, cryptographic covenants, and query execution characteristics.
| Architectural Dimension | KOLMOS Engine | Apache Parquet | PostgreSQL | ClickHouse | Snowflake |
|---|---|---|---|---|---|
| Storage Footprint vs Parquet-zstd | 1.66× – 2.12× smaller | 1.0× (Baseline) | 3.5× – 6.0× larger | ~1.2× smaller | ~1.15× smaller |
| Self-Documenting Explanation Engine | Built-in (Formulas & Prototypes) | None | None | None | None |
| Cryptographic Decode Covenant | WASM bytecode hash-pinned | Format spec dependent | Engine binary dependent | Engine binary dependent | Proprietary cloud locked |
| Cloudflare R2 & Object Storage Native | CAS chunks + Conditional PUT | Filesystem abstraction | Block storage (EBS) only | S3 disks support | Managed S3/GCS |
| Wire Protocol Drop-in Support | PostgreSQL, MySQL, MongoDB | File format only | PG only | HTTP + Native TCP | JDBC / ODBC / REST |
| Live Database Incremental Ingest | PG + MySQL + MongoDB built-in | Requires external ETL (Spark) | Source database | Kafka / S3 engines | Fivetran / Snowpipe ($$) |
| Full Scan Speed (Honest Evaluation) | 1.6× – 8× slower than Parquet | Fastest analytical baseline | Slow on large scans | Ultra-fast vectorized | Fast cloud warehouse |
* Note: KOLMOS is designed for massive storage reduction and self-documenting data; analytical full scans are 1.6×–8× slower than uncompressed Parquet scans.
Ready to store the
explanation of your data?
Start experiencing 1.66×–2.12× storage reduction with bit-exact recovery on your PostgreSQL, MySQL, or MongoDB workloads today.