Roadmap¶

This roadmap outlines the planned development of Alopex DB from the current state to production readiness.

Current Status¶

# Rust
cargo add alopex-embedded alopex-sql alopex-server alopex-dataframe alopex-cluster

# Python
pip install alopex

cargo add alopex-skulk

Timeline¶

gantt
    title Alopex Product Family Timeline
    dateFormat  YYYY-MM
    axisFormat  %Y-%m

    section Alopex DB
    v0.1-v0.2 Core          :done, 2025-01, 2025-10
    v0.3 SQL + HNSW         :done, 2025-10, 2025-12
    v0.4 Server + DataFrame :done, 2026-01, 2026-01
    v0.5 GROUP BY + JOIN    :done, 2026-01, 2026-01

    section Practical Foundation
    v0.6 SQL JOIN + Nim Parser :done, 2026-07, 2026-07

    section Distributed
    v0.7 Cluster-aware      :done, 2026-07, 2026-07
    v0.8.0-v0.8.10 Streaming + SQL :done, 2026-07, 2026-08
    v0.8.11 Transactions + vectors :active, 2026-09, 2027-01
    v0.9 Distributed parity (frozen) :2027-01, 2027-02
    v1.0 GA                 :milestone, 2027-03, 0d

    section Chirps
    v0.5 Raft Consensus     :done, 2025-12, 2026-01
    v0.6 Raft + Transport   :done, 2026-08, 2026-08
    v0.7 Iggy + Durable     :2027-01, 2027-02

    section Skulk
    v0.1-v0.2 TSM + Lifecycle :done, 2025-10, 2025-12
    v0.3 Arrow + Parquet    :done, 2026-07, 2026-07
    v0.3.1 Throughput       :done, 2026-07, 2026-07
    v0.4 Query Engine       :done, 2026-07, 2026-07
    v0.8 Sharding on Chirps :2027-01, 2027-03

    section Trail
    Design                  :done, 2026-07, 2026-07
    v0.1 Append path        :2026-09, 2026-11
    v0.3 Query + DSL        :2026-12, 2027-02
    v0.6 Joins + TraceQL    :2027-03, 2027-05

Published Crates¶

The following crates are available on crates.io:

Crate	Version	Description
	latest	Embedded database API
	latest	SQL parser, planner, executor
	latest	Core storage engine
	latest	HTTP/gRPC server
	latest	Polars-compatible DataFrame API
	latest	Cluster-aware distributed mode
	latest	CLI with TUI / admin console
	v0.6.3	Cluster messaging layer
	v0.4.0	Time-series storage and ingest core

Version Compatibility Matrix¶

Phase 1: Foundation (v0.1 - v0.2)¶

Status: Complete

v0.1 — Embedded KV Core¶

• LSM-Tree storage engine
• Write-Ahead Log (WAL) with crash recovery
• Key-Value API (open/put/get/delete)
• Transactions (begin/commit/rollback)
• MVCC with Snapshot Isolation

v0.2 — Vector Core + Columnar¶

• Vector type (VECTOR(dimension))
• Flat search (cosine, L2, inner product)
• Columnar segment storage with compression
• In-memory mode support
• Vector delete/compaction

Phase 2: SQL & HNSW (v0.3)¶

Status: Complete — crates.io Published

v0.3 — SQL Frontend + HNSW Index¶

The first public release on crates.io with full SQL support and HNSW indexing.

Completed Features¶

• SQL Parser (DDL: CREATE/DROP TABLE/INDEX, DML: SELECT/INSERT/UPDATE/DELETE)
• Query Planner with Catalog and LogicalPlan
• SQL Executor (iterator-based execution)
• vector_similarity() function with Top-K optimization
• HNSW Index for high-performance similarity search
• Columnar COPY/Bulk Load (Parquet/CSV → ColumnarSegment)
• Embedded Integration (Database::execute_sql, Transaction::execute_sql)

SQL Examples¶

-- Create table with vector column
CREATE TABLE documents (
    id INTEGER PRIMARY KEY,
    content TEXT,
    embedding VECTOR(1536)
);

-- Insert with vector
INSERT INTO documents (id, content, embedding)
VALUES (1, 'Hello world', [0.1, 0.2, ...]);

-- Hybrid search with vector similarity
SELECT id, content, vector_similarity(embedding, ?) AS score
FROM documents
ORDER BY score DESC
LIMIT 10;

-- Create HNSW index
CREATE INDEX idx_emb ON documents USING HNSW (embedding);

Phase 3: Python & Server (v0.3.3 - v0.4)¶

v0.3.3 — Python Wrapper (alopex-py)¶

Status: Complete — PyPI Published Released: December 2025

Python bindings via PyO3 for the embedded database.

Features¶

• PyO3 module structure with error handling
• Database / Transaction bindings
• SQL API bindings (execute_sql, QueryResult)
• Type stubs (.pyi files) for IDE support
• CI/CD with maturin + pytest + TestPyPI verification
• Vector/HNSW API bindings — planned for v0.3.4
• NumPy integration (zero-copy arrays) — planned for v0.3.5

Preview¶

import alopex

# Open database
db = alopex.Database.open("./my_data")

# Execute SQL
results = db.execute_sql(
    "SELECT * FROM docs WHERE vector_similarity(embedding, ?) > 0.8",
    [query_embedding]
)

# HNSW search
similar = db.search_hnsw("docs", query_embedding, k=10)
for doc_id, score in similar:
    print(f"{doc_id}: {score:.4f}")

v0.4.0 — Server Mode + DataFrame¶

Status: Complete — crates.io Published Released: January 2026

Standalone server and Polars-compatible DataFrame API foundation.

Server Features¶

• alopex-server binary with HTTP/gRPC API
• SQL API (DDL/DML/SELECT with streaming)
• Vector API (HNSW/Flat search)
• Session Manager with TLS support
• Observability (metrics, tracing)
• Runtime-agnostic async facade with tokio adapter

DataFrame Features (alopex-dataframe v0.1.0)¶

• DataFrame / LazyFrame / Expr types
• I/O: read_csv, read_parquet, scan_* variants
• Operations: select, filter, with_columns
• Aggregations: group_by, agg, sum, mean, etc.
• Lazy evaluation with query optimization
• Predicate/Projection Pushdown

DataFrame Preview¶

use alopex_dataframe::{DataFrame, col, lit};

let df = DataFrame::read_parquet("data.parquet")?;

let result = df
    .lazy()
    .filter(col("score").gt(lit(0.5)))
    .select([col("id"), col("content")])
    .collect()?;

Phase 4: GROUP BY, DataFrame P1, Practical Foundation (v0.5 - v0.6)¶

v0.5 — GROUP BY + DataFrame P1¶

Status: Complete — crates.io Published Released: January 2026

• GROUP BY / Aggregation (alopex-sql v0.5.0)
• DataFrame P1: JOIN (inner, left, right, full, semi, anti)
• DataFrame P1: sort / head / tail / unique
• DataFrame P1: fill_null / drop_nulls / null_count
• Server API extensions (alopex-server v0.5.0)
• WAL/Crash recovery hardening — planned for v0.5.x
• Performance benchmarks — planned for v0.5.x

v0.6 — SQL JOIN/Subquery + Nim FFI Parser¶

Status: Complete — crates.io Published Released: July 7, 2026

The SQL frontend was re-architected: the Rust SQL parser was replaced with a Nim FFI parser communicating over MessagePack, and JOIN/subquery support landed.

• SQL JOIN support (alopex-sql v0.6)
• SQL Subquery support
• Nim FFI SQL parser (MessagePack protocol) — parser shared libraries ship with each release
• Workspace version alignment across all crates and alopex-py

Carried forward toward v1.0: DataFrame P2 (cast / pivot / unpivot / window functions), embedded durability hardening, and server operation hardening.

WASM/web runtime support is deferred and will be re-evaluated after v1.0 based on adoption and product priorities.

Phase 5: Distributed (v0.7 - v0.9)¶

v0.7 — Cluster-Aware¶

Status: Complete — crates.io Published Released: July 14–18, 2026 (v0.7.0 → v0.7.6)

• alopex-cluster module published on crates.io
• Mode-parity verification suite (v0.7.1)
• gRPC cluster administration (v0.7.6)
• Release packaging hardening (rpath propagation, Nim parser vendoring)

v0.8 — Single-Node Compatibility Closure¶

Status: Active — v0.8.10 published; v0.8.11 remains Released so far: July 23–August 28, 2026 Uses: Chirps v0.5.2 (the standalone Chirps release series is currently v0.6.3)

• Cluster metadata, lifecycle, routing diagnostics, and authenticated distributed-read contracts
• Multi-statement and streaming SQL across HTTP, gRPC, and CLI
• Bounded/incremental CSV and Parquet DataFrame streaming
• Sync/async Python local APIs, SQL streams, transactions, and DataFrame bindings
• PromQL and SQL-TS parser contracts (Nim ABI 0.2.0)
• SQL correctness and name-resolution hardening (v0.8.2–v0.8.3)
• Parser contract 0.4.0 and release-asset hardening (v0.8.4–v0.8.5)
• Alias, REAL, CASE, set operations, CTE, and basic window contracts (v0.8.6)
• Recursive CTE and window correctness closure (v0.8.7)
• Portable relational grammar, single-file convergence, and Python server client (v0.8.8)
• Portable SQL functions and bounded KV glob/regex search (v0.8.9)
• Exact DECIMAL, native temporal, JSON/JSONB, nested, and full-text types (v0.8.10)
• SQL application and administration surface — transaction control, parameters, introspection, schema evolution, constraints, advanced DML, COPY, and identity/sequences (v0.8.11)
• HNSW correctness, recall, and latency closure with published conformance evidence (v0.8.11)

Remote execution, distributed transactions, and client/connection-pool APIs remain outside the v0.8 supported scope.

v0.9 — Distributed Capability and Parity¶

Status: Frozen until v0.8.11 is published and verified Depends on: Chirps v0.7

• Multi-Raft (range partitioning)
• CRDT (Counter, Set for eventual consistency)
• Changefeed via Durable profile
• Distributed transactions

Phase 6: Production (v0.10 - v1.0)¶

v0.10 — Hardening¶

Status: Planned Target: Q1 2027

• Single-node regression suite
• Cluster consistency tests
• Partition/recovery tests
• Configuration freeze
• API stability guarantee

v1.0 — General Availability¶

Status: Planned Target: Q1 2027

• 3-10 node production support
• Rolling upgrades
• Complete documentation
• API/ABI compatibility tests
• Federation support

Chirps Roadmap¶

Alopex Chirps (cluster messaging layer) has its own development track:

Version	Status	Features
v0.1-v0.3	Complete	Gossip, SWIM, Membership API
v0.4	Complete	Raft-ready transport, QoS streams
v0.5	Complete	Raft Consensus API, WalRaftStorage
v0.5.1	Complete	File Transfer API
v0.6.1	Released (Aug 9, 2026)	Raft storage/node APIs, observability, hardened QUIC/SWIM and resumable file transfer
v0.6.2	Released (Aug 11, 2026)	Production QUIC per-connection memory bounds
v0.6.3	Released (Aug 11, 2026)	Memory-contract and mTLS conformance
v0.7	Planned	IggyBackend, Durable profile
v0.8+	Planned	Federation profile, cross-cluster mTLS

Skulk Roadmap¶

Alopex Skulk is the time-series storage and ingest core. It has its own repository and version series.

Version	Status	Features
v0.1.0	Released (Dec 18, 2025)	TSM storage, Gorilla compression, WAL, MemTable
v0.2.0	Released (Dec 23, 2025)	Retention/TTL, time partitions, TSM compaction
v0.3.0	Released (Jul 28, 2026)	Arrow + Parquet wide columnar storage, durability, three ingest decoders
v0.3.1	Released (Jul 29, 2026)	Durable ingest throughput fix (~3–4×), unchanged on-disk format
v0.4.0	Released (Jul 31, 2026)	Embedded PromQL/SQL-TS query engine, Arrow streams, pruning, out-of-order policy
v0.5	Planned	Downsampling, continuous queries, Python bindings
v0.6	Planned	HTTP server, Prometheus-compatible endpoints
v0.7	Planned	Alerting
v0.8	Planned	Sharding via Chirps membership
v0.9	Planned	Shard Raft groups via Chirps Raft API
v1.0	Planned	Stable

Trail Roadmap¶

Alopex Trail is a proposed log and event store with late-bound schema, reusing Skulk's append-only storage machinery. It is a published design; implementation has not started.

Trail's WAL and ingest design can now build on the stabilized Skulk v0.3.1 write path and the v0.4 query contracts. The storage, manifest, and locking pieces remain independently reusable.

Python Across the Family¶

Every engine in the family is reachable from Python, built the same way: PyO3 with abi3 wheels on PyPI, results handed to pandas or Polars over Arrow.

alopex-otel is the different one: it doesn't just read storage from Python, it embeds the whole observability platform in a Python process — receiver, pipeline, storage, and dashboard — so a desktop app or a notebook can be observed without deploying anything.

Package	Engine	Status
alopex	Alopex DB — SQL, vector, DataFrame
alopex-skulk	Skulk — time-series	Skulk v0.5
alopex-trail	Trail — logs and events	Trail v0.4
alopex-otel	OTel — instrumentation, embedded platform, cross-signal queries	OTel M1

Every engine holds data as Arrow internally, so query results cross into pandas or Polars without a conversion step.

alopex-dataframe Roadmap¶

Polars-compatible DataFrame engine in pure Rust. Since v0.6.0 the crate is versioned with the workspace; the remaining feature phases below are tracked by phase name.

Phase	Features	Status
DF-0	DataFrame/Series types, Arrow integration, CSV/Parquet I/O	Complete
DF-1	JOIN (all types), sort/head/tail/unique, fill_null/drop_nulls	Complete
DF-2	cast, pivot/unpivot, window functions (over, rolling, shift, rank)	Planned
DF-3	str.* namespace, dt.* namespace, list.* + explode/implode	Planned
DF-4	Bounded CSV/Parquet streaming, expressions, projection, concat	v0.8 baseline shipped; further optimization planned

alopex-py Roadmap¶

Python bindings with NumPy and DataFrame support. Up to v0.4.0 alopex-py followed its own versioning scheme; since v0.6.0 it is released in lockstep with the Rust workspace, with wheels attached to every release.

Version	Phase	Features	Status
v0.3.3	Phase 1	Database/Transaction/SQL basic API	PyPI Published
v0.3.5	Phase 1+	NumPy integration (zero-copy arrays + GIL release)	PyPI Published
v0.4.0	Phase 1+	Catalog API (Polars Unity Catalog compatible)	PyPI Published
v0.6.0–v0.8.10	Aligned	Workspace-aligned releases (SQL, cluster-aware streaming, vectors, DataFrame, server client)	PyPI Published
Phase 2	—	DataFrame API MVP via alopex-dataframe	Planned
Phase 3	—	DataFrame namespaces (str/dt/list)	Planned
GA	v1.0	Polars-compatible DataFrame + API stabilization	Planned

Contributing¶

We welcome contributions! Priority areas:

Contributing Guide

Changelog¶

Recent Updates¶

• 2026-08-28: Alopex DB v0.8.10 released — exact DECIMAL/NUMERIC, native DATE/TIME/INTERVAL, JSON/JSONB, nested ARRAY/MAP/STRUCT, and deterministic full-text search
• 2026-08-26: Alopex DB v0.8.9 released — portable SQL functions and bounded KV byte-glob/regex key search
• 2026-08-26: Alopex DB v0.8.8 released — portable relational grammar, single-file convergence, process locking, and Python server client
• 2026-08-18: Alopex DB v0.8.7 released — recursive CTEs and window correctness closure
• 2026-08-17: Alopex DB v0.8.6 released — alias resolution, REAL, CASE, set operations, CTEs, and basic window functions
• 2026-08-15: Alopex DB v0.8.5 released — public-surface, security, package-parity, and release-verifier hardening
• 2026-08-10: Alopex DB v0.8.4 released — parser contract 0.4.0 and target-qualified native parser assets
• 2026-08-11: alopex-chirps v0.6.3 released — restored memory-contract and mTLS conformance after v0.6.2 bounded production QUIC memory
• 2026-08-09: alopex-chirps v0.6.1 released — Raft APIs and hardened QUIC/SWIM/file-transfer release contract
• 2026-08-03: Alopex DB v0.8.3 released — SQL JOIN/name-resolution correctness and performance fixes
• 2026-08-01: Alopex DB v0.8.2 — SQL correctness fixes across timestamps, numeric types, casts, subqueries, and JOINs
• 2026-07-31: Alopex Skulk v0.4.0 released — embedded PromQL/SQL-TS query engine and Arrow result streams
• 2026-07-29: Alopex Skulk v0.3.1 — durable ingest throughput improvement with unchanged on-disk format
• 2026-07-23: Alopex DB v0.8.0 released — cluster-aware streaming across server, CLI, DataFrame, and Python surfaces
• 2026-07-22: Alopex Enterprise entered the design phase — commercial middleware on the Alopex OSS foundation (Security Suite, Unified Query Model, Observe/SRE, Infrastructure packages)
• 2026-07-22: Alopex Data UI development started — concept site published (a consistent UI model for data applications on one DataSource contract)
• 2026-07-18: v0.7.6 released — gRPC cluster administration; CLI binaries and Python wheels attached to the release
• 2026-07-15 〜 07-18: v0.7.1–v0.7.5 — mode-parity verification suite, release packaging fixes (rpath propagation, Nim parser vendoring)
• 2026-07-14: v0.7.0 released — cluster-aware release; alopex-cluster published on crates.io
• 2026-07-07: v0.6.0 released — SQL JOIN/Subquery support; Rust SQL parser replaced with Nim FFI parser (MessagePack); workspace versions aligned across all crates and alopex-py
• 2026-01-29: v0.5.0 released — GROUP BY/Aggregation, DataFrame P1 (JOIN/sort/null), Server API extensions published on crates.io
• 2026-01-24: alopex-cli v0.4.2 released — TUI default, Admin TUI, security fixes, SELECT literal support
• 2026-01-14: v0.4.0 Server + DataFrame published on crates.io
• 2026-01-01: alopex-chirps v0.5.1 released — File Transfer API
• 2025-12-27: v0.3.3 alopex-py published on PyPI
• 2025-12-23: alopex-skulk v0.2.0 released — time-series database on Alopex Core, published on crates.io
• 2025-12-18: alopex-skulk v0.1.0 released — initial public release
• 2025-12: v0.3 SQL Frontend + HNSW published on crates.io
• 2025-11: HNSW index implementation complete
• 2025-10: alopex-sql Parser/Planner/Executor complete
• 2025-09: Columnar-based Vector Store complete
• 2025-08: LSM-Tree file mode complete
• 2025-06: In-memory mode complete
• 2025-01: Project started

For detailed changes across the family, see the Release History.