Roadmap¶

This roadmap outlines the planned development of Alopex DB from the current state to production readiness.

Current Status¶

# Rust
cargo add alopex-embedded alopex-sql alopex-server alopex-dataframe alopex-cluster

# Python
pip install alopex

cargo add alopex-skulk

Timeline¶

gantt
    title Alopex Product Family Timeline
    dateFormat  YYYY-MM
    axisFormat  %Y-%m

    section Alopex DB
    v0.1-v0.2 Core          :done, 2025-01, 2025-10
    v0.3 SQL + HNSW         :done, 2025-10, 2025-12
    v0.4 Server + DataFrame :done, 2026-01, 2026-01
    v0.5 GROUP BY + JOIN    :done, 2026-01, 2026-01

    section Practical Foundation
    v0.6 SQL JOIN + Nim Parser :done, 2026-07, 2026-07

    section Distributed
    v0.7 Cluster-aware      :done, 2026-07, 2026-07
    v0.8 Metadata Raft      :2026-10, 2026-12
    v0.9 Multi-Raft         :2027-01, 2027-02
    v1.0 GA                 :milestone, 2027-03, 0d

    section Chirps
    v0.5 Raft Consensus     :done, 2025-12, 2026-01
    v0.6 Multi-Raft + TSO   :2026-10, 2026-12
    v0.7 Iggy + Durable     :2027-01, 2027-02

    section Skulk
    v0.1-v0.2 TSM + Lifecycle :done, 2025-10, 2025-12
    v0.3 Arrow + Parquet    :done, 2026-07, 2026-07
    v0.3.1 Throughput       :active, 2026-07, 2026-08
    v0.4 Query Engine       :2026-08, 2026-10
    v0.8 Sharding on Chirps :2027-01, 2027-03

    section Trail
    Design                  :done, 2026-07, 2026-07
    v0.1 Append path        :2026-09, 2026-11
    v0.3 Query + DSL        :2026-12, 2027-02
    v0.6 Joins + TraceQL    :2027-03, 2027-05

Published Crates¶

The following crates are available on crates.io:

Crate	Version	Description
	v0.7.6	Embedded database API
	v0.7.6	SQL parser, planner, executor
	v0.7.6	Core storage engine
	v0.7.6	HTTP/gRPC server
	v0.7.6	Polars-compatible DataFrame API
	v0.7.6	Cluster-aware distributed mode
	v0.7.6	CLI with TUI / admin console
	v0.5.1	Cluster messaging layer
	v0.3.0	Time-series storage and ingest core

Version Compatibility Matrix¶

Phase 1: Foundation (v0.1 - v0.2)¶

Status: Complete

v0.1 — Embedded KV Core¶

• LSM-Tree storage engine
• Write-Ahead Log (WAL) with crash recovery
• Key-Value API (open/put/get/delete)
• Transactions (begin/commit/rollback)
• MVCC with Snapshot Isolation

v0.2 — Vector Core + Columnar¶

• Vector type (VECTOR(dimension))
• Flat search (cosine, L2, inner product)
• Columnar segment storage with compression
• In-memory mode support
• Vector delete/compaction

Phase 2: SQL & HNSW (v0.3)¶

Status: Complete — crates.io Published

v0.3 — SQL Frontend + HNSW Index¶

The first public release on crates.io with full SQL support and HNSW indexing.

Completed Features¶

• SQL Parser (DDL: CREATE/DROP TABLE/INDEX, DML: SELECT/INSERT/UPDATE/DELETE)
• Query Planner with Catalog and LogicalPlan
• SQL Executor (iterator-based execution)
• vector_similarity() function with Top-K optimization
• HNSW Index for high-performance similarity search
• Columnar COPY/Bulk Load (Parquet/CSV → ColumnarSegment)
• Embedded Integration (Database::execute_sql, Transaction::execute_sql)

SQL Examples¶

-- Create table with vector column
CREATE TABLE documents (
    id INTEGER PRIMARY KEY,
    content TEXT,
    embedding VECTOR(1536)
);

-- Insert with vector
INSERT INTO documents (id, content, embedding)
VALUES (1, 'Hello world', [0.1, 0.2, ...]);

-- Hybrid search with vector similarity
SELECT id, content, vector_similarity(embedding, ?) AS score
FROM documents
ORDER BY score DESC
LIMIT 10;

-- Create HNSW index
CREATE INDEX idx_emb ON documents USING HNSW (embedding);

Phase 3: Python & Server (v0.3.3 - v0.4)¶

v0.3.3 — Python Wrapper (alopex-py)¶

Status: Complete — PyPI Published Released: December 2025

Python bindings via PyO3 for the embedded database.

Features¶

• PyO3 module structure with error handling
• Database / Transaction bindings
• SQL API bindings (execute_sql, QueryResult)
• Type stubs (.pyi files) for IDE support
• CI/CD with maturin + pytest + TestPyPI verification
• Vector/HNSW API bindings — planned for v0.3.4
• NumPy integration (zero-copy arrays) — planned for v0.3.5

Preview¶

import alopex

# Open database
db = alopex.Database.open("./my_data")

# Execute SQL
results = db.execute_sql(
    "SELECT * FROM docs WHERE vector_similarity(embedding, ?) > 0.8",
    [query_embedding]
)

# HNSW search
similar = db.search_hnsw("docs", query_embedding, k=10)
for doc_id, score in similar:
    print(f"{doc_id}: {score:.4f}")

v0.4.0 — Server Mode + DataFrame¶

Status: Complete — crates.io Published Released: January 2026

Standalone server and Polars-compatible DataFrame API foundation.

Server Features¶

• alopex-server binary with HTTP/gRPC API
• SQL API (DDL/DML/SELECT with streaming)
• Vector API (HNSW/Flat search)
• Session Manager with TLS support
• Observability (metrics, tracing)
• Runtime-agnostic async facade with tokio adapter

DataFrame Features (alopex-dataframe v0.1.0)¶

• DataFrame / LazyFrame / Expr types
• I/O: read_csv, read_parquet, scan_* variants
• Operations: select, filter, with_columns
• Aggregations: group_by, agg, sum, mean, etc.
• Lazy evaluation with query optimization
• Predicate/Projection Pushdown

DataFrame Preview¶

use alopex_dataframe::{DataFrame, col, lit};

let df = DataFrame::read_parquet("data.parquet")?;

let result = df
    .lazy()
    .filter(col("score").gt(lit(0.5)))
    .select([col("id"), col("content")])
    .collect()?;

Phase 4: GROUP BY, DataFrame P1, Practical Foundation (v0.5 - v0.6)¶

v0.5 — GROUP BY + DataFrame P1¶

Status: Complete — crates.io Published Released: January 2026

• GROUP BY / Aggregation (alopex-sql v0.5.0)
• DataFrame P1: JOIN (inner, left, right, full, semi, anti)
• DataFrame P1: sort / head / tail / unique
• DataFrame P1: fill_null / drop_nulls / null_count
• Server API extensions (alopex-server v0.5.0)
• WAL/Crash recovery hardening — planned for v0.5.x
• Performance benchmarks — planned for v0.5.x

v0.6 — SQL JOIN/Subquery + Nim FFI Parser¶

Status: Complete — crates.io Published Released: July 7, 2026

The SQL frontend was re-architected: the Rust SQL parser was replaced with a Nim FFI parser communicating over MessagePack, and JOIN/subquery support landed.

• SQL JOIN support (alopex-sql v0.6)
• SQL Subquery support
• Nim FFI SQL parser (MessagePack protocol) — parser shared libraries ship with each release
• Workspace version alignment across all crates and alopex-py

Carried forward toward v1.0: DataFrame P2 (cast / pivot / unpivot / window functions), embedded durability hardening, and server operation hardening.

WASM/web runtime support is deferred and will be re-evaluated after v1.0 based on adoption and product priorities.

Phase 5: Distributed (v0.7 - v0.9)¶

v0.7 — Cluster-Aware¶

Status: Complete — crates.io Published Released: July 14–18, 2026 (v0.7.0 → v0.7.6)

• alopex-cluster module published on crates.io
• Mode-parity verification suite (v0.7.1)
• gRPC cluster administration (v0.7.6)
• Release packaging hardening (rpath propagation, Nim parser vendoring)

v0.8 — Metadata Raft¶

Status: Planned Target: Q4 2026 Depends on: Chirps v0.6

• Metadata Raft Group
• MultiRaftManager integration
• Shard/range metadata management

v0.9 — Multi-Raft + CRDT¶

Status: Planned Target: Q1 2027 Depends on: Chirps v0.7

• Multi-Raft (range partitioning)
• CRDT (Counter, Set for eventual consistency)
• Changefeed via Durable profile
• Distributed transactions

Phase 6: Production (v0.10 - v1.0)¶

v0.10 — Hardening¶

Status: Planned Target: Q1 2027

• Single-node regression suite
• Cluster consistency tests
• Partition/recovery tests
• Configuration freeze
• API stability guarantee

v1.0 — General Availability¶

Status: Planned Target: Q1 2027

• 3-10 node production support
• Rolling upgrades
• Complete documentation
• API/ABI compatibility tests
• Federation support

Chirps Roadmap¶

Alopex Chirps (cluster messaging layer) has its own development track:

Version	Status	Features
v0.1-v0.3	Complete	Gossip, SWIM, Membership API
v0.4	Complete	Raft-ready transport, QoS streams
v0.5	Complete	Raft Consensus API, WalRaftStorage
v0.5.1	Complete	File Transfer API
v0.6	Planned	Multi-Raft, TSO, Observability
v0.7	Planned	IggyBackend, Durable profile
v0.8+	Planned	Federation profile, cross-cluster mTLS

Skulk Roadmap¶

Alopex Skulk is the time-series storage and ingest core. It has its own repository and version series.

Version	Status	Features
v0.1.0	Released (Dec 18, 2025)	TSM storage, Gorilla compression, WAL, MemTable
v0.2.0	Released (Dec 23, 2025)	Retention/TTL, time partitions, TSM compaction
v0.3.0	Released (Jul 28, 2026)	Arrow + Parquet wide columnar storage, durability, three ingest decoders
v0.3.1	In progress	Ingest throughput and p99 latency against unchanged targets
v0.4	Planned	Query engine — PromQL, SQL-TS, predicate pushdown. Parsers are shared with Alopex DB and gated behind a feature flag
v0.5	Planned	Downsampling, continuous queries, Python bindings
v0.6	Planned	HTTP server, Prometheus-compatible endpoints
v0.7	Planned	Alerting
v0.8	Planned	Sharding via Chirps membership
v0.9	Planned	Shard Raft groups via Chirps Raft API
v1.0	Planned	Stable

Trail Roadmap¶

Alopex Trail is a proposed log and event store with late-bound schema, reusing Skulk's append-only storage machinery. It is a published design; implementation has not started.

Trail's WAL and ingest layers reuse code that Skulk v0.3.1 is actively changing, so that part of the work follows v0.3.1. The storage, manifest, and locking pieces are independent of it.

Python Across the Family¶

Every engine in the family is reachable from Python, built the same way: PyO3 with abi3 wheels on PyPI, results handed to pandas or Polars over Arrow.

alopex-otel is the different one: it doesn't just read storage from Python, it embeds the whole observability platform in a Python process — receiver, pipeline, storage, and dashboard — so a desktop app or a notebook can be observed without deploying anything.

Package	Engine	Status
alopex	Alopex DB — SQL, vector, DataFrame	v0.7.6 on PyPI
alopex-skulk	Skulk — time-series	Skulk v0.5
alopex-trail	Trail — logs and events	Trail v0.4
alopex-otel	OTel — instrumentation, embedded platform, cross-signal queries	OTel M1

Every engine holds data as Arrow internally, so query results cross into pandas or Polars without a conversion step.

alopex-dataframe Roadmap¶

Polars-compatible DataFrame engine in pure Rust. Since v0.6.0 the crate is versioned with the workspace (currently v0.7.6); the remaining feature phases below are tracked by phase name.

Phase	Features	Status
DF-0	DataFrame/Series types, Arrow integration, CSV/Parquet I/O	Complete
DF-1	JOIN (all types), sort/head/tail/unique, fill_null/drop_nulls	Complete
DF-2	cast, pivot/unpivot, window functions (over, rolling, shift, rank)	Planned
DF-3	str.* namespace, dt.* namespace, list.* + explode/implode	Planned
DF-4	Streaming execution, CSE optimization, concat	Planned

alopex-py Roadmap¶

Python bindings with NumPy and DataFrame support. Up to v0.4.0 alopex-py followed its own versioning scheme; since v0.6.0 it is released in lockstep with the Rust workspace (currently v0.7.6 on PyPI, wheels attached to every release).

Version	Phase	Features	Status
v0.3.3	Phase 1	Database/Transaction/SQL basic API	PyPI Published
v0.3.5	Phase 1+	NumPy integration (zero-copy arrays + GIL release)	PyPI Published
v0.4.0	Phase 1+	Catalog API (Polars Unity Catalog compatible)	PyPI Published
v0.6.0–v0.7.6	Aligned	Workspace-aligned releases (SQL JOIN/subquery, cluster-aware engine)	PyPI Published
Phase 2	—	DataFrame API MVP via alopex-dataframe	Planned
Phase 3	—	DataFrame namespaces (str/dt/list)	Planned
GA	v1.0	Polars-compatible DataFrame + API stabilization	Planned

Contributing¶

We welcome contributions! Priority areas:

Contributing Guide

Changelog¶

Recent Updates¶

• 2026-07-22: Alopex Enterprise entered the design phase — commercial middleware on the Alopex OSS foundation (Security Suite, Unified Query Model, Observe/SRE, Infrastructure packages)
• 2026-07-22: Alopex Data UI development started — concept site published (a consistent UI model for data applications on one DataSource contract)
• 2026-07-18: v0.7.6 released — gRPC cluster administration; CLI binaries and Python wheels attached to the release
• 2026-07-15 〜 07-18: v0.7.1–v0.7.5 — mode-parity verification suite, release packaging fixes (rpath propagation, Nim parser vendoring)
• 2026-07-14: v0.7.0 released — cluster-aware release; alopex-cluster published on crates.io
• 2026-07-07: v0.6.0 released — SQL JOIN/Subquery support; Rust SQL parser replaced with Nim FFI parser (MessagePack); workspace versions aligned across all crates and alopex-py
• 2026-01-29: v0.5.0 released — GROUP BY/Aggregation, DataFrame P1 (JOIN/sort/null), Server API extensions published on crates.io
• 2026-01-24: alopex-cli v0.4.2 released — TUI default, Admin TUI, security fixes, SELECT literal support
• 2026-01-14: v0.4.0 Server + DataFrame published on crates.io
• 2026-01-01: alopex-chirps v0.5.1 released — File Transfer API
• 2025-12-27: v0.3.3 alopex-py published on PyPI
• 2025-12-23: alopex-skulk v0.2.0 released — time-series database on Alopex Core, published on crates.io
• 2025-12-18: alopex-skulk v0.1.0 released — initial public release
• 2025-12: v0.3 SQL Frontend + HNSW published on crates.io
• 2025-11: HNSW index implementation complete
• 2025-10: alopex-sql Parser/Planner/Executor complete
• 2025-09: Columnar-based Vector Store complete
• 2025-08: LSM-Tree file mode complete
• 2025-06: In-memory mode complete
• 2025-01: Project started

For detailed changes, see the GitHub Releases.