Developers

Get started

In X2, everything is a stream.

Unix made everything a file. X2 makes everything a stream: your data, your cluster's state, your metrics.

Join the closed beta

Overview

  • One system for Kafka, NATS, MQTT, and Redis, with compute in the cluster.
  • Simple clients. Nothing to tune: no batching, no linger, no timeouts.
  • An order of magnitude faster: under 1 ms in the cloud, and lower on capable hardware.

Built by the team that architected NYSE Pillar, which powers $100B+ in daily transactions.

  • /
    • public
      • helloa stream: your data
      • ordersa partitioned topic
        • part-0
        • part-1
        • part-2
    • homeone directory per user
      • ana
        • events
    • cutconsumer positions
    • topicKafka topics, as links
      • orders
    • sys
      • cmdevery cluster change
      • txnthe partitions themselves
One namespace holds your streams, consumer positions, Kafka topics and the cluster's own streams. Stream creates, ACLs, metrics and alerts are all records on streams.

What’s in the SDK

The X2 SDK has what you need to run X2 and build applications on it:

ModuleWhat’s Included
ServerThe X2 server, shipped in the SDK container image and as Linux tarballs for x86_64 and aarch64. Runs directly on Linux, or in Docker.
Client LibrariesC++, Go, Rust, Python, TypeScript, and Java. Message types are generated from one schema, so every language reads and writes the same records with the same client semantics.
Cluster UI and ObservabilityA UI that shows the full state of your cluster, with fully programmable dashboards for traffic flows, backlogs, and per-client performance.
Cluster ToolsTools for managing a cluster and for reading and writing data, plus tools for provisioning, benchmarking, and diagnostics.
Blueprints & Samples7+ samples to help you get started, each a complete app with a code walk-through.
DocumentationQuick-start tutorials, guides for pointing Kafka and NATS apps at X2, tool and protocol references, architecture docs, and sample Terraform and Ansible configs for deploying a test cluster on AWS.

You can start without changing your existing applications. Point your Kafka or NATS clients at an X2 gateway and leave your producer and consumer code alone.

Kafka, NATS, Redis, Postgres, HTTP, and WebSocket clients publishing and consuming through AlgoX2 gateways publishersKafka producersNATS publishersRedis clientsPostgres (CDC)HTTP / WebSocketgatewayprotocol ingressone stream modelstream layerordered oncegatewayprotocol egressconsumersKafka consumersNATS subscribersRedis subscribersPostgres subscribersHTTP / WebSocket
Scroll sideways for the whole diagramIngress and egress gateways translate each edge protocol into the same stream model. A producer can publish through one protocol while a consumer reads the ordered history through another, without a bridge service between them.

To build directly on X2, use the client libraries: they give you the APIs for reading, writing, and managing streams. But X2 goes beyond moving data. It is the foundation for a broad range of ambitious applications, from financial exchanges to distributed data systems. It can index streams and retrieve records by key, which makes it easier to transform and materialize streams.

For applications that need the best performance and throughput, the SDK lets you build apps that run inside the cluster, over a shared-memory ring. It also offers integrations for keeping state (for example, in RocksDB), and the cluster restarts your processes when they fail.

Architectural Blueprints

To help you learn X2, the SDK includes a set of samples that serve as blueprints. Each one is a complete application with a walk-through that you can copy and make your own:

samp_exch

Resource Allocation / Matching Engine with Hot-Hot Failover

Matching engines are everywhere: financial exchanges, schedulers, ride sharing, delivery. This sample shows how to build one, modeled here as a financial exchange.

In hot-hot mode, two matchers run the same input and write to one output stream, and positional deduplication keeps one copy, so losing a matcher costs nothing. In hot-warm mode, the standby takes over from a snapshot of the book.

samp_fleet

Cross-System and Cross-Protocol Interoperability

Clients speaking different protocols read and write the same streams with their stock libraries, using one X2 cluster as the broker, log, key-value store, and database replication source.

samp_cdcsamp_agg

Real-Time Materialized Views from Streams

Checkpointed readers run in the cluster to maintain live views and event-time window aggregates, writing the results to output streams for downstream subscribers or key lookups.

The input can be a Postgres change log captured with x2cdc or an event stream such as page clicks.

Performance Baselines

X2 is designed for efficiency and performance, not just for the sake of performance, but to unlock new use cases for streams.

p99 Produce Latency

on high-end hardware
<5µs
in the cloud
<600µs

Measured from publish to acknowledgement, with no client-side batching delay.

p99 End-to-End Latency

on high-end hardware
<15µs
in the cloud
<1ms

Measured from publisher to subscriber through the cluster.

Throughput (4-Node Cluster)

messages/sec delivered
180M
messages/sec produced
20M

Multiple readers can follow the same stream at high fan-out without slowing down producers.

Closed beta

What would you build first?

We're letting in a small group, picked so the SDK gets tested on many kinds of problems.

Join the closed beta