System Architecture
Logix uses an event-driven micro-architecture capable of ingesting high-throughput log streams, persisting them to column-oriented storage, and fanning out real-time streams.
End-to-End Event Pipeline
┌──────────────────────────────────────────────────────────────────┐
│ Application Layer (Next.js / Node.js) │
│ logger.info() / logger.error() / logger.log() │
└────────────────────────────────┬─────────────────────────────────┘
│ HTTP POST /logs/send (Batch JSON)
▼
┌──────────────────────────────────────────────────────────────────┐
│ Logix Ingestion Gateway │
│ - Dual Auth Guard (Clerk JWT / API Key SHA-256 Hashing) │
│ - Plan Usage & Quota Verification Guard (Redis LRU Cache) │
└────────────────────────────────┬─────────────────────────────────┘
│ Publish to Subject: "logs.ingest"
▼
┌──────────────────────────────────────────────────────────────────┐
│ NATS JetStream (Stream: OML_LOGS) │
│ Worker Durable Push Queue: OML_LOGS_WORKER │
└────────────────────────────────┬─────────────────────────────────┘
│ Process Transformed Batch
▼
┌────────────────────────────────┼─────────────────────────────────┐
│ │ │
│ 1. Columnar Persistence │ 2. Real-Time Broadcaster │ 3. Rule Engine
│ ClickHouse Storage │ SSE Client Registry │ Alert Consumer
│ Table: logs.log_events │ data: { type: "live" } │ Webhook Worker
│ │ │
└────────────────────────────────┴─────────────────────────────────┘Core Infrastructure Components
1. NATS JetStream (Message Broker)
Acts as the shock absorber for incoming logs. When your application experiences sudden traffic spikes, logs are safely buffered inside the OML_LOGS JetStream stream instead of crashing database workers.
2. ClickHouse Database Engine
Logs are stored in a columnar schema optimized for time-series aggregation. ClickHouse allows querying millions of events with full-text search and environment filtering in under 100 milliseconds.
3. SSE Broadcaster Registry
Maintains active HTTP response streams (`EventSource`). As NATS consumers process incoming log batches, matching logs are filtered dynamically and streamed to connected live tail viewers.