Self-Hosted Agentic Observability with Superlog
Self-Hosted Agentic Observability with Superlog
Traditional monitoring tells you a service is down. But what if your observability stack could also tell you why — without you having to open five different dashboards and trace through logs manually?
Enter Superlog: an open-source, self-hosted agentic telemetry system. It ingests OpenTelemetry traces, logs, and metrics, groups noisy signals into actionable incidents, and runs AI agents to investigate root causes automatically. Think of it as a self-healing observability platform that watches your infrastructure while you sleep.
Why Is Superlog Trending?
Superlog has gathered over 900 GitHub stars in less than a month, and it's backed by Y Combinator (P26 batch). The convergence of OpenTelemetry standardization and LLM-powered agents makes this the right idea at the right time:
- OpenTelemetry-native — no proprietary agents, no vendor lock-in. Your services emit OTLP, Superlog ingests it.
- AI-driven incident grouping — the worker process groups related alerts into incidents and runs an LLM agent to investigate, producing a plain-English summary of what happened.
- Self-hosted with Docker — one
docker compose upgets you a complete stack: ClickHouse for telemetry, PostgreSQL for metadata, and an OTel collector for ingestion. - Pluggable agent runners — the default "community" agent runner records a local incident summary, and you can swap in custom runners for deeper integrations.
- GitHub and Linear integrations — auto-create issues and tickets from incidents directly.
Prerequisites
- A Linux server (or VM) with Docker and Docker Compose installed
- Node.js 20+ and pnpm 9+ (for local development)
- 4 GB RAM minimum (ClickHouse is hungry)
- An Anthropic or OpenAI API key (optional, for AI investigation)
Installation and Setup
Step 1: Clone the Repository
git clone https://github.com/superloglabs/superlog.git
cd superlogStep 2: Install Dependencies
Superlog uses pnpm workspaces and Turborepo:
pnpm installStep 3: Start the Backing Services
Bring up PostgreSQL, ClickHouse, and the OpenTelemetry Collector:
docker compose up -dThis starts three containers:
- PostgreSQL (
postgres:16) — port5434, stores metadata, incidents, and users - ClickHouse (
clickhouse/clickhouse-server:26.1) — ports8123and9000, stores telemetry (traces, logs, metrics) - Collector (
otel/opentelemetry-collector-contrib) — ports4317and4318, routes OTLP intake to ClickHouse
Step 4: Run Database Migrations
pnpm --filter @superlog/db db:migrateStep 5: Start the Development Stack
pnpm devThis starts four services:
- Web UI —
http://localhost:5173 - API Server —
http://localhost:4100 - OTLP Intake Proxy —
http://localhost:4101 - Worker — background incident processing
Step 6: Configure an API Key (Optional)
For AI-powered incident investigation, set an LLM provider key in the worker's environment:
# apps/worker/.env
ANTHROPIC_API_KEY=sk-ant-...Architecture
The architecture follows a clean ingestion → storage → processing → visualization pipeline.
Data Ingestion: Your applications emit OpenTelemetry telemetry (traces, logs, metrics) via OTLP gRPC/HTTP. The OpenTelemetry Collector receives this data, strips any tenant-spoofing attributes, stamps each signal with a project ID, and exports everything to ClickHouse.
Storage Layer: ClickHouse stores all telemetry data in optimized columnar tables (one per signal type). PostgreSQL holds metadata: projects, users, incidents, and configuration.
Application Layer: The API Server provides an HTTP API for the web frontend and handles authentication. The Worker runs two critical processes: incident grouping (correlating related telemetry signals) and the agent runner (invoking an LLM to investigate each incident). The Web UI is a React dashboard built with Vite.
AI & Integrations: When an incident is created, the agent runner can call an LLM (Anthropic or OpenAI) for root-cause analysis. It can also auto-create GitHub issues or Linear tickets from incidents.
Sending Telemetry to Superlog
Point your OpenTelemetry exporter to the collector:
# otel-collector-config.yaml (in your app)
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
exporters:
otlp:
endpoint: your-server:4317 # or :4318 for HTTP
tls:
insecure: true # disable for productionOr use any OTLP-compatible SDK (Python, Node.js, Go, Java, etc.):
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
provider = TracerProvider(
resource=Resource.create({"service.name": "my-app"})
)
provider.add_span_processor(
BatchSpanProcessor(OTLPSpanExporter(endpoint="http://your-server:4317"))
)
trace.set_tracer_provider(provider)Verification Checklist
- All four services (PostgreSQL, ClickHouse, Collector, API) are running:
docker compose ps - Web UI loads at
http://localhost:5173 - API responds at
http://localhost:4100/health - You can send a test trace and see it in the dashboard
- Incidents appear when the worker groups related signals
Resources
- GitHub: https://github.com/superloglabs/superlog
- Website: https://superlog.sh
- Documentation: https://superlog.sh/docs
- Discord: https://discord.gg/wJ56aRh8hx
- OpenTelemetry: https://opentelemetry.io
- ClickHouse: https://clickhouse.com