How It Works
Three steps to give your AI agents full Kafka access.
Install
go install github.com/josedab/kafkamcp/cmd/kafkamcp@latestSingle Go binary, no JVM or CGO. Signed Docker images and pre-built binaries begin with the first tagged release.
Configure
kafkamcp init --output kafkamcp.yamlPoint it at your Kafka clusters. Add Schema Registry, access policies, and audit logging as needed.
Connect
kafkamcp --config kafkamcp.yamlWire it into Claude Desktop, LangChain, CrewAI, or any MCP-compatible client. Agents get instant Kafka access.
Why KafkaMCP
Everything agents need to work with Kafka — built into a single server.
🔍 Discover
Agents list topics, browse schemas, inspect consumer groups, and explore cluster topology — with zero prior knowledge.
📨 Consume & Produce
Read messages with flexible positioning (offset, timestamp, latest) and publish with delivery confirmation and schema validation.
🔒 Secure by Default
Per-agent access control with default-deny policies, rate limiting, TLS 1.3, and a full audit trail of every interaction.
📋 Schema-Aware
Auto-discovers Avro, Protobuf, and JSON schemas from Schema Registry. Agents understand message structure without human annotation.
🏗️ Multi-Cluster
Connect to multiple Kafka clusters. Each tool accepts a cluster parameter. Agents discover clusters via MCP resources.
📊 Observable
Prometheus metrics, OpenTelemetry tracing, Grafana dashboards, and a built-in web UI for real-time monitoring.
Your Agent, Connected to Kafka
Once KafkaMCP is running, agents interact with Kafka using natural language.
kafka_list_topics — Discover your entire Kafka topologykafka_consume — Read messages with flexible positioningkafka_describe_consumer_group — Monitor consumer health and lagkafka_analyze_dlq — Categorize and diagnose DLQ errorsRead-only by default (≤15 tools in core-read pack). Additional packs — ops-write and governance — unlock real, guarded mutations and policy controls behind approval enforcement. Full tool catalog and 9 MCP resources cover discovery, consumption, schema management, consumer groups, connectors, and more.
server:
transport: stdio
clusters:
- name: production
bootstrap_servers: "broker:9092"
default: true
schema_registry:
url: "http://schema-registry:8081"
policies:
default_deny: true
agents:
- id: "my-agent"
topics:
- pattern: "orders.*"
permissions: [read, write]
rate_limit:
requests_per_minute: 120
audit:
enabled: trueArchitecture
A purpose-built bridge between AI agents and Apache Kafka.
┌──────────────────────────────────────────────────────────┐
│ AI Agents (Claude, LangChain, CrewAI, AutoGen, Custom) │
└──────────────┬─────────────────────┬─────────────────────┘
│ stdio │ SSE / HTTP
┌──────────────▼─────────────────────▼─────────────────────┐
│ KafkaMCP Server │
│ ┌─────────┐ ┌─────────┐ ┌──────┐ ┌────────┐ ┌────────┐ │
│ │ ≤15 │ │ 18 │ │ Auth │ │ Rate │ │ Audit │ │
│ │ Tools * │ │ Rsrcs │ │Engine│ │Limiter │ │ Logger │ │
│ └────┬────┘ └────┬────┘ └──────┘ └────────┘ └────────┘ │
│ └───────────┼──────────────────────────────────────│
│ ┌────────────────▼────────────────────────────────────┐ │
│ │ Kafka Client (franz-go) │ Schema Registry │ Connect│ │
│ └────────────────┬────────────────────────────────────┘ │
└───────────────────┼──────────────────────────────────────┘
┌──────────────┼──────────────┐
▼ ▼ ▼
┌─────────┐ ┌───────────┐ ┌──────────┐
│ Kafka │ │ Schema │ │ Kafka │
│ Brokers │ │ Registry │ │ Connect │
└─────────┘ └───────────┘ └──────────┘
* Default core-read pack. ops-write/governance packs available.Join the Community
KafkaMCP is open source under the Apache 2.0 license. We welcome contributions of all kinds.