Skip to main content

The Schema Blindness Problem: Why Your Agent Hallucinates Field Names

· 4 min read
Jose D.

I watched an agent produce a message to a payments topic with a field called total_price. The schema expected total_amount. The message was accepted — no validation, no error, no warning — serialized as JSON, and dropped into the topic. A downstream consumer tried to deserialize it, failed on the missing total_amount field, and routed it to the dead-letter queue. Thirty thousand messages later, someone noticed.

The agent didn't make a mistake in reasoning. It made a mistake in vocabulary. It had no way to know that the field was called total_amount and not total_price, because nothing told it the schema. The agent was schema-blind.

This is the default state of every AI agent interacting with Kafka today.

Connect Claude to Your Kafka Cluster in 5 Minutes

· 4 min read
Jose D.

I'm going to show you how to go from "Claude has no idea my Kafka cluster exists" to "Claude lists topics, consumes messages, and inspects consumer groups" in under five minutes. No Python scripts. No REST wrappers. No Docker Compose stack. One binary, one YAML file.

Here's what we're building:

Claude Desktop ←→ MCP (stdio) ←→ KafkaMCP ←→ Your Kafka Cluster

Your AI Agents Are Blind to Kafka — And That's Costing You Weeks

· 5 min read
Jose D.

Last month I watched a senior data engineer at a Series C fintech spend eleven days building a pipeline so their incident response agent could read from a single Kafka topic.

Eleven days. One topic.

She wrote a Python consumer that subscribed to orders.dlq, deserialized Avro messages using a hand-managed schema cache, dumped the results into a REST endpoint, wrote an MCP tool definition that called the REST endpoint, added error handling for when the consumer fell behind, and then spent two more days debugging a memory leak in the consumer process. When she was done, the agent could sample dead-letter queue messages. Read-only. No schema awareness. No consumer group inspection. And when the team wanted the agent to also read from payments.failed, the whole process started over.

This is the default experience for every team that wants AI agents to interact with Kafka. It's a hidden tax that compounds with every new topic and every new agent, and it's entirely unnecessary.