Developer Knowledge MCP Server

The Question

Could an AI coding agent trust Google's documentation as much as it already trusts its own training data, and would grounding it in the real page actually change anything?

Context

This came out of a broader push to make Google's developer documentation AI-first, across 40+ product teams. Part of that work had already shipped: adopting the llms.txt standard so legacy docs could be read cleanly by AI tools, which saw 3k+ daily copy-as-markdown operations in its first 90 days. But adoption of the standard wasn't the same as an agent actually reasoning correctly. Developers were increasingly asking a coding agent instead of reading the doc themselves, and too often the agent answered from stale training data instead of the page that shipped last week: confidently, and wrong.

The Constraint

An agent's confident wrong answer looks identical to a confident right one. There's no built-in signal that training data has gone stale, so whatever fixed this had to prove itself against a real bar, not just look better in a demo.

My Hypothesis

My first instinct was that a bigger, more current model would eventually solve this on its own. It wouldn't. Training data is always a snapshot, and Google's documentation updates faster than any model retrains.

What I Tried

I guided the server through private preview and internal dogfooding first, using early user feedback to refine what search and retrieval actually needed to return. In parallel, I ran the cross-functional build of a RAG pipeline (vector search plus a knowledge graph) to see whether grounding answers in retrieved source material would measurably reduce the agent's tendency to improvise.

The Decision

I built it as retrieval, not a bigger model. The server searches for the real documentation page first, retrieves the full text, then grounds its answer in that source material instead of letting the model improvise from memory. And I committed to a hard gate before shipping: a blind eval score had to clear 8 out of 10, not just show improvement.

The Build

Three tools (search, retrieve, grounded answer), re-indexing docs every 24 hours so grounding stays current, with Model Armor plus RAI filters keeping the corpus safe to query at scale.

Tools it exposes

Meets developers in

The Evidence

The RAG pipeline cut LLM API hallucinations by 35% and improved chat response semantic accuracy by 40%. The eval itself: a side-by-side comparison, the same developer questions answered with the server on and off, checked against a golden question set, scored blind by a technical writer who didn't know which version they were reading. The first pass came back at 7.6 out of 10, short of the 8 we'd set as the bar. It took several more rounds to close that gap, and we landed at 8.5. That gap, not a demo, is what made the case for public preview, which is where it shipped live across Firebase, Android, and Cloud documentation.

What I Learned

The eval mattered more than the demo. A demo shows you the version that worked; a blind score against a threshold tells you whether it actually works, and 7.6 said no, clearly enough that there was no argument about whether to keep iterating.

Where I think this is headed

Docs are content. MCP is infrastructure. The strongest MCP servers, Shopify's, Google's, aren't valuable because they answer questions better: they're valuable because they give agents a structured, authenticated API into the platform. I think of MCP servers as the agent-native interface to a platform. The real unlock is orchestration: specialized MCP servers collaborating so an agent can get things done across an entire ecosystem, not just answer questions about one product.