How do you give a chatbot a memory bigger than its brain?

Imagine you run Nimbus, a coffee subscription company. You want a support bot that knows everything — shipping, refunds, every coffee, every brewing guide. But a chatbot can only read so much text at once. Your help docs don't fit. So what do you do? This page is the answer you can click.

The problem, in one picture

A language model has a fixed reading limit per message (its "context window"). Your knowledge base does not.

Try it yourself — this is real retrieval

Ask the Nimbus bot a question. On the right, watch exactly what happens under the hood: which words it pulls out, which docs it grabs, and the actual message it builds. Nothing here is faked — the panel runs the same keyword-ranking math (BM25) that ContextForge uses internally.

The magic trick: grow the knowledge base to 10,000 docs

Here's the part that matters. Drag the slider to add more help docs to Nimbus. Watch what happens to the size of each message the bot sends.

And the honest part: where it breaks

This isn't magic, it's keyword matching — so it has a real blind spot. It finds docs that share words with your question. Ask the same thing in different words, and it can whiff. Click this and watch it miss:

The whole thing, in real code

Everything you just clicked is about 15 lines. This is the actual ContextForge API:

# Once: load your whole knowledge base
from contextforge import ContextForge

cf = ContextForge(
    db_path="nimbus.db",
    min_score=2.0,   # ignore weak matches
)
await cf.ingest("./help_docs")  # 10,000 files

# Per question: it grabs only what's needed
reply = await cf.chat("How do I get a refund?")
# → only the Returns doc enters the prompt.
# The other 9,999 stay on disk. ~150 tokens.

# Paste the entire knowledge base into
# every single message, forever:

docs = load_all("./help_docs")   # 10,000 files

prompt = (
    SYSTEM +
    "\n".join(docs) +        # 👈 everything
    user_question
)
reply = await llm.chat(prompt)
# → ~600,000 tokens. Every turn.
# Slow, expensive, and won't even fit. 💸

So… how is this useful to you?

Swap "Nimbus coffee" for anything with more docs than fits in a prompt. Same trick, instant payoff:

• Customer support bot — chat over your entire help center without re-pasting it every message.
• Internal "ask the handbook" — HR policies, runbooks, wikis. Answers cite the one doc they came from.
• Product / API assistant — hundreds of pages of docs, the user gets only the relevant slice.
• Personal knowledge agent — years of notes on disk, each question pulls the few that matter.