RAG Is Not the Whole Toolkit: The NLP Techniques Real Problems Still Need

A support request lands in the queue, and something has to work out what it is about before anyone can answer it. The hand goes to the prompt. That usually works, which is exactly why it became the reflex. It is also the slowest way to do it, the most expensive, and the one that can least explain itself afterwards.

Six cheaper methods sit under that reflex. An exact match, when the request already carries a clean identifier. A spelling fix, when one wrong letter is all that stands between the question and the answer. A keyword search over a vocabulary an expert wrote. Embeddings, for the wording that vocabulary does not cover. Most requests are settled by one of these, in milliseconds, and each one can name the rule it fired on.

Knowing the whole ladder, and reaching for the lowest rung that solves the case, is the engineering. These articles walk that ladder on real document problems: classifying a request, matching free text to a reference list, reading a table, absorbing OCR noise, and running the model on your own machine when the corpus cannot leave it.

🧭 New to the series? Every article in this series sits on our two Towards Data Science author pages, Angela Shi and Kezhan Shi. That is the shortest way to see what is covered and where this one sits.

1. Why a bonus series at all

A working RAG pipeline produces real engineering questions the main spine cannot host without losing its line of argument. Four patterns recurred during the work on Volume 1, each of them surfacing a piece of work that did not fit into a single article of the spine. The bonus tier is what made each one land properly instead of decaying into a Twitter thread or a comment on a vendor blog.

Some concerns cut across several bricks. Spelling and OCR noise touch parsing (the corpus is noisy), question parsing (the user’s wording is noisy), and retrieval (the matching has to absorb both). Justifying “I don’t know” asks each of the four bricks for one piece of evidence. Tables in PDFs need a representation choice in parsing, a retrieval rule, and a generation contract. The discipline of writing faithful mocks touches every brick that has a contract worth testing. A single main-spine article on any of these would have either pulled too much from one brick or broken the spine’s rhythm. The bonus article carries the cross-cutting view in one place, so the reader can see all the bricks line up under one concern.

Some pipelines have a different shape from the start. The series defends one shape: read a corpus the team did not author, with adversarial structure, with an expert in the loop. A few real cases invert that. The FAQ corpus is one: the team designs the corpus, so parsing is trivial, retrieval doubles as a cache, and few-shot prompting becomes a retrieval problem. The dispatched architecture is another, an editorial choice (named routes over autonomous agents) that the spine references but does not stop to defend on its own. Both deserve a piece that walks the alternative shape fully, and that piece is the bonus. The reader gets the contrast explicit, not implied.

Some work is a reproducible benchmark. Picking an LLM for a RAG pipeline is not a position to argue, it is an experiment to run. Picking a CV parser is the same. Bonus articles run the same pipeline on a fixed set of inputs while varying one dimension (the model, the parser, the embedder) and report the numbers per question type. The methodology bonus articles add is exactly the discipline the main spine practises: vary one thing, measure on real questions, report per failure mode. The numbers age with the model lineup, the discipline does not.

A small local-LLM stack. Three articles run the cascade’s expensive stages against self-hosted Ollama models when the cloud is rate-limited, behind a VNet, or off-limits for the corpus. The numbers come from real runs against a single GPU box on the author’s desk, not against a vendor’s white paper. Self-hosting is one of the rare topics where the cost of not publishing concrete numbers is a reader making the wrong call; the local-LLM stack closes that gap.

The criterion is consistent across all of them: the piece would derail the spine if inlined, and matters too much to leave to a footnote.

2. What every bonus article shares

A bonus article in this series is not a freer-form blog post next to a stricter spine. It follows the same discipline as the spine, just on a different shape of question. Five invariants run through every bonus.

Real fixtures and real numbers. Every benchmark in the bonus series runs on documents the reader can find: NIST publications, arXiv papers, openly licensed corpora, or fictional cases the series authored itself (synthetic CVs in B08, the FAQ in B02). No screenshot of a result without the file that produced it. No anecdotal “in our experience” without the run that backs it.

Brick-aware framing. Every bonus names which of the four bricks it touches and explains its claim against that brick’s contract. The article does not float in “RAG in general” space; it lands inside the relational tables, the typed contracts, and the audit trail the spine established. That is why the pairings at the end of each bonus point back to specific spine articles.

Per-failure-mode reporting. Aggregate numbers lie (Manifesto M4, Position 8). The bonus benchmarks report per question type the same way the spine evaluation does. “This model wins on point lookups, that one wins on listings, both fail on cross-references” beats “model X scored 87% overall”.

No magical tool claim. A bonus article that walked in saying “this new framework solves the table problem” would belong on a vendor blog, not in this series. The bonus articles run plain code, lean on the relational tables the spine defines, and report what the engineering produced. The frameworks each bonus does mention (SymSpell in B01, Ollama in B10 to B12, Pydantic in B07) are tools the series already uses; their role is to demonstrate a discipline, not to sell a stack.

The article would survive the spine being rewritten. Each bonus answers a question independent of the exact wording of any main-spine article. If the spine’s parsing article were rewritten tomorrow, B01 (spelling) and B04 (tables) would still hold. That independence is what makes the bonus a bonus and not an appendix to a specific article.

3. The four families

3.1 Cross-cutting practical concerns

The most common bonus shape. A concern that real production pipelines hit and that touches more than one brick. Four pieces sit at the seams where the spine’s four-brick decomposition is silent because the concern crosses several seams at once. Each one walks the concern through every brick it touches and proposes a discipline (the spelling cascade, the four-bricks-of-evidence pattern, the table-level escalation, the faithful-mock contract) that the spine can adopt without rewriting any single brick.

• B01: Noisy text in RAG, typos, OCR, and the gap classical spell-check leaves. Three sources of one problem: user typos, fast-typing transcription noise, OCR character errors. Forty years of classical spell-correction (Levenshtein, BK-tree, Soundex, SymSpell) handle one of the three. Embeddings and LLMs absorb the rest. The practical split the article defends: spell-correct the question against the corpus vocabulary at parse time; leave volume noisy; design retrieval around the noise. Includes the corpus-vocabulary cascade that turns a generic spell-check into a domain-aware one without training anything. Pairs with Articles 5 (parsing), 6 (question parsing), 7 (retrieval).

• B03: When the RAG says “I don’t know”, justifying the absence of an answer. A confident wrong answer is a bug. A bare “no answer” with no justification is almost as bad. Each of the four bricks owes the user one piece of evidence: what was parsed, which vocabulary was searched, which pages were swept, why nothing matched. The “I don’t know” becomes auditable instead of opaque, and the user (or the auditor six months later) can tell whether the absence is real or whether the system simply did not look. Pairs with Articles 5 to 8.

• B04: Tables in PDFs for RAG, don’t flatten the grid. Tables are where most RAG pipelines silently fail. A linear decision tree across table types does not work because the dimensions cross (a table can be column-typed and small, column-typed and large, mixed-row, header-only, deeply nested). The right pattern is four levels of representation (row-as-line in line_df, separate table_df, columnar with named and typed columns, columnar but heterogeneous), a per-table diagnostic on five orthogonal axes, and a handful of idempotent operations that move tables between levels. Most tables stay at the simplest level; only the few that need it pay the cost of escalation. Pairs with Articles 5, 7, 8.

• B07: Faithful mocks, tests that don’t lie about types. A mock that simplifies a return type for convenience is a mock that hides a production bug. A real incident on this very project, a one-line fix, a principle worth locking in: every mock must have the exact shape of the object it stands in for, no matter how tempting the convenience shortcut. The discipline cuts across every brick of the pipeline that provides a typed contract (parsing returns DataFrames, question parsing returns Pydantic, retrieval returns frames with provenance, generation returns typed JSON), so it sits in cross-cutting concerns alongside the spelling / no-answer / tables triad. Pairs with Article 18 (code architecture).

3.2 Alternative pipeline shapes

The corpus is not the inherited adversarial PDF set the series defends against by default. The whole pipeline inverts.

These two pieces are counterpoints to the main shape, not contradictions of it. The series does not retract its position when the corpus is FAQ-shaped; it acknowledges that the trade-offs flip, and walks the alternative end to end. Reading them sharpens the reader’s sense of when the spine’s defaults apply and when they do not.

• B02: FAQ as RAG, when you get to design the corpus. The controlled-corpus counterpoint to the rest of the series. Standard RAG assumes you inherit a chaotic corpus; FAQ flips it. Parsing becomes trivial (the entries are structured input, not extracted output), retrieval doubles as a cache (the same questions hit the same FAQ rows every day), and few-shot prompting itself becomes a retrieval problem (which past examples to inject given the current question). Closes with the feedback loop that turns the FAQ into a living corpus driven by the question stream, so the corpus grows with what users actually ask. Pairs with Articles 1, 6, 7, 14.

• B06: How a dispatched RAG architecture makes most token-saving tricks unnecessary. Most of the agent-shaped token-saving tricks people reach for (multi-step planners, prompt-pruning agents, context-compression layers) are working around an architecture choice. Pick the architecture first (a deterministic dispatcher routing each question to one named handler), and most of the tricks become unnecessary. The piece walks three real cases where the architecture’s clarity made the framework patch redundant: chunk-size tuning, multi-step query rewriting, agent-driven retry. The position the article defends is the editorial backbone of Manifesto M4 Position 3, applied at the token-economics level. Pairs with Article 13.

3.3 Reproducible benchmarks

The same pipeline, one varying dimension, the numbers per question type.

These are the bonus articles that age. The exact models tested in B05, the exact CV parsers in B08, will be superseded. The methodology those benchmarks defend (vary one thing, measure on real questions, report per question type) is what survives. Reading them today gives the reader the current numbers; reading them in two years gives the reader the template for running the same comparison on the model lineup of the day.

• B05: Picking a model for enterprise RAG, the same pipeline, thirteen models, side by side. The model choice is one variable in the pipeline, not the whole system. The companion benchmark runs the V1 four-brick pipeline on the same documents and the same questions, changing only the LLM: OpenAI tiers, Anthropic, self-hosted Mistral, Llama, Phi, Qwen. Cost, latency, faithfulness, and structured-output reliability measured per question type. The takeaway is rarely “the biggest model wins”; the same pipeline plus a strong dispatcher narrows the gap between a $20/M-token model and a free self-hosted one. The article ends with the model-selection table the broker corpus actually uses in production. Pairs with Article 13.

• B08: We ran four PDF parsers on one CV. One flipped ‘do not redistribute’ into ‘document redistribution’. A head-to-head between two layout-aware OCRs and two vision LLMs on synthetic CVs the series wrote itself (open-license, reproducible, no client data). Same eight-DataFrame parser contract on all four engines. Where each one wins, where each one loses, and what to read in a CV that any of them might break on. The synthetic-CV authoring choice (rather than using a public CV dataset) is also defended in the article: it gives the reader a fixture they can extend on the spot. Pairs with Articles 5, 5bis to 5septies.

3.4 The local-LLM stack

Three pieces that read together as a small sub-series. When the cloud is off-limits, here is the cascade.

These three are the only bonus articles that form a tight sub-series. Each one tests one stage of the cascade (last-stage LLM, embeddings, smallest-size sweep) against self-hosted Ollama models. Read in order, they answer one bigger question end-to-end: “can the whole cascade run on a single GPU on my desk when the cloud is not an option?” The current answer is yes, with one named model per stage and a documented gotcha.

• B10: The one RAG step that still needs an LLM, run locally on Ollama. When the cloud API is rate-limited, behind a VNet, or off-limits for the data, a local LLM keeps the pipeline alive. Walks qwen2.5:7b on the cascade’s last stage (the LLM-confirm step the spine introduces in Article 7), shows clean structured outputs, then exposes a reasoning-model gotcha with qwen3:4b that silently drops the schema. Closes on the operational lesson: not every smaller-but-fancier model is a drop-in replacement. Pairs with B01 and Article 8.

• B11: Local embeddings for RAG, which Ollama model rescues OCR noise when the cloud is off. Same noisy chunks as B01, run against two local Ollama embedders instead of text-embedding-ada-002. The local lineup is good enough for the cascade, and on the retrieval-relevant test (separation between answer-bearing and decoy chunks on OCR-noisy text) one local model has better separation than the cloud reference. The piece closes with the local-vs-cloud trade-off explicit: cloud wins on raw clean-text quality, local wins on residency and on a specific retrieval-noise band. Pairs with B01 and Article 7.

• B12: How small can a local LLM be for RAG extraction, eleven Ollama models sorted by size. Eleven Ollama models pulled locally, from 815 MB to 9.1 GB. JSON structural validity passes from 1 B onwards. Literal extraction without fabrication kicks in at 7 B. The smallest production-ready pick is qwen2.5:7b. The sweep makes the “how small can I go” question answerable from first principles for the reader who wants to repeat it on next month’s lineup. Pairs with B10 and Article 8.

4. Three reading paths

“Read in any order” is true but unhelpful. Three concrete paths fit recurring needs.

The parsing-noise path. Read B01 (spelling) and B04 (tables) back to back. The two of them define the operational discipline for the half of a RAG pipeline most reference material skips: what to do when the input is noisy on the character level (B01) and what to do when the input is structured on the grid level (B04). Both touch parsing, retrieval, and generation. Together they cover most of the “the parser returned something useless” failure modes a production team will hit.

The architecture-counterpoint path. Read B02 (FAQ as RAG) and B06 (dispatched architecture) together. Both articles are editorial pieces that sharpen the spine’s defaults by walking the case where the default does not apply. B02 inverts the corpus shape; B06 inverts the routing layer. After reading the two, the spine’s choices read as choices instead of as defaults, and the reader knows under which conditions they would pick the other side.

The self-hosted cascade path. Read B10, B11, B12 in order. The three answer one larger question end-to-end: “can I run the V1 cascade on a single GPU on my desk when the cloud is not an option?” B10 tests the last-stage LLM, B11 tests the embedder, B12 sweeps the smallest viable size. Together they give a production-ready local stack with one named model per stage. If self-hosting is on the table for compliance, residency, or budget reasons, this is the path.

The non-paths matter too. B03 (justifying “I don’t know”), B05 (model benchmark), B07 (faithful mocks), B08 (CV parsers) each stand on their own and do not need a companion. Drop in when the matching spine article raises the question.

5. What earns a piece a bonus slot

The bonus tier is not where leftover work goes. Three operational tests decide whether a piece deserves to be a bonus or whether it should land in the spine, in a footnote, or not at all.

Cross-brick reach without a single brick home. If the piece’s argument touches two or more bricks of the four-brick architecture and cannot be located in any one of them, it is a bonus candidate. If the argument lives entirely inside one brick, it belongs in that brick’s spine article.

Standalone readability under the spine’s vocabulary. A bonus assumes the spine’s vocabulary (the four bricks, line_df, the typed contracts) but does not require any other bonus. If the piece needs another bonus to make sense, the two should fuse or one of them was not justified to begin with.

Replaceable conclusion is fine; replaceable discipline is not. A bonus benchmark can have its concrete numbers age out of date and still earn its place if the methodology it taught is reusable. A bonus that depends on a specific 2026 model lineup to stay accurate is at risk of becoming a stale footnote; one that defends a benchmarking discipline is durable. The local-LLM trio (B10, B11, B12) is durable because the model names change but the cascade-stage framing does not.

The reader who wants to spot the next bonus the series will publish can apply these three tests to any topic that is “in the air” around enterprise RAG. A piece on prompt versioning that touches generation alone is a spine concern. A piece on prompt versioning that asks how document parsing, question parsing, and retrieval all carry their own prompt-history dimension is a bonus shape.

6. What this bonus series does not include

The bonus articles cover what surfaced during Volume 1 of the series. Several adjacent topics are deliberately out of scope here and will land in later volumes:

• Other document formats (Word, Excel, PowerPoint, email): Volume 2.

• Other intents on documents (translation, summarization, side-by-side comparison, redaction): Volume 2.

• Document production with a tool catalog (the agent picks tools per brick at runtime): Volume 3.

• Agentic loop on top of the audited bricks: Volume 4.

• Multi-tenant SaaS-specific operational concerns (signed-URL upload, per-tenant Postgres isolation, regional residency): Volume 7.

Each Volume opens its own bonus side-aisle when it ships, with a sibling announcement. This announcement covers Volume 1’s set.

7. How to follow

The bonus articles publish out of band, between numbered articles of the main spine, on the same Towards Data Science feed. Following the author page catches all of them as they ship. The series-level outline keeps the running table of every bonus in the Bonus (B-series) section, including this announcement at the top.

If you read one bonus that paired with a main-spine article you have not read yet, the pairing pointer is the place to start backtracking. The bonus assumes the main article’s vocabulary (the four bricks, line_df, the four-brick contracts) but does not assume any other bonus. New readers can also start from the spine’s series introduction, come back here, and pick the family that matches their current question.