Building Document Structure with Loop Engineering: Recovering a PDF’s Outline from Body Typography for RAG

With the four cases mapped, here is how the cascade actually turns a document’s pages into a toc_df, one stage at a time, from the signals it reads off the page to the LLM check at the end.

Every rule the body-typography detector reads is a column of the extracted table. So the question is not “what are the six signals” first, it is “which columns does the parser give me at all”. The answer depends on how the PDF was parsed. The four bricks live one level up; here we live inside brick 1, and brick 1 has (already, in the earlier articles) several implementations.

Two practical points that shape the code:

1. PyMuPDF exposes typography per span, not per line. A span is one contiguous run of characters in one font, size, weight, italic and color. A line that mixes bold and non-bold text expands to multiple spans. So the module provides a helper enrich_line_df_with_style(line_df, span_df) that aggregates the spans of each line down to one row and appends font_size (character-weighted), bold_ratio, is_bold, is_italic, dominant_font_name. Consumers pass a line_df when the parser has no typography, or a line_df plus a span_df when it does.
2. Missing typography does not break the loop. Each of the six signal functions checks the columns it needs and returns a zero series when they are absent. Font size and bold contribute 0.0 to the score in that case; the four position and text signals (numeric prefix, short length, left alignment, blank-line-above) carry the load. Azure OCR Layout and EasyOCR both fall in this regime.

Mistral OCR is a special case: it returns the document already in markdown, and its # / ## / ### headings are the reconstructed outline. When you have a Mistral output you skip Case 4 entirely and read the markdown structure. That is the parser matrix’s shortcut column.

Now the six signals themselves. Each is one function of one input frame, each returns a pd.Series[float] in [0, 1], and the combined score is a plain weighted sum.

1. Font size ratio. A heading uses a larger font than body prose. Score = (font_size / median_body_size) - 1, clipped to [0, 1]. A line at 12pt in a body of 10pt scores 0.2; at 16pt in a 10pt body it saturates to 1.0.
2. Boldness. Headings are often bold. When the enriched line_df carries bold_ratio, we use that; otherwise is_bold as a 0/1 fallback.
3. Numeric prefix. 1., 1.2, 1.2.3, I., A.. Regex ^s*(?:d+(?:.d+)*.?|[IVXLCDM]+.|[A-Z].)(?:s|$). Present ⇒ 1, absent ⇒ 0. Also carries the level signal: 1.2.3 is a level-3 heading. Fitz artefact. A LaTeX-exported PDF often emits "1" and "Introduction" as two separate lines; the loop pre-merges bare-number lines with their next line via merge_split_headings before scoring, so the numeric prefix is recovered on the merged line.
4. Short length. A heading is short. Score linearly decays from 1 for lines shorter than 20 characters to 0 at 90 characters. A paragraph rarely lives on a single line; a heading rarely spills across two.
5. Left alignment. A heading starts at the left margin (or the block’s own left indent). Score = 1 when x0 matches the page’s median left margin within a small tolerance, decays with distance.
6. Blank line above. Headings sit in visual isolation. Score = 1 when the vertical gap above the line is wider than the median gap between prose lines on that page.

A weighted sum turns six scores into one. Weights come from prior belief, not from a training run. Numeric prefix is the strongest single signal (1.6). Font size ratio next (1.4). Boldness middle (1.0). Blank-line-above (0.8), short length (0.6), left alignment (0.4) are supporting evidence. Sum of weights is 5.8. The default threshold is 3.0 (about half the sum), which on the attention paper picks up every real heading plus a handful of table-cell false positives.

candidates_df = detect_body_headings(line_df, threshold=3.0)

candidates_df[["text", "heading_score", "candidate_level",
               "signal_font_size_ratio", "signal_has_numeric_prefix"]].head()

On the attention paper (Vaswani et al. 2017, an arXiv LaTeX export), the deterministic pass at threshold 3.5 finds 24 candidates: 21 real section headings (from 1 Introduction to 7 Conclusion, all subsections included) and 3 false positives (28.4, 4.33, 26.4, BLEU scores from the results table on page 8-9 that happen to be bold and short). Recall on real headings is 91 % (21 of 23 real headings, 1 Introduction sits just below the threshold and comes back at 3.0). Precision is 88 %. Level accuracy from numeric prefix is 100 % on the recovered rows. Those are the numbers from a real run, not a projection.

Here is where the LLM does real detection work. Not free-form. Fixed schema. The prompt describes what a heading is, what a false positive looks like (figure caption, table row label, bold in-body emphasis, BLEU score in a results table), gives the candidates in reading order with page and snippet, and asks for a JSON list of the kept entries. Missed headings (small caps, italics, an unnumbered epilogue) can be added by the same pass because it also sees a slice of the surrounding lines.

The loop is bounded. At most max_passes iterations (3 by default). Each pass reads the current keep list and can drop entries or propose additions. Convergence is when a pass changes nothing.

def my_llm_parse(system_prompt: str, user_content: str) -> list[dict]:
    ...  # user's LLM client, returns the kept-entries JSON list

toc_df = reconstruct_toc_from_body(
    line_df,
    span_df=span_df,
    mode="no_toc",
    max_passes=3,
    llm_parse=my_llm_parse,
)

Two things worth spelling out. First, the LLM is told what it may return. The response schema is fixed (title, page, level, source), matching the table of content cascade so downstream code does not branch on which case fired. Second, llm_parse is an injected callable. Tests hand in a mock; CI can replay from a JSON cache; the module never opens a socket by itself. That is the discipline every LLM-touching module in this series follows.

Some PDFs have a native outline that stops at level 2. 1. Introduction, 1.1 Motivation, 2. Method, 2.1 Data, and the file is done. But the body clearly runs 2.1.1, 2.1.2, 2.2.1. The retrieval brick wants those. So does the chunker, if the section runs long.

reconstruct_toc_from_body handles this with mode="extend_native". The native outline is preserved verbatim (each row carries source="native"). The body-typography pass runs on top, and any candidate whose title-and-page pair does not already exist in the native outline is added with source="body_structure". The reader can see, row by row, which entries came from the file and which came from the body pass. That is what source is for.

deeper_toc = reconstruct_toc_from_body(
    line_df,
    existing_toc_df=native_toc,   # from doc.get_toc(), stops at level 2
    span_df=span_df,
    mode="extend_native",
)

deeper_toc[deeper_toc["source"] == "body_structure"].head()

mode="auto" (the default) picks this branch when existing_toc_df is non-empty and its max level is ≤ 2, so most callers do not have to think about which mode to pass.

The other case: a single PDF file that concatenates two or more documents. An arXiv paper followed by a supplementary memo. A request-for-proposal bundle where each vendor answer is its own document. A review dossier of the letter and the response side-by-side. The native outline (if any) is flat and confusing: two "1. Introduction" entries, page numbers that do not increase monotonically, a re-init of the numbering half-way through.

detect_document_boundaries looks for three deterministic signals: numbering re-init (a "1." on a page after "N.something"), style rupture (a jump in median font size between pages N-1 and N), and cover pages (a page with a large title, low line count, no body text). Each returns a small frame of boundary rows. When any boundary is confirmed, reconstruct_toc_from_body(..., mode="composite") re-roots the outline: each internal document becomes a top-level entry ("Document 1", "Document 2", …) with the original outlines nested under.

This is the case where the merge is worth an LLM pass, because a re-init in numbering could also just be a document that legitimately restarts a counter inside a chapter. The MERGE_PROMPT asks the LLM to confirm each boundary or reject it, then to return the merged outline as a JSON list. The whole subpackage funnels through one public entry point:

toc_df = reconstruct_toc_from_body(line_df, span_df=span_df, mode="no_toc")
toc_df = reconstruct_toc_from_body(
    line_df, existing_toc_df=native_toc, span_df=span_df, mode="extend_native",
)
toc_df = reconstruct_toc_from_body(
    line_df, existing_toc_df=native_toc, span_df=span_df, mode="composite",
)
toc_df = reconstruct_toc_from_body(line_df, existing_toc_df=native_toc, span_df=span_df, mode="auto")

For the pipeline caller who already uses reconstruct_toc_df and wants the body-typography case as a cascade fallback rather than a separate call, add "body_structure" to the methods tuple: reconstruct_toc_df(pdf_path, methods=("links", "contents_text", "llm", "body_structure")). The cascade will try the table of content methods first and fall through to Case 4 when they return empty.

The honest way to score this is against ground truth. Take six PDFs that DO carry a native outline, hide it, run the body-typography loop on the body, and compare the reconstruction to the outline the file was hiding. The six fixtures are all tier-1 open sources with different flavours of structure:

The best way to make the decision visible is to overlay the algorithm’s per-line output on the raster of the page. Green boxes are lines the deterministic loop scored above the 3.0 threshold and kept as heading candidates. Grey outlines are the other lines the algorithm looked at (body prose, page headers, footnotes) but did not keep. The tag on the right shows what span_df exposed for each candidate: (level, bold flag, font size, heading score).

Two things read off the overlay. First, the number of green boxes per page is small (four on Vaswani p2, two on NIST p10), which confirms the threshold is picking headings rather than sprinkling them across the page. Second, the tags show the algorithm sees the right numbers: 11.9 pt bold for NIST section titles, 11.9 pt bold for Vaswani chapter titles, both with heading scores near 4.28 (well above the 3.0 threshold). The signals span_df exposes match what a human reader would flag as a heading, and the scorer combines them without needing per-document tuning.

Here is the exact code that ran to produce these overlays. It reads the same line_df + span_df the rest of the pipeline reads, then loops over the candidates for the page and draws the boxes.

pdf = "data/nist/NIST.SP.800-207.pdf"
line_df = fitz_pdf_to_line_df(pdf)
span_df = build_span_df(pdf)
enriched = enrich_line_df_with_style(line_df, span_df)   # per-line font + bold
candidates = detect_body_headings(enriched, threshold=3.0)

page = 10
for _, row in candidates[candidates["page_num"] == page].iterrows():
    print(row["text"], row["heading_score"], row["candidate_level"],
          row["font_size"], bool(row["is_bold"]))
# 1 Introduction 4.28 1 12.0 True

Every category in the taxonomy above is one the LLM is designed to catch on the first pass.

• Page numbers alone: a candidate whose text is a single digit is never a heading.
• Table of content dot-leader lines: "1 Introduction ..........." is a contents-page entry, not a body heading.
• Cover-page author names: no numeric prefix, wrong page.
• Table cells: bold and short but not a section title.
• The HEADING_VALIDATION_PROMPT names each category explicitly. Here is the measured effect of a single validation pass with gpt-4.1 (Azure OpenAI, cache on) on the six-fixture eval.

Three things read off the measurement.

Precision lifts massively. From 15% up to 96% on the attention paper, 38% to 98% on NIST SP 800-207, 21% to 100% on FIPS 199, 31% to 82% on NIST SP 1800-32, 7% to 72% on SP 800-171r2. The LLM removes exactly the FP categories the taxonomy above named.

Recall is preserved on five of six fixtures. The LLM validation is conservative in the good sense: it drops false positives without touching the real headings the deterministic pass already found. On Vaswani, 100% recall is preserved; on SP 800-207, 75%; on SP 800-171r2, 97%; on FIPS 199, 60%; on SP 1800-32, 62%. The micro-aggregate on those five docs stays close to before.

FEMA NFIP is the outlier where the LLM over-drops. Recall collapses from 69% to 22%. The Roman-numeral appendices (IV. PROPERTY NOT INSURED) and named sub-forms (National Flood Insurance Program Dwelling Form) do not fit the LLM’s mental model of “section heading with a numeric prefix”, and it drops them alongside the false positives. This is a known limit: the LLM prompt was tuned on decimal-numbered outlines. A per-domain prompt or a two-pass loop with a corrective critic would recover here.

The catch that even the LLM does not fix is deeper. Look at what SP 800-207 has that the deterministic pass genuinely misses: unnumbered top-level titles like "References", "Acronyms", and the front-page "NIST SP 800-207, Zero Trust Architecture". These do not carry a numeric prefix, and their font signal blends with the body. The LLM can propose them when it sees the surrounding context, but the deterministic recall does not include them by construction. That is the ceiling of the method: a document that wants a perfectly-nested reconstructed outline has to carry visible signals for every heading.

The numbers above are honest but small. Four caveats that a rigorous reader should know:

• The threshold 3.0 is a default, not a tuned choice. No cross-validation, no ROC / precision-recall sweep. A per-corpus sweep would likely lift precision meaningfully on the noisier fixtures.
• The matching step is loose. Titles normalize by stripping leading numbers, then compare exactly. A document with two sections named Introduction (a body intro and an appendix intro) can match either, and the greedy assignment picks the closest page.
• The page-within-1 metric is almost tautological. Native and recon both derive page from fitz’s page_num, so an off-by-one at that stage is rare by construction. A stress test would randomize the ground-truth page assignment or use an OCR fixture where the page number is inferred.
• Six fixtures, English only, tier-1 open sources only. No confidence intervals. No multilingual document. No RTL script. No handwritten / scanned insurance CG (which is tier-2 anyway). The pattern generalizes in ways this eval does not confirm.

Section 3.4’s LLM validation pass has now been measured on the six-fixture eval (see Section 4.2 above). Micro precision moves from 20% to 87%, recall from 72% to 51% (the drop is entirely FEMA NFIP, whose Roman appendices confuse the LLM). Per-corpus prompt tuning is a real follow-up; the current prompt was written against decimal-numbered outlines.

A recovered TOC is one dimension of a document’s structure. It is not the whole thing. Two examples make this clear.

Insurance auto contract. A section titled 3.2 Collision Guarantee explains coverage, then names the plafond, then names the exclusions that apply. The exclusions do not necessarily live in a separate chapter titled Exclusions. They live inside the guarantee section. A retrieval that scopes by section (via toc_df.start_page, end_page) and asks “what exclusions apply to the collision guarantee?” pulls back the whole section. The answer is there, but so is a lot of noise, and the query cannot filter on exclusion alone.

Composite / poorly-written docs. A transcription, a meeting minute, a legal opinion, a forensic report. None of these have a clean hierarchical TOC. Themes recur across paragraphs, actors appear and reappear, and the useful navigation is not “which section” but “which topic”. A hierarchical outline of a transcription is close to useless.

The idea, in one line: the algorithm recovers the level (5octies loop), and the business adds tags on top. Two layers stack on the same line or paragraph. Neither replaces the other.

• Layer 1: LEVEL. From the body-typography loop above. Chapter 3, section 3.2, subsection 3.2.1. This is toc_df and the section anchors on every paragraph via start_page/end_page.
• Layer 2: TAGS. A list[str] per paragraph, drawn from a domain taxonomy. In the insurance example: ["garantie", "garantie:collision", "plafond", "exclusion", "exclusion:vitesse"]. Tags come from a fixed taxonomy defined by the business (or bootstrapped from the corpus and validated by an expert), then an LLM proposes tags per paragraph.

Retrieval implications: the query “what exclusions apply to the collision guarantee?” becomes an intersection on Layer 2 (garantie:collision AND exclusion:*), which is exact even when the answer lives inside a guarantee section, not an exclusions section. The query “section 3.2 in full” still works on Layer 1. The two are complementary.

Illustration on a synthesized auto-contract paragraph (fabricated for the article, no real insurer named, the point is the pattern, not the fixture):

The example is synthetic on purpose. Real insurance conditions générales are tier-2 documents and cannot be published verbatim on TDS; the pattern this figure shows is representative, not a specific insurer’s wording.

The idea is powerful. It is also hard. Four costs to name upfront:

• Redundancy with Layer 1. Tagging a paragraph garantie when it lives inside a section titled Auto guarantees often duplicates information already in toc_df.breadcrumb. Useful tags are the ones that cross the section boundary: exclusion inside a guarantee section, condition that applies to two different guarantees.
• Taxonomy chicken-and-egg. Without a fixed taxonomy, an LLM invents new tags each pass, producing label proliferation and cross-document inconsistency. With a fixed taxonomy, the tag set is brittle across domains (an auto taxonomy will not work on a health-insurance contract without adaptation).
• Paragraph unit is fuzzy. Fitz’s paragraph boundary is not the semantic paragraph a human would draw. A single displayed paragraph can span multiple line_df blocks, be broken by page footers, or run across pages. Layer 2 needs a paragraph model, not just a line stream.
• Cost per document. One thousand paragraphs is one thousand LLM calls if the tagger runs one paragraph at a time. Batching, cache, and hierarchical clustering (tag a group of similar paragraphs together) matter for practical use.

The tagging pass belongs to a later brick in the series. Two natural homes:

• Volume 2 (Multi-format, multi-intent documents). The classification intent tags each paragraph against a fixed taxonomy. That is Layer 2, packaged as a first-class RAG intent.
• Volume 3 (Agentic Bricks). An agent reads the taxonomy, walks the paragraphs it has not tagged yet, proposes tags, checks consistency across paragraphs of the same section, and re-runs on the paragraphs whose tags were rejected. That is Layer 2 with a feedback loop.

Tables are a third axis I did not open here. The parsing brick keeps each table entire; the retrieval brick decides whether to score the table as one chunk or as individual rows serialised with their column headers. A follow-up article in Part III builds that row-level retrieval loop and pairs it with this parsing choice.

The TOC loop this article built stays useful on its own. It is the first axis of the super structure. Layers 2 and 3 come on top when the use case calls for them.

A PDF that has neither an outline nor a printed contents page is not a dead end. The body carries every signal a heading gives off: font size, boldness, numeric prefix, short length, left alignment, blank line above. Six per-line scoring functions read from an enriched line_df (the parser matrix says which parsers give what), a weighted sum surfaces candidates, and the LLM validates them in a bounded loop that stops on convergence. Rules propose, LLM validates. The same subpackage extends a partial native outline (mode extend_native) and reconciles composite files (mode composite), all through one entry point that returns the same toc_df shape as the table of content cascade, with a source column so an audit can see which case fired. Retrieval, chunking, summarisation keep working unchanged. Case 4 is the first place in the parsing brick where the LLM does real detection work rather than checking coherence, and it needs the harness discipline around it: injected llm_parse callable, fixed response schema, bounded pass count, cached raw responses.

The next article (5nonies, in the queue) closes the parsing brick with an agentic parsing loop: given a document, pick which parsing methods to run at all (fitz, Azure, Docling, EasyOCR, vision LLM), execute them in the right order, synthesise their outputs into one enriched corpus. Case 4 of this cascade is one of the tools that agent will pick from.

Earlier in the series (payable-window links, out-of-window titles listed without a link):

External sources (worked examples and prior art):