How we replaced Docling with pdfplumber: 2× faster parsing

PipesHub is open source: github.com/pipeshub-ai/pipeshub-ai

TL;DR

• The problem: Docling extracted PDFs well, but it was expensive on CPU (a median of 0.98 seconds per page in our runs) and ran as a separate service with its own models in memory.

• The approach: we rebuilt the PDF path on pdfplumber and OpenCV. A 4.9 MB gate model decides which pages need deeper layout analysis, and only those pages go to Docling’s own heron layout model, which we run without the rest of Docling.

• Speed and memory: median parsing time fell from 0.98 to 0.48 seconds per page (both parsers run back to back on a 58-document development set). Peak worker memory at the 95th percentile fell from 2.91 GB to 1.97 GB (full 1,228-document set).

• Quality: the new parser kept more of each PDF’s text layer, found more tables against human labels, and kept the evidence for benchmark answers at least as well as Docling on all three question-answer benchmarks.

• Trade-offs: Docling is still better at finding figures, rebuilding table cell structure and placing captions. The results and limitations sections give the numbers.

• End to end: a fresh PipesHub instance with the new parser indexed 88 of 91 benchmark PDFs and answered 83% of 156 single-document questions correctly.

Why we replaced Docling

Every document connected to PipesHub passes through the same pipeline: extract its content, organise it into blocks such as paragraphs and tables, enrich tables with summaries, and index the result for search.

PDFs were the expensive exception. Docling produced good structured output, running a layout model, a table-structure model and OCR. On CPU, where most self-hosted deployments run, our measurements showed a median of 0.98 seconds per page, and minutes for a long filing. A 300-page 10-K could take longer to parse than every other indexing step combined. Docling also needed its own service with its own models in memory.

Most enterprise PDFs are born digital: reports, filings, manuals and contracts exported from Word or a reporting tool. Their text is already in the file. Running vision models over every page to rediscover it is mostly unnecessary work.

So we changed the strategy: read the PDF’s existing text first, use layout models only where they add value, and fall back to PipesHub’s OCR path when the text layer is unusable.

PipesHub already had an experimental pdfplumber and OpenCV parser behind a feature flag. It was fast, but nobody trusted its output. The job was not only to make it fast. It was to make it fast enough to matter without losing the evidence users rely on.

Figure 1. Where the PDF parser sits in PipesHub’s indexing pipeline, before and after. One environment flag picks the parser; DOCX and PPTX still go through Docling.

How we benchmarked PDF parsing

We built the evaluation harness before changing the parser. It runs both parsers on the same documents, converts their output into a common schema, and measures four things: extraction quality, answer evidence, text loss and performance.

Each measure answers a different question. Human annotations tell us whether the output is correct. Benchmark questions tell us whether useful evidence survives. The PDF’s own text layer reveals silent text loss. Back-to-back runs separate parser performance from changes in the machine’s load.

All of this is a parser-level evaluation: it scores what the parser emits. Whether PipesHub as a whole then retrieves the right evidence and answers correctly is a separate, end-to-end test, described in its own section after the results.

Figure 2. The evaluation harness: one corpus, two parsers, four kinds of score.

Datasets

• Question-answer benchmarks: FinanceBench (84 filings), LongDocURL (382 documents) and MMLongBench-Doc (101 documents).

• Human layout labels: OmniDocBench (661 labelled pages, covering text, reading order, tables and figures).

• Tables: OmniDocBench has 307 readable tables, so we added 761 tables from DocLayNet, drawn from financial reports, laws, patents, tenders, manuals and scientific articles, for 1,068 in total.

Together that is 1,228 documents and 47,360 pages, plus the 1,068-table set. A 58-document development set drawn from the same sources served for quick iterations and for the speed comparison.

What we measured

Table detection is scored with recall and precision, and table cell structure with TEDS.

TEDS (Tree Edit Distance-based Similarity) measures how closely a predicted table’s structure matches the reference structure, from 0 (nothing in common) to 1 (identical). See the original paper.

Text loss needs no reference parser. It catches the failure where a whole page is emitted as one image and its text disappears, which is invisible if you only compare one parser against another. For the same reason, agreement with Docling was a diagnostic rather than a goal: we tracked it to find differences, but scored against human labels and answers.

On a shared machine, a run started an hour later can be 30% slower for reasons unrelated to the code. So every speed comparison in this post comes from two parsers run back to back on the development set; full-set runs are used for quality and memory only.

Release targets

“About as good as Docling” is not a release criterion, so we set targets before changing code:

Retrieval quality is what users feel, so the first target is about answers rather than how the output looks. Tables get their own target because financial and operational questions live in tables. Lost pages get one because a page that silently disappears is the worst failure a RAG system can have: it produces wrong answers with confident citations.

Where we started

The first run of the existing pdfplumber parser, on the development set, was humbling:

• 12.7% of pages lost more than half their text, against 1.2% for Docling on the same documents. Financial statements were often emitted as a single image, so their numbers never reached the index.

• It found 17–30% of the tables that Docling found.

• Evidence for table questions survived only 44–46% of the time, against 92–100% for Docling.

It was fast, and it was not usable. The next section describes what we built to fix it.

How we built the new PDF parsing pipeline

The new parser reads each page’s text layer with pdfplumber and runs rule-based layout analysis on every page. A small gate model decides which pages also go to heron, the layout model that finds tables, figures and captions. Where heron and the rules both find something, heron’s boxes win; where heron finds nothing, the rules’ results stand. If a model fails to load or errors on a page, the parser logs one warning and continues with the rules, so the models are never a hard dependency.

Figure 3. How each PDF page is parsed. The 13 ms gate decides whether the page goes to heron; rules always run, and model boxes win where they overlap.

For each page:

1. pdfplumber reads the characters, lines and rectangles.

2. Pages without a readable text layer go to PipesHub’s OCR path.

3. The page is rendered at 72 DPI, the size both models expect, and the gate checks it. If the gate fires, heron returns table, picture and caption boxes.

4. The rule-based finders run on every page. A model table or picture replaces any rule region it overlaps; rule regions the model did not see stay.

5. Table cells are built from the PDF’s text, reading order is resolved, and the page’s blocks go to indexing.

The subsections below cover how each part got there, in the order we tackled them.

Extracting text without losing content

Failure: 12.7% of pages lost more than half their text.

Diagnosis: layout analysis treated any region with enough drawn vector graphics as a figure, and figures were emitted as pixels, so a financial statement with ruled lines lost its text.

Fix: a region becomes an image only if it does not mostly hold text, and any text inside an image is emitted too, so chart labels stay searchable.

Result: pages losing more than half their text fell to about 1%, level with Docling on the development set, and stayed there through every later change. On the full set the final rate is 1.2%, against 2.9% for Docling.

A second kind of text loss surfaced later, while we were chasing a caption gap: text layers that are not really text. Some PDFs embed fonts without a Unicode mapping, so every character comes out as a glyph code such as (cid:22). We found 226 such pages across 24 documents. They were worse than lost pages: garbage was indexed and occasionally matched real searches (an answer of “22.2%” can match the digits in (cid:22)). Pages where most glyphs are unmapped now go through PipesHub’s existing OCR path, like scanned pages, which in our test set covers 168 of the 186 affected pages. Glyph codes left in otherwise readable documents are removed before indexing, and the harness’s data-loss report gained an “unreadable pages” column so this failure stays visible.

Detecting tables and choosing when to use a layout model

Failure: with text fixed, tables were the gap: 0.60 recall against human labels, where Docling reached 0.84.

Diagnosis: production parsers share one pattern. A vision model finds where the tables are, and the cells come from the PDF’s text. Rules alone (ruling lines, aligned columns, whitespace channels) handle ruled financial tables and break on unruled tables, tables beside prose and stacked tables. We improved our rules anyway: we counted left and right column edges in the style of Nurminen’s Tabula work, stopped text blocks from absorbing tables, and kept grid-less tables as text instead of dropping them. Recall on the 1,068-table set moved only from 0.48 to 0.51. Rules had reached their ceiling.

Comparing detectors: we tested every table detector we could run on CPU under a permissive licence, on the same 1,068-table benchmark. Each detector only proposed table regions, and the same pdfplumber code extracted cells from the PDF’s text layer. That isolates the detection step instead of comparing whole parsing pipelines.

Figure 4. Table detector comparison on 1,068 tables: detection recall, with each detector’s time per page under its name. Highlighted are the two we chose: heron as the detector, PP-DocLayout-S as the gate. Cells always come from the PDF text, so only detection differs.

heron offered the best balance: detection recall close to Docling’s reference result, at about 0.19 seconds per page on four threads. It is Docling’s own layout model, and exported to ONNX it runs on onnxruntime without the rest of Docling. PP-DocLayout-S was much faster but found fewer tables, which made it useful as a gate rather than as the final detector. We excluded AGPL-licensed detectors, such as DocLayout-YOLO, because PipesHub is Apache-licensed.

The recall in this table is detection recall: whether a detector’s box lands on each table. The parser’s final table recall also depends on how the rules and the model boxes are combined, so the two numbers are related but not the same.

When the layout model runs: the gate is PP-DocLayout-S, run on every page in 13 ms. If it sees a table with confidence 0.2 or higher, or a picture or chart with 0.3 or higher, heron runs on the page: roughly a third of pages for tables alone, and about half once figures count too. The thresholds are deliberately low, so a doubtful page goes to heron rather than being skipped.

For each table box heron returns, pdfplumber builds the cell grid from the text. If pdfplumber’s own table finder covers only part of the box, or finds a single column in an unruled table, we split the box by whitespace instead.

Result: table recall on the 1,068-table set rose from 0.48 to 0.77. Against OmniDocBench’s human labels it rose from 0.60 to 0.857, slightly ahead of Docling’s 0.842.

Figure 5. Table recall after each change, on the 1,068-table benchmark. Most of the gain comes from adding the layout model. For reference, Docling reaches 0.816 on the same set.

Handling figures, reading order and captions

Figures. pdfplumber found 59% of the figures humans labelled, against Docling’s 91%. Almost every miss fell into four groups: boxes that were too large or too small, several figures merged into one, charts drawn as vector graphics, and embedded pictures that were never emitted. heron already runs on figure pages and returns picture boxes, so we use them. A model picture replaces the rule-based image regions it overlaps and is never merged with neighbouring images afterwards. The gate now also fires on pictures and charts. A rule-based “table” that sits inside a heron picture is replaced by the picture, because those are almost always charts whose gridlines looked like a table. Figure recall rose from 0.59 to 0.86; Docling is at 0.91.

Captions. Captions taught us a lesson about metrics. Cutting captions out as separate blocks did not move the score. The real gap was that PipesHub’s Docling path stores each figure’s caption on its image block (ImageMetadata.captions), where search and the image describer read it, and our path left that field empty. We now link each figure to the nearest caption-like text above or below it and store it there. Scored the same way for both parsers, “caption attached to its figure” rose from 0.38 to 0.52; Docling is at 0.60.

Reading order. On single-column pages our reading order already beat Docling’s (0.962 against 0.941), but two-column pages were worse. The right-hand column often absorbed the running header at the top of the page, so its top edge sat higher than the left column’s and the whole right column was read first. A final ordering pass fixed it: when two blocks sit side by side (overlapping vertically, barely overlapping horizontally), the left one comes first. Reading-order agreement with human labels rose from 0.839 to 0.875 (Docling: 0.884). On two-column pages it rose from 0.831 to 0.898, ahead of Docling’s 0.872.

Keeping speed and memory in check

Speed. Adding heron made parsing slower, as expected, and our first version made it worse: it skipped the gate on pages where the rules had already found a table, so heron ran on 64% of pages. Running the gate on every page made parsing 16% faster, for a table recall cost of 0.002.

Memory. Pages were first rasterised all at once, so memory grew with document length: a 549-page filing peaked at 10.5 GB, and long documents were killed. Rasterising in 16-page windows brought that filing to 2.9 GB. The first full-scale run still showed pdfplumber using more memory than Docling on our 100 heaviest documents: 6.05 GB at the 95th percentile against 3.44 GB. Profiling found two causes, neither a leak in our code:

1. pdfplumber keeps every page it has opened, with its parsed characters and layout, until the whole PDF is closed: about 5 MB per page. Closing each page as soon as its regions are extracted is a one-line change.

2. The C allocator kept memory that Python had freed, so a long-running worker grew from 1.2 GB to 4.1 GB over a sequence of long PDFs. The parse worker now asks glibc to return freed memory (malloc_trim) after each document.

Figure 6. Peak memory per worker: median, p95 and max on the 100 heaviest documents, and the peak on the 1,216-page PDF. pdfplumber now is green; Docling and pdfplumber before closing pages are grey. Output was identical before and after.

On the 100 heaviest documents, peak memory per worker went from 4.85 / 6.05 / 6.88 GB (median / p95 / max) to 1.86 / 2.19 / 3.51 GB, below Docling’s 2.07 / 3.44 / 3.95 GB. The 1,216-page PDF now peaks at 1.9 GB instead of 6.5 GB and parses in under four minutes.

What we tried and what didn’t work

• OpenCV ruling-line detection: recall was only 0.335, because many tables have no visible grid lines. This is why tables come from a layout model, with rules as the fallback.

• INT8 quantisation of heron: it made heron slower on our CPUs (2.8 s per page), so we kept the FP32 model.

• A stricter gate: it cut model calls but missed real tables, exactly in the documents where tables matter. This is why the gate thresholds are low.

• Fewer threads per model: no faster under load.

• Captions as separate blocks: no change in the metric. Storing captions on the figure’s metadata was the fix that worked.

• Unicode normalisation for full-width text: no measurable gain.

• Chasing every DocLayNet “table” in legal documents: most were tables of contents, indexes and legend sheets. Their text is kept as text, and Docling misses them too.

Results: pdfplumber vs Docling

The final parser met every quality target. It missed our original median-speed target of 0.365 seconds per page: its measured median is 0.48 seconds per page, still about twice as fast as Docling in the back-to-back comparison. We set that target before adding a layout model, and chose table and figure quality over the last tenth of a second.

The table below brings the parser-level results together. Each row says where it was measured, because the speed, memory and quality numbers come from different runs.

“Human labels” means OmniDocBench’s 661 annotated pages. The full set is all 1,228 documents (47,360 pages); the dev set is its 58-document development subset.

So the new parser is faster, uses less memory, keeps more of the PDF’s text and keeps answer evidence at least as well. It finds more tables than Docling against OmniDocBench’s labels, but fewer on the wider 1,068-table set. Docling is closer to the human labels on text F1 and reading order, by small margins, and further ahead on figures, table structure and captions.

Figure 7. Where each parser is ahead: every quality measure in the table above, as a score out of 100%. pdfplumber is green, Docling grey.

Figure 8. Speed (back-to-back development-set run) and peak memory at p95 (full 1,228-document set); lower is better.

End-to-end evaluation in PipesHub

Parser-level benchmarks tell us whether text, tables and figures survive extraction. They do not tell us whether the complete application retrieves the right evidence and answers questions correctly.

To test that, we started a fresh PipesHub instance with the pdfplumber path enabled, uploaded 91 benchmark PDFs (4,959 pages) through the API, and asked 178 benchmark questions. A separate model graded every answer against the benchmark’s reference answer.

• 88 of the 91 PDFs indexed. Two were over the storage service’s 30 MB upload limit, which applies with Docling too. One was routed to OCR, and the test instance had no multimodal model configured to run it.

• The 178 questions split into two sets: 156 limited to their own source document, and 22 FinanceBench questions that search across all 88 indexed documents.

Answers took a median of 5.2 seconds. Most wrong answers were counting or arithmetic questions (“how many website URLs does the document have?”) or values that exist only inside chart images, which our test model could not read. Text the parser lost was not a cause.

What this shows: with the new parser, PipesHub indexes real benchmark PDFs and answers most of their questions from the right document. What it does not show: a comparison with Docling, since we did not run the same test with Docling, or performance on a larger collection where retrieval is harder.

The run also found a real bug that no offline benchmark could have shown. PipesHub stores a document’s parsed blocks with every figure inline, and its storage service refuses files over 30 MB. Our parser saved figure crops as PNG, which is large for photos: one 94-page PDF produced 55 MB of blocks. Figures are now saved as PNG or JPEG, whichever is smaller, so charts stay lossless and photos shrink. That document now produces 12 MB.

Limitations and trade-offs

The new pipeline does well where most enterprise PDFs live: born-digital documents with a usable text layer, long filings, and documents whose questions depend on text and tables. Docling remains stronger in a few places:

• Figures, by about 4 points of recall.

• Table cell structure: Docling runs a dedicated table-structure model; our cells come from the text layer and whitespace.

• Captions, mostly on Chinese newspaper layouts and captions printed inside images.

• Tables in manuals, tenders and legal documents on DocLayNet, by 11–13 points.

Two cases still need special handling. Scanned pages and pages whose text layer is mostly unmapped glyphs go to PipesHub’s OCR path, which needs a multimodal model configured. A few garbled pages inside otherwise readable documents are not OCR’d page by page, as Docling does; their glyph codes are stripped, and the words on them are lost.

Lessons for building PDF pipelines

1. Direct text extraction is enough for most born-digital pages. The text layer already holds the words and their positions. What it lacks is structure, and that is where models earn their cost.

2. Use layout analysis where rules hit a ceiling, and gate it. Our rules plateaued at 0.51 table recall. A 13 ms gate in front of one good layout model gave us model-level table recall without running the model on every page.

3. Reuse the best model even when replacing its framework. Docling’s heron was the best table detector we tested, and it runs without Docling.

4. Build the evaluation before the parser. We changed the parser about twenty times. Without a harness that scores against human labels and real questions, half of those changes would have been guesses.

5. Weigh speed against the evidence that survives, not against output similarity. “Does the evidence survive?” and “is any page silently lost?” predicted user-visible quality better than similarity to another parser’s output. Measure speed with both parsers run back to back: on shared hardware, a parser can look 30% slower because a neighbour started a job.

6. Distrust metrics that improve too easily. A higher evidence score once came from answers matching digits inside garbled glyph codes.

7. Run the real system end to end. Our image-size bug only appeared in the storage layer.

The biggest lesson was not that pdfplumber is always better than Docling. It is that a document parser should be judged by the evidence it preserves, the questions a system can answer and the resources it consumes, not by extraction speed or output similarity alone. For PipesHub, direct text extraction, targeted layout models and a repeatable evaluation harness gave us a faster PDF pipeline with measured trade-offs, and a benchmark to keep closing the gaps with.

How to enable the pdfplumber parser

The pdfplumber path is switched on with one environment variable. Docling remains the default, so you enable the new path explicitly and can compare the two on your own documents.

# backend/env.template
ENABLE_PDFPLUMBER_PROCESSOR=true   # parse PDFs with pdfplumber instead of Docling

# Optional tuning; the defaults are shown
# PDF_TABLE_MODEL=heron                 # "off": rules only, fastest, lower table recall
# PDF_TABLE_GATE_MODEL=                 # bundled gate by default; "off" sends every page to heron
# PDF_TABLE_GATE_TABLE_SCORE=0.2        # gate score for a table that sends a page to heron
# PDF_TABLE_GATE_PICTURE_SCORE=0.3      # gate score for an image or chart
# PDF_TABLE_MODEL_THREADS=4             # ONNX threads per worker

# backend/env.template
ENABLE_PDFPLUMBER_PROCESSOR=true   # parse PDFs with pdfplumber instead of Docling

# Optional tuning; the defaults are shown
# PDF_TABLE_MODEL=heron                 # "off": rules only, fastest, lower table recall
# PDF_TABLE_GATE_MODEL=                 # bundled gate by default; "off" sends every page to heron
# PDF_TABLE_GATE_TABLE_SCORE=0.2        # gate score for a table that sends a page to heron
# PDF_TABLE_GATE_PICTURE_SCORE=0.3      # gate score for an image or chart
# PDF_TABLE_MODEL_THREADS=4             # ONNX threads per worker

# backend/env.template
ENABLE_PDFPLUMBER_PROCESSOR=true   # parse PDFs with pdfplumber instead of Docling

# Optional tuning; the defaults are shown
# PDF_TABLE_MODEL=heron                 # "off": rules only, fastest, lower table recall
# PDF_TABLE_GATE_MODEL=                 # bundled gate by default; "off" sends every page to heron
# PDF_TABLE_GATE_TABLE_SCORE=0.2        # gate score for a table that sends a page to heron
# PDF_TABLE_GATE_PICTURE_SCORE=0.3      # gate score for an image or chart
# PDF_TABLE_MODEL_THREADS=4             # ONNX threads per worker

• The gate model ships with PipesHub, so it runs by default. Lower gate scores send more pages to heron: slower, with slightly better recall.

• The Docker image already includes heron (164 MB, downloaded at build time), so nothing is fetched at runtime, including in air-gapped deployments built from the image.

• Running from source, heron downloads once from Hugging Face on the first PDF parse and is cached in HF_HOME. If it cannot be downloaded, the parser logs one warning and uses the rule-based tables.

• With the flag on, PDFs no longer need the Docling service. DOCX and PPTX still go through the Docling library inside PipesHub.

PipesHub is open source. Explore the PipesHub repository, try the parser on your own documents, and tell us where it falls short.

Benchmark methodology and limitations

• Corpus sizes: FinanceBench 11,878 pages, LongDocURL 30,250 pages, MMLongBench-Doc 4,574 pages, OmniDocBench 661 single pages. The table benchmark counts only tables with readable text, from DocLayNet v1.2’s test split.

• Docling: run in-process through PipesHub’s own Docling integration, on the same documents.

• pdfplumber: heron ONNX (4 threads) with the PP-DocLayout-S gate, 4 worker processes. LLM table summaries were skipped for both parsers in the offline runs and enabled in the end-to-end test.

• Memory: the worker process plus its child processes while parsing one document, sampled every 50 ms.

• Caveats: OmniDocBench is 57% Chinese, which lowers both parsers’ text scores. heron was trained on DocLayNet-style pages, so DocLayNet table numbers likely favour both parsers.

• Models and licences: heron (Docling project, Apache-2.0), PP-DocLayout-S (PaddlePaddle, Apache-2.0, converted to ONNX), pdfplumber (MIT), onnxruntime (MIT).

Frequently asked questions

Is pdfplumber always better than Docling?

No. It is faster, uses less memory, keeps more of the PDF’s text and finds more tables against OmniDocBench’s labels. Docling is better at finding figures, rebuilding table structure and placing captions, and finds more tables on the wider 1,068-table set.

Do I still need the Docling service?

Not for PDFs once ENABLE_PDFPLUMBER_PROCESSOR=true is set. DOCX and PPTX still use the Docling library, which runs inside PipesHub rather than as a separate service.

Does the new parser handle scanned PDFs?

Scanned pages, and pages whose text layer is mostly unmapped glyphs, go to PipesHub’s existing OCR path, which needs a multimodal model configured.

Will it work offline?

Yes with the Docker image, which includes both models. From source, heron downloads once on the first PDF parse; without network access the parser falls back to rule-based tables.