pdfmd.devPDF to Markdown benchmark: pdfmd vs Marker vs Docling vs PyMuPDF
Compare published prices per 1,000 pages, then inspect a separate local conversion benchmark. Prices are each vendor's published rate, checked October 4, 2026. Quality and speed come from six public domain PDFs run through each engine on one machine and scored by a script, shown as percentages. The reference tables were checked cell by cell by an AI model, not yet by a person.
Published service prices and local conversion results are separate comparisons. The hosted services were not tested. Their quality and speed cannot be inferred from the local engine results below.
The market, per 1,000 pages
| Service | Mode | Output | Price | At volume | pdfmd costs |
|---|---|---|---|---|---|
| pdfmd.dev | Starter plan | Markdown | $0.89 | $0.75 Pro plan | n/a |
| Mathpix | Files API (batch) | Markdown | $1.50 | $1.00 over 30M pages a month | 41% less |
| Mistral | OCR 4.1, Batch API | Markdown | $2.00 | none published | 55% less |
| LlamaParse | Cost-effective | Markdown | $3.75 | none published | 76% less |
| AWS Textract | Layout | JSON | $4.00 | $3.00 over 1M pages a month | 78% less |
| Datalab (hosted Marker) | Fast or balanced | Markdown | $4.00 | $3.00 if you let them retain data | 78% less |
| IBM (hosted Docling) | Pay as you go | Markdown | $4.00 | none published | 78% less |
| Mistral | OCR 4.1 | Markdown | $4.00 | none published | 78% less |
| Mathpix | v3/pdf | Markdown | $5.00 | $3.50 over 1M pages a month | 82% less |
| Nutrient | Vision, Starter plan | Markdown | $5.80 | $2.49 Pro plan billed yearly, 100,000 pages a month | 85% less |
| Azure Document Intelligence | Layout | Markdown | $10.00 | $8.00 500K pages a month commitment | 91% less |
| Datalab (hosted Marker) | Accurate | Markdown | $10.00 | $7.50 if you let them retain data | 91% less |
| Google Document AI | Layout Parser | Layout blocks | $10.00 | $8.00 3-year savings plan | 91% less |
| Reducto | r-1 Parse | Markdown | $10.00 | $8.00 batch queue, 12-hour completion | 91% less |
| TextIn | xParse, Parse | Markdown or JSON | $10.00 | none published | 91% less |
| Upstage | Document Parse, Standard | Markdown or HTML | $10.00 | none published | 91% less |
| LlamaParse | Agentic | Markdown | $12.50 | none published | 93% less |
| AWS Textract | Tables and Layout | JSON | $15.00 | $10.00 over 1M pages a month | 94% less |
| Unstructured | All strategies | JSON elements | $15.00 | none published | 94% less |
Published pay-as-you-go prices in US dollars per 1,000 pages, read from each vendor's own pricing page or docs. US regions. A volume price needs the stated monthly volume, commitment or option. pdfmd is priced by monthly plan, so its rate per 1,000 pages assumes the plan's pages are used; the Starter rate is a launch price for a limited time. Left out: Text-only OCR (Azure Read, Google Enterprise Document OCR, AWS Textract DetectDocumentText, Upstage Document OCR) and LlamaParse Fast: they return text without tables or headings, so they are not comparable. LandingAI ADE: the price depends on how many characters come out, so there is no per-page price to compare. Adobe PDF Extract: no paid price is published. Nutrient Standard mode: free, so it has no per-page price. Nutrient's pay-as-you-go rate is per credit, and its pricing page does not say how many credits a Vision page uses, so the Vision row uses the Starter plan rate ($29 a month for 5,000 pages).
Where pdfmd loses
- Speed, pages per second as a share of pdfmd's: Plain PyMuPDF text 3,076%, pdfmd 100%.
- Whole-text ordered recall of the table values: Docling 99.3%, pdfmd 97.9%.
- Whole-text ordered recall of the table values: Plain PyMuPDF text 100.0%, pdfmd 97.9%.
- Headings preserved: Docling 99.0%, pdfmd 98.1%.
- Ordered table-cell recall on Salient Statistics, United States (lithium): Marker 100.0%, pdfmd 95.8%.
Quality, every engine
These results cover six public domain PDFs and three selected tables. They measure local conversions, not hosted API performance or general accuracy. The comparison engines used their default settings.
| Engine | Ordered table-cell recall | Whole-text ordered recall | Headings kept | Speed vs pdfmd |
|---|---|---|---|---|
| pdfmd.dev | 95.2% (138/145) | 97.9% | 98.1% (101/103) | 100% |
| Marker | 57.2% (83/145) | 84.8% | 94.2% (97/103) | 6% |
| Docling | 86.2% (125/145) | 99.3% | 99.0% (102/103) | 3% |
| Plain PyMuPDF text | 0.0% (0/145) | 100.0% | 0.0% (0/103) | 3,076% |
Plain PyMuPDF writes text, not Markdown, so it has no tables or headings by construction: it is here as the floor, and its whole-text recall shows the text itself survives. Neither recall measure checks that a value sits in the right column; see Method.
Tables
| Table | Cells | pdfmd.dev | Marker | Docling | Plain PyMuPDF text |
|---|---|---|---|---|---|
| Salient Statistics, United States (lithium) page 1. Five year columns, withheld values written as W, a two line row label, and a > prefix on percentages. | 48 | 95.8% (46) | 100.0% (48) | 91.7% (44) | 0.0% (0) |
| Table 5: Page-level latencies for document indexing page 19. A spanning header over three columns and a bold, rule-separated header block in a LaTeX paper. | 25 | 92.0% (23) | 68.0% (17) | 80.0% (20) | 0.0% (0) |
| Table A-15. Alternative measures of labor underutilization page 27. Two level column header, nine data columns, and row labels wrapped over up to six lines with dot leaders. | 72 | 95.8% (69) | 25.0% (18) | 84.7% (61) | 0.0% (0) |
Headings
Scored on the three documents whose PDF outline lists real section headings: SEC Form 10-K, general instructions and form; ColPali: Efficient Document Retrieval with Vision Language Models (arXiv 2407.01449v6); NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0).
| Engine | sec-form-10k | colpali | nist-ai-rmf | Words run together | Missed, first few |
|---|---|---|---|---|---|
| pdfmd.dev | 39/41 | 32/32 | 30/30 | 0 | Item 1B. Unresolved Staff Comments.; Item 7. Management’s Discussion and Analysis of Financial Condition and Results of Operations. |
| Marker | 36/41 | 32/32 | 29/30 | 0 | C. Preparation of Report; D. Signature and Filing of Report; E. Disclosure With Respect to Foreign Subsidiaries |
| Docling | 40/41 | 32/32 | 30/30 | 14 | Part I |
| Plain PyMuPDF text | 0/41 | 0/32 | 0/30 | 0 | Form 10-K, ANNUAL Report Pursuant to Section 13 or 15(d) of the Securities Exchange Act of 1934; General Instructions; A. Rule as to Use of Form 10-K |
Method
- Ordered table-cell recall. Every table in an engine's output is a candidate, and so is every run of up to three adjacent tables, because some engines split one table at a header or page break. Ground truth rows are aligned to output rows in order, and cells within a row are matched in order. A value counts if it is inside a table, in the right row and in the right order. It is recall, not accuracy: empty cells are ignored on both sides, extra cells in the output cost nothing, and a value is not checked for the column it sits in. A row reading "WRONG | 2025 | 100" under "Year | Revenue" still scores 100 percent, because 2025 and 100 appear in order. Footnote markers, dot leaders, emphasis and dash styles are normalised away.
- Whole-text ordered recall. The same values looked for anywhere in the output text, in reading order, table or not. It shows whether the text survived at all, and is the only table measure plain text can score on.
- Headings kept. The PDF's own outline (its bookmarks) is the list of headings its authors declared. One is kept when a Markdown heading line matches it, keeping the order of both (the longest in-order match), ignoring section numbers, case and spacing. Spacing is ignored because the measure is whether the heading survived as a heading; an engine that glues a heading's words together ("TEXTUALRETRIEVALMETHODS") is counted as keeping it, and the number of such headings is shown in its own column as a text defect.
- Speed. Pages per second over the six documents, from the median of three timed runs per document, shown as a share of pdfmd's on the same machine. Models are loaded, and Marker's and Docling's first conversion done, before timing starts. Network and hosted service latency are not included.
- Price. Each vendor's published pay-as-you-go price per 1,000 pages, read from its own pricing page on October 4, 2026. The market median uses each vendor's cheapest mode that keeps tables or layout. pdfmd's price is read from the plans Stripe charges.
- Choice of tables. The three tables were picked for structure that is hard to convert (a wrapped row label, a spanning header, a two level header with long wrapped labels) after looking at how pdfmd converted them, and before running the other engines.
Ground truth
Reference table values received an automated check against the source documents. This is not independent human validation. Human sign-off: pending.
Documents
- Federal Reserve issues FOMC statement, 17 September 2025, Board of Governors of the Federal Reserve System. Board website policy: "Unless otherwise indicated, information on Board's website is in the public domain and may be copied and distributed without permission." Source. sha256 74f7ac6262051726
- USGS Mineral Commodity Summaries 2025: Lithium, U.S. Geological Survey. USGS policy: USGS-authored or produced data and information are considered to be in the U.S. public domain. Source. sha256 7ec5654133aa5ef6
- The Employment Situation, August 2025 (BLS news release, 5 September 2025), U.S. Bureau of Labor Statistics. BLS: "everything that we publish, both in hard copy and electronically, is in the public domain, except for previously copyrighted photographs and illustrations." Source. sha256 d91f1329ce9e7a13
- SEC Form 10-K, general instructions and form, U.S. Securities and Exchange Commission. A work of the U.S. Government (17 U.S.C. 105). SEC site policy: information on sec.gov "is considered public information and may be copied or further distributed by users of the web site without the SEC's permission." Source. sha256 65e8ce4f1364c735
- ColPali: Efficient Document Retrieval with Vision Language Models (arXiv 2407.01449v6), arXiv (authors: Faysse et al.). Released by its authors under CC0 1.0 Universal (public domain dedication), as stated on the arXiv abstract page. Source. sha256 852ec724f3d48316
- NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0), National Institute of Standards and Technology. A work of the U.S. Government (17 U.S.C. 105). NIST: "information presented on NIST sites are considered public information and may be distributed or copied", except material marked as copyrighted. Source. sha256 7576edb531d98488
Test conditions
The tests used one Apple M5 Pro machine with 18 CPU cores and 48 GB of memory. Other tasks were active during the timed runs. The engines ran sequentially, so background load could vary between runs. Speed ratios are specific to this test and are not a service latency guarantee. This benchmark covers PDFs with a text layer; it does not establish scanned-document OCR quality.
See also the tables guide and the API docs.