Merge feat/pdf-markdown: PDF→Markdown endpoint + geometric structure mode

- POST /ocr/pdf?markdown=1 (& ?as_pages=1) on GPU and CPU routes
- pdf_job.h split into header + pdf_job.cpp
- PDFium char-flow text-layer extraction (surrogates, control chars)
- markdown exporter plain-text fallback for unrecognized formula/table regions
- geometric mode: exact text layer + tables→HTML / formulas→LaTeX, never OCRs
  (image-only pages return empty prose; use mode=auto/ocr to OCR them)

Author: nataell95

.gitignore

@@ -60,3 +60,5 @@ site/
 
 # TRT/model scratch cache
 model.cache
+# Local scratch outputs (demo/markdown test results) — never tracked
+/tmp/

CMakeLists.txt

@@ -417,6 +417,7 @@ if(USE_CPU_ONLY)
         src/server/cpu_main.cpp
         src/routes/common_routes.cpp
         src/routes/pdf_routes.cpp
+        src/pipeline/pdf_job.cpp
         ${PROTO_GEN_CC}
         ${GRPC_GEN_CC}
     )
@@ -685,6 +686,7 @@ else()
         src/routes/common_routes.cpp
         src/routes/image_routes.cpp
         src/routes/pdf_routes.cpp
+        src/pipeline/pdf_job.cpp
         ${PROTO_GEN_CC}
         ${GRPC_GEN_CC}
     )

README.md

@@ -109,7 +109,7 @@ Then opt in per request (combine freely; `tables`/`formulas` auto-enable layout)
 ```bash
 curl -X POST "http://localhost:8000/ocr/raw?layout=1&tables=1&formulas=1" \
   --data-binary @paper.png -H "Content-Type: image/png"
-# PDF:  POST /ocr/pdf   ·   Markdown export: POST /ocr/markdown   ·   gRPC: port 50051
+# PDF:  POST /ocr/pdf   ·   PDF → Markdown: POST /ocr/pdf?markdown=1   ·   page → Markdown: POST /ocr/markdown   ·   gRPC: port 50051
 ```
 
 `GET /capabilities` reports which stages a running server has loaded.
@@ -268,7 +268,7 @@ One binary serves HTTP and gRPC from a shared GPU pipeline pool.
 | `POST /ocr` | OCR base64 image in JSON |
 | `POST /ocr/pixels` | Zero-decode raw pixel buffer |
 | `POST /ocr/batch` | Batch of images |
-| `POST /ocr/pdf` | PDF → text (optional page images & auto-rotate) |
+| `POST /ocr/pdf` | PDF → text (optional page images & auto-rotate); `?markdown=1` → whole PDF as Markdown |
 | `POST /ocr/markdown` | Page → faithful Markdown (GPU build; requires layout) |
 | `POST /infer` | OCR + layout / reading-order / blocks in one structured response |
 | `GET /capabilities` | Runtime feature & route discovery |

docs/api/http.md

@@ -291,10 +291,14 @@ embedded text is trustworthy.
     - multipart with field name `file` or `pdf`.
 - **`mode` query** (default `ocr`):
     - `ocr` — always render + OCR every page.
-    - `geometric` — use the PDF text layer for content; render only when
-      `layout=1` or as a safety net.
+    - `geometric` — prose comes **only** from the PDF text layer; the page is
+      **never OCR'd**. A page with no usable text layer (image-only / scanned)
+      returns empty prose — use `auto` or `ocr` for those. The page is still
+      rendered for layout and table/formula recognition when requested (those
+      are vision-recognized for born-digital pages too), so `?tables=1`/
+      `?formulas=1` work in geometric mode and keep the exact text-layer prose.
     - `auto` — text layer when trusted (`text_layer_quality == "trusted"`),
-      OCR otherwise.
+      OCR otherwise. This is the mode that recovers prose on image-only pages.
     - `auto_verified` — GPU only. Runs OCR, then cross-checks every
       detection against the PDF text layer; replaces matches with the
       native string (`source: "pdf"`). On CPU this aliases to `auto`.
@@ -304,6 +308,40 @@ embedded text is trustworthy.
     `MAX_PDF_PAGES` defaults to `2000`. Exceeding returns
     `400 PDF_TOO_LARGE` with the limit echoed back in the message.
 
+### PDF → Markdown (`?markdown=1`)
+
+One call converts the whole PDF to the same faithful Markdown as
+`POST /ocr/markdown`, using the parallel page pipeline — no client-side
+page splitting. Available on both the GPU and CPU builds:
+
+```bash
+curl -X POST 'http://localhost:8000/ocr/pdf?markdown=1' \
+  --data-binary @paper.pdf -H 'Content-Type: application/pdf'
+```
+
+- Returns `text/markdown`: pages concatenated in order, each prefixed with an
+  invisible `<!-- page N -->` marker (safe to render, easy to split on).
+- `&as_pages=1` returns JSON instead — `{"pages":[{"page_index":0,
+  "markdown":"…"}, …]}` — for chunked/RAG consumers; per-page
+  `text_degraded` / `table_degraded` / `formula_degraded` flags appear when set.
+- Implies `layout=1&reading_order=1` (requires the layout model —
+  `400 LAYOUT_DISABLED` otherwise). In the default `ocr` mode, tables and
+  formulas are recognized whenever their backends are loaded (→ HTML / LaTeX);
+  pass `tables=0` / `formulas=0` to opt out.
+- **Mode choice.** `mode=ocr` (default) renders and OCRs every page — best for
+  scanned PDFs. `mode=geometric` reads a born-digital PDF's embedded text layer
+  directly for the prose (exact text, faster, no OCR errors) while **still**
+  recognizing tables → HTML and formulas → LaTeX on the rendered image — so a
+  born-digital paper exports exact text *and* structured math/tables. `auto`
+  picks per page: text layer when trustworthy, OCR otherwise.
+- Figure/chart crops are embedded as base64 `data:` URIs with their OCR'd text
+  as alt text; tables come back as HTML, formulas as `$…$` / `$$…$$` LaTeX.
+- `dpi`, `mode` and `autorotate` work as usual. `text=0` and `images=` are
+  rejected (`400 INVALID_PARAMETER`) — markdown needs the text, and the figure
+  crops are already embedded.
+- Per-stage degradation is aggregated in the `X-OCR-Degraded` response header
+  with page numbers (e.g. `table(p3,p7)`), same contract as `/ocr/markdown`.
+
 ### Inline page-image export
 
 `?images=inline` adds each rendered page back to the response as a

include/turbo_ocr/pipeline/ocr_pipeline.h

@@ -209,6 +209,20 @@ class OcrPipeline : public IOcrPipeline {
   [[nodiscard]] OcrPipelineResult run_layout_only(const cv::Mat &img,
                                                    cudaStream_t stream);
 
+  // Layout + table/formula recognition WITHOUT detection/recognition. Used by
+  // /ocr/pdf mode=geometric/auto when the client asked for structure: the page
+  // text comes from the PDFium text layer (passed in as `text_results`, already
+  // in this image's pixel space), so det/rec is skipped, but tables (→ HTML) and
+  // formulas (→ LaTeX) still need the rendered image and the layout regions.
+  // Uploads the image once, runs layout, then the CUA router over the supplied
+  // text boxes — the same dispatch_router_ the OCR path uses. Returns the text
+  // results verbatim plus populated `.layout` / `.tables` / `.formulas`. A no-op
+  // (returns just the text) when layout isn't loaded.
+  [[nodiscard]] OcrPipelineResult run_layout_and_structure(
+      const cv::Mat &img, cudaStream_t stream,
+      std::vector<OCRResultItem> text_results, bool want_tables,
+      bool want_formulas, const routing::RequestRouting &routing = {});
+
   // Ensure the GPU upload buffer can hold an image of the given size.
   // Returns {d_img_buf_, d_img_pitch_} after grow-only reallocation.
   // Useful for callers that want to decode directly into the pipeline's GPU buffer.