Introduction
Document parsing is an important computer vision task that involves recognizing text, identifying document structures, and reconstructing information in the correct reading order. It supports applications such as document digitization, financial document processing, academic research, and automated information extraction.
Traditional OCR systems typically use multi-stage pipelines. These systems detect document layouts, identify individual regions, and apply specialized recognition models to extract text, formulas, and tables. Although this approach can be reliable, it often requires complex processing pipelines and substantial customization.
Vision-language models (VLMs) offer an alternative by generating structured document content directly from visual inputs. These models simplify the processing pipeline and can generalize across different document types.
However, even advanced VLM-based OCR systems face difficulties with format-intensive content. Plain text is generally easier to recognize because its structure is comparatively straightforward. Mathematical formulas and tables introduce additional complexity because the model must understand relationships between symbols, expressions, rows, columns, and hierarchical structures.
The researchers observe that this difficulty is reflected in the models' output entropy. As the proportion of formatted content increases, uncertainty in the generated output also increases. This suggests that improving OCR performance requires more than increasing training data or optimizing conventional token-level prediction.
FD-RL addresses this limitation by identifying difficult examples and providing reward signals tailored to the structural requirements of different content types.
Methodology
1. Training Data Construction
The researchers construct a comprehensive training corpus from three primary sources: enhanced open-source datasets, real-world PDF documents, and synthetically generated OCR samples.
Open-source dataset enhancement
Existing OCR datasets provide broad coverage but may contain missing content, incorrect reading order, duplicated sentences, and inconsistent annotations.
The researchers improve these datasets through quality inspection, re-annotation, and similarity-based filtering. A lightweight OCR model is used to generate comparison annotations, while the original labels are retained as ground truth for samples that pass the quality checks.
Real-world PDF processing
Real-world documents provide examples of different layouts, fonts, color schemes, and resolutions.
The researchers construct layout-aware data at three levels:
Page-level data for complete document understanding.
Region-level data for recognizing selected portions of a document.
Multi-page data for processing sequences of two to six pages.
They also construct content-aware samples focused on accurate recognition of text, mathematical formulas, and tables. Formulas are represented in LaTeX, tables are represented in structured formats, and document content is organized into Markdown.
Synthetic OCR data generation
To expand coverage of educational and academic documents, the researchers generate synthetic OCR examples from online educational resources and academic question-and-answer content.
The collected material is cleaned, categorized, and rendered into document images using HTML templates, CSS, MathJax, and Playwright. Each generated image is paired with its corresponding Markdown representation.
The resulting dataset covers nine document categories, including books, notes, slides, examination papers, academic papers, magazines, newspapers, financial reports, and synthetic documents.
2. Supervised Fine-Tuning
FD-RL uses a two-stage training strategy, beginning with supervised fine-tuning.
The researchers use Qwen3-VL-4B as the pretrained vision-language backbone. During this stage, the visual encoder and projector are frozen, while the language model is updated.
This configuration preserves the pretrained visual representations while adapting the language model to recognize document content and generate structured outputs.
The SFT stage establishes the foundation for the subsequent reinforcement learning process.
3. Entropy-Based Data Filtration
After supervised fine-tuning, the model is used to identify samples that produce high output uncertainty.
The researchers calculate the average token entropy of each sample using the model's generated token probabilities. Samples with entropy above a selected threshold are retained for reinforcement learning.
Before entropy filtering, the candidate dataset is also adjusted to increase the representation of formulas and tables, remove samples containing only plain text, and balance Chinese and English examples.
This strategy concentrates reinforcement learning on challenging examples where the model is more likely to make structural or recognition errors.
The underlying principle is that uncertain outputs can reveal where the model needs additional learning. By focusing training on these examples, the method aims to improve its ability to interpret complex document structures.
4. Format-Decoupled Rewards
A central contribution of FD-RL is its use of different reward functions for different types of document content.
Instead of evaluating every output through a single text similarity metric, the framework separates recognized content into plain text, mathematical formulas, and tables.
Plain text recognition
Normalized Edit Distance (NED) is used to measure the similarity between predicted text and the reference. It provides character-level feedback for recognition errors.
Mathematical formula recognition
BLEU is used to evaluate formula predictions through n-gram matching. This provides feedback on differences in formula expressions and their local structure.
Table structure recognition
TEDS-S is used to evaluate the structural similarity of tables represented in HTML. It accounts for the organization of table elements rather than considering only their textual content.
The rewards are calculated for the content types present in the reference document and combined into an overall score. When parsing fails, the framework falls back to string matching to avoid assigning an uninformative zero reward.
This design allows the model to receive feedback that better reflects the specific requirements of each content type.
5. Reinforcement Learning Optimization
The reinforcement learning stage uses Group Relative Policy Optimization (GRPO).
For each input, the model generates multiple candidate responses. Their rewards are calculated using the format-specific evaluation functions, and the responses are compared within the group to estimate relative advantages.
The model is then optimized to favor responses that achieve higher rewards.
By combining GRPO with entropy-based filtering and format-decoupled rewards, FD-RL aims to improve structural validity and recognition accuracy without relying solely on memorizing reference token sequences.
Experimental Evaluation
The researchers evaluate FD-RL on OmniDocBench, a document parsing benchmark containing 1,355 pages across nine document types, two languages, and a variety of layouts.
The benchmark evaluates text recognition, formula recognition, table structure recognition, and reading order prediction.
Overall performance
FD-RL achieves an overall score of 90.41 on OmniDocBench.
Among the end-to-end models included in the paper's comparison, the results demonstrate improvements over several established OCR systems.
Model
Overall score
FD-RL
90.41
dots.ocr
88.41
DeepSeek-OCR
87.01
Nanonets-OCR-s
85.59
olmOCR-7B
81.79
Scores are reproduced from the paper's reported comparison table.
Performance across content types
The reported results show that FD-RL performs particularly well on formulas and tables.
Text recognition: NED of 0.049, with dots.ocr reporting 0.048 in the comparison.
Formula recognition: CDM score of 88.67, the highest among the end-to-end models in the reported comparison.
Table recognition: TEDS score of 87.35 and TEDS-S score of 92.10.
Reading order: ROEdit score of 0.055, ranking second among the compared end-to-end specialized models.
The results suggest that the format-specific optimization strategy is particularly useful for content where structural correctness matters alongside textual accuracy.
Performance across document types
The paper reports that FD-RL achieves the best performance in four of the nine document categories and ranks second in the remaining five.
Examples include a text edit distance of 0.0235 for slides and 0.0464 for examination papers. The model also reports edit distances of 0.0300 for books and 0.0881 for notes.
These results indicate that the method generalizes across multiple document categories rather than improving performance only on a single type of formatted content.
Ablation Studies
The researchers conduct ablation experiments to evaluate the contribution of the training strategy and reward design.
The reported results show the following progression:
A baseline using a unified NED reward achieves an overall score of 88.64.
Separating content types while retaining NED improves the score to 89.61.
Using BLEU for formula evaluation raises the score to 89.80.
The complete reward design, including TEDS-S for tables, achieves 90.41.
These experiments support the importance of matching the reward function to the structure of the content being recognized.
The study also evaluates different training data sources, entropy filtering rates, and reinforcement learning configurations. Its findings indicate that data selection and reward design both influence the effectiveness of the final model.
Discussion
FD-RL approaches document OCR as a problem involving both recognition and structural interpretation.
A model may correctly identify many individual characters while still producing an incorrect formula or malformed table. Conventional token-level objectives may not adequately reflect these differences.
The proposed framework addresses this issue by concentrating reinforcement learning on uncertain examples and evaluating different content types with specialized metrics.
This approach is relevant to document processing systems that need to preserve the structure of academic papers, financial reports, examination papers, and other documents containing complex layouts.
However, benchmark performance does not guarantee identical results across every document type, language, or real-world deployment environment. Practical performance will depend on the input distribution, document quality, and recognition requirements.
Conclusion
Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCR introduces a reinforcement learning framework designed to improve recognition of structurally complex documents.
FD-RL combines supervised fine-tuning, entropy-based data filtration, and format-specific rewards to improve text recognition, formula interpretation, and table structure reconstruction.
The reported OmniDocBench score of 90.41 and the ablation results support the effectiveness of its format-aware training strategy. More broadly, the research demonstrates how reinforcement learning can be used to optimize structural correctness in document understanding systems.
Research Resources
Project repository: https://github.com/DocTron-hub/FD-RL
Conference: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)