ReTeX

Work in progress · paper in preparation · model not released yet

ReTeX reads the image of a document page and writes LaTeX code that compiles back into that page: the same text, equations, tables and figures, in the same place, with the same fonts and margins. Document conversion becomes an image-to-code problem, and the LaTeX compiler is the judge.

This is a personal research project. The results below come from the current model, evaluated on 400 real arXiv pages from papers submitted between July and September 2026, after every paper the model was trained on.

Why LaTeX

Most document converters output plain text, Markdown or a JSON tree, and lose what makes a page a page: equations stop being math, tables lose their structure and the layout disappears. LaTeX keeps all of it, and it can be compiled. Since the same page can be written in many different ways, ReTeX is judged on what its code produces, not on the code itself.

How it works

• Model. Qwen2.5-VL-3B fine-tuned with LoRA. It writes the whole document, from \documentclass to \end{document}, in one pass.
• Training data. Synthetic pages from a programmatic generator that samples layouts, fonts and content and keeps only what compiles, plus about 59,000 real arXiv pages paired with LaTeX that reproduces them exactly.
• Figures. Photos and plots cannot be written as code, so the model only decides where each figure goes and how big it is.

How it is measured

Both the original page and the model’s output are compiled, and fid_v3, a metric built for this project, compares them: it lists every visible word, image and line on each page, aligns them, and splits the error into content, graphics, geometry, placement and style. A perfect copy scores 1 and a page that does not compile scores 0.

The benchmark holds 400 pages from 400 arXiv papers submitted between July and September 2026, so the model has never seen any of them.

Results

When the output compiles, it is usually very close to the original. Most of the remaining gap comes from pages that do not compile and from figures and floats that land in the wrong place.

Examples

Real outputs of the model on pages from the benchmark, chosen to show what works and what does not. ● good, ● partial, ● failure. All pages come from papers under a CC BY 4.0 licence.

fid_v3 = 0.991 · One column, custom geometry. Theorem environments, aligned equations and an inline sum are reproduced almost exactly.

Loss by family:content 0.007graphics 0.000geometry 0.002placement 0.000style 0.000

Source: G. Andrews, M. Ghosh Dastidar, “The Partition Pairing Theorems I”, arXiv:2608.22291, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.993 · Two columns. A full two-column page: every word lands in the right column and line, section headings included.

Loss by family:content 0.001graphics 0.000geometry 0.000placement 0.005style 0.000

Source: M. Vyvlecka, R. Joos, B. Breiholz et al., “Rack-integrated quantum dot-based source of single and entangled photons at telecom C-band”, arXiv:2607.21454, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.964 · One column with floats. A two-column table with math in the cells, followed by propositions and numbered equations.

Loss by family:content 0.009graphics 0.001geometry 0.019placement 0.007style 0.000

Source: M. Planat, “Symmetry-Fixed Holonomies and Spectral Isolation in Two-Cycle Photonic Geometries”, arXiv:2608.16501, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.906 · Two columns. The two plots, their caption and the equations are reproduced, but the column break falls in a different place, so some text changes column.

Loss by family:content 0.034graphics 0.007geometry 0.007placement 0.038style 0.009

Source: C.-F. Kam, “Hudson's theorem fails for the SU(1,1) discrete series”, arXiv:2609.07242, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.904 · Two columns. Text and figure are right, but the right column starts a few lines too early: a placement error.

Loss by family:content 0.010graphics 0.016geometry 0.005placement 0.065style 0.000

Source: E. Abelev, R. Romero, N. P. Armitage, “Derivation and application of a general scaling relation between the dc and asymptotic high-frequency optical Hall responses”, arXiv:2607.15043, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.733 · One column with floats. The text is complete, but the model draws the plot at 90% of the text width instead of 100%, which shifts the caption and everything below it.

Loss by family:content 0.002graphics 0.142geometry 0.036placement 0.087style 0.000

Source: Y. Kini, “NeuroSynth: A Biologically Inspired Continual Reinforcement Learning Architecture for Mitigating Catastrophic Forgetting”, arXiv:2607.28663, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.570 · Two columns. The text is complete, but the plots end up in different positions and sizes than in the original, which pushes both columns around.

Loss by family:content 0.003graphics 0.098geometry 0.180placement 0.143style 0.006

Source: J. C. Getelina, A. Cox, M. Asaduzzaman et al., “Large-scale quantum simulations of dissipative spin-1/2 Heisenberg chains”, arXiv:2609.16108, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

fid_v3 = 0.495 · One column with floats. The original figure is a TikZ drawing; the model writes an external image instead (grey placeholder) that is too large and spills the page onto a second one.

Loss by family:content 0.031graphics 0.317geometry 0.123placement 0.034style 0.000

Source: F. Gunsilius, “A condition for the identification of multivariate models with binary instruments”, arXiv:2607.01429, CC BY 4.0. The page was re-rendered from the paper's LaTeX source.

Show the LaTeX written by the model

Loading…

What we learned along the way

• On real pages, the main failure was never finishing the preamble. Real arXiv preambles are long and full of custom macros, and the model learned to keep writing \newcommand lines until it ran out of tokens. Rewriting the training preambles into a canonical form that renders the same page removed almost all of these loops.
• Measure the render, and measure it carefully. An earlier metric gave a blank page 0.47. Replacing it changed which experiments looked like progress.

Next steps

• Placement, especially for figures and two-column pages: the text is usually right, but it does not always land where it should.
• Reinforcement learning with the compiler in the loop, using the fidelity of the compiled page as the reward.
• Multi-page documents.

Demo

A public demo will come with the paper. Until then, the examples above are real, unedited outputs of the model.