Xunjian Yin / Research

ACL 2026Reverse language modeling

LEDOM: Reverse Language Model

Xunjian Yin1, Sitao Cheng3, Yuxi Xie5, Xinyu Hu1, Li Lin1, Xinyi Wang2, Liangming Pan1, William Yang Wang4, Xiaojun Wan1

1 Peking University · 2 Princeton University · 3 University of Waterloo · 4 University of California, Santa Barbara · 5 National University of Singapore

Paper arXiv Model (7B, preliminary) Code

Train a language model to predict the past. Then use its backward probabilities to help forward models choose better answers.

Result
With QwenMath on AIME 2024, Reverse Reward reranking reaches 23.3% vs 16.7% for greedy decoding and 11.3% for random selection among the same samples.
New vs prior
TRLM reranks forward outputs with a small reverse model, and Golovneva et al. train forward then reverse; this work pretrains a purely reverse 2B/7B model and uses it as a posterior scorer.
Limitation
Reverse Reward is compared only with greedy decoding and random selection, not with learned outcome or process reward models.
One sentence. Two directions.
Thecapitalof FranceisParis.

P(earlier text | later text)

Given the ending, predict what comes before it.

Given contextText to predict

Illustrative word-level example. LEDOM reverses tokens during training; this is not a live model output.

01 / The research question

An answer can be fluent and still fail to explain the question

A forward language model scores how plausible a response is after a prompt, but a plausible continuation can quietly drop one of the problem’s constraints. LEDOM asks the complementary question: given the response, how well can a model reconstruct the prompt? The intended signal is sensitivity to the premises, not fluency.

Both factorization directions are exact by the chain rule, but with finite data and capacity they learn different approximations. LEDOM isolates the asymmetry: reverse and forward models share the architecture, the tokenizer and a 435B-token corpus of general text, mathematics and code, at 2B and 7B parameters. The public 7B checkpoint (Reverse-Model-7B-348B) is a preliminary release at about 348B training tokens, according to the code repository.

Closest prior work

Earlier reverse-generation work regularized forward models with forward-backward agreement (Serdyuk et al., 2017; Zhang et al., 2019), trained small reverse models to find worst-case inputs (Pfau et al., 2023), or trained forward then reverse to mitigate the reversal curse (Golovneva et al., 2024). Closest to its application, TRLM (Varun et al., 2025) uses a small reverse model for unsupervised feedback and best-of-N reranking. LEDOM pretrains a purely reverse model at 2B and 7B scale, matched to forward models, and extends reranking with a noisy-channel analysis and step-level verification.

02 / Inside the method

Reconstruct the prompt, then combine the two scores

For a question x and a candidate response y, the forward model estimates P(y | x); the reverse model reads the reversed response, then the reversed prompt, and estimates P(x | y). The prompt tokens are what is scored, so the method does not ask a second model whether an answer looks correct.

Published examples · Table 2, Figures 5–6

Give LEDOM an ending and it writes what came before

Each example pairs a given later text with the earlier text LEDOM generated for it, exactly as printed in the paper. Press Play to replay the output in LEDOM’s order: the word next to the given text comes first and the opening comes last.

Question generation

Published example · Table 2 · as printed · 16 words

Earlier text · written by LEDOM, last word first

Alyssa’s cat loves eggs. If she eats 3 eggs a day, how many in a week?

Later text · given to LEDOM

Answer: She eats 3 eggs every day. 7 days in a week. 3*7 = 21. 21

The question keeps the answer’s three eggs a day and seven-day week, and adds Alyssa and her cat, which the answer never mentions. Reverse Reward scores this direction, P(question | answer).

All 16 words generated. LEDOM wrote “week?” first and “Alyssa’s” last.

Reversal curse

Published example · Table 2 · as printed · 2 words

Earlier text · written by LEDOM, last word first

J.K. Rowling

Later text · given to LEDOM

is the author of Harry Potter.

Text usually names the entity before describing it. Read backwards, the description comes first, so recovering the name is an ordinary prediction for LEDOM (§3).

All 2 words generated. LEDOM wrote “Rowling” first and “J.K.” last.

Abductive reasoning

Published example · Appendix C, Figure 6 · full output · 104 words

Earlier text · written by LEDOM, last word first

Mike worked very hard in school and was the first college graduate in his family. His family did not have a lot of money, so he had to find a way to pay for his college education. Mike had a part-time job while he was in high school. He also did odd jobs to earn extra money. After a few years, Mike was able to save enough money to go to college. When Mike graduated from college, he got a job at a bank. It was a good job, but it was not what he wanted to do for the rest of his life.

Later text · given to LEDOM

That is why Mike gave up his job and started his own business.

The paper’s example of abductive inference: an antecedent story that motivates the stated outcome.

All 104 words generated. LEDOM wrote “life.” first and “Mike” last.

Story generation

Published example · Appendix C, Figure 6 · full output · 107 words

Earlier text · written by LEDOM, last word first

"Goodnight, my princess," he said, leaning down and kissing her on the forehead. Then he turned and went to his room where he lay down on his bed and fell into a deep sleep. The next morning he awoke and found that he couldn’t go back to sleep, so he got up and walked into Princess Lara’s room and saw that she was still asleep.
He walked over to her and gently placed his hand on her cheek. She opened her eyes and looked up at him. He smiled and leaned down to kiss her. As he kissed her, he whispered in her ear, "I love you."

Later text · given to LEDOM

And so, Prince Adrian and Princess Elara lived happily ever after, their hearts forever entwined, their love a beacon of hope and inspiration for all.

Look at the names: the output says “Princess Lara”, while the given ending says “Princess Elara”.

All 107 words generated. LEDOM wrote “you.'” first and “'Goodnight,” last.

GSM8K evaluation

Published example · Appendix B, Figure 5 · full prediction · 70 words

Earlier text · written by LEDOM, last word first

16 is the answer of the following question.
Since Janet sells her fresh duck eggs at the farmers’ market for $1 per fresh duck egg, the number of dollars Janet makes per fresh duck egg is 3-1=$2.
Since she sells 8 eggs every day, the total number of dollars Janet makes at the farmers’ market every day is 8*2=16
Therefore, Janet makes 16 dollars at the farmers’ market every day.

Later text · given to LEDOM

The above is the thought chain.
Question: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers’ market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers’ market? [few-shot demonstrations: Figure 5 prints one and elides six]

Incorrect: the reference answer is 16 − 3 − 4 = 9 eggs, times $2 = $18. The chain reaches 16 from premises the question does not contain ($1 per egg, 8 eggs). LEDOM scores 1.74% on GSM8K at both 2B and 7B (Table 3).

All 70 words generated. LEDOM wrote “day.” first and “16” last.

Published examples: Table 2, Appendix B, Figure 5 and Appendix C, Figure 6, in the paper’s reading order; five of its cases. Measured: GSM8K accuracy from Table 3. Illustrative: the word-by-word replay. LEDOM generates subword tokens in reverse, so the pieces within each word are also produced last to first.

Reverse Reward combines the two as P(y | x)^(1−λ) × P(x | y)^λ, a weighted sum of log-likelihoods. The paper’s Proposition 1 states when this helps: an incorrect chain with comparable forward likelihood but poorer backward reconstruction is ranked lower.

The main experiments rerank 64 sampled complete responses; a step-level beam search scores partial paths. The forward generator is unchanged, the reverse scorer is fine-tuned on math data, and each candidate costs one extra reverse forward pass.

  1. 01

    Reverse the token order

    Train a standard causal Transformer on reversed sequences. The architecture stays the same; each prediction now depends on text that originally came after it.

  2. 02

    Learn backward reconstruction

    Given an ending, reconstruct plausible earlier text. The paper illustrates this with abductive reasoning, question synthesis, and inverse relation completion.

  3. 03

    Score in both directions

    Reverse Reward combines forward P(response | prompt) with reverse P(prompt | response) to rerank candidate answers, without updating the forward model’s weights.

View the paper’s figure
Forward and reverse language modeling use complementary factorizations of the same sequence. Original Figure 1 from the paper.
Forward and reverse language modeling use complementary factorizations of the same sequence. Original Figure 1 from the paper. Full size ↗
03 / Reading the evidence

Reverse scores choose better answers than greedy or random selection

Measured · Table 4

Does Reverse Reward help beyond sampling more answers?

Reverse Reward ranks candidate solutions by P(y | x)1−λ · P(x | y)λ, the forward model’s likelihood combined with LEDOM’s probability of reconstructing the question. The paper reports no per-candidate scores, so this chart uses its twelve measured model–benchmark accuracies. Each mark shows how far a strategy lands from the chosen baseline.

Random pick (Best-of-N)Reverse Reward (Best-of-N)Reverse Reward (beam search)

Accuracy minus greedy decoding, percentage pointsRR Best-of-N

MATH-500

DeepSeekMath+1.6
QwenMath+2.8
OpenMath2+1.0

GSM8K

DeepSeekMath+2.3
QwenMath+0.5
OpenMath2+1.2

AIME 2024

DeepSeekMath+3.3
QwenMath+6.6
OpenMath2+6.7

AMC 2023

DeepSeekMath+15.0
QwenMath+2.5
OpenMath20.0

Compared with greedy decoding:

  • Random pick (Best-of-N): higher in 1 of 12 pairs (DeepSeekMath · AMC 2023); equal in 1 (OpenMath2 · AIME 2024); lower in 10.
  • Reverse Reward (Best-of-N): higher in 11 of 12 pairs; equal in 1 (OpenMath2 · AMC 2023).
  • Reverse Reward (beam search): higher in 3 of 4 pairs; lower in 1 (OpenMath2 · AIME 2024).

Table 4 as reported: accuracy (%). Bold marks the best value per model; shaded rows are the current baseline.

Model and strategyMATH-500GSM8KAIME 2024AMC 2023
DeepSeekMath
Greedy decoding42.081.810.012.5
Random pick (Best-of-N)40.781.18.9718.6
Reverse Reward (Best-of-N)43.684.113.327.5
QwenMath
Greedy decoding78.095.616.755.0
Random pick (Best-of-N)73.994.711.348.3
Reverse Reward (Best-of-N)80.896.123.357.5
OpenMath2
Greedy decoding64.089.810.040.0
Random pick (Best-of-N)56.287.110.024.8
Reverse Reward (Best-of-N)65.091.016.740.0
Reverse Reward (beam search)65.491.86.742.5

Derived: every greedy, Reverse Reward and beam-search value in the AIME 2024 and AMC 2023 columns is a whole number of problems out of 30 or 40, consistent with the standard sets. QwenMath’s +6.6 points on AIME 2024 would then be 5 → 7 solved problems. The paper does not state set sizes, and most random-pick values (e.g. 8.97) are not whole-problem fractions.

Measured: Table 4, accuracy (%) for DeepSeekMath-7B, QwenMath-7B and OpenMath2-8B, with LEDOM fine-tuned on 100K OpenMathInstruct-2 examples as the reverse scorer (Appendix D.2). Derived: differences in percentage points. Table 4 and §6.5 state N = 64 candidates, while Appendix D.2.3 lists N = 4 for reranking and beams of k = 4 with n = 3 expansions. The paper does not report λ. The case studies in Tables 7–8 give responses and orderings but no score values, and Figure 3 plots accuracy against N without numeric labels, so neither is reproduced as numbers here.

In the paper’s download case, a 200 GB file at 2 GB/minute fails at 40% and needs a 20-minute restart and a fresh download: 40 + 20 + 100 = 160 minutes. The forward model’s top candidate downloads only the remainder and adds a second restart; Reverse Reward ranks it lower because that chain cannot reconstruct the problem.

Random selection from the same samples is usually below greedy decoding, so the gain comes from the reverse score, not from sampling diversity. Finer search is not always better: on AIME 2024, OpenMath2’s step-level beam search falls below greedy decoding, which the paper attributes to pruning partially correct beams early.

Scope. The backward-generation examples are qualitative, and the reasoning gains are limited to mathematics, with a reverse scorer fine-tuned on mathematical data. Read the study ↗

Further reading in the paperReverse factorization · §2 ↗Reverse Reward · §5 ↗Math results, controls and costs · §6 ↗
Research summary

LEDOM studies right-to-left autoregressive pretraining at 2B and 7B parameters on 435 billion tokens. It reconstructs earlier text from later context, with qualitative examples of abductive reasoning, question synthesis, and inverse relation completion. Reverse Reward combines forward response likelihood with reverse prompt likelihood to rerank candidate responses. In the reported experiments, this improves mathematical reasoning over greedy baselines, while standalone reverse models remain weaker on many forward-oriented benchmarks. Models, code, and data are released for further research.

Full paper ↗
Citation BibTeX 
@inproceedings{yin-etal-2026-ledom,
    title = "{LEDOM}: Reverse Language Model",
    author = "Yin, Xunjian  and
      Cheng, Sitao  and
      Xie, Yuxi  and
      Hu, Xinyu  and
      Lin, Li  and
      Wang, Xinyi  and
      Pan, Liangming  and
      Wang, William Yang  and
      Wan, Xiaojun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.975/",
    doi = "10.18653/v1/2026.acl-long.975",
    pages = "21306--21326",
    ISBN = "979-8-89176-390-6"
}