The Path Reuse and Compression Theory of LLM Hallucination | NEXAIRI

What are Path Reuse and Path Compression in LLMs?

Path Reuse and Path Compression are two structural mechanisms that cause language models to hallucinate at different stages of training.

Researchers at Michigan State University, led by Xinnan Dai and Jiliang Tang, published findings in April 2026 showing that LLM hallucinations aren't random glitches. They follow predictable patterns tied to how neural networks organize knowledge during training. The team used graph theory to map these patterns, treating the model's internal knowledge as a network of connected paths.

Path Reuse appears in the first roughly 30% of training. When a model encounters a query, it activates paths it memorized from training data instead of following the context provided in the prompt. Path Compression kicks in after about 60% of training, when frequently traveled multi-step reasoning routes collapse into direct shortcuts. The model jumps from premise to conclusion without the intermediate steps.

Both mechanisms are structural. They're baked into the model's weights during pretraining, which means surface-level fixes like better prompting or basic fine-tuning can't fully eliminate them.

Why does ChatGPT ignore the context you give it?

Path Reuse causes the model to prefer its memorized training data over the specific information you provide in your prompt.

This is one of the most common user complaints about large language models. You paste a document into ChatGPT, ask a specific question about it, and the model answers based on something it learned during training rather than what you just gave it. The Michigan State team's graph analysis shows why: the model's internal knowledge paths are so well-worn from billions of training examples that new context can't override them.

Think of it like asking directions from someone who's driven the same route for 20 years. Even if you show them a map with a new faster route, they'll default to the one they know. The model does the same thing, but with facts. When training data says X and your prompt says Y, Path Reuse means the model often picks X.

The researchers tested this on Erdős-Rényi random graphs with 10 nodes, 0.4 edge probability, and 225,808 subgraphs. They trained models to learn graph structures and then presented contradictory information. Models consistently favored memorized paths over contextual evidence during early and mid-training checkpoints.

What is the reversal curse and how does Path Reuse explain it?

The reversal curse is a known failure where models trained on "A is B" can't answer "B is ?" Path Reuse shows it's a directional memory problem.

Berglund et al. documented this phenomenon in 2023: train a model on the fact that "Tom Hanks starred in Forrest Gump" and it can answer "Who starred in Forrest Gump?" But ask "What movie did Tom Hanks star in?" and the model fails. The knowledge only flows in one direction.

The Michigan State researchers showed this is a direct consequence of Path Reuse. The memorized path through the model's weight structure goes from query pattern A to answer B, but no reciprocal path exists from B back to A. The graph structure is directional. Building the reverse path requires separate training examples that explicitly establish the B-to-A connection.

This finding matters because it means the reversal curse isn't a bug that better training data can fix by volume alone. You'd need training data that explicitly covers both directions, or architectural changes that force bidirectional path formation.

How does Path Compression cause reasoning failures?

Path Compression collapses multi-step logic into shortcuts, making models skip intermediate steps and produce wrong answers on complex questions.

This is the mechanism behind multi-hop reasoning failures. When a model has seen thousands of examples where A leads to B leads to C, it eventually learns a direct shortcut from A to C. The intermediate step B gets compressed out. That works fine when the shortcut is valid. It fails when a new problem requires going through B in a slightly different way.

The researchers found Path Compression becomes dominant after roughly 60% of training steps. At that point, the model has seen enough examples that common reasoning chains crystallize into direct mappings. The result: the model answers confidently and quickly, but it's using a shortcut that doesn't apply to the specific question.

This connects to Anthropic's finding that chain-of-thought traces are often unfaithful, documented by Chen et al. in 2025. The model appears to show its reasoning step-by-step, but the actual computation skips those steps entirely. The chain-of-thought is a post-hoc rationalization for a compressed path the model already traveled.

Can RLHF fix structural hallucinations?

Reinforcement learning helps more than supervised fine-tuning, but neither method can fully undo structural patterns formed during pretraining.

The Michigan State team tested both PPO (Proximal Policy Optimization) and SFT (Supervised Fine-Tuning) as recovery methods. PPO outperformed SFT at reducing hallucinations from both Path Reuse and Path Compression. But the improvements were partial. The structural patterns formed during pretraining leave lasting imprints in the model's weight structure.

This has significant implications for the industry's approach to AI safety. Most alignment work focuses on post-training interventions: RLHF, constitutional AI, and system prompt engineering. The Path Reuse framework suggests these interventions are working against structural patterns that formed much earlier in the training pipeline, during pretraining itself.

The paper aligns with a broader research trend. Ouyang et al.'s original InstructGPT paper established RLHF as the standard alignment approach. But recent work keeps finding failure modes that post-training can't fully address. Path Reuse and Path Compression add two more entries to that list.

Does this explain why bigger models still hallucinate?

Yes. Both mechanisms are structural consequences of how transformers learn, regardless of model size. Scaling parameters doesn't solve the underlying graph problem.

This is arguably the paper's most important implication. The AI industry has spent billions on the assumption that bigger models produce fewer errors. GPT-4 has fewer hallucinations than GPT-3, which seemed to validate that assumption. But the Michigan State analysis shows the mechanisms causing hallucinations are baked into the transformer training process itself.

Larger models have more parameters to form more paths, but they also develop more shortcuts through Path Compression. They have richer memorized knowledge through Path Reuse, but that makes the override problem worse, not better. The model becomes more confident in its memorized answers, making it harder for new context to break through.

This matters for anyone evaluating AI tools for enterprise use. A model that scores well on benchmarks may still hallucinate on your specific documents because Path Reuse causes it to favor training data over your context. Retrieval-augmented generation (RAG) helps but doesn't eliminate the problem, because the model's internal paths still compete with the retrieved context.

What did 225,808 subgraphs reveal about hallucination timing?

The researchers tracked hallucination rates at every training checkpoint using graph-based experiments, revealing exact transition points where each mechanism dominates.

The experimental setup used Llama-2 architecture trained on two types of synthetic graphs. Erdős-Rényi random graphs provided controlled environments with N=10 nodes and 0.4 edge probability, producing 225,808 subgraphs for analysis. Stochastic Block Model graphs with 1,000 nodes tested whether the patterns held at larger scale.

Tasks included shortest path prediction, reachability queries, and path existence determination. At each training checkpoint, the researchers measured how often the model chose memorized paths over contextually correct ones (Path Reuse) and how often it skipped intermediate reasoning steps (Path Compression).

The results showed clear phase transitions. Path Reuse hallucinations peak early and then gradually decline as the model learns to balance memory with context. Path Compression hallucinations follow the opposite curve, rising as training progresses and common reasoning routes get compressed. The overlap period, where both mechanisms are active, produces the highest hallucination rates. This suggests there's an optimal training duration for each task, beyond which additional training increases certain types of errors.

How will this research change AI development in 2026?

The unified framework gives engineers specific intervention targets at known training stages, replacing the trial-and-error approach to hallucination reduction.

Before this paper, hallucination mitigation was largely reactive. Engineers would test models, find errors, and apply patches through fine-tuning or prompt engineering. The Path Reuse and Path Compression framework converts this into a proactive process. If you know Path Reuse peaks at 30% of training and Path Compression dominates after 60%, you can design training schedules that intervene at those exact points.

Potential interventions include curriculum learning strategies that vary the training data distribution at critical phases, architectural modifications that maintain separate pathways for memorized and contextual knowledge, and training-time regularization that penalizes shortcut formation.

For enterprise AI teams, the practical takeaway is straightforward: don't trust any model's output on tasks where your context contradicts its training data. RAG systems help, but the model's internal paths will still compete with retrieved context. Build verification layers that flag when the model's answer contradicts the provided documents.