Why does the model miss things buried in a long prompt?

It is tempting to paste everything into the prompt and trust the model to find what matters. With a long input, that backfires. The model can read all of it, but it does not use all of it equally well.

A widely cited study tested where relevant information sits inside a long input. The finding: “performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts,” and this held “even for explicitly long-context models.”^[1] The pattern is often described as U-shaped: the edges of a long prompt are used well, the middle is the blind spot.

This has two practical consequences. First, more context is not automatically better. If you paste ten documents to answer a question that needs one, the nine extra ones are noise that can bury the signal. Trim to what the task actually needs.

Second, placement matters. Put the most important material where the model attends best. For large inputs, Anthropic’s guidance is to “put longform data at the top” of the prompt, above your question and instructions, and notes that putting the query at the end “can improve response quality by up to 30%… especially with complex, multi-document inputs.”^[2] For long-document tasks it also suggests asking the model to quote the relevant parts first, which “helps Claude cut through the noise of the rest of the document’s contents.”^[2]

So the fix is less about clever wording and more about layout: keep the prompt as short as the task allows, and place the parts that matter at the start and the end.

References

1. Lost in the Middle: How Language Models Use Long Contexts — Liu et al., Transactions of the ACL
2. Prompting best practices — Anthropic

Related

• What are the common ways prompting goes wrong? →
• Why do AI models make up facts, and how do I reduce it? →