Long documents have been a problematic area for LLMs based on our exploratory conversations. The issue is not a singular mode of failure, but a collection of current technical realities:
- This is for the large part, a minor issue for general world knowledge, which is where LLMs produce blatant false information for basic questions; this is harmful but also relatively easy to spot.
- LLMs can mix general knowledge with proprietary information to produce unusable content. This happens when LLM adds outside knowledge to its synthesized answers. While it can be helpful, the traceability and repeatability aspects are reduced, resulting in a whiplash effect for downstream information processing.
- Even when using RAG, LLMs can lose important content in the retrieval process, known as the “needle in the haystack” issue (Nelson, 2024).
- Furthermore, the output could be true, but the sourcing is opaque and retrofitting the assumed intention, which can also render the output unreliable and unfit for downstream processing in a complex system (Saxena, 2025).
Our patent-pending architecture addresses these various levels of hallucinations by deploying a knowledge network-centric approach where the sequence of logic and content priorities are emphasized. This means LumIntelligence is not bound by the size or the proprietary properties of the original content. It does not produce results prior to finding sources or deviate from given information that reflects the current dataset. To demonstrate this, we have results from two tests: the first one shows how LumIntelligence performs over the regular RAG over a long document; the second one expands on that to parse proprietary information and produces answers based on targeted sources.
For the first test, we ran RAG and LumIntelligence on long regulation documents, such as GDPR. Given it is a well-known document with general world knowledge content, RAG generally performed well to find obvious chunks before synthesizing an answer. But the differences happen when the content is more buried in unrelated context. One such example includes when asking for encryption-related requirements, LumIntelligence can consistently find content on page 53 while RAG struggled to produce content linking back to the same source. The flip side of this example is when the question was phrased to be similar to the original text, traditional RAG performed well regardless of the LLM used; this behavior is in line with Nelson’s paper.
For the second test, we created a mock document with proprietary terms to describe an internal project and its product specifications. Some of the terms clash with well-established norms, for example, SLA is usually known as “Service Level Agreement” whereas in this document, it stands for “Saltwater Load Assessment.” Some terms are internal to the project and not available as part of the general world knowledge of LLMs. LumIntelligence was able to discover custom terms and apply the appropriate glossary when building the knowledge base, which in turn produced close to 100% accuracy in finding the right sources. General RAG approach failed to adapt to the changing environment and found less than half of the relevant information.
With these tests, we are making further progress to generalize the platform while making it more adaptive to meet industry and business specific needs. It is important to not only trust the outcome of AI, but to be able to verify the claims as well, and that’s why we are building it.
Yours in illumination,
Peter @ LumIntelligence
Nelson, Elliot, et al. “Needle in the haystack for memory based large language models.” arXiv preprint arXiv:2407.01437 (2024).
Saxena, Yash, et al. “Generation-Time vs. Post-hoc Citation: a holistic evaluation of LLM attribution.” arXiv preprint arXiv:2509.21557 (2025).