GovernedRAG: Risk-Adaptive Retrieval-Augmented Generation for Enterprise Compliance Decision Support
Article Sidebar
Main Article Content
Abstract
When enterprises rely on LLMs for compliance decision support because they are under sustained regulatory scrutiny, retrieval-augmented generation without control allows access-sensitive regulatory, contractual, and audit content to be surfaced indiscriminately, and it does not tailor its response to the risk level of the query. This paper combines the latest research in retrieval-augmented generation, access control, adversarial threats, and AI risk management and introduces GovernedRAG, an architecture for enterprise compliance decision support that adapts to risk. The architecture integrates a four-stage risk classifier, a retrieval gate that limits the scope of candidate evidence retrieved to the scope of the requesting user before they are ranked, a grounded generation stage with an evaluation loop based on the reference-free assessment frameworks, and a continuous evaluation loop. A formalization is provided for the risk-weighted relevance, access gating, and faithfulness scoring, and the proposal is compared to five related retrieval-augmented paradigms, and the proposal is juxtaposed with them. It shows that governance and risk adaptivity are still not considered as first-class architectural concerns in previous retrieval-augmented systems, that access control is still mostly outside of retrieval ranking and not embedded in the ranking function, and that reference-free evaluation protocols are practical foundations for ongoing, compliance-driven monitoring. The proposed framework is organized into four risk tiers, four architectural stages and six comparative synthesis tables. Consequences for auditing, regulation and empirical confirmation are considered.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
[1] A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, “Self-RAG: Learning to retrieve, generate, and critique through self-reflection,” in Proc. Twelfth Int. Conf. Learning Representations (ICLR), 2024, doi: 10.48550/arXiv.2310.11511.
[2] B. C. Das, M. H. Amini, and Y. Wu, “Security and privacy challenges of large language models: A survey,” arXiv preprint, 2024, doi: 10.48550/arXiv.2402.00888.
[3] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson, “From local to global: A graph RAG approach to query-focused summarization,” arXiv preprint, 2024, doi: 10.48550/arXiv.2404.16130.
[4] S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, “RAGAs: Automated evaluation of retrieval augmented generation,” in Proc. 18th Conf. European Chapter Assoc. Comput. Linguistics: System Demonstrations, 2024, pp. 150-158, doi: 10.18653/v1/2024.eacl-demo.16.
[5] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, Q. Guo, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” arXiv preprint, 2023, doi: 10.48550/arXiv.2312.10997.
[6] S. Hassani, “Enhancing legal compliance and regulation analysis with large language models,” arXiv preprint, 2024, doi: 10.48550/arXiv.2404.17522.
[7] Y. Huang and J. X. Huang, “A survey on retrieval-augmented text generation for large language models,” arXiv preprint, 2024, doi: 10.48550/arXiv.2404.10981.
[8] I. Iaroshev, R. Pillai, L. Vaglietti, and T. Hanne, “Evaluating retrieval-augmented generation models for financial report question and answering,” Appl. Sci., vol. 14, no. 20, Art. no. 9318, 2024, doi: 10.3390/app14209318.
[9] S. H. Jayasundara, N. A. G. Arachchilage, and G. Russello, “RAGent: Retrieval-based access control policy generation,” arXiv preprint, 2024, doi: 10.48550/arXiv.2409.07489.
[10] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Comput. Surv., vol. 55, no. 12, pp. 1-38, 2023, doi: 10.1145/3571730.
[11] V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih, “Dense passage retrieval for open-domain question answering,” in Proc. 2020 Conf. Empirical Methods Natural Lang. Process. (EMNLP), 2020, pp. 6769-6781, doi: 10.18653/v1/2020.emnlp-main.550.
[12] H. Kwon and W. Pak, “Text-based prompt injection attack using mathematical functions in modern large language models,” Electronics, vol. 13, no. 24, Art. no. 5008, 2024, doi: 10.3390/electronics13245008.
[13] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Kuttler, M. Lewis, W. Yih, T. Rocktaschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 9459-9474, 2020, doi: 10.48550/arXiv.2005.11401.
[14] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Trans. Assoc. Comput. Linguistics, vol. 12, pp. 157-173, 2024, doi: 10.1162/tacl_a_00638.
[15] C. Novelli, F. Casolari, A. Rotolo, M. Taddeo, and L. Floridi, “AI risk assessment: A scenario-based, proportional methodology for the AI Act,” Digital Society, vol. 3, Art. no. 13, 2024, doi: 10.1007/s44206-024-00095-1.
[16] S. Rossi, A. M. Michel, R. R. Mukkamala, and J. B. Thatcher, “An early categorization of prompt injection attacks on large language models,” arXiv preprint, 2024, doi: 10.48550/arXiv.2402.00898.
[17] J. Saad-Falcon, O. Khattab, C. Potts, and M. Zaharia, “ARES: An automated evaluation framework for retrieval-augmented generation systems,” in Proc. 2024 Conf. North American Chapter Assoc. Comput. Linguistics: Human Lang. Technol. (NAACL-HLT), vol. 1, 2024, pp. 338-354, doi: 10.18653/v1/2024.naacl-long.20.
[18] R. S. Sandhu, E. J. Coyne, H. L. Feinstein, and C. E. Youman, “Role-based access control models,” Computer, vol. 29, no. 2, pp. 38-47, 1996, doi: 10.1109/2.485845.
[19] B. Sarmah, D. Mehta, B. Hall, R. Rao, S. Patel, and S. Pasquali, “HybridRAG: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction,” in Proc. 5th ACM Int. Conf. AI in Finance, 2024, pp. 608-616, doi: 10.1145/3677052.3698671.
[20] E. Tabassi, “Artificial intelligence risk management framework (AI RMF 1.0),” NIST AI 100-1, National Institute of Standards and Technology, 2023, doi: 10.6028/NIST.AI.100-1.