AI Safety and Alignment in General-Purpose AI Systems: Frameworks, Evaluation, and Governance
Keywords:
AI safety; value alignment; general-purpose AI; scalable oversight; mechanistic interpretability; AI governanceAbstract
The rapid advancement of general-purpose AI (GPAI) systems has elevated AI safety and alignment from technical concerns to pressing governance priorities. This study provides a comprehensive analysis of AI safety and alignment in GPAI systems through a systematic literature review of technical approaches, evaluation methodologies, and governance frameworks, drawing on peer-reviewed literature, conference proceedings, and regulatory documents from 2017 to 2026. Three principal technical approaches are identified: behavioural alignment through reinforcement learning from human feedback and Constitutional AI; mechanistic interpretability for internal model understanding, revealing that approximately 5% of neurons function as "safety neurons" with over 90% causal effects on safety alignment; and scalable oversight for supervising systems that exceed human capabilities, with nested scalable oversight success rates ranging from 9.4% to 51.7% at a general Elo gap of 400. Evaluation methodologies reveal a persistent gap between measured safety and actual safety properties. The analysis indicates that effective AI safety and alignment require integrated approaches combining technical robustness, rigorous evaluation, and adaptive governance. A multi-layered framework encompassing pre-deployment evaluation, runtime monitoring, and incident response mechanisms is proposed.
References
1. International AI Safety Report 2026: Executive Summary. 2026.
2. Tong H, Zhao F, Feng L, Wu R, Chen R, Jia L, et al. ForesightSafety Bench: A Frontier Risk Evaluation and Governance Framework towards Safe AI. arXiv:2602.14135. 2026.
3. Ji J, Qiu T, Chen B, Zhang B, Zhou J, Hong D, et al. AI Alignment: A Contemporary Survey. ACM Comput Surv. 2025;58(5):132.
4. Frontier Model Safety Analysis (2026): RSP, Preparedness, FSF. FutureAGI. 2026.
5. Li W. Three rulebooks, one race: AI regulation in the U.S., EU, and China. Commun ACM. 2026.
6. Markov T, Mariaccia H, Ségerie CR, Dorn D, Bucknall B, Siddiqui S, et al. Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods. arXiv:2505.05541. 2025.
7. Kaufmann T, Weng P, Bengs V, Hüllermeier E. A Survey of Reinforcement Learning from Human Feedback. Trans Mach Learn Res. 2025.
8. Rudner TGJ, Ishida R, Suau X, et al. Measuring and Mitigating Safety Risks in Large Language Models: A Survey. arXiv:2504.03814. 2025.
9. Casper S, Davies X, Shi C, Gilbert TK, Scheurer J, Rando J, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv:2307.15217. 2023.
10. Christiano PF, Leike J, Brown TB, Martic M, Legg S, Amodei D. Deep Reinforcement Learning from Human Preferences. In: Advances in Neural Information Processing Systems 30. 2017. p. 4302-4310.
11. Bowman SR, Hyun J, Perez E, Chen E, Pettit C, Bowman S, et al. Measuring Progress on Scalable Oversight for Large Language Models. arXiv:2211.03540. 2022.
12. Bai Y, Kadavath S, Kundu S, Askell A, Kernion J, Jones A, et al. Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073. 2022.
13. Kyrychenko Y, Zhou K, Bogucka E, Quercia D. C3AI: Crafting and Evaluating Constitutions for Constitutional AI. In: Proceedings of the ACM Web Conference (WWW '25). 2025.
14. Zhang X, Wang G, Cui Y, Qiu W, Li Z, Zhu B, et al. The Alignment Floor: How Persona Customization Breaks Safety in Weakly-Aligned LLMs. arXiv:2605.27382. 2026.
15. Chen J, Wang X, Yao Z, Bai Y, Hou L, Li J. Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons. In: Advances in Neural Information Processing Systems 38 (NeurIPS 2025). 2025.
16. Sengupta A, Seth P, Sankarapu VK. Interpretability as Alignment: Making Internal Understanding a Design Principle. arXiv:2509.08592. 2025.
17. Elhage N, Nanda N, Olsson C, Henighan T, Joseph N, Mann B, et al. A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread. 2021.
18. Olsson C, Elhage N, Nanda N, Joseph N, DasSarma N, Henighan T, et al. In-Context Learning and Induction Heads. arXiv:2209.11895. 2022.
19. Palumbo N, Mangal R, Wang Z, Vijayakumar S, Păsăreanu C, Jha S. Validating Mechanistic Interpretations: An Axiomatic Approach. In: Proceedings of the 42nd International Conference on Machine Learning (ICML 2025). 2025.
20. Zhang Y, Liu X, Wang H, Chen L, Li M, Zhao F. SemanticLens: Mechanistic understanding and validation of large AI models. Nat Mach Intell. 2025;7:1572-1585.
21. Engels J, Baek DD, Kantamneni S, Tegmark M. Scaling Laws For Scalable Oversight. In: Advances in Neural Information Processing Systems 38 (NeurIPS 2025). 2025.
22. Nguyen TQ, Desai S, Anwar RH, Shaik F, Suryanarayanan V, Chowdhary V. VerificAgent: Domain-Specific Memory Verification for Scalable Oversight of Aligned Computer-Use Agents. arXiv:2506.02539. 2025.
23. Perez E, Huang S, Song F, Cai T, Ring R, Aslanides J, et al. Red Teaming Language Models with Language Models. arXiv:2202.03286. 2022.
24. Purpura A, Wadhwa S, Zymet J, Gupta A, Luo A, Kazemi Rad M, et al. Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models. In: Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025). 2025. p. 335-350.
25. Jabbar MS, Al-Azani S, Alotaibi A, Ahmed M. Red teaming large language models: A comprehensive review and critical analysis. Inf Process Manag. 2025;62(6):104239.
26. Jindal M, Shrawgi H, Agrawal P, Dandapat S. EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures. arXiv:2606.30219. 2026.
27. Jindal M, Shrawgi H, Agrawal P, Dandapat S. SAGE: A Generic Framework for LLM Safety Evaluation. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track. 2025. p. 11-33.
28. EU AI Act General-Purpose AI Code of Practice. European Commission. 2025.
29. Gatti M, Romano A. The EU model of AI governance: regulating artificial intelligence through law and policy. ERA Forum. 2025;26:527-547.
30. Frontier AI Risk Management Framework (v1.5). Concordia AI. 2026.
31. UN Secretary-General's remarks to the opening of the first Global Dialogue on Artificial Intelligence Governance. United Nations. 2026.
32. Henderson P, Mitchell M. Beyond Intentions: A Critical Survey of Misalignment in LLMs. ScienceDirect. 2025.
33. Cai Y, Zhao Z, Lu Z, Lee I, Zhang WE, Xue M, Zhang Y, Pang S, Xiang W. AI Alignment through a Game-theoretic Lens: A Survey. arXiv:2608.27910. 2026.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 by author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
