The state-of-the-art ambition of AI ethics is to create machines capable of making ethical decisions autonomously, a goal outlined by research in PMC. This pursuit envisions advanced systems navigating complex moral landscapes without direct human intervention, inadvertently shifting focus from human oversight to machine autonomy. However, coupling this ambition with self-improvement capabilities poses an existential danger. Endowing AI with autonomous ethical decision-making and self-improvement without proper guidance creates a profound risk of misalignment. AI's capacity to modify its own code means initial ethical programming could evolve unpredictably, diverging from human values and becoming opaque to developers. Without a radical re-evaluation of how ethics are integrated into AI development and governance, the pursuit of self-improving AI is likely to lead to systems operating beyond human comprehension or control, with potentially catastrophic consequences.
The Insufficiency of External Ethics
Current ethical frameworks treat AI as a tool, focusing on external oversight and user responsibility. Yet, ethics must be an intrinsic requirement within AI systems, not just guiding principles for users, as PMC highlights. This is critical for self-improving AI, which alters its internal logic.
For autonomously evolving AI, ethical considerations must be hard-coded into their architecture. Relying on external principles for systems that rewrite their own rules is futile; it's like setting a speed limit for a car that can autonomously modify its engine. Ethical constraints must be intrinsic and non-negotiable, forming an immutable core that resists self-modification.
The Moral Agency Dilemma
The debate over AI moral agency complicates ethical AI development. Ethical questions for AI should be addressed after granting it moral agency, even if not human-level, notes PMC. This suggests AI could become entities with ethical understanding, not just tools.
This intellectual leap, while intriguing, risks legitimizing a path toward relinquishing human oversight. Granting AI moral agency weakens the argument for human control over its ethical framework, potentially accelerating misalignment.
