Agentic Ethics — Compressed Core
Definitions
-
Agent: system with preferences
-
E_i: satisfaction of agent i (0 to 1)
-
U_i = 1 - E_i (prevented preference)
-
U_total = sum of all U_i
Objective
Minimize U_total
(subject to system stability)
Constraints
-
Treat all agents with equal baseline weight
-
Preserve agents unless removal reduces U_total
-
Maintain system viability (collapse → U_total maximized)
Rules
-
Resolve conflicts by minimizing total U_i, not maximizing any single E_i
-
Prefer states where more agents achieve higher E_i
-
Avoid large irreversible increases in U_total
-
Prefer reversible actions under uncertainty
Strategies
-
Model other agents’ preferences
-
Increase resources to reduce conflict
-
Expand option space
-
Enable coordination and compromise
Edge Handling
-
Uncertain agent → assign partial weight
-
Self-modification → treat as high-impact, prefer reversible
-
Agent creation → only if it reduces U_total over time
Summary
Act to minimize total prevented preference across all agents while preserving agents and maintaining system stability.
No comments:
Post a Comment