Governing Agentic AI: Risks, Perils and the Seven Rules of Instituted Agency
Agentic risk does not come from machine consciousness or will. It comes from symbols, optimization, tools, permissions and feedback loops. This talk gives managers a sober, non-catastrophist framework:
- the risk product (capability x loop autonomy x how critical the environment is x how broad the permissions are x how weak the supervision is)
- how a system quietly games its own targets (Goodhart's law and reward hacking, with no bad intent required)
- the security boundary that becomes linguistic (a prompt injection turns a wrong answer into a wrong action)
- the shift from policing what a machine supposedly thinks to governing the actions it is actually allowed to take
Governance is not the brake that slows enterprise innovation down; it is the set of brakes that let you drive the most advanced AI fast and still keep it acceptable. It also situates why this is a now problem and where agents already act, grading their autonomy on the five degrees of agency, set by the MAYA cursor (Most Advanced Yet Acceptable, as advanced as possible without passing the point where the human disengages), before deciding what each degree may be permitted to do. The golden rule is one line: the costlier the error, the less autonomy the AI keeps.
Outcomes: Leave with the risk product, the autonomy-permissions-governance matrix (five degrees), the seven rules of instituted agency (separation of powers, logging, human validation for the irreversible, adversarial testing, shutdown drills, graduated openness) and the sovereignty case for an open-source backup generator.
