Getting machines to disobey
Marija Slavkovik (University of Bergen)
Getting machines to disobey
Marija Slavkovik (University of Bergen)
An intelligent agent should be able to disobey the norms of its environment. A smart device should be able to refuse compliance with a user request. A chatbot should not be sycophantic. We define disobedience as an act of intentional norm violation and we postulate the distinctions among six types of disobedience: direct violation, justified exception, civil disobedience, trolling, non-compliance and whistleblowing. Each type requires a distinct monitoring workflow, but most importantly each requires a reasoning process. How does a machine decide to disobey and how should an environment handle this via a good governance framework. This talk considers the need for machine disobedience, the types of disobedience and the dual perspective: agents use reason based practical reasoning to decide whether to obey or disobey, while the governance framework processes observable outcomes and routes them into differentiated institutional responses.