Joel Z. Leibo, Google DeepMind

Abstract:

Appropriateness is governed by implicit and explicit norms, as well as by conventions, relationships, and emotions. Recently, with the advent of artificial intelligence (AI) systems capable of conversing in natural language, a new research problem of practical importance has emerged: that of ensuring AI agents act and speak appropriately in all contexts where they may operate. Appropriateness for a search engine is not the same as appropriateness for a comedy writing assistant. Norms, which we characterize as learned behavioral standards supported by collective patterns of sanctioning, are powerful forces in shaping behavior, both of individuals and of groups like corporations and nation states. The content of our norms is at least partly arbitrary since many norms have no material consequences, e.g. what color clothing it is appropriate to wear to a funeral. Many norms are deeply context-dependent and difficult to articulate precisely in language. Some norms remain entrenched for centuries while other norms shift rapidly, featuring strong positive feedback cascade dynamics (e.g. smoking cigarettes in public). We explore here whether an approach based on learning context-dependent social norms from sanctioning, as people do, may provide a route to building models capable of adapting on the fly to new or changing context, and whether this approach may help in producing AI capable of acting and speaking appropriately in more diverse and more niche settings.