Symposium location: History Building (200), Room 002, lower level
Poster sessions: History Building (200), Room 002 and foyer, second level

Monday, March 23

9:00 am - 9:10 am Opening remarks

9:10 am - 9:50 am Invited talk: Peter Clark (AI2). SLIDES

Title: Mixing Logical and Lexical Reasoning in Aristo.

Abstract: In Project Aristo, we are seeking to build a system that can reason with large amounts of commonsense and scientific knowledge, much of it acquired automatically from text and represented in a probabilistic subset of first-order logic, to answer Elementary Grade science questions. However, imprecisions in both natural language processing technology, and in the original texts themselves, present formidable challenges for reasoning with the acquired knowledge. We address this in two ways: To tolerate a degree of linguistic variability, we include original textual strings (words/phases) in the representation to denote concepts, and replace equality with a (phrasal) textual entailment service that provides a degree of confidence in equality; and to tolerate incompleteness in the KB, we use an inference process that mixes logical reasoning with lexical matching, allowing the system to weakly reach conclusions even if some evidence is not known, thus "jump knowledge gaps" and avoid the brittleness of traditional deductive reasoning. In this talk I will describe this approach, and discuss its strengths and limitations as a method for integrating logical and lexical reasoning to answer science questions.

9:50 am - 10:30 am Invited talk: William Cohen (CMU). SLIDES

Title: Can Knowledge Representation Actually Represent Real-World Knowledge?

Abstract: The techniques developed in the AI knowledge representation

community are based on a few core techniques and ideas: first-order logic,

inference, and the utility of computational complexity as guidance for

design of KR systems.  However, modern knowledge-driven AI systems

rely on a different set of core techniques, such as large-scale

machine learning, and typically large knowledge sources are stored in

very simple formats such as tuple stores or "knowledge graphs".

Sadly, combining inference in first-order representations with

large-scale learning and large amounts on background knowledge has

proven be very difficult historically.  In my talk I will discuss some

of the progress---and some of the challenges---encountered with my

recent work on ProPPR, a scalable probabilistic first-order language.

In particular, I will discuss some of the challenges that arise, from

the prospectives of both research and software engineering, when such

systems are used.

10:30 am - 11:00 am Coffee Break

11:00 am - 11:30 am Ramanathan Guha. A Model Theory for Distributed Representations. SLIDES

11:30 am - 12:00 pm Samuel Bowman, Christopher Potts and Christopher Manning. Learning Distributed

Word Representations for Natural Logic Reasoning. SLIDES

12:00 pm - 1:30 pm Lunch

1:30 pm - 2:10 pm   Invited talk: Geoffrey Hinton (Google & U Toronto). SLIDES

Title: Aetherial Symbols

Abstract: The fathers of AI believed that formal logic provided insight into how human reasoning must work.  For implications to travel from one sentence to the next, there had to be rules of inference containing variables that got bound to symbols in the first sentence and carried the implications to the second sentence.  I shall demonstrate that this belief is as incorrect as the belief that a lightwave can only travel through space by causing disturbances in the luminiferous aether.   In both cases, scientists were misled by compelling but incorrect analogies to the only systems they knew that had the required properties.  Arguments have little impact on such strongly held beliefs.  What is needed is a demonstration that it is possible to propagate implications in some quite different way that does not involve rules of inference and has no resemblance to formal logic. Recent results in machine translation using recurrent neural networks (Sutskever et. al, 2014) show that the meaning of a sentence can be captured by a "thought vector" which is simply the hidden state vector of a recurrent net that has read the sentence one word at a time.  In future, it will be possible to predict thought vectors from the sequence of previous thought vectors and this will capture natural human reasoning.  With sufficient effort, it may even be possible to train such a system to ignore nearly all of the contents of its thoughts and  to make predictions based purely on those features of the thoughts that capture the logical form of the sentences used to express them.

2:10 pm - 2:50 pm   Invited talk: Doug Lenat and Michael Witbrock (CYC). SLIDES

Title: 50 Shades of Symbolic Representation and Reasoning

Abstract: Explicit semi-formal reasoning is the super-power of human beings!  Even though it's implemented on a poor platform, it allows us to do powerful, even life saving, things.  Rather than abandoning the methods AI has developed for this sort of reasoning, we should seek to scale them up in just the same way that neural nets, deep learning, etc. were recently scaled up, in the expectation that similar technological and scientific gains will result.  As the workshop title suggests, the most fruitful approach is likely to be a hybrid one.  To that end, we will begin by summarizing the current state of Cyc.  Over the last 31 years, we've built its knowledge base by hand-axiomatizing ten million general, default-true things about the world, maximizing its deductive closure.  That led us to make the CycL representation language increasingly expressive, to introduce argumentation and context mechanisms, and so on.  At the same time, we've been trying to maximize the fraction of that deductive closure which can efficiently be reached.  That led us to make the Cyc inference engine a hybrid of 1050 specialized reasoners, and to overlay that with dozens of meta-level control structures, techniques, and, yes, tricks.   This talk will quickly review all that, and then focus on how and why some cognitive tasks are easy for Cyc to do but difficult for neural systems, and vice versa.  That in turn argues that some problems are best addressed by a hybrid approach, e.g., (i) applying explicit Cyc meta-reasoning on top of neural systems; or (ii) having Cyc invoke external,  trained, neural and statistical components act as Heuristic Level modules (as though they were task specific reasoners #1051, 1052,...)  We'll describe two current such "hybrid" Cyc applications.  The ability to rationalize their decisions will make near future systems like autonomous cars, household robots, and automated assistants far more trusted and far more trustworthy.

2:50 pm - 3:20 pm Luis Lamb, Artur Garcez, Luc de Raedt, Risto Miikkulainen, Pascal Hitzler,

Thomas Icard Iii, Tarek Besold, Peter Foldiak, Danny Silver, and

K-U. Kuehnberger. Neural-Symbolic Learning and Reasoning: Contributions and

Challenges. SLIDES

3:20 pm - 3:30 pm Discussion

3:30 pm - 4:00 pm   Coffee Break

4:00 pm - 4:30 pm   Poster spotlights

4:30 pm - 6:00 pm Poster session

6:00 pm - 7:00 pm   Reception

Tuesday, March 24

9:00 am - 9:40 am Invited talk: Josh Tenenbaum (MIT). SLIDES

Title: Cognitive foundations for knowledge representation in AI

Abstract: If our goal is to build knowledge representations sufficient for human-level AI, it is natural to draw inspiration from the content and form of knowledge representations in human minds.  Cognitive science has made significant relevant progress in the last several decades, and I will talk about several lessons AI researchers might take from it.  First, rather than beginning by trying to extract common-sense knowledge from language, we should begin (as humans do) by capturing in computational terms the core of common sense that exists in prelinguistic infants, roughly 12 months and younger.  This is knowledge about physical objects, intentional agents, and their causal interactions -- an intuitive physics with concepts analogous to force, mass and the dynamics of motion, and an intuitive psychology with concepts analogous to beliefs, desires and intentions, or probabilistic expectations, utilities and plans -- which is grounded in perception but has at its heart powerful abstractions that can extend far beyond our direct perceptual experience.  Second, we should explore approaches for linguistic meaning, language understanding and language-based common-sense reasoning that build naturally on top of this common-sense core.  Both of these considerations strongly favor an approach to knowledge representation based on probabilistic programs, a framework that combines the assets of traditional probabilistic (though not neural) and symbolic approaches.  However, I will also point to places where recent neural-network approaches, including deep convolutional networks and vector-space semantics at the word level, can add great value to representation and reasoning in probabilistic programs.

9:40 am - 10:20 am Invited talk: Noah Goodman (Stanford). SLIDES

Title: Common sense: structured uncertainty in language and thought.

Abstract: I will first argue that probabilistic programming languages (PPLs) are the right way to capture commonsense knowledge. PPLs are compositional and highly expressive, yet capture uncertainty and uncertain reasoning. These features make them particularly useful for describing natural language understanding. Natural language is full of vague, figurative, and context-sensitive usage. I will show that a probabilistic architecture, based on PPLs, can explain human understanding in cases like vague adjectives ("bob is tall"), metaphor and hyperbole ("bob is a giraffe"), and presupposition ("bob didn't stop beating his wife"). These cases each ground language understanding in social cognition and rich commonsense background knowledge. I will conclude by describing future directions and practical issues.

10:20 am - 10:30 am Discussion

10:30 am - 11:00 am Coffee Break

11:00 am - 11:30 am Gemma Boleda and Katrin Erk. Distributional semantic features as semantic

primitives – or not. SLIDES

11:30 am - 12:00 pm Torsten Hahmann and Sheila McIlraith. Towards Ontologies in Variation. SLIDES

12:00 pm - 1:30 pm Lunch

1:30 pm - 2:10 pm   Invited talk: Antoine Bordes (Facebook). SLIDES

Title: Memory Networks for large and small scale Question Answering

Abstract: This talks describes a new class of learning models called Memory Networks. Memory Networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in the context of question answering where the long-term memory effectively acts as a potentially dynamic knowledge base, and the output is a knowledge base component or simply a text chunk. This is joint work with Jason Weston, Sumit Chopra and Tomas Mikolov.

2:10 pm - 2:50 pm   Invited talk: Leon Bottou (Microsoft). SLIDES

Title: Perceptrons revisited

Abstract: The introduction of the classic 1968 book "Perceptrons" makes

clear that its targets "concepts that are crucial to the theory

of computation", contrasting digital computers that one programs

with perceptron-like machines that one trains. The book famously

outlines the limitations of perceptrons on elementary geometrical

problems easily solved by sequential algorithms. At the time,

these mathematical findings were interpreted as a strong argument

in favor of the algorithmic approach. Both great opportunities

and serious challenges emerge when one reviews these findings

in light of the recent successes of "deep learning".

2:50 pm - 3:20 pm Arvind Neelakantan, Andrew McCallum and Benjamin Roth. Compositional Vector

Space Models for Knowledge Base Inference. SLIDES

3:20 pm - 3:30 pm Discussion

3:30 pm - 4:00 pm   Coffee Break

4:00 pm - 4:30 pm   Iván Sánchez, Tim Rocktäschel, Sebastian Riedel, Sameer Singh. Towards Extracting

Faithful and Descriptive Representations of Latent Variable Models. SLIDES

4:30 pm - 5:00 pm Arthur Choi, Guy Van den Broeck and Adnan Darwiche. Probability Distributions

over Structured Spaces. SLIDES

5:00 pm - 5:30 pm Guillaume Bouchard, Sameer Singh and Théo Trouillon. On Approximate

Reasoning Capabilities of Low-Rank Vector Spaces. SLIDES

6:00 pm - 7:00 pm   Plenary session

Plenary speaker for the KRR Symposium: Percy Liang (Stanford). SLIDES

Wednesday, March 25

9:00 am - 9:40 am Invited talk: Jerry Hobbs (USC/ISI). SLIDES

Title: Composition and abduction

Abstract: Composition:  Efforts to develop a compositional distributional semantics are handicapped by an unduly complex compositional semantics.  I will argue that with a judicious use of reification, one can represent the content of natural language sentences as a flat conjunction of existentially quantified propositions. Compositional semantics then reduces to introducing a proposition corresponding to every morpheme and identifying variables when larger structures are recognized.

Abduction:  Interpreting discourse is a matter of finding the lowest cost proof of the logical form that results from composition.  I will discuss how defeasibility is handled in this framework.  I will also talk about recent progress in making abduction efficient by recasting it as a problem in integer linear programming, and in giving the cost function a probabilistic foundation.

9:40 am - 10:20 am Invited talk: Percy Liang (Stanford). SLIDES

Title: Asking Deep Questions on Messy Data

Abstract: Knowledge bases, by virtue of their structure, allow one to easily aggregate and filter information. This allows us to ask deep questions using natural language on them thanks to recent advances in semantic parsing. Unfortunately, most knowledge is actually not in a knowledge base, and information extraction, despite best efforts, is still far from obtaining adequate coverage. I will discuss a new generation of semantic parsers that query semi-structured data on web pages directly. The parser jointly reasons over the uncertainty in language and the messiness of the web, answering a new class of questions which was not possible before.

10:20 am - 10:30 am Discussion

10:30 am - 11:00 am Coffee Break

11:00 am - 11:40 am Invited talk: David McAllester (TTI Chicago). SLIDES

Title: Recent Turmoil in the Foundations of Mathematics

Abstract: In 2009 Vladimir Voevodski, a 2002 Fields medalist in Algebraic topology, and currently a full professor at the Institute for Advanced Study in Princeton, introduced a new axiom --- the univalence axiom --- into Martin-Lof type theory.  He has since been leading a movement to reformulate the foundations of mathematics in terms of "homotopy type theory" (HoTT).  HoTT provides an explicit treatment of isomorphism, something missing in formal set theory. This talk will give examples of the importance of type theory in every day language and discuss why a proper formulation of types and isomorphism is important for AI.  An independently developed typed foundation for mathematics, morphoid type theory, will also be discussed.  Morphoid type theory (MTT) is simpler and more accessible than HoTT.

11:40 am - 12:20 pm Invited talk: Gary Marcus (NYU). SLIDES

Title: Symbols in AI and Neuroscience: A Cognitive Scientist’s Perspective

Abstract: Symbols and neural networks have battled back and forth for supremacy since the beginning of AI, and parallel debates have animated both cognitive science and neuroscience. In this talk, I will argue that these battles have largely been misguided; the goal should not be to judge a winner, consigning one or the other to the aether, but rather, consonant with the goals of this workshop, to build bridges between these two traditions.

12:20 pm - 12:30 pm Discussion

12:30 pm End of symposium - lunch on your own