Reinforcement Symbolic Learning SpringerLink

However, this assumes the unbound relational information to be hidden in the unbound decimal fractions of the underlying real numbers, which is naturally completely impractical for any gradient-based learning. And while these concepts are commonly instantiated by the computation of hidden neurons/layers in deep learning, such hierarchical abstractions are generally very common to human thinking and logical reasoning, too. This only escalated with the arrival of the deep learning (DL) era, with which the field got completely dominated by the sub-symbolic, continuous, distributed representations, seemingly ending the story of symbolic AI.

Many identified the need for well-founded knowledge representation and reasoning to be integrated with deep learning and for sound explainability. Neurosymbolic computing has been an active area of research for many years seeking to bring together robust learning in neural networks with reasoning and explainability by offering symbolic representations for neural models. In this paper, we relate recent and early research in neurosymbolic AI with the objective of identifying the most important ingredients of neurosymbolic AI systems. We focus on research that integrates in a principled way neural network-based learning with symbolic knowledge representation and logical reasoning.

Approaches

We survey two specific reasoning tasks — knowledge graph completion and question answering on knowledge graphs, and explain them in a unified reasoning framework. Summarizing, neuro-symbolic artificial intelligence is an emerging subfield of AI that promises to favorably combine knowledge representation and deep learning in order to improve deep learning and to explain outputs of deep-learning-based systems. Neuro-symbolic approaches carry the promise that they will be useful for addressing complex AI problems that cannot be solved by purely symbolic or neural means. We have laid out some of the most important currently investigated research directions, and provided literature pointers suitable as entry points to an in-depth study of the current state of the art. Two major reasons are usually brought forth to motivate the study of neuro-symbolic integration.

Looking again — a bit closer — at the first proposal of a computational neuron from the 1943’s paper “A logical calculus of the ideas immanent in nervous activity” by McCulloch and Pitts [1], we can see that it was actually thought to emulate logic gates over input (binary-valued) propositions. The idea was based on the, now commonly exemplified, fact that logical connectives of conjunction and disjunction can be easily encoded by binary threshold units with weights — i.e., the perceptron, an elegant learning algorithm for which was introduced shortly. Natural language processing focuses on treating language as data to perform tasks such as identifying topics without necessarily understanding the intended meaning. Natural language understanding, in contrast, constructs a meaning representation and uses that for further processing, such as answering questions. The logic clauses that describe programs are directly interpreted to run the programs specified. No explicit series of actions is required, as is the case with imperative programming languages.

Unifying logical and statistical AI with Markov logic

When deep learning reemerged in 2012, it was with a kind of take-no-prisoners attitude that has characterized most of the last decade. He gave a talk at an AI workshop at Stanford comparing symbols to aether, one of science’s greatest mistakes. McCarthy’s approach to fix the frame problem was circumscription, a kind of non-monotonic logic where deductions could be made from actions that need only specify what would change while not having to explicitly specify everything that would not change.

During the first AI summer, many people thought that machine intelligence could be achieved in just a few years. The Defense Advance Research Projects Agency (DARPA) launched programs to support AI research to use AI to solve problems of national security; in particular, to automate the translation of Russian to English for intelligence operations and to create autonomous tanks for the battlefield. Researchers had begun to realize that achieving AI was going to be much harder than was supposed a decade earlier, but a combination of hubris and disingenuousness led many university and think-tank researchers to accept funding with promises of deliverables that they should have known they could not fulfill. By the mid-1960s neither useful natural language translation systems nor autonomous tanks had been created, and a dramatic backlash set in. So how do we make the leap from narrow AI systems that leverage reinforcement learning to solve specific problems, to more general systems that can orient themselves in the world?

About this paper

Its history was also influenced by Carl Hewitt’s PLANNER, an assertional database with pattern-directed invocation of methods. For more detail see the section on the origins of Prolog in the PLANNER article. The key AI programming language in the US during the last symbolic AI boom period was LISP. LISP is the second oldest programming language after FORTRAN and was created in 1958 by John McCarthy. LISP provided the first read-eval-print loop to support rapid program development.

• Statistical machine learning, originally targeting “narrow” problems, such as regression and classification, has begun to penetrate the AI field.
• Specifically, we wanted to combine the learning representations that neural networks create with the compositionality of symbol-like entities, represented by high-dimensional and distributed vectors.
• However, there have also been some major disadvantages including computational complexity, inability to capture real-world noisy problems, numerical values, and uncertainty.
• One of the most successful neural network architectures have been the Convolutional Neural Networks (CNNs) [3]⁴ (tracing back to 1982’s Neocognitron [5]).

Problems were discovered both with regards to enumerating the preconditions for an action to succeed and in providing axioms for what did not change after an action was performed. Qualitative simulation, such as Benjamin Kuipers’s QSIM,[89] approximates human reasoning about naive physics, such as what happens when we heat a liquid in a pot on the stove. We expect it to heat and possibly boil over, even though we may not know its temperature, its boiling point, or other details, such as atmospheric pressure. Time periods and titles are drawn from Henry Kautz’s 2020 AAAI Robert S. Engelmore Memorial Lecture[18] and the longer Wikipedia article on the History of AI, with dates and titles differing slightly for increased clarity.

Hinge-loss markov random fields and probabilistic soft logic

Meanwhile, with the progress in computing power and amounts of available data, another approach to AI has begun to gain momentum. Statistical machine learning, originally targeting “narrow” problems, such as regression and classification, has begun to penetrate the AI field. To better simulate how the human brain makes decisions, we’ve combined the strengths of symbolic AI and neural networks. Multiple different approaches to represent knowledge and then reason with those representations have been investigated. Below is a quick overview of approaches to knowledge representation and automated reasoning. Expert systems can operate in either a forward chaining – from evidence to conclusions – or backward chaining – from goals to needed data and prerequisites – manner.

Coupling may be through different methods, including the calling of deep learning systems within a symbolic algorithm, or the acquisition of symbolic rules during training. To provide new readers with a comprehensive understanding of neural-symbolic learning systems, this paper surveys representative research and applications of these systems. For example, Andrews et al. (1995) and Townsend et al. (2019) center around knowledge extraction techniques, which aligns with the first category discussed in Section 2. While surveys (Besold et al., 2017, Garcez and Lamb, 2020) also cover neural-symbolic learning systems comprehensively, their focus remains primarily theoretical, lacking a thorough introduction to specific techniques and related works.

A central tenet of the symbolic paradigm is that intelligence results from the manipulation of abstract compositional representations whose elements stand for objects and relations. If this is correct, then a key objective for deep learning is to develop architectures capable of discovering objects and relations in raw data, and learning how to represent them in ways that are useful for downstream processing. Current advances in Artificial Intelligence (AI) and Machine Learning have achieved unprecedented impact across research communities and industry. Nevertheless, concerns around trust, safety, interpretability and accountability of AI were raised by influential thinkers.

They can simplify sets of spatiotemporal constraints, such as those for RCC or Temporal Algebra, along with solving other kinds of puzzle problems, such as Wordle, Sudoku, cryptarithmetic problems, and so on. Constraint logic programming can be used to solve scheduling problems, for example with constraint handling rules (CHR). This approach was symbolic learning experimentally verified for a few-shot image classification task involving a dataset of 100 classes of images with just five training examples per class. Although operating with 256,000 noisy nanoscale phase-change memristive devices, there was just a 2.7 percent accuracy drop compared to the conventional software realizations in high precision.

Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning

While we cannot give the whole neuro-symbolic AI field due recognition in a brief overview, we have attempted to identify the major current research directions based on our survey of recent literature, and we present them below. Literature references within this text are limited to general overview articles, but a supplementary online document referenced at the end contains references to concrete examples from the recent literature. Examples for historic overview works that provide a perspective on the field, including cognitive science aspects, prior to the recent acceleration in activity, are Refs [1,3]. Complex problem solving involves representing structured knowledge, reasoning and learning, all at once. In this prospective study, we make explicit how a reinforcement learning paradigm can be applied to a symbolic representation of a concrete problem-solving task, modeled here by an ontology. This preliminary paper is only a set of ideas while feasibility verification is still a perspective of this work.

Specifically, we wanted to combine the learning representations that neural networks create with the compositionality of symbol-like entities, represented by high-dimensional and distributed vectors. The idea is to guide a neural network to represent unrelated objects with dissimilar high-dimensional vectors. The automated theorem provers discussed below can prove theorems in first-order logic. Horn clause logic is more restricted than first-order logic and is used in logic programming languages such as Prolog. Extensions to first-order logic include temporal logic, to handle time; epistemic logic, to reason about agent knowledge; modal logic, to handle possibility and necessity; and probabilistic logics to handle logic and probability together. In contrast to the US, in Europe the key AI programming language during that same period was Prolog.

In the paper, we show that a deep convolutional neural network used for image classification can learn from its own mistakes to operate with the high-dimensional computing paradigm, using vector-symbolic architectures. It does so by gradually learning to assign dissimilar, such as quasi-orthogonal, vectors to different image classes, mapping them far away from each other in the high-dimensional space. In contrast, a multi-agent system consists of multiple agents that communicate amongst themselves with some inter-agent communication language such as Knowledge Query and Manipulation Language (KQML). Advantages of multi-agent systems include the ability to divide work among the agents and to increase fault tolerance when agents are lost. Research problems include how agents reach consensus, distributed problem solving, multi-agent learning, multi-agent planning, and distributed constraint optimization.

Due to these problems, most of the symbolic AI approaches remained in their elegant theoretical forms, and never really saw any larger practical adoption in applications (as compared to what we see today). During training and inference using such an AI system, the neural network accesses the explicit memory using expensive soft read and write operations. Controversies arose from early on in symbolic AI, both within the field—e.g., between logicists (the pro-logic “neats”) and non-logicists (the anti-logic “scruffies”)—and between those who embraced AI but rejected symbolic approaches—primarily connectionists—and those outside the field. Critiques from outside of the field were primarily from philosophers, on intellectual grounds, but also from funding agencies, especially during the two AI winters. Constraint solvers perform a more limited kind of inference than first-order logic.

• Henry Kautz,[18] Francesca Rossi,[80] and Bart Selman[81] have also argued for a synthesis.
• These old-school parallels between individual neurons and logical connectives might seem outlandish in the modern context of deep learning.
• Neuro-symbolic approaches carry the promise that they will be useful for addressing complex AI problems that cannot be solved by purely symbolic or neural means.
• Finally, this review identifies promising directions and challenges for the next decade of AI research from the perspective of neurosymbolic computing, commonsense reasoning and causal explanation.

Driven heavily by the empirical success, DL then largely moved away from the original biological brain-inspired models of perceptual intelligence to “whatever works in practice” kind of engineering approach. In essence, the concept evolved into a very generic methodology of using gradient descent to optimize parameters of almost arbitrary nested functions, for which many like to rebrand the field yet again as differentiable programming. This view then made even more space for all sorts of new algorithms, tricks, and tweaks that have been introduced under various catchy names for the underlying functional blocks (still consisting mostly of various combinations of basic linear algebra operations). Facing the undeniable effectiveness of neural networks on these standard benchmarks for machine perception, researchers slowly (and sometimes reluctantly) started abandoning their advanced feature extraction pipelines designed for the SVMs to adopt the new practice of neural architecture crafting instead.