What is AI missing?

This is a post-question, in which I tried to formulate the main problems of neural networks, the solution of which can make a breakthrough in AI technology. Basically, we are talking about networks that work with text (GPT, BERT, ELMO, etc.). As you know, a good formulation of a problem is half of its solution. But I myself cannot find these solutions. I hope for the “help of the audience”, as there are many people here who face the same problems and possibly “see” their solution.



So.



1. The most seemingly simple, but the neural network does not take into account the facts. The neural network learns from particular facts, but does not seem to know about them. In cognitive language, NN has semantic, not episodic memory, roughly speaking.

The solution may be simple, but a neural network is a classifier, and precedents cannot be classes, a contradiction. And often just such a response from bots is needed, they work very badly with facts, if we are not talking about the "response" template. The problem is compounded by the fact that there are always exceptions that the network cannot take into account if it hasn't had enough exception examples. And if there are enough examples, this is no exception. In general, NN can say that this is a hat, but he cannot say which hat is mine (there was only one example).



2. "Common Sense". A well-known problem, even called "AI dark matter". There are interesting approaches to the solution, for example, in this article, which describes an attempt to combine symbolic (logical) AI and neural network approaches. But this is an attempt to go back instead of forward. The problem is that "common sense" is implicit knowledge about the world, which was not in the training dataset. Nobody even pronounces such platitudes, they are recognized at 4-6 years old, when they still cannot write. The high-profile failures of the Kompreno and Cyc projects show that it is impossible to clearly describe all the facts. They are somehow displayed on the fly. There are no good solution ideas yet, except for the limitation of the vocabulary. For example, a “schoolboy” should “direct” such “filters” to the vocabulary of the answer so that the chosen options do not contain the words “army” or “marriage” if it is about himself, and not about the presence of his elder brother at the wedding. How to do this in NN is not (to me) clear.



3.An equally important problem, and possibly related to the previous one, is the problem of constructing reasoning. Neural networks do not know how to make syllogisms, that is, the simplest conclusions with consistent reasoning (intermediate conclusions). The same problem, on the other hand, is the inability to pursue the goal of reasoning or at least adhere to a certain meaning. GPT can build news text on a given topic, but it is useless to say, "write news to denigrate X". In the best case, she will write about the denigration by others, and in an explicit form, and not like we humans, between the lines. The conclusion of the syllogism is also a goal - it is necessary to correlate the premises with the conclusion. Have it in mind at the first utterance (premise). It is not even clear "from which side" this should be put into the network. Maybe someone knows?



4.And another problem, which is not even dark matter, but an AI black hole. These are analogies and metaphors. AI understands everything only literally. It is useless for her to say "like X". The network can add to the description, but not describe the analogue. Maybe it's just a problem with the corresponding dataset. But it seems to me that it is deeper and shows the root "flaw" of the current AI architectures as well as item 3. Our language is entirely metaphorical, and hence the "curse of linguists" - homonymy, grows from here. The same lexemes are used through metaphors in a bunch of different "concepts". And we can easily navigate in this. This is partially solved in the task of defining intents, but this is again the definition of the "theme", and not the whole concept, which consists not only of the name of the intent and associated response templates as in bots.



So far, these four will be enough for discussion, although there are more specific, but no less important problems in building bots, for example. It is enough to chat with Alice and they become intuitively obvious. But with their wording, everything is not so simple - to guess what the problem is to guess and how to solve it. This is more difficult. Thanks for the constructive comments on the topic.