Why is it impossible for a program or AI to have semantic understanding?

Question

relatively new to philosophy.

This question is based on John Searle's Chinese Room Argument.

I find it odd that his main argument for why programs could not think was that because programs could only follow syntax rules but could not associate any understanding or semantics to words( or any object/symbol).

This point seems contestable to me (although I can't quite word it well enough). How is John so certain that it would be impossible for a program to understand semantics? Is mimicking semantic understanding actually different from genuine semantic understanding?

What does philosophy say about whether it is truly impossible for mankind to one day develop a program capable of semantic understanding? According to Turing's Same-Evidence Argument, if a computer can pass the test, we would have to assume that it is capable of understanding. Could we even distinguish between simulated mimicking understanding and actual understanding?

Edit: wow this really blew up. Thank you for the answers!

Sidenote:

I posited this question partly because John's visualization of what a program looks like seems flawed to me. He is able to clearly break-down and visualize a program (the room) with a 'core' (the man in the room) in the middle that handles the inputs and produces the outputs.

However, complex algorithmic programs aren't designed in such a simple manner. Take Artifical Neural Networks for example, which are said to be "black-boxes" due to the fact that we can't break down a neural network into components to deduce how it decides to give certain outputs. John's arguement seems built on the fact that we could 'peer' into how programs/algorithms make decisions when that isn't necessarily true.

Chess algorithms such as the infamous Deep Blue and Alpha Zero sometimes produce moves that professional chess players fail to consider. Would John argue that these algorithms "fail to understand Chess?". It seems flawed to say that the program fails in semantic understanding when it can display creativity which human chess players themselves may lack.

Answer 1

There is a blatant problem with Searle’s argument and it’s quite hard to understand why it hasn’t been pointed out before: None of Mr. Searle’s brain cells understands English, yet he claims that he can? What argument can he make that an AI can’t reverse and throw right into his face?

Answer 2

I find it odd that his main argument for why programs could not think was that because programs could only follow syntax rules but could not associate any understanding or semantics to words( or any object/symbol).

That was more his conclusion than his argument. His actual argument about the Chinese Room thought-experiment was that if the room was occupied by a conscious agent who is perfectly capable of semantic understanding, like a person, and they were to execute the syntactical rules of a Chinese-speaking program by hand (or from memory), they would nevertheless lack any semantic understanding of Chinese. For example, in the SEP article on the Chinese Room that you linked to, it quotes Searle giving a summary of the argument in 1999 where he says (emphasis mine):

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

And it quotes a later 2010 statement where he said:

A system, me, for example, would not acquire an understanding of Chinese just by going through the steps of a computer program that simulated the behavior of a Chinese speaker

I also found his original 1980 paper on the subject online here, where he imagined that he was a native English speaker in the room responding to English questions in a natural way, and responding to Chinese questions based on hand-simulating an elaborate computer program, and his argument was based on the contrast between his own understanding in the first case with his lack of understanding in the second:

Now the claims made by strong AI are that the programmed computer understands the stories and that the program in some sense explains human understanding. But we are now in a position to examine these claims in light of our thought experiment.

1. As regards the first claim, it seems to me quite obvious in the example that I do not understand a word of the Chinese stories. I have inputs and outputs that are indistinguishable from those of the native Chinese speaker, and I can have any formal program you like, but I still understand nothing. For the same reasons, Schank's computer understands nothing of any stories, whether in Chinese, English, or whatever, since in the Chinese case the computer is me, and in cases where the computer is not me, the computer has nothing more than I have in the case where I understand nothing.

2. As regards the second claim, that the program explains human understanding, we can see that the computer and its program do not provide sufficient conditions of understanding since the computer and the program are functioning, and there is no understanding. But does it even provide a necessary condition or a significant contribution to understanding? One of the claims made by the supporters of strong AI is that when I understand a story in English, what I am doing is exactly the same -- or perhaps more of the same -- as what I was doing in manipulating the Chinese symbols. It is simply more formal symbol manipulation that distinguishes the case in English, where I do understand, from the case in Chinese, where I don't.

There have been various responses to the argument by philosophers who don't find it convincing, see section 4 of the SEP article. The one I think is the most convincing refutation is the "systems reply", which basically says that the boundaries of "systems" are somewhat arbitrary and that a given named physical system may have multiple computational sub-processes going on within it that could be sufficiently independent that each might individually have semantic understanding of certain things and yet lack understanding of things that the other sub-process does understand. To pick an extreme case, imagine some alien species that is naturally two-headed, with independent brains that have no neural connections between them--although both brains might be considered to be part of a single biological "system" we wouldn't be surprised if one brain could understand something (say, the Chinese language) that the other was ignorant of. And even if there were some neural connections between them, they might not be of the right configuration to ensure that high-level conceptual understanding of any arbitrary topic would necessarily be shared by both brains.

Here is David Chalmers giving this type of argument on p. 326 of his book The Conscious Mind, where the agent inside the room is a "demon" who may have memorization capabilities far beyond those of real-life human:

Searle also gives a version of the argument in which the demon memorizes the rules of the computation, and implements the program internally. Of course, in practice people cannot memorize even one hundred rules and symbols, let alone many billions, but we can imagine that a demon with a supermemory module might be able to memorize all the rules and the states of all the symbols. In this case, we can again expect the system to give rise to conscious experiences that are not the demon's experiences. Searle argues that the demon must have the experiences if anyone does, as all the processing is internal to the demon, but this should instead be regarded as an example of two mental systems realized within the same physical space. The organization that gives rise to the Chinese experiences is quite distinct from the organization that gives rise to the demon's experiences. The Chinese-understanding organization lies in the causal relations between billions of locations in the supermemory module; once again, the demon only acts as a kind of causal facilitator. This is made clear if we consider a spectrum of cases in which the demon scurrying around the skull gradually memorizes the rules and symbols, until everything is internalized. The relevant structure is gradually moved from the skull to the demon's supermemory, but experience remains constant throughout, and entirely separate from the experiences of the demon.

Answer 3

As I see it, Searle is getting at the point that syntax is algorithmic — a system driven by predefined rules and procedures — but semantics is (as far as we can tell) not. In other words, it's easy enough to create and recognize a syntactically well-formed sentence on purely procedural grounds, but judging the meaningfulness of a sentence requires something beyond that. I mean, compare the following utterances:

• Jarod loves potato chips
• Loves potato Jarod chips
• Jarod chips potato loves

The first is syntactically correct and clearly meaningful. The second is syntactically incorrect (it doesn't follow the procedural sentence construction rules of English). The third is syntactically correct (treating 'chip' as a verb), but of questionable meaning. What does it mean to 'chip potato loves'? Now, if you imagine those three phrases passed into the inverse of the Chinese room (a room in which a Mandarin-only speaker is processing algorithmic rules for English), that man would recognize °2 as structural nonsense, but he would make no distinction between °1 and °3. How could he?

Note that this is akin to the distinction in logic between the validity and the truth-value of a series of propositions. The first tells us nothing about the second, and vice versa.

What's missing from syntactical analysis is the ability to make meaning from ambiguity (through non-procedural processes like extension, analogy, metaphor, simplification, correlation...). You and I can sit and ponder what it means to 'chip potato loves', and sooner or later we'll assign a meaning to it. But in order to be able to assign meaning we have to assess the meaning of the individual words and find some correspondence within them. That is more a function of the practical use of words than their syntactic structure or overt dictionary definitions.

This might be clearer to see if we think in terms of humor. For instance, if we take a couple of (clearly stupid) jokes:

• Why should you never fight a dinosaur? 'Cuz you'll get jurasskicked!
• Whoever invented knock-knock jokes should get a no bell prize.

... we can see that they are both syntactically correct, but the joke lies in their odd correspondences: the link from fights to getting your ass kicked, and dinosaurs to Jurassic; the similarity of 'no bell' (meaning no doorbell, hence the need to knock) to 'Nobel' (the archetypal prize for smart people). We can program a computer to repeat these jokes, obviously, but if we fed them into our inverse Chinese room the man inside would not laugh, and would not output 'hah-hah' unless he was explicitly told to do that for these sets of symbols. To get a computer to understand the humor of these jokes (or at least the stupidity of them), we'd have to make the computer capable of wide-ranging fuzzy associations between otherwise unrelated concepts, and no one has yet developed an algorithm to do that. If they do, it will require more than syntactical analysis, so Searle's Chinese Room problem will still hold.

Answer 4

Short Answer

There's a number of positions outlined in your SEP link to Searle's Room that make clear that philosophy has not decided by consensus one way or another the question of human and semantic understanding. The history of AI is an ongoing debate, in fact, about the question. A great introduction into that history is Nils Nilsson's The Quest for Artificial Intelligence. I'll caution you that anyone who answers you strongly negatively or in the affirmative hasn't even picked up and read this book. Philosophy is undecided largely because philosophy has reached no strong consensus on what constitutes understanding. Science has not reached a consensus on semantics in the brain. That being said, computers have made gains in the last couple of decades, perhaps not demonstrative of human-level intelligence, but certainly enough to listen to you and fulfill some of your needs. Essentially, though, besides agnostics, there are two camps: those who believe in a Cartesian notion of understanding that rejects anything other than humans as being capable of human-esque intelligence, and an upstart crowd that is interested in artificial general intelligence and believes it is possible in theory. (Warning that my bias is the latter.) Whatever the case, you are firmly in the philosophy of artificial intelligence, a relatively new branch of philosophical inquiry less than 100 years old given the emergence of digital computation in the late 1930's and early 1940's.

Long Answer

Getting Your Feet Wet

The dream to animate the inanimate to conduct itself as a human goes back thousands of years straight into the mythology of the Proto-Indo-Europeans. The abridged accounting thereof also seems to be an intro in every book today that seeks to introduce AI. Obviously, you have two questions at play, one about Searle and the Chinese Room, but the larger issue of what does philosophy say about developing machines that think.

You've cited the Chinese Room Argument, which has a number of posts on this site. Start with a review of those:

• Why doesn't the Chinese room learn Chinese?
• An English room inside Searle's Chinese Room?
• Does Searle's Chinese Room model computers correctly?
• Do relationships rebut the Chinese room argument?
• Does adding structure make the Chinese room semantic?

Understanding Searle's argument and the arguments that reply to it, particular the systems response are necessary for orienting yourself. Once you've done so, if I were you, I'd reach out and get a copy of The Philosophy of Artificial Intelligence by Margaret Boden and What Computer Cant Do by Hubert Dreyfus. If you want to know see what the AGI camp is cooking up, a good recent publication called Artificial General Intelligence by Ben Goertzel and Cassio Pennachin (Eds.) offers into some of the (IMNSHO) flailing attempts to create architectures to imbue software with human-level intelligence traits.

As to the question of how can Searle be so sure? Well, John Searle is renowned for his philosophy, and that confidence may be a function of his success as a philosopher and his lack of technical sophistication as a computer scientist. John Searle's continuation of the successes of the linguistic turn in philosophy is tough to dispute. He's written a lot about how language reflects on reality, both personal and social, but I would point out that Searle has a tool he uses to deal with the complexity of the mind called The Background. He often dismisses details right into that nebulous, diffuse thing to simplify to make his points. Overall, it's an excellent strategy for narrowing down his argumentation to what deserves focus, but the downside to that is that runs the danger of making it too easy to sweep aside relative propositions since informal argument is governed by non-monotonic logic and defeasible propositions.

Human-Level Intelligence and the Nature of Thinking

The other part of your question revolves around coming to terms with just what it means to simulate understanding, particularly of language. As you are likely aware, Alan Turing is famous for many things, but among them is his Turing Test which is an attempt to operationalize human semantic intelligence. As we approach 100 years, no one has been able to do it, which in the history of artificial intelligence has often been touted as just around the corner much as fusion reactors have been constantly 30 years (Discovery Magazine) away. In fact, when Hubert Dreyfus began attacking the AI program on campus and with RAND, he noted the outright hostility of his fellow thinkers almost immediately.

Why has the promise of AI been so slow to materialize (though gains in machine intelligence have accomplished some fantastic goals lately)? Well, it boils down to what the science of linguistics has discovered about semantics. The easiest way to explain it is to say that meaning is rooted in physical embodiment, and that processing strings in a serial ALU falls short of some of the connectist nature of the human brain. These are the computational details where the question of human intelligence gets gritty and where an ignorance of the materials science and the mathematical structures of computation start to have a bearing on the philosophy of mind.

In fact, the question of what constitutes human intelligence is not only an open question in the philosophy of mind, but in psychology itself, where there are two, roughly speaking, models vying for approval, the Cattell-Horn model which is related to the G factor and is operationalized through intelligence quotient testing, and what might be called a pluralistic notion of intelligence that is most famous through Howard Gardner's MI theory which is popular with humanists and educators. As there are adherents to harder and softer "sciences of the mind", so too does this bias reflect itself in the notion of intelligence.

Semantics and Understanding

Ultimately, the question you are after is rooted more in the philosophy of language more than anything else, because the discussion of the syntax-semantics dichotomy rests there with the philosophers and scientists of language. There are a number of competing models for how exactly this stuff, thing, experience called "meaning" happens, and if you really want to understand what's involved in how semantics functions with people, I'd recommend two books to start you on your way, though they aren't easy reads. First, Ray Jackendoff has his Foundations of Language which is highly technical, but makes a specific architectural argument about how embodied systems of the brain give rise to what we recognize as meaning. The second is also a tough read, but worthwhile if you really want to understand why the promise of human-level AI and language use has failed to materialize, Cognitive Linguistics by Evans and Green which offers a comprehensive picture of how language and meaning are grounded in bodily experience.

Summation

Now, what I've offered here isn't an easy answer so much as a blueprint for understanding why most philosophers are out of their element when discussing how to implement an aspect of actual human cognition rooted in neural computation on systems designed to implement the von Neumann architecture of a Turing machine. Searle's contributions to language, semantics, and intentionality are indisputable, however, in some regards, the question of engineering semantic understanding has begun to move out of philosophy and into the scientific domains of machine learning, software engineering, and neurology. As such, you will see resistance to abandoning classical notions in philosophy like truth-conditional semantics and Platonic mathematics that are adduced in favor of transcendental forms of metaphysics. In fact, Searle himself concedes the brain is a biological computer, but just remains skeptical that our current computer technology can mimic them, which is a measured conservatism.

Answer 5

TL:DR;

If we view brains as computing machines (which, for all we know, they are), there is no basis for Searle's claim.

According to the Church-Turing thesis, which is a very respected result in computer science, there is no computation that cannot be performed by an ordinary computer.

You can view it as a challenge: show me a solvable problem that cannot be solved by a computer program. So far nobody has been able to do it.

The significance of that result is that (if we don't account for speed and space) any computer that might exist in the universe be it electric, quantum or one based on technology that we cannot imagine would be just as capable of solving a problem as the phone in your pocket.

If we consider the brain as falling in that category, then the difference between the brain and any other computer is just the software that it runs.

If we agree with all that we can easily refute Searle's claim that, because he can perform computation that outputs chinese without understanding Chinese, the computer doesn't understand it. The response is that it is the software that "understands" anything, not the hardware e.g. silicone chips cannot play chess, and neither can neurons, but if we arrange them in the correct way then they both can play chess.

The Chinese room experiment only works if we don't consider brains as computing machines i.e. if we think that there is something happening in our brains that cannot happen in any other type of system. However, nobody has been able to provide evidence for such a thing happening.

Answer 6

I think the simplest way to explain it: syntax can be parsed computationally, yet computation can be abstracted to ridiculous or "funny" instantiations. Since we don't know how mental states (e.g. semantic understanding, consciousness, awareness, etc) arise from the physical--"the unfathomable gap between physical process and subjective awareness which mocks our search for the filaments that bind the corporeal and the mental together", should we put any weight at all on the idea that this produces mental states (e.g. semantic understanding)? "There is no more amazing and puzzling fact than that of consciousness". We claim ignorance for how neurons and the brain give rise to consciousness, but of all things we strongly believe about consciousness, the brain is involved: "there is little doubt that humans have a mental life, because we have brains".

We are not prepared to make the jump for something other than brains giving rise to consciousness, even though we do not know how brains do so. Surely computation alone can't be it, just think all the "funny instantiations" of computation beyond a person in a room shuffling cards: water troughs, moving grains of sand around, etc. The Chinese room experiment is just another "funny instantiation" of computation.

[1] all quotes taken from: Maudlin, T. (1989). Computation and Consciousness. The Journal of Philosophy, 86(8), 407. doi:10.2307/2026650

Answer 7

A machine could conceivably have its own semantic. This would only require that it had its own internal representation of the world. However, what would be the use of that? Each human obviously has his or her own private mental representation of the world. However, despite this, we do share most of it and this simply because we are essentially biologically very similar to each other and we are gregarious, so that we share large chunks of our lives. We all understand what is the Sun because there is only one Sun and we have broadly the same experience of it. Thus, we end up with broadly the same semantic. There are differences, but they represent a small subset of the whole--contrary to what controversies on the Internet or indeed in real life may suggest.

So, the problem is not so much of a machine having its own semantic but of having a semantic sufficiently close to that of a human being, at least if we want humans and machines to understand each other. The difficulty, then, becomes that the production of a human semantic remains largely an unknown process. It may not be impossible to do something comparable in principle, but it is probably for now at least way beyond our technical capabilities, in particular in terms of the massive data that the human brain processes continuously.

Answer 8

Let us call a comprehending agent a thinking being possessing "semantic understanding" of the meaning of words arranged propositionally. Suppose now there is in the input stream of a comprehending agent a word which the agent hasn't encountered before. Now when a translation machine for instance may encounter an "original" or nonsense word for which an appropriate translation may be inferred from contextual clues, but doesn't reflect any particular instance of text in the "real corpus" it is intended to translate -- what should its output be? A translation machine might simply output a null result, or enter undefined behavior; yet what should the output of a comprehending agent be, in that case? "Perfect" translation ability seems to perhaps imply, in other words, an extra creative step for which it would seem challenging to specify an explicit rule.

The significance of translation, the difficulty and complexity involved, is as a rule maybe sort of understated in my view -- and involves all the problematics that the phenomenologists, deconstructionists and psychoanalysts have raised surrounding the 'profound depths' at work in the genesis of local transcendental structure.

Perhaps the limits of so-called "undefined behavior" for software are suggestively similar here to those of language's own outer penumbra -- that is, of nonsense and hapax legemonon, which maybe play a more important role in the construction of "sense" than we might imagine. But suffice to say all these analyses of language do seem to me to raise the question of the origin and value of sense as a distinct entity; and moreover it seems to me that the most diligent efforts of philosophers of mathematics, Frege and Russell, do not exactly succeed in solving the ambiguity at the heart of some of these axiomatic, sense-grounding systems of reasoning like ZFC.

Answer 9

Another argument I’ve seen against the experiment is that “together with a book of instructions for manipulating the symbols (the program),” capable of interpreting Mandarin like a native speaker, could not in fact exist. Natural human languages don’t work that way, and there are an infinite number of possible sentences in Chinese. Even if you somehow did find a large enough subset of Chinese that you could fit into your book of rules, the man in the box would never be able to pass himself off as a native speaker because he could only give identical answers to identical questions. This approach failed to work for English, and my understanding is that it would not work for Chinese either.

This might be more a limitation of the analogy than a decisive refutation of the underlying point, but: it turns out that a system that works by looking up and following a list of grammar rules doesn’t produce convincing enough responses that we regard it as “understanding” a human language.

Answer 10

The Chinese Chinese Room

The main problem with the Chinese Room argument is that it presupposes a massive, massive thing: an algorithm which provides "Chinese language responses". We are just supposed to accept this black box without question so we can focus on the "real issues" in the debate. But we can utterly destroy the Chinese Room argument with one simple trick! We just define where this algorithm comes from!

You see, the implicit assumption is that the "algorithm" is somehow unnatural...a cold and lifeless product of human ingenuity which cannot possibly reflect the beauty and glory of human consciousness. But why not? What if the "algorithm" were nothing more than a precise description of an actual Chinese brain??? What if the "Chinese Room" were nothing more than an actual, ordinary, Chinese-speaking brain which was replaced by an English-speaking homunculus which otherwise executes the exact same actions as the corresponding Chinese brain? Is Searle still going to insist that the homunculus-in-the-shell really doesn't "understand" Chinese? Of course, the homunculus doesn't necessarily understand Chinese, but it obviously doesn't need to.

The Searle Room

Of course, we don't need to bring Mandarin (or Cantonese, or any of thousands of other Chinese languages) into it. We can just replace John Searle's brain with an alien-speaking homunculus and an algorithmic description of his brain. Then, we can turn his argument on its head and insist that brains don't understand English either. And if that's the case, then brains aren't special, and are thus on the same level as computers/AI.

Of course, Roger Penrose implicitly understood the danger of this argument, which is why he went down the long road of trying to show that there is no algorithmic description of the brain because brains are special, by harnessing quantum effects. That's a whole different thread, so I'll just leave it at that.

Answer 11

Searle's argument was framed in times when we only had Symbolic AI which is built on rule-based logic. This sort of system is inflexible and non-dynamic. It works statically through proof-theoretic systems and/or truth tables. Every extension and addition of rules need to be implemented manually by hand. One of the examples of a Symbolic logic environment that had high hopes in terms of AI at the time, but failed, was Prolog programming language.

The limitations of rule-based systems do not apply in this day and age. We now mimic and reproduce machine learning systems based on neural networks of the brain that are able to learn semantical contexts via either supervised, semi-supervised, or even unsupervised learning processes (Example). One example is Python-based software SpaCy which works in the area of NLP (Natural Language Processing).

Through the computational linguistic approach, the machine can now utilise neural networks to constantly learn semantic contexts of say, biological papers, scientific papers, or even newspapers and also any other pieces of text. For instance, it can extract what you need based on the meaning of the text (semantic similarity) which is achieved through creating word-embeddings, vectors that act like "maps" for terms (See demo).

To conclude, it becomes more and more feasible to make "hard" claims about AI understanding semantics than it was in the age of Searle's Chinese Room argument. Even more so, the field of computational neuroscience now implements the digital parts of the brain that model human processes of learning, and even cognition. There is indeed hope that machines would be able to "understand" meaning in ways beyond Searle's argument.

Answer 12

The problem with the chinese room argument is that the man only receives input from a single source, the message where as semantic understanding requires that the message be associated with other inputs.

So almost by definition the room cant have semantic understanding.

However, if you added extra inputs say time of day, weather and memory the man might soon come to associate "good morning" with sunny mornings.

You could go further and remove the external inputs, simply encoding the association in the instructions using english. This would amount to a translation and the man, having learned the translation, would obviously have a semantic understanding of chinese.

Its easy to see that a computer program can also be given external inputs and memory. Indeed you can imagine a very simple program "good morning" = sunPosition && noClouds which on the face of it is semantic understanding

Answer 13

Searle isn't making the point that programs cannot be semantic, he is starting from the definition of a computer program - which is formal, a.k.a syntactical - and then drawing parallels to human usage of formal systems (mechanical instructions) and comparing that to actual understanding.

A good example would be - learning the material on an exam by heart (this would be syntactical/mechanical) vs taking your time to actually understand the material (this would be the semantics). Understanding is not the 'aha!' feeling you get from deeply consuming the material, but rather it comes with that feeling. When you understand the material on the exam, it is absolutely not the same as having learnt it by heart.

This is basically the same case as in the Chinese room argument. One could give any other example of people following some kind of computer program - some kind of specific step-by-step set of instructions - and arrive at the same conclusion as Searle did with his Chinese room, but with some intuition from personal experience to back it up (we've all had to follow some kind of step-by-step list of instructions at some point, I am sure).

Answer 14

There are many answers already posted here, which offer significant insight. However, I don't see any that seem to fully understand the point of Searle's thought problem, and many that take implausible/untenable positions.

I fully answered this question for a different asker, here: How does the Chinese Room Argument handle the pile of sand paradox?

Searle's use of "semantics", which is a purely linguistics term, may be distracting some of the posters here from understanding his point. One CAN do language without consciousness, but Seale is working with a definition of understanding, such that one cannot do "understanding" without consciousness, and he was using "semantics" as a stand in for understanding.

Understanding involves competence, awareness of the answer provided by competence, and I/O. The "books" in the room have competence, but no awareness. The operator has awareness, and operator plus mechanisms in the room do I/O.

Most of the answers here that reject Searle's argument, also reject that understanding requires awareness. They are explicitly behaviorist answers. They treat consciousness as irrelevant, and humans as reductionist machines. Note, however, that philosophy, psychology, and science in general have rejected behaviorism, and also reductionism, see the SEP section 5 on scientific reduction: https://plato.stanford.edu/entries/scientific-reduction/ Note also that the very clear evolutionary tuning of consciousness require that consciousness be causal, so this dismissal of causal consciousness is in contradiction of essential biologic principles.

I also noticed that most of the rejecters of Searle's thought problem, accused Searle of being anti-science and a dualist. This is explicitly untrue. Searle is an advocate of non-reductive physicalism. He considers consciousness to be emergent, and non-computational, hence consciousness cannot emerge from the computations performed by functionalism. His alternative position is that there is some as yet unknown feature of biological systems that allows consciousness to emerge from them.

Of the responses to Searle's thought problem cited here, the only one that addresses the issue of awareness are the citations of Chalmers. Chalmers understands Searle's argument, that by separating the competence from the awareness and I/O, Searle was highlighting the implausibility of tying consciousness to demonstrated competence. Chalmers is the rare philosopher who grabbed this bull by the horns. He basically admitted that competence alone (the books) did not have consciousness, but instead claims the room ASSEMBLY has consciousness. That the room system is conscious when the operator implements the functions in the books, and does the I/O. This is a rare view, as the room SYSTEM is not the sort of thing that most philosophers will admit can support consciousness.

Answer 15

tl;dr– The Chinese room argument is a pure silliness, on-par with flat-Earth theory. Without having done a formal survey, it's my general understanding that those in the field largely disregard it as an anti-intellectual position.

---

The "Chinese room argument" is pure silliness.

Searle's argument is basically:

1. Assume that AI are mindless machines.

2. That assumption can't be disproven because any evidence to the contrary might be the consequence of a script-like algorithm that merely sounds human (or sounds like it understands Chinese).

3. Because the assumption can't be disproven, it must be correct.

Searle's argument is just a copy/paste of the Solipsist argument:

1. Assume that other people are mindless zombies.

2. That assumption can't be disproven because any evidence to the contrary might be a consequence of a script-like behavior that merely sounds human (or sounds like it isn't a zombie).

3. Because the assumption can't be disproven, it must be correct.

Wikipedia lists a bunch of criticisms of Searle's argument. Not because it needs further debunking, but rather because there're so many things wrong with it.

Basically, it's Vitalism.

Answer 16

It's not impossible for an AI to have semantic understanding at all. All semantics are preceded by a strict syntax, of sorts, but instead of such an AI "reading" the input, it is the input.

If I prick your finger and your body has learned to react, has it not understood the semantics of the event? Such an event is associated with pain or damage, or whatever, yet it is strictly an operant-conditioned response -- there was no other "meaning".

This subtle inflection (reads the input vs. is the input) is the basis for understanding. Modify Searle's argument as follows: a conscious agent scans the Chinese characters and reads each brushstroke of each glyph -- in order to perform the right categorization (of the glyph), they will need semantic understanding of the glyph (via it's "syntax" of brushstrokes).

Answer 17

At the risk of appearing naive, I would just note to the questioner that in all these very good answers, it is not entirely clear what is meant by "semantics" and "meaning."

I believe Searle was, many years ago, introducing into a much more buttoned-up analytical philosophy questions that are now more highly developed with the reintroduction of some versions of metaphysics and even idealism into Anglo-American curricula.

Questions of "meaning" are strictly excluded from Shannon's model of information, upon which computer "syntax" is still based. Like Newton's theory of gravity or Jevon's theory of utility, it offers a mathematical modeling and forbears from any "substantialist" or "essentialist" hypothesis of what the models quantify.

Searle's argument may have been necessary within the woefully constricted tradition in which he taught, but I believe would have been thought silly and redundant in the so-called "Continental" traditions since Husserl.

It is a bit like trying to compare Shannon's mathematical definition of information with Walter Ong's. In the latter, there is an irreducible bearer of "experience," the body, the physical vibrations produced by the Word spoken between bodies.

This experience is tragically firewalled. I cannot actually "feel your pain." It can be "sympathetically" but not syntactically transferred. Nor can the machine mimic actually "living" in the sense of undergoing the complex semantic interactions that enable it to reproduce itself both physically and "inexactly." (Von Neumann's machine replicators aside.)

In my own informal reading, Searle is merely indicating the exclusion of these many traditional philosophical issues from the path taken by analytical "philosophy" in his day. Though I have not read him much, Searle's fellow Californian Hubert Dreyfus may have somewhere written a contemporaneous rejection of the whole basis of the Chinese Room "problem."

Numbers are very mysteriously "equal" and can thus fit into "equations." But experiences of bodies over time at some level defy equivalence, cannot ever be exactly the same, and so can never be fully modeled in discontinuous quantities.

Answer 18

What is consciousness ?

In simplified terms, if you have an AI machine (no matter how complex it is ), it could be described by three things : state of the machine (both internal and external) S , input into machine I , and output O . Output would be a function of input and state O=f(I,S) , and any randomness would be modeled trough state (pseudo-randomness) . In other words output would be deterministic.

Model described above is true even for most complex neural networks, and as we can see it simply has no free will. Even if we assume it could learn (i.e. change its own algorithm) this is still done deterministically - with certain input and state our AI machine would change itself but only in prescribed manner. Again note that even if we include randomness into this change, this randomness is still part of the state S, therefore included in our basic equation.

Since our AI machine does not have free will, from that proceeds that it cannot create, or in other words any new structure made by this machine would simply be pre-programmed into it. In any given point of time, machine would have set of patterns P and set of modifications M. Machine could apply those modifications to patterns, creating set of "new" patterns Pn, but that set would be already pre-determined by initial set of P and M.

What this have to do with semantics ? Semantics is simply study of meaning. You have certain phenomenon (words, sounds, letters, pictures ...) standing instead of something else. For example, word "dog" (both written or spoken) symbolizes certain animal species. Dogs have hair, four legs, they are carnivorous mammals. Same as cats. Yet, (almost) no human would call dog a cat. Ships on the other hand have no legs, hair and they are not animals at all, yet humans call some of them iron dogs ! How would you explain to AI what is a meaning of word "dog" ?

Humans are irrational and illogical. This has been mathematically proven by Gödel’s Incompleteness Theorems and Tarski's undefinability theorem. In any strong formal system you cannot completely define the truth. You will have some truths (and some falsehoods) that are unprovable. Everything could not be defined. Yet, in some strange way (intuitively) humans would would differentiate between the two. Zen Buddhist call that clap of one hand. It defies explanation because it cannot be defined - it is absurd but yet compelling. And completely incomprehensible to AI because it cannot be translated into formal language.