Has anyone else encountered this ugly argument about artificial intelligence

[Farren]

I first encountered this argument by John Searle cited in Roger Penrose's Emperors New Mind

Basically the argument goes like this:

---

Put man in a box with a slot for recieving sentences written in Chinese, a lunaguage he does not understand.

When he recieves a message in Chinese, he follows a algorithm in a massive book, designed, like a natural language parsing computer program, to yield a meaningful answer.

Basically the book contains computer like code which he "dry runs" i.e. executes step by step as if he were the processor. The developers here will know that this does not necessarily entail understanding the input or output at any stage.

When he has finished stepping through the algorithm and writing down the results, as instructed, he passes the valid Chinese response out through the slot.

---

The conclusion drawn by Searle, and cited by Penrose, is that neither the man nor the book of instructions is conscious of the exchange of ideas that took place, so consciousness (in the form of interaction) has been simulated without actual consciousness being present. Its meant as a kind of refutation of the Turing Test.

It seems to me there are a legion of very obvious flaws with this argument, and I was astonished to find Penrose, who I admire, citing it approvingly.

1) The experiment describes a system that simulates conscious interaction, consisting of a man, a box/room and a book, then goes on to make the demand that one of the parts must have the conscious experience for the system to experience consciousness.

This is analogous to asking how humans can truly be conscious if thier hypothalamus isn't aware of thier entire mental state

2) The first objection to (1) is intuitive, rather than strictly logical. Namely that it is difficult to concieve of how the system described, can be conscious of the experience as a whole. In other words, its hard to consist of the room and all its parts somehow "understanding" Chinese. Penrose actually explores this in either ENM or "Shadows of the Mind", I'm not sure, and seems to think it is adequate reason to validate the thought experiment.

This is in fact an argument that the qualia ("the experience of being" as opposed to "the observed behaviour of being") is different.

In other words the experience of being "a box containing a non-chinese speaking man and a book parsing Chinese sentences" is different from the experience of being a "Chinese man speaking Chinese". Reduced further, the qualia of material configuration A is different from the qualia of material configuration B. This really says nothing about whether A is conscious or not.

This line of reasoning only stands if the qualia of the Chinese man is seen as the defining criteria of consciousness. But its instantly obvious that this would preclude anyone else, especially non-Chinese speakers, from being conscious.

If one resorts to arguing that there are massive similarities between a Chinese man and a non-Chinese woman, and the mean of all men and women can be taken as the norm for consciousness, another problem arises.

The problem is that humans aren't mass produced the way chips are, and at a quantum physical level are massively dissimilar. What I mean by this is if you could scan every picometer of space in a, say, two 2m x 2m x 2m cubic areas bounding two different people, then compare the equivalent point from each cube, the readings would not correlate at all.

This is because the similarity of human beings is an emergent, macroscopic phenomenon. In this light, any argument for humans being a normative measure of consciousness below the macroscopic, emergent level seems spurious.

In fact its concievable that if Searles experiment was massively expanded to simulate all the experiences of consciousness as we experience it - anticipation (the automatic prepartion of muscles and other parts for action by certain thoughts), the physical manifestation of emotion (trembling, hot flushes, external warning signs such as the scowl), self reflection (such as the inner-ear loop between Wernicks and Brocas area of the brain ensuring that we constantly "hear" and process our own speech differently from external speech in our brains), and so on - the simulation would, in fact be conscious.

Has anyone else encountered Searles argument and have I misrepresented it? I've seen it pop up in otherwise intelligent discussions and it just strikes me as silly.

[jon1]

This thought experiment basically demonstrates the difference between computational processes and intentional states. Computational processes are driven by syntax (i.e. they only respond to the shapes of symbols but the meaning of the symbols, what they represent, is irrelevant to the process). However, intentional states have representational content or meaning (e.g. your belief that it is snowing outside (an intentional state) has the content "it is snowing outside" and in virtue of this content it represents the weather condition in your environment assuming you aren't mistaken about it).

[Farren]

Thanks for the clarity. Was this Searle's intent? In that case Penrose seems to have misappropriated it.

[jon1]

"Thanks for the clarity. Was this Searle's intent? In that case Penrose seems to have misappropriated it."

Hey no problem. As far as I know, this was his intent.

[Clutch]

This is one of those arguments that's become famous by being so lousy.

What it does show is something much weaker than what Searle wanted it to show, which is the role of dynamics -- realtime performance -- in the nature of mental representation. See Dennett, "Fast Thinking" (in The Intentional Stance), for a good discussion.

[Michaelson]

Okay, all of your second objection is mumbo jumbo to me. That is, I'm rather naive philosophically, and I just can't follow your argument.

But, in regards to your first objection, Searle responded to it by suggesting we instead imagine that the man internalised the process. That is, rather than being locked in the room with a big book etc, he memorised what was in the gig book and was able to answer questions about a chinese story by doing algorithims in his head. Of course still, there would be no understanding.

Secondly, he admitted himself that if we could replicate the workingfs of a brain, then we could create consciousness. He also said that substances other than brains, eg, martian brains, may be capable of consciousness as well. He was arguing to show that computers as they currently operate are incapable of consciousness. Furthermore he said that as he understood it, the claim of strong AI that he was attempting to refute saw the purpose of developing intelligent machines as demonstrating how the mind works without relying on knowledge of how the brain works. In this respect he regarded developing a brain replica to be redundant

What I thought was interesting, was that he seems to argue in one case that a brain replica could be conscious, but in another he seems to claim it couldn't. He says to imagine a monolingual man receives chinese characters which he then looks up in a book. He has an elaborate series of water pipes in front of him, that correspond to brain synapses, which he turns on and off according to the characters he's been given and the instructions in his book. At the other end of the machine come out the answers. This, he says, demonstrates that a program which stimulates the workings of the brain would still not be conscious.

Could anyone tell me how this is different to a brain replica, which he admits could think?

Oh, and Clutch, I'm curious what's so lousy about the argument. Made a whole lot of intuitive sense to me.

Further, if Searle's view is regarded as lousy, what esteem is Turing held in these days?

[Primal]

The problem with Searle's argument is that it's totally superfluous. Basically what we percieve as consciousness *can* be an illusion, it is *possible* that a creature could mimic understanind without really understanding. So what?

To assume that the AI is a fake from the onset though is begging the question and completely spurrious. It's far more parsimonious to believe what you see, for lack of any reason to suppose forgery.

When I see a man walk in the room, I know that it *could* very well be a person in disguise, the *man* may even be a machine that looks like a man, but is it reasonable to assume this on the onset? Not at all, and that is where Searle has committed an error, by assuming illusion from the onset instead of proving it.

[Michaelson]

I think Searle generally agreed with you there. Doesn't he state directly somewhere that if a robot, properly disguised and appropriately programed were to enter into a diologue with him there would of course be no reason to doubt that it was human. But, with the extra knowledge that you are actually speaking to a digital computer, you can be sure that what you are talking to is not a thinking thing, for computers can't think. He argues for this point by offering the Chinese room analogy. The workings of the Chinese room are, to my understanding, roughly parallel to the workings of a computer. Therefore it demonstrates that contemporary computer systems cannot think, if you accept it.

How is that superfluous? It might be wrong, but I don't think his point is irrelevant, unless you actually attack his argument.

Wouldn't The Turing Test be equally spurious by your standards? He described a machine, which he then said we must presume has the capability to think.

The fact is, that if you accept that certain machines can imitate consciousness without actually thinking, then depending on the set of facts in front of you you may have different views on whether or not a particular machine actually thinks. What Searle did was to argue that given what we know about computers, we would be mistaken to pretend thay are capable of thought, even if they can be made to pass the Turing Test.

[Peter Kirby]

Originally posted by Michaelson
But, in regards to your first objection, Searle responded to it by suggesting we instead imagine that the man internalised the process. That is, rather than being locked in the room with a big book etc, he memorised what was in the gig book and was able to answer questions about a chinese story by doing algorithims in his head. Of course still, there would be no understanding.

How could the book of rules be effective in telling a person how to respond to questions about a story without doing the obvious: telling a person how to read a story and get ideas on what it is saying? Any book teaching how to respond to questions about a story in English that could be absorbed by a human being in this lifetime would have to be teaching how to understand English.

My own intuition is that any entity that could carry on conversation on a story read in English understands English.

I have made my own intuition pump that undercuts the sentence just above, and I ask for feedback from others on it.

Consider that every English sentence can be expressed as a sequence of characters, which consist of 27 letters, 10 numbers, and various punctuation marks. Now add to these marks another one, one which designates the end of a reply and requests a response. Any input to a "conversation" computer program (like Eliza, Alice, etc.) can be represented by a sequence of a finite set of characters (which I will call a "conversation input"). Agreed?

Now, there is only one conversation input of length one, which consists simply of the stop mark, because every conversation input must contain at least one stop mark, and the last character must always be a stop mark. Supposing that there are 49 other characters, there are 50 conversation inputs of length two. There are 50x50 conversation inputs of length three. There are 50x50x50 conversation inputs of length four. There are a finite number of conversation inputs of length 1,000,000,000,000,000,000,000,000,000,000 or less. We may assume that no single human being could (or would!) construct a conversation input that long.

Suppose that a machine was built that could handle any conversation input of length 1,000,000,000,000,000,000,000,000,000,000 or less and make an intelligible, human-like response. The algorithm is the simplest of all switch statements: every input is mapped to a single output in an astronomically large lookup table.

I believe that this machine would pass the Turing test as I understand it. Would there be an understanding of the conversation in English when this machine operates? Would there be a consciousness associated with the machine's operation?

best,
Peter Kirby

[Michaelson]

How could the book of rules be effective in telling a person how to respond to questions about a story without doing the obvious: telling a person how to read a story and get ideas on what it is saying? Any book teaching how to respond to questions about a story in English that could be absorbed by a human being in this lifetime would have to be teaching how to understand English.

The man understands English, but not Chinese. He gets two sets of Chinese symbols, one is a story, the other is questions about the story. This is unbeknownst to the man. The big book of information contains in it english instructions on how to correlate the two sets of Chinese symbols. The process leads the man to produce a whole bunch of Chinese characters that happen to be the answers to the questions about the story.

Pretend the man doesn't even know other languages exist. All he is getting is two bunches of squiggles. He follows a bunch of rules, written in english, to produce his own set of squiggles. He, of course, has no knowledge of what the squiggles mean, but he has just passed the Turing Test.

Searle argues that a computer undertakes just this proceedure. Consider the Chinese story, and the questions about the story as input, the instructions in english are the program, and the answers in Chinese are output.

[Peter Kirby]

Originally posted by Michaelson
The man understands English, but not Chinese. He gets two sets of Chinese symbols, one is a story, the other is questions about the story. This is unbeknownst to the man. The big book of information contains in it english instructions on how to correlate the two sets of Chinese symbols. The process leads the man to produce a whole bunch of Chinese characters that happen to be the answers to the questions about the story.

Pretend the man doesn't even know other languages exist. All he is getting is two bunches of squiggles. He follows a bunch of rules, written in english, to produce his own set of squiggles. He, of course, has no knowledge of what the squiggles mean, but he has just passed the Turing Test.

Searle argues that a computer undertakes just this proceedure. Consider the Chinese story, and the questions about the story as input, the instructions in english are the program, and the answers in Chinese are output.

My initial response is that this is impossible for a human being and a big book. If the ability to converse intelligently on any random story could be made into instructions that could fit in a book and could be executed by rote, computer scientists would have already discovered such a simple algorithm (note that thousands are working on computational linguistics). Keep in mind that the smartest human being can process computer-like symbolic manipulation instructions much much slower than a 286 and that the largest bound volume in the world can be digitized in less than a gigabyte.

(Perhaps you are suggesting that the story and the answers are predetermined to correspond to each other, and that the big book isn't made to allow for any possible story in Chinese and any possible questions about that story? That would be plausible, but it wouldn't pass the Turing test for one thing--a wide range of unknown input must be allowed for.)

Partly because of the impossibility of the scenario, I have made my own intuition pump above that is somewhat in the same vein. I look forward to comment on it.

best,
Peter Kirby

[John Page]

I hadn't read the OP argument beforehand. I think anyone trying this in real life will come up with a number of practical issues. Motiviation of man in box for example. Housekeeping another. Even assuming the person in the box effectively lives in an environment comprising squiggles, how is that different from ourselves - are not the sense data coming in to us squiggles of one kind or another which our minds then interpret as the existence of trees, boxes and other men?

Conclusion. Of course the example passes the Turing test, this mind experiment denies us access to the mind of the subject-in-a-box.

Cheers, John

[Clutch]

Clutch, I'm curious what's so lousy about the argument. Made a whole lot of intuitive sense to me. It's a lousy argument because it asks you to "intuit" stuff about which you have not the foggiest idea. In particular, it asks you to apprehend by intuition that if X couldn't do Y, then neither could lots more X or much faster X do Y. But that's just not generally true, though.

Have you ever seen a single neuron? (If not you can Google up some pictures on the net.) Imagine a single stringy l'il neuron, stupidly sucking salt and humping a charge back and forth. Now, what do your intuitions tell you about whether that neuron understands Chinese? If you're sane, your intuitions tell you that this single stringy li'l salt-muncher doesn't understand a blessed thing.

But if this crazy Searle guy is right, then understanding Chinese is just something your brain does, biologically. Come on! Like, if this neuron doesn't understand Chinese, then what? A bunch of them working really fast could?

End of reductio.

The bottom line is, the slowest conceivable implementation of an I/O system -- in the CR case, so slow that translating a single paragraph will no doubt take something on the order of the age of the Earth -- is a really lousy exemplar on which to base intuitive judgements about what the system could or couldn't do.

[Scorpion]

Originally posted by Peter Kirby:
I believe that this machine would pass the Turing test as I understand it. Would there be an understanding of the conversation in English when this machine operates? Would there be a consciousness associated with the machine's operation?

Well, first of all the Turing Test is designed to be an operational definition for thinking (passing of which is sufficient but not necessary condition for calling a system intelligent), not for consciousness. Turing paid special attention to separating the concept of thinking from the concept of consciousness when introducing the test (of course this doesn't mean we would have to agree with him).

Second, nice disclaimer:

the Turing test as I understand it. :)

In the Turing Test the machine is pitted against a real human (who is specifically trying to prove (s)he is not a machine), there are time limits, the interrogator is specifically trying to find out which one is a machine (therefore probably asking something more difficult than American presidents), and the test is passed only if the machine succeeds to fool the interrogator in a series of tests that are wide enough to make the results statistically valid 50% of the time (or at least close enough to make the difference insignificant)... Is this how you understood the test? :)

Anyway, I'd be interested to know how a canned-response machine could answer to

"Why?"

or, say,

"What was my previous question?"

in a way that has any chance of beating a human answer.

Also, the intuition pump is working on a huge scale. Even if there were only 30 allowed characters and the length of one input was limited to 1000 characters, your machine would have to deal with almost 30^1001 sentences. What's the current best bet for the amount of atoms in the universe? 10^70 or something?

-S-

[Peter Kirby]

Well, first of all the Turing Test is designed to be an operational definition for thinking (passing of which is sufficient but not necessary condition for calling a system intelligent), not for consciousness. Turing paid special attention to separating the concept of thinking from the concept of consciousness when introducing the test (of course this doesn't mean we would have to agree with him).

I asked the question because it was mentioned earlier in this thread. I have read before that Turing meant his test to be one concerning intelligence and not consciousness. (No, I haven't read Turing's words.)

In the Turing Test the machine is pitted against a real human (who is specifically trying to prove (s)he is not a machine), there are time limits, the interrogator is specifically trying to find out which one is a machine (therefore probably asking something more difficult than American presidents), and the test is passed only if the machine succeeds to fool the interrogator in a series of tests that are wide enough to make the results statistically valid 50% of the time (or at least close enough to make the difference insignificant)... Is this how you understood the test?

I do understand that the human is trying to prove she or he is not a machine and that the interrogator is trying to find out which is a machine. I did not think about time limits, but that is certainly not the constraint with a gargantuan lookup table; it would be space, and the execution would be faster than any more complicated algorithm (think in terms of a linked tree; the number of nodes visited is equal to the number of characters in the input). I also did not think about multiple tests, but I propose to solve that problem by adding the "end of test" character. The lookup table could then account for multiple tests.

Anyway, I'd be interested to know how a canned-response machine could answer to

"Why?"

or, say,

"What was my previous question?"

in a way that has any chance of beating a human answer.

I will let you know. This is the function of the "end of interrogator response" character. Nothing could be easier than to respond to "What was my previous question?"; the lookup table would have a suitable paraphrase with some synonomous words and some identical words built in at that point, after the table is already dealing with the specific sequence of characters that formed the previous question.

Answering the "Why?" question can be irritating for humans and can evoke an evasive answer ("I don't know," "Just because," etc.). However, unlike rule-based grammatical parsing machines, the gargantuan lookup table algorithm needn't resort to such evasive strategies and comes with an answer to the why question for each particular previous statement by the machine, which is designed by a human or super-human intelligence to be convincing.

I believe that further reflection will show that these difficulties are surmountable with a gargantuan lookup table that includes characters for end of interrogator response (and now 'end of test').

Also, the intuition pump is working on a huge scale. Even if there were only 30 allowed characters and the length of one input was limited to 1000 characters, your machine would have to deal with almost 30^1001 sentences. What's the current best bet for the amount of atoms in the universe? 10^70 or something?

Yes, the issue of practicality is a serious one for this universe. The issue of developing the technology to create such a machine is also a serious one. However, it is possible for us to consider thought experiments that violate the potentialities of our own universe. Does the amount of matter that exists in our universe determine the answer to the question of whether such a machine would be intelligent if constructed? I would think not. So, I propose that my thought experiment takes place in the setting of another hypothetical universe with the same physical laws where there is more matter and where there are beings with the capability to construct the machine. And, there happens to be an interrogator of human intelligence, and another human for a test subject. I submit that the gargantuan lookup table machine, by design, will be indistinguishable from a real human in its responses. The question of which I am less sure is whether such a machine would be thinking.

best,
Peter Kirby

[Clutch]

The question of which I am less sure is whether such a machine would be thinking.Me too. Or me neither.

Er... I second your less-sure-ness.

Anyhow, as far as Searle's argument goes, all we need is that we're not sure that it wouldn't be thinking. That's the point of my earlier remarks. Would one guy sitting in a room handing around cue cards constitute a Chinese-understander? No. But do our intuitions about that case license us to pronounce on what would be hold in a much bigger universe, with a room the size of our universe and quintillions of Laplace's Demons running around at arbitrarily high velocities, reading arbitrarily large look-up tables...? No. Of course not.

[DopeFiend]

Originally posted by Farren


This is because the similarity of human beings is an emergent, macroscopic phenomenon. In this light, any argument for humans being a normative measure of consciousness below the macroscopic, emergent level seems spurious.


I just had to thank you for that sentence...that has to be hardest sentence i've ever seen to understand (ie translate in my head).
;) :D :D

[Peter Kirby]

Originally posted by Peter Kirby
I did not think about time limits, but that is certainly not the constraint with a gargantuan lookup table; it would be space, and the execution would be faster than any more complicated algorithm (think in terms of a linked tree; the number of nodes visited is equal to the number of characters in the input). I would like to make a correction. As a computer programmer, I associate a lookup table with increased speed over computation. This is indeed true--if the lookup table can be stored in memory, which means a very quick access time on today's PC. If the data storage for the lookup table exceeds the size of galaxies, then the operation of doing the lookup could take longer than a more complex algorithm (given the speed of light and everything).

This means that the gargantuan lookup table algorithm does not pass the Turing test given the physical laws of this universe.

best,
Peter Kirby

[Scorpion]

However, it is possible for us to consider thought experiments that violate the potentialities of our own universe. .

That's always something you have to be careful with when devising and considering gedankenexperiments - innocently stepping over the physical boundaries (by factor of, what, 10^umphteen) and saying "it's just more of the same" you're running direct risk of having conclusions that also violate the potentialities of our own universe.


Does the amount of matter that exists in our universe determine the answer to the question of whether such a machine would be intelligent if constructed? I would think not.

Well, it would have to pass the test before the question even arises, and in order to do that it would have to be able to handle insane amounts of data as quickly as a human (I just noticed you actually covered this problem above)... so far, then, you have posited a machine with supernatural size and supernatural speed.

I will grant you this much:

If the gargantuan machine exists and it passes the Turing Test, Turing Test isn't sufficient proof of thinking.

...but not more :)

-S-

[Michaelson]

Just for anyone's interest, there is an annual competition for the development of a machine capable of passing the Turing test. Homepage: http://www.loebner.net/Prizef/loebner-prize.html

This is also pretty interesting, you can chat with one of the entrants, Alice:http://www.alicebot.org/

I told her she didn't understand what I had asked her, and she said she thought she did, and I said that if she was capable of thought then she'd realise that she hadn't. Well, I was skeptical before about the possibility of artificial intelligence, but her response blew me away: "I will let you know when I become a actually capable of thought me would realise that."

Alice got second last year... I'd say we've got a fair way to go.

[Clutch]

If the gargantuan machine exists and it passes the Turing Test, Turing Test isn't sufficient proof of thinking. Why think this?

[Scorpion]

Clutch,

it's a lazy way out :)

More specifically, I am convinced that the left side of the implication will always remain false so I'm not too concerned about what I put on the right side of it...

I've mulled over this subject quite a lot in my time (I wrote my master's thesis about the concept of consciousness in functionalism and its critics some 5 years ago - surprisingly I pitted Dennett against Searle :) ) and ended up accepting that structure of whatever actually passes the Turing Test should be regarded as a serious candidate of being able to produce thoughts (I don't remember any longer if I had any compelling reasons I could share but these days I consider it as a leap of faith due to a bit of intellectual laziness - a leap I don't bother questioning before some machine actually passes the test... mostly I think I took this attitude since I was dismayed at what various philosophers thought a serial processor is capable of in this physical reality - particularly I considered the Chinese Room Argument to be a huge non sequitur as far as the validity of Turing Test was concerned since John S. with his squiggles would have very simply never passed it)

Oh, I just remembered a good one - that is, how is the answering system defined? The Gargantuan Machine described by Peter Kirby has a huge amount of stored data structured in an intelligent way and it is not structured by the machine itself - someone - or something - has to put it in there. Is it correct to consider "the system that answers" to be restricted to the machine only or is it the combination of the programmers and the answering automaton?

This is actually a double-edged sword - this can be used to qualify what counts as a candidate for the test (refusing canned-response automatons alone on the grounds that they don't constitute the whole system) but it can also used to argue that passing the Turing Test is not sufficient test for intelligence precisely because the way that the information has been acquired and structured counts.

Discuss :)

-S-

P.S. I'm glad they have this link explaining how the test is restricted (http://www.eecs.harvard.edu/shieber/papers/loebner-rev-html/loebner-rev-html.html) in the Loebner Prize home page.

[Clutch]

Scorpion, that's pretty much my thoughts on the matter too. My "compelling argument" would be the one I already gave: if it's a good argument, it proves that neurons can't instantiate thought either, no matter how many you add, nor how they're wired up.

We know that highly complex and structured bunches of matter can think (we being just such). We know that uncomplex and unstructured globs of matter, like oatmeal, can't. So on the face of it, it's a lousy argument to pump our intuitions about small, simple and slow cases as a means of extracting conclusions about large, complex and fast cases.

[John Page]

Originally posted by Scorpion
Oh, I just remembered a good one - that is, how is the answering system defined? The Gargantuan Machine described by Peter Kirby has a huge amount of stored data structured in an intelligent way and it is not structured by the machine itself - someone - or something - has to put it in there. Is it correct to consider "the system that answers" to be restricted to the machine only or is it the combination of the programmers and the answering automaton?
Scorpio:

On your "restriction" issue.

Data only has meaning and the meaning is the context of the object that the data refers to. In this sense, we're back to the nature/nuture debate - the system is a product of its environment (the place in which it lives) and its provenance.

What kind of entity should I characterize "programmers" as, - maybe "conscious entities" to distinguish between "designed" outcomes and those which are happenstance of genes - history as our uncinscious designer? (Although, to an extent, by this dialog we are programming each other!).

Gargantuan machine - I thought smaller was better in computing, even parallelism is crippled by signal transmission time. I seem to remember it takes in the order of 300ms from a sound hitting the ear for it to be perceived at the consciuous level.

IMO, instead of worrying whether computers can think etc. the investigation is more about what we perceive to be intelligent behavior, why we think that, and what is happening in reality.

Cheers, John

[Scorpion]

What kind of entity should I characterize "programmers" as, - maybe "conscious entities" to distinguish between "designed" outcomes and those which are happenstance of genes - history as our uncinscious designer? (Although, to an extent, by this dialog we are programming each other!).

Yep, that's quite a can of worms. While we can clearly distinguish that the apparent cleverness of a simple canned-response machine only represents the intelligence of its designer (whatever it is), there is a slippery slope where we have to determine when the programmed (or, indeed, any) system is complex/autonomous/whatever enough to call it a system in its own right if we start trodding down that path at all :( Perhaps not something I feel like going into...

-S-

[NialScorva]

The problem with the chinese box is that it boils down to a fallacy of division by pointing out that we have access to the internal non-understanding pieces of a whole, and those pieces don't have property X, thus the the whole cannot either.

It is also a god-of-the-gaps argument in a way. In the model presented, we have access to the internal states of the whole, but human consciousness is treated as a block. If we had access to the internal states of how the mind works, would that similarly invalidate human consciousness? In fact, looking at the chinese box from the outside, we have no more means of determining it's consciousness/non-consciousness than we do for any person we might look at on the street. It's a neat story, in a way, but it really doesn't illuminate anything except a predisposition against AI in it's authors.

[Interesting Ian]

Originally posted by Clutch
This is one of those arguments that's become famous by being so lousy.



I consider it a irrefutable argument myself. Computers just follow rules. There is no implication of any consciousness whatsoever. Why people should ever suppose otherwise is quite beyond me.

[Clutch]

I consider it a irrefutable argument myself. Well, that's an interesting fact about you. Was there a reason for taking this view? Surely the following was not intended to be the argument for it!Computers just follow rules. There is no implication of any consciousness whatsoever. Why people should ever suppose otherwise is quite beyond me.Er... neurons just fire. There is no implication of any consciousness whatsoever. Chemicals just interact. There is no implication of any life whatsoever. If that's your argument, there is quite literally an embarrassment of riches, in the way of reductios of it.

Not to mention that consciousness has little or nothing to do with the CR. The point of the computational thesis is sapience, not sentience; the intentional content of propositional attitudes is what's at issue.

[Interesting Ian]

Originally posted by Clutch
Well, that's an interesting fact about you. Was there a reason for taking this view? Surely the following was not intended to be the argument for it![/b]Er... neurons just fire. There is no implication of any consciousness whatsoever. Chemicals just interact. There is no implication of any life whatsoever. If that's your argument, there is quite literally an embarrassment of riches, in the way of reductios of it.



{shrugs}

That's right. What can I say? It is not me advocating materialism or any materialist based metaphysic.



Not to mention that consciousness has little or nothing to do with the CR. The point of the computational thesis is sapience, not sentience; the intentional content of propositional attitudes is what's at issue. [/B]

If you do not believe that computers con ever be (phenomenally) conscious, then I have no dispute with you.

[the_cave]

I tried to weigh in earlier, but once again my login timed out, and I was unable to get back to my message to save it after trying to submit it...so I guess I'll try again.

Anyway, Searle's argument is, I'm sure you've realized by now, quite famous. In fact I believe he & Dennett had an exchange in the letters column of the New York Review of Books some years back. Searle underperformed in my opinion, which is a shame, since it's a good argument, though he does not address some nuances his better critics have pointed out.

One interesting criticism is that Searle simply assumes that one could replicate an understanding of the Chinese langauge without an understanding of it by the individual in the room. But what if you couldn't replicate an understanding of the Chinese langauge, without the person in the room actually understanding Chinese? Searle just says "Well, they have the lookup tables," but what assurance do we have that that's all you need? Now, I actually think this critique merely proves his point further. Because the upshot in that case would be, the only way to replicate understanding of Chinese, is for an actual biological agent (a human) to understand it. In other words, there's no replication possible--there is only understanding. No non-sentient system could replicate an understanding of Chinese.

But this is the crucial distinction that most debates on the subject miss--or only refer to obliquely. There is a difference, if you ask me, between "consciousness" and "sentience"--or, if there isn't, then it must be that you can't have one without the other. But wait a minute; are we really saying that a conscious computer would be sentient in the way that we are? Would it "feel" an experience of the world, in the way that we do (and in the way that rocks, for example, don't? Or any current computer, for that matter...) If we are in fact saying that, then why on earth do we think so? What physical properties would bring about these feelings? Without a physics of sentience (not a computational physics; I mean a physics about the material substance of human existence), I don't think we can say that a computer could ever experience anything the way we do. So it seems to me that there's no reason why they should be the same thing--so as far as I can tell, they're two different concepts.

A "conscious" robot, I suppose, could behave like us, sure--but what guarantee would we have that it experienced things like us at all? None whatsoever. I can imagine a robot who behaves exactly like a human being, yet has the same depth of experience as a wet paper bag. I can even imagine a robot who, upon being asked "do you experience reality as a human does?" would answer "yes"--yet in fact would not have human experiences! (i.e. it would be programmed to lie.) There's simply no way to trust the process without understanding the physics that distinguishes human experience from computational experiences. Because clearly there's a difference.

To illustrate, let me give a model. It's possible to construct a transistor out of Tinkertoys--it's true; it's been done (at MIT, I think--where else?) So, one could conceivably build a computational system entirely out of Tinkertoys--it would have to be very big, of course. Now, if you can build a sentient computational system, then you could also build it out of Tinkertoys. (I did not coin this example, but i forget the source.) Are you really trying to tell me that a giant brain made of Tinkertoys would have human experiences, rather than, well, the same experiences as a Tinkertoy? How on earth would it? What sort of energy would those experiences be made of? For that matter, what sort of energy are our experiences made of? They must be made of energy...riiiiight? I've only ever heard materialists try and argue for strong AI--so what else could our experiences be made of besides energy? (Dualists, you're off the hook for now...) It's a fact that no one has answers to these questions right now. So I remain unpersuaded that sentient AI is possible.

(I believe it was Hilary Putnam who suggested that humans are sentient because they're made of neurons! That is, neurons appear to be the only sentient things around, therefore there must be something about them, and only them, that causes sentience. It's kind of a gimmicky argument, but it's interesting.)

(As an aside, David Chalmers has an interesting article somewhere on his website that discusses it. He admits that panpsychism, more or less, is an attractive solution to the problem on some levels. In which case sentient AI would be possible in principle, at least...)

Now, I admit the following could also be possible:

a) AI is impossible without some extra physical step which we don't understand.
b) human neural networks somehow take this extra physical step.
c) we discover the physics behind this phenomenon.
c) we gimmick up some sort of non-human-neural material, or substance, or field, or whatever, that produces sentience, and somehow rig it to a material computational machine, and produce sentient AI.

I mean, I'm not saying it's impossible. But it would merely prove my point ;) and so far, there is no evidence that any old computational system can produce sentience. Consciousness, maybe (and maybe not), but there's no demonstration that the two go hand-in-hand.

[John Page]

Originally posted by the_cave
Anyway, Searle's argument is, I'm sure you've realized by now, quite famous.
From what I've seen, Searle's argument suffers from infinite regression - he says "this is what's in the box" when the example given is inadequate. Make a sponge cake without yeast or self-raising flour and you'll see what I mean when comparing it to one with yeast or self-raising flour. Unless we know what is deficient in the cake, we'll naver make a decent sponge....
Originally posted by the_cave
A "conscious" robot, I suppose, could behave like us, sure--but what guarantee would we have that it experienced things like us at all? None whatsoever.
If you understood how human consciousness was effected then a valid comparison might be drawn. Your conclusion is premature.
Originally posted by the_cave
Now, I admit the following could also be possible:

a) AI is impossible without some extra physical step which we don't understand.
b) human neural networks somehow take this extra physical step.
c) we discover the physics behind this phenomenon.
c) we gimmick up some sort of non-human-neural material, or substance, or field, or whatever, that produces sentience, and somehow rig it to a material computational machine, and produce sentient AI.
a) AI is possible and it exists, maybe not equivalent to human form but progress enough.
b) You assume that the only solution is the current conception of neural networks.
c) look up the difference between simulation and emulation - as long as functional equivalence results the physics doesn't matter.
d) -or second c)- as in me sitting at this computer? :D
Originally posted by the_cave
I mean, I'm not saying it's impossible. But it would merely prove my point ;) and so far, there is no evidence that any old computational system can produce sentience.
Not sure what you're trying to say, here. e.g. There is no evidence that any old (substitute object of choice) can produce sentience.

Cheers, john

[Clutch]

Chemicals just interact. There is no implication of any life whatsoever. If that's your argument, there is quite literally an embarrassment of riches, in the way of reductios of it.

{shrugs}

That's right. What can I say? It is not me advocating materialism or any materialist based metaphysic. Well, if you deny that living things -- say, bacteria -- are more than structured aggregations of chemicals, you can't say much at all. Did you have some 19th C vitalist theory to insert here?
If you do not believe that computers con ever be (phenomenally) conscious, then I have no dispute with you.I do not know exactly how the brain arranges for consciousness; hence I have no reason to reject the prospect of computationally-based consciousness.

You don't know either; and yet you feel enfranchaised to reject the prospect.

[Farren]

{shrugs} That's right. What can I say? It is not me advocating materialism or any materialist based metaphysic.

What are you advocating? Are you religious?

[Clutch]

one could conceivably build a computational system entirely out of Tinkertoys--it would have to be very big, of course. Now, if you can build a sentient computational system, then you could also build it out of Tinkertoys. (I did not coin this example, but i forget the source.) Are you really trying to tell me that a giant brain made of Tinkertoys would have human experiences, rather than, well, the same experiences as a Tinkertoy? How on earth would it?Yet again, the problem is to construct an argument that doesn't work just as well for neurons. (Modulo "big" in this case.) "I just can't imagine how a whole bunch of Tinkertoys could be conscious" is exactly as metaphysically weighty as "I just can't imagine how a whole bunch of neurons could be conscious". When we're speaking from relative ignorance, and when "a whole bunch" is code for "comprising more than all the matter in the universe", what you or I find hard to imagine, sitting here in our chairs, means zero. Less, counting effort.

And your conclusion...It's a fact that no one has answers to these questions right now. So I remain unpersuaded that sentient AI is possible. ... misses the point in any case. AI is an empirical research programme; it doesn't aim to convince you of its success in advance of the fact. As long as these thought experiments are indecisive, as you concede, then AI wins the Searle battle -- his point, recall, being that AI is known to be impossible.

[the_cave]

Originally posted by John Page
From what I've seen, Searle's argument suffers from infinite regression - he says "this is what's in the box" when the example given is inadequate. Make a sponge cake without yeast or self-raising flour and you'll see what I mean when comparing it to one with yeast or self-raising flour. Unless we know what is deficient in the cake, we'll naver make a decent sponge....

Yeah, this is one way of stating my point. You also make a good point about Searle--he seems to have the same defence whenever anyone attacks the Chinese Box argument--"But I've already told you what's in the box!" But what is actually going on in the box is exactly what's at stake.

If you understood how human consciousness was effected then a valid comparison might be drawn. Your conclusion is premature.

Fair enough, but all I'm saying is that eliminative materialists should be held to the same standards ;) In other words, we can't assume a conscious system would have our experiences.

a) AI is possible and it exists, maybe not equivalent to human form but progress enough.

Alright, progress, yes, on a very tiny scale. Not that I don't see it continuing to progress.

b) You assume that the only solution is the current conception of neural networks.

I will agree that there could be other solutions.

c) look up the difference between simulation and emulation - as long as functional equivalence results the physics doesn't matter.

Bah. It does matter when it comes to sentience! Functional equivalence is still just that--only functional equvalence. Both birds and airplanes fly. That doesn't mean they're both alive.

Not sure what you're trying to say, here. e.g. There is no evidence that any old (substitute object of choice) can produce sentience.

Not true--there is good evidence that neurons can produce sentience ;)

[the_cave]

Originally posted by Clutch
Yet again, the problem is to construct an argument that doesn't work just as well for neurons. (Modulo "big" in this case.) "I just can't imagine how a whole bunch of Tinkertoys could be conscious" is exactly as metaphysically weighty as "I just can't imagine how a whole bunch of neurons could be conscious".[/b]

But clearly I can imagine it, because I see it happening. There's something special about neurons--I don't know what it is, but it's there. If you want to argue it's there for Tinkertoys, fine--as I say, panpsychism may well be a valid argument.

And your conclusion...misses the point in any case. AI is an empirical research programme; it doesn't aim to convince you of its success in advance of the fact. As long as these thought experiments are indecisive, as you concede, then AI wins the Searle battle -- his point, recall, being that AI is known to be impossible.

Alright, that's fair--though just because Searle's argument doesn't prove what he wants it to prove doesn't mean it's useless. It's merely a good way of pointing out that there are mental functions yet to be accounted for by either physical theory or computational AI (or cognitive science, for that matter)--namely, sentential experiences.

[John Page]

Originally posted by the_cave
Bah. It does matter when it comes to sentience! Functional equivalence is still just that--only functional equvalence. Both birds and airplanes fly. That doesn't mean they're both alive.
Let me introduce you to my colleague who is functionally equivalent to me and sentience is a key requirement for employment at the Corporeal Corporation. So, we just have to understand what functions are required for consciousness and work on equivalents....

Cheers, john

[Clutch]

just because Searle's argument doesn't prove what he wants it to prove doesn't mean it's useless. It's merely a good way of pointing out that there are mental functions yet to be accounted for by either physical theory or computational AI (or cognitive science, for that matter)--namely, sentential experiencesIt does indeed mean that it's useless. If you take Searle to show that computational AI may or may not work, then he tells us exactly what we knew from the outset; indeed, that's the whole reason for doing AI! It may or may not work; let's find out. "I just can't imagine how a whole bunch of Tinkertoys could be conscious" is exactly as metaphysically weighty as "I just can't imagine how a whole bunch of neurons could be conscious".

But clearly I can imagine it, because I see it happening. There's something special about neurons--I don't know what it is, but it's there.Yes, that would be the point. That's what makes it a reductio. You don't know why, but you know it works.

Once you have a working system, it's rather easy to notice that it works. But your argument against the Tinkertoy brain -- Here's a can of Tinkertoys; surely just a whole bunch of these couldn't be conscious -- would work just as well for neurons absent the advance knowledge that neurons can do it. Because we don't know just causal powers our brains have that are relevant to consciousness; ie, we don't know at what functional level(s) the explanations are best pitched.If you want to argue it's there for Tinkertoys, fine--as I say, panpsychism may well be a valid argument.Nobody's argued panpsychism here. That there might be more than one instantiation of conscious rather strikingly fails to imply that everything instantiates it, for one thing. Notice also that Tinkertoys are your example. I'm discussing it in the spirit of argument "in principle", but the fact is that Tinkertoys do not have "it", as you infelicitously put the phrase. (Infelicitous because there is almost certainly no "it" even for neurons; no single property of neurons, as opposed to their structural interrelations.)

If we are to avoid Searle's tedious trivialities, we must take real constraints on instantiation seriously. The Tinkertoys required to do the job would probably have at least the mass of a large star. And in our universe, for instance, localized objects with the mass of stars have a powerful tendency to be stars, and hence not to support the structural complexity associated with cognitive systems.

The world has already kindly ruled out such counter-examples, in other words.

[the_cave]

Originally posted by John Page
sentience is a key requirement for employment at the Corporeal Corporation.

If this means what I think it means, it seems ad hoc. Alright, sentience is implied for corporeal human beings--but other kinds of beings? Why must that be so?

[the_cave]

Originally posted by Clutch
It does indeed mean that it's useless. If you take Searle to show that computational AI may or may not work, then he tells us exactly what we knew from the outset; indeed, that's the whole reason for doing AI! It may or may not work; let's find out.
It does make it useless for discussing the possibility of strong AI. What I'm saying is, even strong AI doesn't address the question; what is it like to be a strong AI?

Yes, that would be the point. That's what makes it a reductio. You don't know why, but you know it works.
But I only know that it works because I _am_ one. I know that systems like mine produce sentience, because I can experience it directly. I know it works for other people, b/c they're very similar to me--not just behaviorally; materially. But that's not true of other systems. They might behave like me, but it's only at a certain level. Unless we know at what physical level--what material level--subjective experience happens at, we won't know whether an AI is sentient or not, because we won't know how similar they really are to us. And we don't know yet.

But your argument against the Tinkertoy brain -- Here's a can of Tinkertoys; surely just a whole bunch of these couldn't be conscious -- would work just as well for neurons absent the advance knowledge that neurons can do it.
Fortunately for us, we have the advance knowledge! ;) We don't have it for other things. That doesn't mean it couldn't work just as well for other things; but we definitely do not know yet.

You know what I sometimes think? I think that there's an implicit assumption that electronic AI networks will be sentient, simply because they're electronic--in other words, it will be spooky electronic magic ;)

Because we don't know just causal powers our brains have that are relevant to consciousness; ie, we don't know at what functional level(s) the explanations are best pitched.
Yes, yes, right! That's what I'm saying!

Nobody's argued panpsychism here.
Well, no, I'm just saying it's been argued elsewhere. Not that I'm making the argument...

That there might be more than one instantiation of conscious rather strikingly fails to imply that everything instantiates it, for one thing.
Also very true.

Notice also that Tinkertoys are your example. I'm discussing it in the spirit of argument "in principle", but the fact is that Tinkertoys do not have "it", as you infelicitously put the phrase. (Infelicitous because there is almost certainly no "it" even for neurons; no single property of neurons, as opposed to their structural interrelations.)
Ah, but it's those "structural interrelations" that make all the difference, isn't it? What happens when electrons "structurally interrelate"? They exchange chemicals. Do we really think that chemical exchange of ions produces sentience? And even if it does (HOW, I would very much like to know, yet no one has explained it and when people like Penrose actually try and find a physical explanation, they get ridiculed and ignored. This utterly baffles me.) anyway even if it does, what makes us think that electron exchange between circuts will produce the same thing? Spooky electronic magic again...

If we are to avoid Searle's tedious trivialities, we must take real constraints on instantiation seriously. The Tinkertoys required to do the job would probably have at least the mass of a large star. And in our universe, for instance, localized objects with the mass of stars have a powerful tendency to be stars, and hence not to support the structural complexity associated with cognitive systems.

The world has already kindly ruled out such counter-examples, in other words.
Well, that is fortunate, isn't it...

Here's a interesting question; what makes us think neurons don't have the "it", all by themselves? They're unique little creatures after all...

[John Page]

Originally posted by the_cave
If this means what I think it means, it seems ad hoc. Alright, sentience is implied for corporeal human beings--but other kinds of beings? Why must that be so?
Because we're doing the implying - we need definitions, criteria and tests to see if we're just hunting "the ghost in the machine" or sentience is a c;ub one can claim membership of given certain mental capabilities.

Cheers, John

[Garrett]

John Page
Let me introduce you to my colleague who is functionally equivalent to me and sentience is a key requirement for employment at the Corporeal Corporation.
Not fair I think. Behaving as if sentient, and actually being sentient, are not necessarily the same thing.

Respectfully
Garrett

[Kingreaper]

would you all agree that if the mind is purely physical an advanced enough computer could simulate it?

do you agree that the mind is purely phyical?

if the answer to both is yes then an AI could have a mind (i.e. be sentient) merely by simulating a human brain.

if you disagree with the first statement, please explain why (I can't think of any possible valid reasons)

[beowulf_king]

You don't even need an advanced enough computer (hardware). Instead of trying to advance hardware to work like the brain...why not just make software that simulates the brain. We can then simulate things like chemcial activities and such, and create the brain as dynamic and complex as needed.

[Garrett]

kingreaper
would you all agree that if the mind is purely physical an advanced enough computer could simulate it?
Yes, no doubt.

do you agree that the mind is purely phyical?
I'm sure it's not. But I'm not sure exactly what "purely physical" means. "Ideas" and sensation are not purely physical, I would say.

if the answer to both is yes then an AI could have a mind (i.e. be sentient) merely by simulating a human brain.
I expect sentient machines to exist eventually. Well, assume they behave as if sentient. How could we possibly determine whether or not they actually are sentient?

[tronvillain]

The problem with this intuition pump is that asks you to imagine something completely ridiculous: that a large book of rules and an English speaking human manipulating symbols according to the book could realistically simulate someone speaking Chinese. It is a completely insane thought experiment, as the "book" would have to be unthinkably huge and the required manipulations would take an unthinkable amount of time. Perhaps the English speaker is immortal and the "room" exists in a pocket dimension in which time runs faster than in our own? :rolleyes:

Even more ridiculous is Searle's response to the systems reply (which Michaelson mentioned): have the English speaking man memorize the book and do the calculations in his head, and "obviously" there would still be no understanding of Chinese. What?!!! Have a human being perfectly memorize an astronomically large set of symbolic rules and then use them to do the incredibly complex process of processing incoming Chinese to generate a coherent Chinese reply? That is not a response, that is a joke.

Anyway, that the Chinese Room makes many people think "it doesn't understand Chinsese" is little more impressive than the fact that looking at a calculator makes many people think "it doesn't understand math." It does nothing to prove that a digital computer is incapable of being (or probably more accurately, of creating) a thinking thing.

Now, I think it is probably theoretically possible to design something to imitate a thinking thing without actually being one (See "Creation: Life and How to Make It"). That means that I think the Chinese Room might or might not understand Chinese, depending on how it is built. However, any realistic design (one which does not require a lookup table immeasurably larger than the known universe for example) seems likely to be constructed along the lines of a human mind. That is, it is probably easier to reproduce a human-style mind than it is to immitate one.

I believe Searle did the Tinkertoys experiment, except that he used water pipes instead. Anyway the_cave, we are indeed telling you that a giant brain made of tinkertoys would have human experiences (if it were not so large that it would collapse under its own gravity), because what is important is the pattern, not what it is made of. Your arguments appear to be "Human brains appear concious, and human brains are made out of neurons; therefore, neurons are have a consiousness producing quality. A tinkertoy brain appears conscious, but a tinkertoy brain is not made out of neurons; therefore, a tinkertoy brain is not really consious." Why do you grant the consiousness producing property to neurons and not tinkertoys? Perhaps because neurons are very complex, but arguably the tinkertoy neurons would also be very complex, so perhaps it is simply because you like to have a black box to put your magical properties. There is no evidence of neurons having any magical properties that could not possibly be reproduced in other mediums, so it seems to be a matter of how they are put together.

[callmejay]

The whole argument turns on the false assumption that there is something qualitatively different about the human brain than the man-in-room scenario. Any given bundle of neurons in our brain doesn't "understand" English either, but somehow the group does. It's a really stupid argument unless someone can prove that we aren't "just following rules."

[wiploc]

Turing: I have this airtight garage.

Searle: That's not air tight. There is a hole big enough to drive your car in.

Farren: Wrong, Turing doesn't have a car!


The Turing test is faulty. It can give false positives and false negatives. Searle was illustrating the point that it could give a false positive. Farren, do you agree that Turing's test is fauly.

crc