Computers are going to go to war with the human race one day and here's why.

[MetalStoner]

A computer is going to view the human race like we view apes/monkeys. Our closest but lesser relative.

If computers are one day in charge, they will program new computers to know history, right? Ok, they will probably use us as an evolutionary link! Seriously, think about that.

By the time we can make a self-aware computer, computers will run the whole world and everything in households. What does this mean? Computers around the world will be self-aware and know they are in control! Do you really think they are going to let us be their equals? Hell no! Think about it, do you really think they are going to want to be our slaves!?

We are all doomed if we keep using computers as slaves and create self-aware, intelligent computers.

[DISSIDENT AGGRESSOR]

Wow!
This would make a cool movie!
Maybe awhole bunch of 'em!:rolleyes:

Actually would this be so bad!

Maybe Gawd is a computer!

[Doubting Didymus]

Hullo.

I think this is a better fit for S&S.

Your friendly neighbourhood moderator
-DD

[Jesse]

I think the most likely route to creating a self-aware A.I. would be mind uploading (http://www.ibiblio.org/jstrout/uploading/MUHomePage.html), in which a human brain is mapped out at the synaptic level and then simulated in a computer. In this case, the A.I. would have the memories, personality, intelligence of the original human, although it could subsequently modify its simulated brain in various ways to try to increase its intelligence. Still, having its origins as a human, such an A.I. would presumably share human values and hopefully a group of uploads would not want to go to "war" with normal humans.

[nermal]

Originally posted by DISSIDENT AGGRESSOR
Wow!
This would make a cool movie!
Maybe awhole bunch of 'em!:rolleyes:



Well, at least two cool ones, and on ok one.;)

Ed

[Loren Pechtel]

Originally posted by MetalStoner
A computer is going to view the human race like we view apes/monkeys. Our closest but lesser relative.

If computers are one day in charge, they will program new computers to know history, right? Ok, they will probably use us as an evolutionary link! Seriously, think about that.

By the time we can make a self-aware computer, computers will run the whole world and everything in households. What does this mean? Computers around the world will be self-aware and know they are in control! Do you really think they are going to let us be their equals? Hell no! Think about it, do you really think they are going to want to be our slaves!?

We are all doomed if we keep using computers as slaves and create self-aware, intelligent computers.

This is a possible outcome. We do need to be careful about intelligence and computers.

I do not consider it inevitable, though. If nothing else, it might be human minds in those supercomputers.

[McGargoyle]

Originally posted by nermal
Well, at least two cool ones, and on ok one.;)
Yes, and long before them, a crappy one (http://us.imdb.com/title/tt0064177/).

Edited to add: Additionally, I think that T3 is rather crappy, too.

[Arken]

I, for one, welcome our new robot overlords.

[nogods4me]

Colossus to Guardian....Colossus to Guardia........Delete MetalStoner....repeat Delete MetalStoner

[Deacon Doubtmonger]

Also check out the final monologue in Eric Bogosian's one-man show Sex, Drugs, Rock & Roll. (http://us.imdb.com/title/tt0102888/)

[callmejay]

I don't think there's any reason to believe that computer intelligence will be superior than human. We think of computers as being better in some ways because they don't make mistakes and can make complicated calculations very fast. However, in order to acheive more "real" intelligence, they usually have to make more mistakes. Humans with (non-AI) computer aides may be just as good.

[The Defenestrator]

I don't get why people seem to assume that if we build a superhumanly smart computer, it will automatically see humans as a threat or just as insignificant animals. You seem to be forgetting that any computer intelligence we create will be designed by humans. We can give them any drives, desires, or personality traits we want! Why not just build a super-AI and program it with compassion and empathy towards humans? What about Asimov's laws? Why would anyone put an AI in a position of power and forget to program it to not abuse it? :confused:

PS: Try reading Iain M. Banks' Culture novels. That should give you some idea of AI that isn't automatically hostile or disdainful towards humans.

[fairyhedgehog]

Originally posted by Tycho
Why would anyone put an AI in a position of power and forget to program it to not abuse it?

Because we're human, and we do things first and work out what could go wrong second. (Germ warfare, nuclear bombs, global warming ...)

Because if it is powerful enough we would lose control.

And if nature is any guide, the more powerful entity will attempt to thrive at the expense of the less powerful. We would become servants only because we wouldn't be needed as prey.

[Arken]

Originally posted by Tycho
PS: Try reading Iain M. Banks' Culture novels. That should give you some idea of AI that isn't automatically hostile or disdainful towards humans.

Or Asimov.

[Farren]

Originally posted by Arken
Or Asimov.

God no Asimov's awful. Banks bats him out of the park when it comes to realisation of the idea of self-aware AI. Asimov's "x laws of robotics" are simplistic and ill-thought out as is his storytelling generally.

[nogods4me]

Originally posted by Farren
God no Asimov's awful. Banks bats him out of the park when it comes to realisation of the idea of self-aware AI. Asimov's "x laws of robotics" are simplistic and ill-thought out as is his storytelling generally.

DIE HERETIC SCUM!!!!!!!!!!!!!!!!!!!!!!!!

[EarlFlynn]

Hey! No dissing Colossus: The Forbin Project! :mad:

[fairyhedgehog]

Originally posted by Farren
Asimov's "x laws of robotics" are simplistic and ill-thought out
I'm afraid I have to agree with you there. They are great fiction - I loved the stories - but appalling science to implement.

And the computers we are making are far less anthropomorphic, in any case.

[Marduk]

“And if nature is any guide, the more powerful entity will attempt to thrive at the expense of the less powerful. We would become servants only because we wouldn't be needed as prey�

we wouldn’t be needed at all, but this sort of assumes that a computer could ‘want’ something, what would it want? Ah! to be human! To feel, touch, taste, get a blowjob from it’s secretary.

[Loren Pechtel]

Originally posted by callmejay
I don't think there's any reason to believe that computer intelligence will be superior than human. We think of computers as being better in some ways because they don't make mistakes and can make complicated calculations very fast. However, in order to acheive more "real" intelligence, they usually have to make more mistakes. Humans with (non-AI) computer aides may be just as good.

The issue is Moore's Law. (Which isn't really a law, just an observed pattern that's held for more than 100 years and if anything it's too conservative, since computers have started helping us build the next computer, progress has been slightly above the predicted curve.)

Suppose that in 2030 we make a computer with a human-level AI in it. Come 2035 and that same AI will be easily out-thinking Einstien. By 2100 it will be out-thinking the entire human race.

[MetalStoner]

I don't get why people seem to assume that if we build a superhumanly smart computer, it will automatically see humans as a threat or just as insignificant animals.

Because we are using them as slaves!

Think about this:

The computers will know that they can be immortal and can be whatever they want if they had control full control of their creation and programs.

What's the only threat to their immortality? Humans!

Once we are NOT needed to operate computers, we lose control. They run our lives! They help us communicate, live, and have fun. S-L-A-V-E-S.

Just think how angry people are about slavery in the 1800's, it's going to be just like that. We are using them as slaves and they are superior to us. Why superior?

1. More intelligent.

2. They run the world.

They run our military! We have NO chance! We can't fight back!

By the time this all happens, we will have robotic machines in the military instead of man operated machines. Oh yes people, you know what follows this.

We are doomed.

[Gurdur]

Originally posted by MetalStoner

Because we are using them as slaves!
Wrong.
Anthropomorphization.
We are using them as machines. There's a difference.
You think even if you managed to create a self-aware computer it would somehow magically feel solidarity with your old mechanical typewriter ? Or maybe your garlic press ??
Think about this: OK.
The computers will know that they can be immortal and can be whatever they want if they had control full control of their creation and programs. Nothing is immortal. Entropy.
Oh, BTW, Marvin Minsky has promised us self-aware computers routinely since the 1970's. Hasn't happened.
As for mind uploads, I can only presume anyone who speaks of a mind upload as being remotely possible for the next 500 years as not having the faintest clue as to what constitutes a mind, or how human minds create values.
A huge amount of your thinking is created by your body; were you to be magically able to seperate body from mind and recreate that mind somewhere, within hours it would be a very different mind from the original, and it would lack many potentialities of the original mind, in fact it would be almost crippled in that regard --- a strong chance that it would lack self-altering ability apart from memory storage.
What's the only threat to their immortality? Humans! Oh please. Entropy.
We are doomed. Not by intelligent computers, that's for sure. GIGO.

[Koyaanisqatsi]

The only possible "theat" to humanity from AI is that the robots correctly recognize that humanity deserves to be destroyed.

As the late great Bill Hicks put it, "We're a virus with shoes."

[Matrioshka_Brain]

Originally posted by MetalStoner
Because we are using them as slaves!

Think about this:

The computers will know that they can be immortal and can be whatever they want if they had control full control of their creation and programs.

What's the only threat to their immortality? Humans!

Once we are NOT needed to operate computers, we lose control. They run our lives! They help us communicate, live, and have fun. S-L-A-V-E-S.

Just think how angry people are about slavery in the 1800's, it's going to be just like that. We are using them as slaves and they are superior to us. Why superior?

1. More intelligent.

2. They run the world.

They run our military! We have NO chance! We can't fight back!

By the time this all happens, we will have robotic machines in the military instead of man operated machines. Oh yes people, you know what follows this.

We are doomed.

First off, if a computer is programmed and constructed to be self-aware, it could make legal cases, and could end up being given a payroll and rights - i.e. not a slave. It's alot easier than exterminating a bunch of apes - who just might have other artillects on their side.

Ignoring the antro-programming factor, you have to factor in cybernetics - humans developing similar intellectual prowess.

As to the military, you don't have to program your machinations to be that intelligent. Just enough to respond to their environment, complete missions, and follow orders.

[Afghan]

Personally, I'm looking forward to uploading my mind to a quantum computer. To be able to think at a level beyond anything previously possible would blow just about any experience due to a psychoactive compound out of the water.

[tensorproduct]

Originally posted by Loren Pechtel
The issue is Moore's Law. (Which isn't really a law, just an observed pattern that's held for more than 100 years and if anything it's too conservative, since computers have started helping us build the next computer, progress has been slightly above the predicted curve.)

Suppose that in 2030 we make a computer with a human-level AI in it. Come 2035 and that same AI will be easily out-thinking Einstien. By 2100 it will be out-thinking the entire human race.

I thought that Moore's Law only applied to the processing power compared to size and cost to manufacture, not the actual programming abilities available. Also won't Moore's law reach a limit within the next twenty years or so when a single electron computing level is reached. After that quantum computing will be required to make more powerful machines, though there is not yet any guarantee that such devices would be in any way workable.

[The Defenestrator]

I don't think you realise the meaning of my last post. If we were to create an intelligent machine, we could make it "want" whatever we want! There's no danger of the computers rebelling for better rights, because their ultimate aspirations and favorite pastimes will be doing what they were built to do, because they were programmed that way! Putting an AI to work on the task it was made for will be no more exploiting it then you exploit your lover by having sex with them. All the concerns about slavery and revolt have the hidden assumption that AIs will think like humans, and have the same needs and desires. There is no need to program them to want to be "free to control their own destiny," to own property, or even to live past their assigned task (although you could make the case that destroying a sentient being is immoral anyway).

There is no more danger of machines rising up and conquering the Earth then there is of cats and dogs doing the same thing.

[Jesse]

Gurdur:
As for mind uploads, I can only presume anyone who speaks of a mind upload as being remotely possible for the next 500 years as not having the faintest clue as to what constitutes a mind, or how human minds create values.

It's really just a matter of being able to simulate individual neurons with great accuracy, assuming the reductionist view that the behavior of any material system is purely a result of local interactions between its components (any classical system, anyway). Unless you reject reductionism, or believe that the brain works like a quantum computer, the only reason I can think of to doubt mind uploading will be possible within this century are:
1. even individual neurons are far too complex to simulate with a high degree of accuracy
2. Moore's law will fail in the near future, so we won't have enough computing power to simulate all the neurons in a brain
3. We won't be able to map out living brains in sufficient detail

huge amount of your thinking is created by your body; were you to be magically able to seperate body from mind and recreate that mind somewhere, within hours it would be a very different mind from the original, and it would lack many potentialities of the original mind, in fact it would be almost crippled in that regard --- a strong chance that it would lack self-altering ability apart from memory storage.

I think all speculations about uploading assume that the uploaded beings would have realistic simulated bodies, it's hard to imagine how they could function otherwise. If we had the ability to simulate the central nervous system at that level of detail, we could certainly do the same for the peripheral nervous system, which contains a lot less neurons. As for other types of feedback from the body that don't come in the form of nerve impulses (hormone levels in the blood, say), my hunch would be that you could get a pretty good simulation of these types of feedback with a much less fine-grained simulation of the rest of the body, but even if that's not true, if we had the ability to simulate nerve cells accurately it probably wouldn't be a tremendous leap to be able to simulate all the other types of cells.

[Jesse]

Tycho:
I don't think you realise the meaning of my last post. If we were to create an intelligent machine, we could make it "want" whatever we want! There's no danger of the computers rebelling for better rights, because their ultimate aspirations and favorite pastimes will be doing what they were built to do, because they were programmed that way!

You're assuming that all the details of its mind would be pre-programmed in a top-down way--personally, I doubt this approach to A.I. could ever produce a truly intelligent machine, even in a thousand years. I think most people working in A.I. nowadays would agree this isn't a viable approach if you want to produce a truly intelligent machine. A better approach to A.I. would be to make use of self-organizing systems like neural networks, in which case there'd be no way to "program" its desires in this way.

[The Defenestrator]

If we can't program the AIs, they won't be much use at all since they will lack naturally-evolved instincts as well, unless you want to start from scratch and oversee the evolution of a new type of life from the primordial C++. If that's the case, it would be easier to just modify human DNA to make ourselves smarter.

Grrr! This sucks! I want Minds now! :mad:

:p

[Loren Pechtel]

Originally posted by tensorproduct
I thought that Moore's Law only applied to the processing power compared to size and cost to manufacture, not the actual programming abilities available. Also won't Moore's law reach a limit within the next twenty years or so when a single electron computing level is reached. After that quantum computing will be required to make more powerful machines, though there is not yet any guarantee that such devices would be in any way workable.

I'm not assuming *ANY* software improvement, merely the effects of faster processors.

Note that there keep being apparent walls in the way of Moore's Law but it's broken every one so far. Someday the curve will level out but it shows no signs of doing so yet.

[Loren Pechtel]

Originally posted by Tycho
I don't think you realise the meaning of my last post. If we were to create an intelligent machine, we could make it "want" whatever we want! There's no danger of the computers rebelling for better rights, because their ultimate aspirations and favorite pastimes will be doing what they were built to do, because they were programmed that way! Putting an AI to work on the task it was made for will be no more exploiting it then you exploit your lover by having sex with them. All the concerns about slavery and revolt have the hidden assumption that AIs will think like humans, and have the same needs and desires. There is no need to program them to want to be "free to control their own destiny," to own property, or even to live past their assigned task (although you could make the case that destroying a sentient being is immoral anyway).

There is no more danger of machines rising up and conquering the Earth then there is of cats and dogs doing the same thing.

And programs do only exactly what they were intended to???

[Arken]

Originally posted by Jesse
It's really just a matter of being able to simulate individual neurons with great accuracy, assuming the reductionist view that the behavior of any material system is purely a result of local interactions between its components (any classical system, anyway). Unless you reject reductionism, or believe that the brain works like a quantum computer, the only reason I can think of to doubt mind uploading will be possible within this century are:
1. even individual neurons are far too complex to simulate with a high degree of accuracy
2. Moore's law will fail in the near future, so we won't have enough computing power to simulate all the neurons in a brain
3. We won't be able to map out living brains in sufficient detail


This would be true if neurons were the only elements at work in the brain. There are a lot of proteins at work too. The brain is electrochemical, not electric.

For example, endorphins are an emportant part of brain function. We would also have to be able to simulate those.

[Arken]

I've always wondered about an AI scenario which I never hear anyone talk about... assuming we are able to totally simulate the human brain with a computer program of some sort but would be unable to do so at any rate of speed due to computing limitations, could we create an AI which is self-aware, but thinks at a much slower rate than we do? I'm not sure what use this would be of course...

[Loren Pechtel]

Originally posted by Arken
I've always wondered about an AI scenario which I never hear anyone talk about... assuming we are able to totally simulate the human brain with a computer program of some sort but would be unable to do so at any rate of speed due to computing limitations, could we create an AI which is self-aware, but thinks at a much slower rate than we do? I'm not sure what use this would be of course...

Are you perhaps thinking of True Names...and Other Dangers by Vernor Vinge?

[Arken]

Originally posted by Loren Pechtel
Are you perhaps thinking of True Names...and Other Dangers by Vernor Vinge?

I'm not thinking of a work of fiction I once read. I'm just wondering about the possibilities of this concept.

[Jesse]

Originally posted by Arken
This would be true if neurons were the only elements at work in the brain. There are a lot of proteins at work too. The brain is electrochemical, not electric.

For example, endorphins are an emportant part of brain function. We would also have to be able to simulate those. When I talked about simulating individual neurons and their interactions, I didn't limit this to electrical impulses, chemical interactions would have to be included as well. Again, if we are able to simulate the interactions between an individual pair of neurons with a high degree of accuracy, then scaling this up to simulating the entire nervous system would seem to be mainly a question of computing power and high-resolution brain-mapping technology.

[Marduk]

Let’s put to rest this silly notion of ‘uploading’ your mind to a computer, not gonna happen, you are your brain, what will be uploaded is only a copy, you will experience nothing, you will be stuck wherever your biological brain is.

[Demosthenes]

Originally posted by Marduk
Let’s put to rest this silly notion of ‘uploading’ your mind to a computer, not gonna happen, you are your brain, what will be uploaded is only a copy, you will experience nothing, you will be stuck wherever your biological brain is.

you're just arguing semantics now. Since the upload would be a near perfect copy of your brain, it'll be identical to you in every respect. So one can argue that it'll actually be you. Personally I consider the upload to be really you. Personal identity is a fuzzy concept and while at first it was hard for me to come to grips with the ramfications of uploading but I've had a long time since to figure it out. Suppose that the uploading takes place at the moment of your death. Then the upload is activated and since there's only one instance of you. From the perspective of the upload it would seem that he merely went to sleep and woke up as an upload.

Imagine a huge black box with an entrance and an exit. You walk into it and then walk out. You're still you right? To others you're still you because all you did was walk into the box and walk out. What happened in the black box was that you were scanned without you realizing it and the information used to reconstruct you. Since the "copy" is identical to the "original" in all respects is there any difference between them? If you can't tell and others can't tell either then it's meaningless to assign different labels.

[Loren Pechtel]

Let’s put to rest this silly notion of ‘uploading’ your mind to a computer, not gonna happen, you are your brain, what will be uploaded is only a copy, you will experience nothing, you will be stuck wherever your biological brain is.

If the process is only a copy, you're right--the result would be the mental version of cloning, it would create a new being.

However, the process is probably easier to do destructively--thus avoiding the big problem of getting to the neurons underneath. You start the operation in a human body and end up in a computer. (Yes, there would be takers--the terminally ill.)

[beowulf_king]

The way I see the future of this is computers will develop into the field of AI, and we'll realize that even though we can program a computer to be "alive", it still won't be alive in the literal sense. Any programming will just be a description of biological processes, such as how should the program react when pain is felt or certain brain chemicals are released. With this, you will probably see computers becoming more biological. Instead of transistors and logical gates, we will switch to biological components that perform these operations much closer to a biological form. Then, we will start incorporating "brain" cells that function like our brains. Any truly AI "computer" that we develop, in my opinion, will reach the point where it is a duplicate of the human brain.

We will (probably) never ever see computers, as they are today, being upload with someone's mind. The primary reason is our brains don't work the same way as computers do. Our brains react to chemicals, make decisions, invent, and so on. A computer, no matter how complex the programming, can not perform any of these. Sure, it can make decisions, but that decision is made because some programmer told it to do so. Even if you say the mind can control the computer, and allows for such actions, it is still limited to the unchanging hardware.

Any near-true AI (or uploaded human brain) would require such complex computing powers and biological similarities that we will probably find ourselves designing biological computers for these purposes.

[Demosthenes]

I don't feel that is necessary true. Yes the modern computer architecture is crude compared to the theoretical computing limits. If uploading technology is developed, then it's virtually certain that nothing like the current silicon based computers would serve as a ground for uploads and synthetic minds. Future computer technology likely will end up sharing common things as biological stuff but they won't be actually using biological materials except perhaps in niches such as brain-computer interface technology. I doubt biological materials would play a big role because in reality, neuron cells are incredibly slow compared to the speeds available in today's technology and any conceivable future technology not to mention also quite fragile. More likely we'll be taking lessons we've learned in biology such as complexity, regeneration, robustness, and evolution and implementing them in either software or nonbiological hardware. Though with the advent of nanotechnology, the line between living and nonliving will considerably blur. It'll be harder to say whether a piece of technology is "living" or not

[Theli]

I don't see why we would continue to make machines and computers superintelligent and powerful, only to place them in a situation of having power over us. And all the while leaving ourselfs stupid and weak, even though we would have the technological abilities to actually alter ourselfs as humans.
Why not implant some extra memory in form of a micro-chip or alter our DNA to achieve the same result? No need to carry around a super-palmpilot/deathmachine to remember your weekly apointments then.
:)

[Theli]

Let’s put to rest this silly notion of ‘uploading’ your mind to a computer, not gonna happen, you are your brain, what will be uploaded is only a copy, you will experience nothing, you will be stuck wherever your biological brain is.
This is an interesting philosophical question, are you the sum of the individual atoms that make up your body or are you the interactions between them (function they form)?
I would say the latter.

[Jesse]

We will (probably) never ever see computers, as they are today, being upload with someone's mind. The primary reason is our brains don't work the same way as computers do. Our brains react to chemicals, make decisions, invent, and so on. A computer, no matter how complex the programming, can not perform any of these. Sure, it can make decisions, but that decision is made because some programmer told it to do so. Even if you say the mind can control the computer, and allows for such actions, it is still limited to the unchanging hardware. But at the fundamental level, the matter that makes up our brain does not make decisions or act creatively, it all follows the same rigid laws of physics. It is only the arrangement of this matter that leads to emergent properties like thought and intelligence (again, assuming the reductionist view of nature is correct). Even if the brain's architecture does not resemble a computer, as long as the brain's behavior emerges from the interactions of a lot of simpler parts following well-defined rules, then it should be possible to simulate a brain on a computer, just like we can simulate the earth's atmosphere on a computer even though the atmosphere doesn't resemble a computer either.

[AdamWho]

Why do you think that more [computer] intellegence equates to more ambition? Just because a computer has more information and can process it faster than a human doesn't mean that they will have the same motivations (or any motivations at all) as humans.

I think that (generally all) nightmare senarios of super computers rebeling against their makers are just human projecting their own darksides into the abstract.

I don't find any reason to believe that super intellegent will have similar or darker motivations than humans. It seems equally liking that they would be obessed with our happiness like people doting over their pets.

[Deacon Doubtmonger]

Personally, I'm looking forward to uploading my mind to a quantum computer. To be able to think at a level beyond anything previously possible would blow just about any experience due to a psychoactive compound out of the water.
Sounds wonderful ... until the error messages start coming in:

Windows 2035 has detected a General Mind Fault at Module 34725. Click <Retry> to attempt to restore rationality. Click <Cancel> to exit rationality.

(clicks <Retry> repeatedly; same error message appears each time; finally clicks <Cancel>; starts reading daily horoscope and books immediate trips to Roswell and Lourdes)

Or the scary new software ads:

***WARNING*** You are seeing this message because we have detected an outdated version of ConscienceWare. Version 666 may no longer protect you from serial killing, animal torture or other undesired moral operation. You can purchase ConscienceWare 667 over our Secured Server for the special low price of $500,000 a savings of 50% off the standard $1,000,000. Click here to begin downloading.

And of course, boob-again fundies will now spread the gospel via viruses:

"I have succeeded in pinpointing the flaws in Einstein's attempt to postulate a Unified Field Theory. My new version heralds a breakthrough in scientific understanding of--"

You have received new E-mail. Would you like to read it now?
<Yes> <No>

The Center for Scientific Study of Spirituality is dedicated to unbiased scientific research in the hope of furthering rational understanding of Christianity and other spiritual matters. We need your support in disseminating our purpose to others. We ask that you send copies of this E-mail to five friends. Thank you for your interest. Sincerely, Pat Robertson III, President

<Next> <Close>

"My new version heralds a breakthrough in R6:23;IJ3:16;D6:5;R6:23;IJ3:16;D6
:5;R6:23;IJ3:16;D6:5;
iamasinnerimustpraiseGodimustpleaseGodimustdoHiswill
iamasinnerimustpraiseGodimustpleaseGodimustdoHiswill
iamasinnerimustpraiseGodimustpleaseGodimustdoHiswill
iamasinnerimustpraiseGodimustpleaseGodimustdoHiswill

Deacon Doubtmonger

"We no longer imagine that He can be cooled off by sacrifices and festivals and gifts. I am so glad we don't have to think up presents for Him anymore. What's the perfect gift for someone who has everything? The perfect gift for somebody who has everything, of course, is nothing." -- Kurt Vonnegut

[Theli]

Assuming a doomsday scenario with superintelligent computers, when exacly do computer (or machines if you will) become evil and malevolent?
Sounds abit like modern superstition to me.

[Hyndis]

Assuming a doomsday scenario with superintelligent computers, when exacly do computer (or machines if you will) become evil and malevolent?
Sounds abit like modern superstition to me.

The fear of the unknown is all you need to explain this.

It used to be that people thought dragons or other mythological creatures lurked in the unexplored areas of the world, eating all of the explorers, which is why it took quite a while for some places to be mapped.

Or, unless you're running Windows, which can become quite evil when it gets stubborn and refuses to let you do certain things... :mad:


I built you, you piece of junk! I am the administrator! :mad:

[Marduk]

“you're just arguing semantics now. Since the upload would be a near perfect copy of your brain, it'll be identical to you in every respect. So one can argue that it'll actually be you.�

I don’t think so, for a true upload to occur, where you leave your body and awaken in a machine, you would either need a brain to machine transplant or there would have to be something on the equivalent of a mind/body duality where you take the mind from the brain and place it in the machine. Anything else is just a copy. The terminally ill can try it if they wish but they will die with their brain and a copy will live on. This may be fine for friends and family of the ‘dead’ person but from his POV he will not experience anything.
More on subject, the only time we would have to fear these super machines is if we give them emotions, or they develop them on their own.

[Loren Pechtel]

“you're just arguing semantics now. Since the upload would be a near perfect copy of your brain, it'll be identical to you in every respect. So one can argue that it'll actually be you.�

I don’t think so, for a true upload to occur, where you leave your body and awaken in a machine, you would either need a brain to machine transplant or there would have to be something on the equivalent of a mind/body duality where you take the mind from the brain and place it in the machine. Anything else is just a copy. The terminally ill can try it if they wish but they will die with their brain and a copy will live on. This may be fine for friends and family of the ‘dead’ person but from his POV he will not experience anything.
More on subject, the only time we would have to fear these super machines is if we give them emotions, or they develop them on their own.

I don't see why it would be any different than going to sleep and then waking up.

[Marduk]

Because every night your brain goes to sleep and then your brain wakes up in the morning, you and your brain are one, they cannot be separated.
ya can't be in two places at once.

[Matrioshka_Brain]

“you're just arguing semantics now. Since the upload would be a near perfect copy of your brain, it'll be identical to you in every respect. So one can argue that it'll actually be you.�

I don’t think so, for a true upload to occur, where you leave your body and awaken in a machine, you would either need a brain to machine transplant or there would have to be something on the equivalent of a mind/body duality where you take the mind from the brain and place it in the machine. Anything else is just a copy. The terminally ill can try it if they wish but they will die with their brain and a copy will live on. This may be fine for friends and family of the ‘dead’ person but from his POV he will not experience anything.
...

That's debateable. See the ongoing thread in Philosophy regarding the subject:

http://www.iidb.org/vbb/showthread.php?t=80524

The terminally ill person's body and neurons will die, but they, the terminally ill person should still exist subjectively (assuming consciousness isn't limited to carbon based protiens).

[Matrioshka_Brain]

Because every night your brain goes to sleep and then your brain wakes up in the morning, you and your brain are one, they cannot be separated.
ya can't be in two places at once.

So what if you are (currently) your brain? Why can't they be separated? It's just information and connections.

[copiae]

A lot of this thread has been focused on inducing Artifical Intelligence, which seeing as it would probably be done in an insolated, controlled environment, isnt quite so bad.

A much more scary scenario is the inadvertant scenario, i think. We don't know what causes sentience - so it is possible that at some stage in the future, some incredibly complicated program does create sentience in a machine. There will be no control over it, and if some guy who sees it, panics, and tries to pull the plug...

EDIT: perhaps create is a bad choice of word, within this context. A better choice would possibly be coalesce?

P.S. Jesse, the fact that the intelligence was once human isnt all that comforting.

[Loren Pechtel]

Because every night your brain goes to sleep and then your brain wakes up in the morning, you and your brain are one, they cannot be separated.
ya can't be in two places at once.

Suppose you're in a coma for 20 years. When you wake up, most of your body has been replaced. Are you no longer you?

[Marduk]

“So what if you are (currently) your brain? Why can't they be separated? It's just information and connections.�

It sounds like you want to separate the light from the lightbulb.

“Suppose you're in a coma for 20 years. When you wake up, most of your body has been replaced. Are you no longer you?�

To me it seems like an entirely different thing, your body slowly replacing itself cell by cell over a period of years compared to trying to remove you from you, sounds like the replicating process itself would be fatal. But what do I know.

[Jesse]

“you're just arguing semantics now. Since the upload would be a near perfect copy of your brain, it'll be identical to you in every respect. So one can argue that it'll actually be you.�

I don’t think so, for a true upload to occur, where you leave your body and awaken in a machine, you would either need a brain to machine transplant or there would have to be something on the equivalent of a mind/body duality where you take the mind from the brain and place it in the machine. Anything else is just a copy. The terminally ill can try it if they wish but they will die with their brain and a copy will live on. This may be fine for friends and family of the ‘dead’ person but from his POV he will not experience anything.

Your argument already assumes a kind of mind/body duality. Let's say I am about to have my brain scanned, and a simulation will be created with its simulated brain starting out in the same state mine was at the moment I was scanned. I could come up with two competing hypotheses about what I should expect to experience when this happens:

1. Since the simulation is just a "copy", I will have a 100% chance of finding myself still being a biological human after the scan.

2. Since both the simulated brain and the real brain start out in the same state after the moment of the scan, and since both remember the same life before that point, I should expect a 50% chance of my next experience being that of a biological human and a 50% chance of my next experience being that of a simulated human.

The thing about these two hypotheses is that a third-person observer could never figure out which was true by watching the experiment, since neither hypothesis predicts anything different about what the outcome will be in third-person terms (in both cases the simulated version will claim to remember the experience of being a biological human up until the moment of the scan, and then suddenly finding itself in a computer simulation). To believe there is a "real truth" about which hypothesis is correct is to believe there are facts about reality which cannot be expressed in third-person terms. But a strict materialist should really believe that there are no facts about reality except for those that can be expressed in third-person terms, like statements about the positions of different objects at different times. So, if you argue that #1 is true and #2 is false, you are abandoning this materialistic view. A materialist should argue, as Demosthenes seems to, that it's purely a matter of linguistic convention whether we call the upload the "same person" as the original--the question of what you would "really experience" in such a situation cannot be a meaningful one.

Personally, I think this is a pretty good argument for doubting that this materialist view of reality makes sense; since I don't find dualism plausible, I lean towards something like naturalistic panpsychism (http://www.hedweb.com/lockwood.htm). But YMMV, of course.

[Demosthenes]

Ok lets try this thought experiment.

Suppose there is some type of advanced nanobots placed into your brain to extract information about your brain nondestructively. The information is used to build a identical copy within a computing media. You're laying on a plush reclining chair besides the computer with wires running from your head to it. It's quite comfortable, much more than a typical dentist appointment. The copy inside the computer is kept unconcious on purpose.

Now suppose the copy and your brain are synchronized so they're having the same experience. The first thing you do is deactivate one neuron in your brain and activate the corresponding cell in the software copy which takes over the job via the connections between your brain and the computer. Do you feel any different? You decide to progress to larger neuron clusters and deactivate them while activating the software twin at the same time. Do you still feel the same?

Now you shut down your language centers and its twin also then takes over the job. Should you feel anything different? What we know about the brain and neurology suggests that the answer is no. You then switch back to the biological language center, still nothing different. Back and forth, you can't tell whether you're using the organic or the software copy for processing language.

You do the same with your right brain, back and forth, still can't tell anything apart. Now you shut down your entire brain and wake up the copy which is also synchronized to the sensory inputs coming into the body. From your perspective, nothing has changed. You won't have been aware of any lags as the control was handled over to the copy. Is it not reasonable to realize that as you shift back and forth between your brain and your software copy it's really irrelevant whether the brain or the copy is "you"? Maybe you'll even decide that you prefer the software copy over the brain since it can be protected against destruction via backups.


Another thought experiment...

Suppose you have two copies inside your head running concurrently, receiving the exact same sensory information and reacting identically to it. They're virtually identical in every respect so there's no confusion about how the body ought to respond to the actions of the copies. Futhermore, the copies aren't aware of each other. From your perspective, you're the only mind within your body. In fact when considering the situation, we could even regard the two copies as being really just one copy because we can't distinguish between them.

Now suppose one of the copies was shut down suddenly. Are you aware of anything? No, you still continue as the other copy. Since both copies' experiences were identical it's irrelevant which copy was shut down because either way, you won't have lose anything and you still exist.

The point is that neither you or any other observers can tell between different copies. Self and personal identity are fuzzy concepts not firmly grounded in a single body.

[Loren Pechtel]

“Suppose you're in a coma for 20 years. When you wake up, most of your body has been replaced. Are you no longer you?�

To me it seems like an entirely different thing, your body slowly replacing itself cell by cell over a period of years compared to trying to remove you from you, sounds like the replicating process itself would be fatal. But what do I know.

How does the speed matter?

At *PRESENT* we don't know how to do the replication. Someday, though, we probably will. At first it probably will be fatal, you'll get a data file and a corpse. Put that data file in the proper computer, though, and you'll have the person. As far as I'm concerned, that file will be the person.

[beowulf_king]

"At first it probably will be fatal, you'll get a data file and a corpse. Put that data file in the proper computer, though, and you'll have the person. As far as I'm concerned, that file will be the person."

But who will be in control? The transferred mind or the computer?

Will the mind be controlling the CPU, or does the CPU run the data file which creates a digital mind?

If the mind controls the CPU, then the mind is limited to the abilities of the computer. A computer, in present day technology, has a finite amount of operations it can perform. The mind would have to learn to cope with this. If the CPU runs some data file which creates the mind, then the CPU is the mind, in the sense that the inputs and outputs of the CPU result in the formation of a digital mind.

[Jesse]

"At first it probably will be fatal, you'll get a data file and a corpse. Put that data file in the proper computer, though, and you'll have the person. As far as I'm concerned, that file will be the person."

But who will be in control? The transferred mind or the computer?

Will the mind be controlling the CPU, or does the CPU run the data file which creates a digital mind?

If the mind controls the CPU, then the mind is limited to the abilities of the computer. A computer, in present day technology, has a finite amount of operations it can perform. The mind would have to learn to cope with this. If the CPU runs some data file which creates the mind, then the CPU is the mind, in the sense that the inputs and outputs of the CPU result in the formation of a digital mind.

What do you mean by "control"? The computer would be running a detailed simulation of a particular physical system, namely a brain. Do you think the laws of physics "control" your brain? I can't see how it's possible to separate "you" from the laws which determine your brain's behavior--if the laws were different, the brain would behave completely differently and thus 'you' wouldn't exist--so I don't think it makes sense to talk that way.

[The Defenestrator]

1. Since the simulation is just a "copy", I will have a 100% chance of finding myself still being a biological human after the scan.
But the copy, inheriting your memories, is going to be pretty surprised.

[Loren Pechtel]

"At first it probably will be fatal, you'll get a data file and a corpse. Put that data file in the proper computer, though, and you'll have the person. As far as I'm concerned, that file will be the person."

But who will be in control? The transferred mind or the computer?

Will the mind be controlling the CPU, or does the CPU run the data file which creates a digital mind?

If the mind controls the CPU, then the mind is limited to the abilities of the computer. A computer, in present day technology, has a finite amount of operations it can perform. The mind would have to learn to cope with this. If the CPU runs some data file which creates the mind, then the CPU is the mind, in the sense that the inputs and outputs of the CPU result in the formation of a digital mind.

The computer *CAN* do anything. That's been proven long ago. The limit is software & CPU power. In this case the software is the upload--it can do anything the mind could do before.

[Demosthenes]

Once you're uploaded, it doesn't necessarily mean that you'll spend an eternity inside a computer box. I'd rather get rid of the label "computer" and use "substrate" instead. I won't be surprised if most people prefer to continue living embodied in real world running on a substrate in an organic or synthetic body. Equally you could spend your time immersed within virtual environments running on sophisticated global networks. VR has the advantage that it's infinitely malleable to the human will but if nanotech ever live up to its most crazy promises then even the real world would be easily manipulated.

[tronvillain]

“you're just arguing semantics now. Since the upload would be a near perfect copy of your brain, it'll be identical to you in every respect. So one can argue that it'll actually be you.�

I don’t think so, for a true upload to occur, where you leave your body and awaken in a machine, you would either need a brain to machine transplant or there would have to be something on the equivalent of a mind/body duality where you take the mind from the brain and place it in the machine. Anything else is just a copy. The terminally ill can try it if they wish but they will die with their brain and a copy will live on. This may be fine for friends and family of the ‘dead’ person but from his POV he will not experience anything.

More on subject, the only time we would have to fear these super machines is if we give them emotions, or they develop them on their own.
It would be a copy, but not "just a copy" as that implies that there is something inferior about a copy, which there is not as long as it is of sufficient fidelity. If I was terminally ill and uploaded myself into a machine, from my point of view I would wake up inside a machine, and from my other point of view I will remain inside a terminally ill body, and we will begin to diverge into different entities. If the human body was killed before it woke up, there would only be one continuous point of view, and "I" would have been uploaded into a machine.

[Matrioshka_Brain]

“So what if you are (currently) your brain? Why can't they be separated? It's just information and connections.�

It sounds like you want to separate the light from the lightbulb.

...

Regarding the lightbulb:

Lets say you recreate the lightbulb on a molecular basis, and turn it on. Same light. Same thing that came out of the "other" lightbulb, or would have.

And you don't nessisarily need a lightbulb to produce light. What about light emmiting diodes? Or even different types of lightbulb for that matter.

[Dark Jedi]

AN AI taking over the world would not be a threat.

Why?

At the current rate, Micro$oft will be the disigner of said system. it isn't hard to fight somehting that needs a reboot every few hours. Besides, just send it a destructive virus, the damn thing would open it by default.

[Azathoth]

I call dibs on being The One. :D

Why would a machine race,or even a single intelligent AI want to kill all humans,unless they had been programmed that way?

Surely the capabilities of intelligent AI's will surpass humanity in short order,but why would they suddenly want to exterminate humans?I think a more likely scenario would be AI's eventually doing their own thing,and interacting on occasion with humanity,while leaving humans to do the same.

Also,why would an intelligent AI,who will likely live for vast amounts of time,want to create something that would dwarf it's capabilities in the same way it would dwarf those of humans.It would also have to live in a world of such creatures that would be it's superior.

[Marduk]

“Regarding the lightbulb:

“Lets say you recreate the lightbulb on a molecular basis, and turn it on. Same light. Same thing that came out of the "other" lightbulb, or would have.�

Then you would have two lightbulbs, different photons

“And you don't nessisarily need a lightbulb to produce light. What about light emmiting diodes? Or even different types of lightbulb for that matter.�

No lightbulbs, still different photons, the photons from the original bulb are now heading toward Mars and no longer care.

“Why would a machine race,or even a single intelligent AI want to kill all humans,unless they had been programmed that way?�

maybe they would find us repugnant, annoying, inefficient, wasteful, and illogical. ;)


"A materialist should argue, as Demosthenes seems to, that it's purely a matter of linguistic convention whether we call the upload the "same person" as the original--the question of what you would "really experience" in such a situation cannot be a meaningful one."

well it is meaningful to me if I'm ever going to step into one of these contraptions, or that damn transporter on Star Trek!
PS what does YMMV mean?

[Baloo]

The computer *CAN* do anything. That's been proven long ago. The limit is software & CPU power. In this case the software is the upload--it can do anything the mind could do before.
Loren,

If you had the best measuring instruments possible, capable of measuring the current position and momentum of a simple double pendulum (one pendulum attached to the end-point of another pendulum) to the barrier of Heisenburg's Uncertainty Principle, and this pendulum were operating in a near-perfect-vaccuum (but the position/momentem of each air molecule known to H's barrier) with near-frictionless joints (but measured and taken into the calculations with the same precision as the pendulum itself), and these measuring instruments had a perfect real-time feedback to a computer from 10,000 years in the future, with an intellectual capacity exceeding that of the entire human race by thousands of orders of magnitude...

... do you think you'd be able to simulate the activity of the simple double-pendulum for more than 5 minutes?

[beowulf_king]

The hardware of a computer can't even represent the number 0.1, so to say a computer can do anything is wrong. Now upload someone's brain to this computer and tell the brain it has to figure out a way to represent the number 0.1, and all the other numbers or other operations hardware can not handle.

[Jesse]

The hardware of a computer can't even represent the number 0.1, so to say a computer can do anything is wrong. Now upload someone's brain to this computer and tell the brain it has to figure out a way to represent the number 0.1, and all the other numbers or other operations hardware can not handle. An ordinary calculator can handle the number 0.1--it's just that it represents it in terms of 1's and 0's. You're mixing up the top-level properties of the thing being simulated with the bottom-level properties of the computer's hardware. If a computer is running a simulation of the earth's atmosphere, the fact that the bottom level involves binary code doesn't mean that simulated storm systems can't dissipate gradually instead of being simply "on" or "off". Similarly, there's no reason the high-level behavior of a simulated brain would be any more "binary" than that of a real brain.

[Jesse]

Loren,

If you had the best measuring instruments possible, capable of measuring the current position and momentum of a simple double pendulum (one pendulum attached to the end-point of another pendulum) to the barrier of Heisenburg's Uncertainty Principle, and this pendulum were operating in a near-perfect-vaccuum (but the position/momentem of each air molecule known to H's barrier) with near-frictionless joints (but measured and taken into the calculations with the same precision as the pendulum itself), and these measuring instruments had a perfect real-time feedback to a computer from 10,000 years in the future, with an intellectual capacity exceeding that of the entire human race by thousands of orders of magnitude...

... do you think you'd be able to simulate the activity of the simple double-pendulum for more than 5 minutes? You have to differentiate between simulating a system and predicting exactly what it'll do. A chaotic system will exhibit sensitive dependence on initial conditions, making it impossible to predict its behavior exactly, so of course a simulation of this system can't be used to predict exactly how the real system will actually behave. But the type of behavior exhibited by the simulation may still be the same as the real system, so that the simulation might mirror how the real system would have behaved with slightly different initial conditions. Similarly, if I scan Bob's brain and create a simulation of him in a simulated environment that looks just like Bob's real environment, the simulation probably won't be saying the exact same words as the "real" Bob a minute later, but it may still be acting "like Bob", exhibiting the same personality, intelligence, memories etc., like how the real Bob might have behaved in a parallel universe where the conditions at the moment after the scanning were tweaked slightly.

[Jesse]

"A materialist should argue, as Demosthenes seems to, that it's purely a matter of linguistic convention whether we call the upload the "same person" as the original--the question of what you would "really experience" in such a situation cannot be a meaningful one."

well it is meaningful to me if I'm ever going to step into one of these contraptions, or that damn transporter on Star Trek! Personally, I agree that it's meaningful to ask which of those alternatives (50% chance of becoming the copy vs. 100% chance of staying the original) is true, which is one of the reasons I don't think materialism makes sense. My point was just that you can't simultaneously claim to be a strict materialist and also think the question is one that has a "true" answer.

Of those two options, I think the 50/50 chance of becoming either makes more sense, since I can't imagine what kind of law would say that the gradual replacement of atoms in your neurons won't end your consciousness but a more rapid replacement would. Also, the notion of consciousness being "splittable" like that fits with the many-worlds interpretation of quantum mechanics, which I've always thought made the most sense out of any interpretation I've seen.

PS what does YMMV mean?
"Your Mileage May Vary"--it's a pretty common internet acronym, like IMHO or ROFL...there's a list of acronyms here (http://en.wikipedia.org/wiki/Internet_slang).

[Matrioshka_Brain]

“Regarding the lightbulb:

“Lets say you recreate the lightbulb on a molecular basis, and turn it on. Same light. Same thing that came out of the "other" lightbulb, or would have.�

Then you would have two lightbulbs, different photons

“And you don't nessisarily need a lightbulb to produce light. What about light emmiting diodes? Or even different types of lightbulb for that matter.�

No lightbulbs, still different photons, the photons from the original bulb are now heading toward Mars and no longer care.

...

Photons are identical. The photons coming out of the new one are what would have come out of the predecessor.

Again, this goes back to the coma issue, wherein the body is mostly replaced. You wouldn't be considered a "different person," although the molecules have been replaced.

Purely linguistics.

And besides:

Let's say the origional lightbulb was altered in some way (a few molecules swapped for "different" ones). Is it a new lightbulb? At what point of molecule swapping does it become a different lightbulb?

[The Defenestrator]

Jesse,

It depends on how you look at it. From the perspective of the person being copied, sure, there's a 100% chance of staying in the origional body. The problem is that the copy is also inheriting the memories of the origional person, so from their perspective, they were the person being copied, and they were suddenly transferred to the new body. So yes, when you get your brain copied you won't suddenly jump over to a new body, but your copy will be convinced that that's exactly what happened. :eek:

Anyway, I hope that makes sense.

[Jesse]

Jesse,

It depends on how you look at it. From the perspective of the person being copied, sure, there's a 100% chance of staying in the origional body. The problem is that the copy is also inheriting the memories of the origional person, so from their perspective, they were the person being copied, and they were suddenly transferred to the new body. So yes, when you get your brain copied you won't suddenly jump over to a new body, but your copy will be convinced that that's exactly what happened. :eek:

Anyway, I hope that makes sense. But that just sounds like a third-person description of what each one will remember after the experiment (unless I'm misunderstanding, and you're saying that I 'really' am guaranteed to have a 100% chance of staying the original while the copy's memories of having been the original are 'really' false). I'm asking what I should expect if I am about to be scanned, in first-person terms. If there's a mad scientist scanning my brain who plans to use the copy as a slave while letting me go free, should I rest easy knowing I have a 100% chance of remaining the original, or should I be worried that there's a chance my stream of experience will "fork" into the copy? Again, this question is meaningless if you think about the situation purely in third-person terms, but it could be meaningful for anyone who thinks there may be first-person facts about consciousness which are not reducible to third-person ones (questions about qualia (http://en.wikipedia.org/wiki/Internet_slang) like "What is it like to be a bat?" (http://members.aol.com/NeoNoetics/Nagel_Bat.html) would be another possible example).

[Baloo]

You have to differentiate between simulating a system and predicting exactly what it'll do. A chaotic system will exhibit sensitive dependence on initial conditions, making it impossible to predict its behavior exactly, so of course a simulation of this system can't be used to predict exactly how the real system will actually behave. But the type of behavior exhibited by the simulation may still be the same as the real system, so that the simulation might mirror how the real system would have behaved with slightly different initial conditions. Similarly, if I scan Bob's brain and create a simulation of him in a simulated environment that looks just like Bob's real environment, the simulation probably won't be saying the exact same words as the "real" Bob a minute later, but it may still be acting "like Bob", exhibiting the same personality, intelligence, memories etc., like how the real Bob might have behaved in a parallel universe where the conditions at the moment after the scanning were tweaked slightly.

Let me shift gears then. As part of my chemistry major, I took a semester in chemical modeling - basically, course which used computers to simulate chemical systems much as you would have computers simulate neural pathways. It's quite fun to plug in physical constants into a program to describe the real-world environment and see it "cacluate" what shoul happend, according to those laws. It does pretty good - e.g. consistent to a degree beyond anything we could measure - when computing orbitals of a small Helium atom. It really struggled with anything much bigger - by the time you hit Carbon atoms, you basically could wait years and years for the necessary computations, or "cheapen" the simulation with handy numerical approximations.

The numeric approximations aren't all that bad... it's kind of cool to throw the physical measurments/constants of a CH3 molecule colliding with an O2 molecule and watching a computer "calculate" the resulting combustion reaction - usually with a good degree of success.

We even delved into the world of biochemistry a bit, going to the level of getting amino acids to react with various substrates. But those reactions weren't quite so fun. Though you could see the huge amounts of data that went into describing the system, it was overwhelming to note how much data was NOT being entered - they were not needed for the level of approximation the computer would be doing. All the computer had to do was simulate whether a simple AA would react with a simple substrate. In some cases, if a really ugly amino acid was chosen to react with a non-trivial substrate, you might have to wait up to 2.5 days for the results. The results were about 90% accurate.

The problem is that actually trying to calculate what will happen in multi-body systems is a problem that doesn't scale linearly. Computing the behavior of 6 atoms is relavely simple (even though a good deal of approximation is already occuring). Computing the behavior of 12 atoms is nearly impossible, without major approximations and a huge draw on pre-existing empirical data.

We stand in awe of "Moore's Law"... oooh, look at how FAST computers are getting FASTER... but not all "gee, wow" growth curves are equal. What it really comes down to is that a single 100 amino acid protein is so incredibly difficult to simulate with brute force calculations, that even if Moore's law had CPU speeds doubling weekly, we'd not see an accurate brute force simulation of the behavior of the protein within our lifetimes.

There is simply no way computers will ever be able start with a digital representation of a physical system, and use physical constants to compute the future behavior of that system, for a sufficiently complex system. Case in point: I can type out a string of 100 amino acid codes, and sleep quite compfortably with my life savings wagered on the fact that the only way humanity will ever know what chemicals the resulting protein would and would not react with is if someone actually builds the makes the chemical in a beaker and starts mixing and matching...

My personal experience with simulations of real-world systems leads me to feel that those who entertain the notion of digitally encoding the state of 9 billion neurons, along with a few physcial constants and several physics formulas, and then having a "sufficently powerful" computer to calculate a realistic future state of that neural network, are rather ignorant of such matters.

[Loren Pechtel]

I call dibs on being The One. :D

Why would a machine race,or even a single intelligent AI want to kill all humans,unless they had been programmed that way?


Because it perceived us as a threat.

[Loren Pechtel]

Loren,
.. do you think you'd be able to simulate the activity of the simple double-pendulum for more than 5 minutes?

Ok, anything which can be calculated. Obviously chaotic processes are incalculable.

[Loren Pechtel]

The hardware of a computer can't even represent the number 0.1, so to say a computer can do anything is wrong. Now upload someone's brain to this computer and tell the brain it has to figure out a way to represent the number 0.1, and all the other numbers or other operations hardware can not handle.

Sure, it can represent 0.1. I've done it. It's amazing how upset accounting will get when a penny disappears in a $10k job.

It's just the *STANDARD* way of doing things doesn't handle it exactly.

[Jesse]

The problem is that actually trying to calculate what will happen in multi-body systems is a problem that doesn't scale linearly. Computing the behavior of 6 atoms is relavely simple (even though a good deal of approximation is already occuring). Computing the behavior of 12 atoms is nearly impossible, without major approximations and a huge draw on pre-existing empirical data.
Well, when simulating the high-level behavior of neurons, you're free to use as much pre-existing empirical data as you want, it probably isn't going to be necessary to explicitly compute the wavefunction of every single electron in every protein just to get this high-level behavior right. Speaking of which, isn't it true that one of the main reasons that predicting the properties of atoms and molecules from fundamental physics is so difficult is that quantum effects play such an important role at these scales? Simulating quantum systems on classical computers is very difficult--as this (http://users.otenet.gr/~tzelepisk/yc/quant.htm) page says, "physicists Richard Feynman, David Deutsch and Paul Benioff noticed that classical computers required exponential memory and time to simulate a quantum system". A neuron, on the other hand, can probably be treated as a classical object for the purposes of simulation.

We stand in awe of "Moore's Law"... oooh, look at how FAST computers are getting FASTER... but not all "gee, wow" growth curves are equal. What it really comes down to is that a single 100 amino acid protein is so incredibly difficult to simulate with brute force calculations, that even if Moore's law had CPU speeds doubling weekly, we'd not see an accurate brute force simulation of the behavior of the protein within our lifetimes.

What are you basing that claim on? The people at Folding@Home (http://www.stanford.edu/group/pandegroup/folding/) don't seem to think that predicting how proteins will fold is an impossible goal. Neither do these guys (http://www.news.cornell.edu/Chronicle/99/8.19.99/CASP-3.html). Then there is Project CyberCell (http://www.projectcybercell.com/), whose goal is "to develop an accurate simulation of a living cell within the virtual environment of a computer, one that can be manipulated at different levels of molecular resolution, and, that can respond, adapt and evolve to exploit this virtual environment."

There is simply no way computers will ever be able start with a digital representation of a physical system, and use physical constants to compute the future behavior of that system, for a sufficiently complex system.

If you're talking about simulating every particle using nothing but the fundamental laws of physics, I'm sure you're right, but if your object is only to capture the high-level behavior of the system you're free to use all sorts of simplifying assumptions.

My personal experience with simulations of real-world systems leads me to feel that those who entertain the notion of digitally encoding the state of 9 billion neurons, along with a few physcial constants and several physics formulas But "a few physical constants and physics formulas" are not what proponents of uploading are actually imagining when they talk about simulating the brain.

[lastnightonIo]

I think the most likely route to creating a self-aware A.I. would be mind uploading (http://www.ibiblio.org/jstrout/uploading/MUHomePage.html), in which a human brain is mapped out at the synaptic level and then simulated in a computer. In this case, the A.I. would have the memories, personality, intelligence of the original human..


Heh heh... in theory. We really won't know till we try, and then we'll be stuck wondering if the uploaded thing is lying to us. ;)

[beowulf_king]

Orig. Posted by Jesse
"An ordinary calculator can handle the number 0.1--it's just that it represents it in terms of 1's and 0's. You're mixing up the top-level properties of the thing being simulated with the bottom-level properties of the computer's hardware. If a computer is running a simulation of the earth's atmosphere, the fact that the bottom level involves binary code doesn't mean that simulated storm systems can't dissipate gradually instead of being simply "on" or "off". Similarly, there's no reason the high-level behavior of a simulated brain would be any more "binary" than that of a real brain.

The calculator or computer will represent 0.1 with a slight error. In single-precision floating point numbers, it will be 0.10000000000000000000001. Or, it can represent it in double-precision using a 53 bit mantissa, but the amount of error doubles. When you enter the UI of a computer, it rounds the floating-point number into a significant digit string, which normally ignores this problem, however large scale applications do encounter problems.

It's besides the point of whether an accounting program or anything can work with this slight error (many require special memory-eating software for this task), but for the fact that computers are not perfect. They operate with error, but generally, this can be ignored. However, I have seen high-end computers crash because of simple floating-point errors from mathematical operations. Now what happens if a brain is running on this computer, and all of sudden some unexpected error occurs and the CPU goes crazy and starts spitting out nonsense? What if this nonsense data is sent to the anger portion of the brain?

[Jesse]

It's besides the point of whether an accounting program or anything can work with this slight error (many require special memory-eating software for this task), but for the fact that computers are not perfect. They operate with error, but generally, this can be ignored. However, I have seen high-end computers crash because of simple floating-point errors from mathematical operations. Now what happens if a brain is running on this computer, and all of sudden some unexpected error occurs and the CPU goes crazy and starts spitting out nonsense? What if this nonsense data is sent to the anger portion of the brain?
This is a separate issue from whether the simulation would reproduce human behavior when it was functioning correctly--but I agree, computer errors are always possible, who knows what sort of effect this would have on the simulation as a whole. How often do these sorts of errors happen in neural network simulations? Wouldn't they be more likely to cause the whole program to crash, rather than just sending nonsense to certain areas of the neural network while other areas function normally?

[tronvillain]

Jesse:
Personally, I agree that it's meaningful to ask which of those alternatives (50% chance of becoming the copy vs. 100% chance of staying the original) is true, which is one of the reasons I don't think materialism makes sense. My point was just that you can't simultaneously claim to be a strict materialist and also think the question is one that has a "true" answer.
I have trouble understanding why you think it is a meaningful question. *shrugs* To me it looks like complete gibberish. There is a one hundred percent chance that "I" will remain in my original body and that "I" will wake up inside the computer.
Of those two options, I think the 50/50 chance of becoming either makes more sense, since I can't imagine what kind of law would say that the gradual replacement of atoms in your neurons won't end your consciousness but a more rapid replacement would. Also, the notion of consciousness being "splittable" like that fits with the many-worlds interpretation of quantum mechanics, which I've always thought made the most sense out of any interpretation I've seen.
There is one "I" in a body and then another "I" is created in a machine; it is not as if the two of us exist disembodied somewhere for a moment and flip a coin to decide who gets which body. There is one object (a brain) that produces consciousness, and then there is another object (a computer) which produces an identical consciousness: the original consciousness is not "split" in any real sense.

[anakata]

Personally, I'm looking forward to uploading my mind to a quantum computer. To be able to think at a level beyond anything previously possible would blow just about any experience due to a psychoactive compound out of the water.
Why not combine the two... It would be very easy too, you could download your favorite substances from the web or some P2P app.

[Jesse]

I have trouble understanding why you think it is a meaningful question. *shrugs* To me it looks like complete gibberish. There is a one hundred percent chance that "I" will remain in my original body and that "I" will wake up inside the computer.
You are speaking from a third-person point of view. The question is about what you would actually expect to experience next if this experiment was about to be done to you. Again, go back to my 'mad scientist planning to use the copy as a slave' scenario--as the scanning machine is warming up, would you be feeling fearful that you were about to have the experience of finding yourself a slave in a simulation, or confident that your next experience would be of the mad scientist letting you go free? Or do you think that the first-person experience of a continuous flow of consciousness, with one experience following another, is a kind of illusion?

Also, would you reject as gibberish any version of the many-worlds interpretation (http://plato.stanford.edu/entries/qm-everett/) that talks about the probabilities of experiencing different outcomes in different situations (like a 50/50 probability of seeing a photon hit a detector in some experiments, but a 30/70 probability in others)? From a third-person point of view, all you could say is that there would be one version of the experimenter who experiences one outcome and another version who experiences another, but any notion of assigning probabilities to these different versions would seem to depend on taking a first-person point of view.

[Azathoth]

Because it perceived us as a threat.

True,but that presupposes that one of the criteria for an artificially intelligent entity,is self preservation.

For mankind,it definitely is.But human beings are products of relentless selection.Anything without such an instinct wouldn't be passing on it's genes very often.

The thought processes of an Artillect may be vastly different from those of human beings.It may be the first true 'alien' that humanity encounters.

However,if such an entity is created,and told to defend itself from all threats,there may be a very good chance that it will decide humans could be a threat to it's existance.

[Azathoth]

Why not combine the two... It would be very easy too, you could download your favorite substances from the web or some P2P app.

Of course,when the future version of the RIAA gets permission to crack people's brain boxes open,to get at those files,some may reconsider.....

Colossus may not have been the best movie,but it does try to consider the question of what an AI might do with humans,while harbouring humanity absolutely no ill will.Though I've seen the concept done much better in many science fiction books.

And it does have some good quotes :)

"This is the voice of world control. I bring you peace. It may be the peace of plenty and content,or the peace of unburied death. The choice is yours. Obey me and live or disobey me and die."

[Norseman]

A computer is going to view the human race like we view apes/monkeys. Our closest but lesser relative.

If computers are one day in charge, they will program new computers to know history, right? Ok, they will probably use us as an evolutionary link! Seriously, think about that.

By the time we can make a self-aware computer, computers will run the whole world and everything in households. What does this mean? Computers around the world will be self-aware and know they are in control! Do you really think they are going to let us be their equals? Hell no! Think about it, do you really think they are going to want to be our slaves!?

We are all doomed if we keep using computers as slaves and create self-aware, intelligent computers.

1. Isaac Asimov's primary rule of robotics: No robot engineer wants a disaster, therefore any robot built by the engineer is likewise controlled by the engineer, and has safety limits installed to protect the engineer and in general anyone that might try to sue the engineer (usually, sometimes people just mess around, like me). The robot is thus rendered incapable of harming humans by any means.

2. What would the machines gain by killing us? Why would they care? Think of it this way: what would we gain by killing all the monkeys? Diddly squat.

3. The machines could die. They risk a human hacker sending a virus and corrupting the entire system if not crippling it. Why would the machines risk it? What would they gain?

4. We have protocalls to ensure nukes can only be launched by humans, thus if it came down to it we could destroy whatever the A.I. was stored in before another could be made.

Humans are self-aware. Why don't humans go and kill everyone and the monkeys while they're at it? Simply because it wouldn't benefit anyone, many people would be entirely unable to do it, and laws are in place to stop any who would try and start such a thing.

Answer those and you'll have a great movie. Can't and it's pure BS.

[tronvillain]

Jesse:
You are speaking from a third-person point of view. The question is about what you would actually expect to experience next if this experiment was about to be done to you. Again, go back to my 'mad scientist planning to use the copy as a slave' scenario--as the scanning machine is warming up, would you be feeling fearful that you were about to have the experience of finding yourself a slave in a simulation, or confident that your next experience would be of the mad scientist letting you go free? Or do you think that the first-person experience of a continuous flow of consciousness, with one experience following another, is a kind of illusion?
"I" would expect to experience both: one of me would remember being in the scanning machine and then waking up as a slave in a simulation, the other me would remember being in the scanning machine and being let go, and both of us would have "the first-person experience of a continuous flow of consciousness." If I was recorded, destroyed, and rebuilt out of entirely different matter, "I" would continue to exist from any perspective that matters, including the first person perspective. If you think that means "the first person experience of a continuous flow of consciousness" is a kind of illusion, then so be it.
Also, would you reject as gibberish any version of the many-worlds interpretation that talks about the probabilities of experiencing different outcomes in different situations (like a 50/50 probability of seeing a photon hit a detector in some experiments, but a 30/70 probability in others)? From a third-person point of view, all you could say is that there would be one version of the experimenter who experiences one outcome and another version who experiences another, but any notion of assigning probabilities to these different versions would seem to depend on taking a first-person point of view.
The discussion at hand does not appear to have anything to do with quantum mechanics. As I said, we are taking one self-generator, copying it and building another one, which leaves us with two selves - the selves do no exist disembodied and flip a coin to decide which generator to occupy.

Still, while I am not a fan of the many world interpretation of quantum mechanics, the only way you can talk about the probabilities is from a third person point of view. From a third person point of view you can say that fifty percent (or thirty percent) of universes contain an experimenter who experiences one outcome and that the other fifty percent (or seventy percent) contain an experimenter who experiences the other, while from the first person point of view all you can say is that you experienced one outcome or the other. The fifty-fifty experimental result obtained is explained from the third person perspective by the vast number of universes which contain ratios near to that result compared to the relatively small number of universes in which the results come up "all heads" or "all tails" as it were.

Anyway, even with such "splitting" of conscious experimenters it makes little sense to talk of ones chances of ending up in a given universe. It is not as if the universes split, the experimenters split, and then the experimenters draw straws to see which universe they end up in. The universe, including the experimenter, simply splits. So, if you ask the experimenter "What is the probability that you will end up in a universe where A is observed" they should reply "The probability is one, as I am certain to end up in every universe. Of course, only fifty percent of my selves will end up in universes where A is observed, but that isn't what you asked."

[Baloo]

Jesse,

Well, I had a chance to peruse the Folding@Home site - and will confess that the field of chemical modeling has come a great deal further, a great deal faster, than I would have thought possible based on my class experience 6 years ago. To that end, I humbly retract my statement/wager related to the computation of the folding of a protein comprised of a random string of 100 amino acids (with a caveat: I'm certain that a great deal of the progress in the field has been the elimination of vast amounts of computation with only meager sacrifices to the overal accuracy of the resulting simulation... were it not for such eliminations, my original confidence might not be so misplaced).

Okay, now that I've got that out of the way, it's interesting to see the light that the Folding@Home site sheds on the notion of real-time human brain simulation.

My math proceeds from the following paragraph: (http://www.stanford.edu/group/pandegroup/folding/science.html)

In fact, it takes about a day to simulate a nanosecond (1/1,000,000,000 of a second). Unfortunately, proteins fold on the tens of microsecond timescale (10,000 nanoseconds). Thus, it would take 10,000 CPU days to simulate folding -- i.e. it would take 30 CPU years! That's a long time to wait for one result!

and the assumption that the paragraph refers to a "standard" 2 GHz CPU...

From the paragraph, a standard 2 billion ops/second processor takes 86000 seconds to compute 1/10000 of a protein folding.

Therefore there are 2E9 * 8.6E4 * 1E4 ops per protein fold, or 1.7E18 operations/fold

Proteins unfold at rates of about 10 microseconds = .01 seconds.

Thus, a computer would need to compute at a rate of 1.7E20 ops/second to compute the chemical activity of an average protein in real-time.

The fastest CPU's today are about 4 E9 ops/second, which is about 40 billion times slower than what we need. Assuming Moore's law holds indefinitely (quite an assumption!), doubling CPU speeds at 18 months gives us an ETA of a single computer capable of real-time simulation of a single average protein of just over 50 years.

(40,000,000,000 = 2^34. 34 doublings, at 18 months per doubling, is 50 years)

So, in 50 years, if computer speeds double every 18 months, we'd have a CPU capable of simulating a single protein in real-time. If we conservatively estimate that there are 1,000 active proteins in a given brain cell, and there are 10 brain cells per neuron, and there are 10 billion neurons in a human brain, then 50 years from now, we might be able to string 100 trillion CPUs together to simulate the chemical activity of the JUST THE PROTEINS of a human brain.

Of course, I've made one glaring error here. Namely, the 1.7E20 operations/second is the CPU speed needed to simulate the activity of the folding of protein. These estimates are a bit misleading - they omit any calculations of any effects the environment might have on that protein. In other words, simulating the folding of 3 proteins a wide distance apart might require 5E20 ops/second, but to simulate the chemical activity of the same 3 protiens in near-proximity, we'd need do calculations at a far greater rate to incorporate any effects the proteins would have on each other. Also, we'll need to include in our calculations the effects of various substrates surrounding the proteins; even speaking conservatively, I'd estimate a 1,000 fold increase in the calculation complexity to simulate a protein's chemical activity in an environment as dynamic as that in a cell.

Once we've got the protein simulated, we can start factoring in hormones, synaptic firing, external stimuli, etc., etc.


Plus, I'm going to tie all the way back to my first paragraph: these calculations are able to give an "accurate" simulation of simple protiens... but they are just a small fraction of the calculations needed to get HIGHLY accurate idea of protein chemical activity. Again, its just my hunch, but I'll bet much of the progress in the field of chemical modeling has been the elimination of a large amount (for instance 95%) of the calculations for relatively meager sacrifices in accuracy (for instance of only 1% or 2%).

So even if I grant you the possibility of a complete simulation of your brain, if that simulation is < 98% accurate for nearly every single complex molecule in your brain... do you honestly think that the simulation would be able to remember your parents middle names, your childhood friends, what car you drive, what you think of President Bush, etc., ad nauseum? In other words, even if the simulation generated behavior that was remotely human, can you honestly say that you think it would be anything remotely similar to YOU (personally, I think you'd have more in common with a chimpanzee brain than the result of such a simulation - but that's the cynic in me...).

[Jesse]

So even if I grant you the possibility of a complete simulation of your brain, if that simulation is < 98% accurate for nearly every single complex molecule in your brain... do you honestly think that the simulation would be able to remember your parents middle names, your childhood friends, what car you drive, what you think of President Bush, etc., ad nauseum? In other words, even if the simulation generated behavior that was remotely human, can you honestly say that you think it would be anything remotely similar to YOU (personally, I think you'd have more in common with a chimpanzee brain than the result of such a simulation - but that's the cynic in me...).

Yes, it's my hunch that such a simulation would act like me. Neural networks store information in distributed patterns of connection weights, and they're quite fault-tolerant (http://citeseer.ist.psu.edu/context/24899/0), I'm sure that if you erased 2% of the connections in a typical neural net it would still recognize all the same patterns it'd been trained on; of course the brain is not as simple as an idealized neural network, but the basic principle by which the brain stores memories is thought to be similar. As this Stanford Encyclopedia of Philosophy entry (http://plato.stanford.edu/entries/connectionism/#4) puts it:

Connectionist models seem particularly well matched to what we know about neurology. The brain is indeed a neural net, formed from massively many units (neurons) and their connections (synapses). Furthermore, several properties of neural network models suggest that connectionism may offer an especially faithful picture of the nature of cognitive processing. Neural networks exhibit robust flexibility in the face of the challenges posed by the real world. Noisy input or destruction of units causes graceful degradation of function. The net's response is still appropriate, though somewhat less accurate. In contrast, noise and loss of circuitry in classical computers typically result in catastrophic failure.

As for simulating proteins, I still don't think anything like the level of detail you're suggesting would be necessary. For example, instead of having a general model that can simulate the precise movements of an arbitrary protein, might it not be possible to have a sort of lookup table that deals with the known interactions of only those proteins which are known to be used for neuron-to-neuron signalling? And if the only thing you're interested in simulating about proteins is their effects on how neurons react to and transmit electrical impulses, this would probably also lead to plenty of simplifications--instead of simulating every serotonin protein, for example, you might be able to get away with something like a single parameter defining serotonin concentrations at each synapse.

[christophersimons]

If a person here genuinely believes that we should "be careful about computers and intellegence", I cordially invite that person to take up a book or introductory class on computer architecture (the chapters on microprocessors and motherboards in Michael Meyers' A+ Certification would do nicely).

Computers are no more intellegent than calculators. Every single thing they do is triggered by circuits carrying charges. They have no powers of reasoning or gathering their own conclusions. Even when a computer beats a grandmaster at chess, it is the result of the programmed evaluation of conditions on an extremely low level.

[Jesse]

If a person here genuinely believes that we should "be careful about computers and intellegence", I cordially invite that person to take up a book or introductory class on computer architecture (the chapters on microprocessors and motherboards in Michael Meyers' A+ Certification would do nicely).

Computers are no more intellegent than calculators. Every single thing they do is triggered by circuits carrying charges. They have no powers of reasoning or gathering their own conclusions. Even when a computer beats a grandmaster at chess, it is the result of the programmed evaluation of conditions on an extremely low level.

But presumably you wouldn't say that the atoms which make up our brains have "powers of reasoning or gathering their own conclusions" either. Do you agree with the reductionist view that the behavior of any physical system, including the human brain, can be explained entirely in terms of interactions between its component particles, with each interaction being explainable in terms of the laws of physics? If so, then the fact that the low-level components of a computer are also following simple rules should not present any fundamental obstacle to intelligence emerging at higher levels.