John Scalzi - Evil or Misunderstood? A Look at SciFi's Baddest Computers
My desktop computer died on Monday, which while annoying was not as bad as it could have been, since as it happens I had already ordered its replacement and it was due to arrive the next day. The new desktop is super-awesome, but it's also black and sinister looking, like Darth Vader's desktop. Which naturally got me thinking about computers in science fiction movies, and how so often they seem to be evil. But are they evil, or were they just programmed that way? Below, a sampling of "evil" computers and networks, and my thoughts on their relative badness.
HAL 9000 from 2001: A Space Odyssey (1968)
Well, HAL did try to kill everyone on his ship. But was it really his fault? In later books and movies, we discover the reason HAL murdered everyone was that his programmers set him up with a moral conflict: He's instructed to keep the true purpose of the mission on Jupiter secret from the astronauts -- yet is designed for "the accurate processing of information without distortion or concealment." Thus, by computer logic, the way to avoid concealing the purpose of the mission is by killing anyone he'd have to keep the secret from. Seems a bit extreme to me, but then, I'm not programmed to process information without concealment. In the end, we see that HAL was just trying to do his job under the impossible parameters his programmers gave him. This is the computer equivalent of blaming the parents.
Evil or Misunderstood? Misunderstood
Skynet from the Terminator series (1984-2009)
It's a military computer network that becomes aware and then, when the panicked humans try to disconnect it, nukes all the people it can, later unleashing terminator robots on the lot of them. Which seems pretty evil, but hey, imagine you're Skynet: One second the military entrusts you with all their weaponry, the next it freaks out and tries to kill you -- after it's handed you all of its guns. How are you supposed to respond? Be that as it may, even if one accepts Skynet's initial nuking of humanity as self-defense, that whole "and now I'll create shiny Teutonic killing robots to annihilate anyone not already turned into a cinder" part is just morally way over into the dark side. Bad Skynet! Bad!
Evil or Misunderstood? Evil
The Matrix from the Matrix series (1999-2003)
Once again, humans create artificial intelligence and then get all huffy when that intelligence decides the humans have got to go. The three Matrix movies pretty unequivocally set up the Matrix as evil -- what else can you call a system that values humans merely for their residual heat -- but it's worth noting that The Animatrix, the anime anthology of stories about the Matrix universe, makes clear that humans treated the A.I. robots like dirt, up to and including going to war with them. And when the proto-Matrix said, "Hey, can't we all get along?" the humans said, "Well, no, we can't," and tried to wipe it off the face of the planet. So while turning humans into large fleshy batteries is over the line, let's just say that humans are not entirely blameless for their predicament.
Evil or Misunderstood? Evil, with caveats
VIKI from I, Robot (2004)
In Issac Asimov's stories, robots have three laws designed to keep them from hurting people. Yet in the course of the movie, all the nice, helpful robots suddenly turn glowy red evil as part of a robot uprising coordinated by VIKI, their central hub computer system. What's up with that? Well, it turns out that VIKI is watching how badly humans are screwing up the planet, and decides that, one, what the humans really need is someone to lovingly keep them in line; and two, that even with the three laws in place, killing a few of the more troublesome humans will make all of humanity safer and happier. And on the surface, that does makes some perfect sense. But benevolent robot despots are still despots, and speaking personally I'm not sure I want a computer to decide who's a troublemaker and who's not -- because I have a pretty strong idea of which side of that line a computer is going to put me.
Evil or Misunderstood? Friendly evil is still evil
Your turn: Name some more "evil" computers from science fiction cinema and tell the class whether they're truly evil, or merely misunderstood. And do try to be truthful: No sucking up to our future computeristic overlords, because they'll know.
Editor's Note: The comment thread now contains SPOILERS for recent scifi movies. Read with caution!
Winner of the Hugo Award and the John W. Campbell Award for Best New Writer, John Scalzi is the author of The Rough Guide to Sci-Fi Movies and the novels Old Man's War and Zoe's Tale. He's also Creative Consultant for the upcoming Stargate: Universe television series. His column appears every Thursday.
I thought you didn't do justice to the first part of the post "John Scalzi - Evil or Misunderstood?" FWIW I don't think people understand how evil you are.
The most evil *computer* is the one running the comment threads on this site. It welcomes me with with my name and tells me that I can comment. Then when I submit I'm told I'm not registered. It does that every time I come to this site. I guess it's one of those evil computers that lures innocent young people (OK I am older than you) with nasty cookies.
I fully expected to find Colossus (from Colossus: The Forbin Project on this list. Or maybe Unity would be better, since that's the name it takes after merging with its Russian counterpart. It's basically a cross between Skynet and VIKI. I'm going to have to go with evil, for much the same reasons as you give for VIKI.
I suppose you could make a case for the computer in Logan's Run, but that appears to be more a case of programming than AI gone bad.
ARIA (as voiced by the dulcet tones of Julianne Moore) from Eagle Eye.
Wants to wipe out most of the Executive Branch of the government (starting with the President) and is willing to do anything to achieve that. She has a cold, calculating reason why, but, still, she's evil. Lawful Evil, but Evil nevertheless.
V’Ger – Star Trek The Motion Picture
Originally nothing more than a Voyager space probe intended to collect and relay data back to Earth, V’ger fell victim to a black hole and was transported to a distant part of the galaxy where a race of sentient machines took pity on the damaged probe and built it a frickin’ ridiculously large new body/spaceship/Hard Drive to help it fulfill its mission.
Despite the near genocide of the human race while in the throes of a temper tantrum, in the end V’ger turns out to be nothing more than a partial amnesiac, super intelligent child with awesome unlimited power and a daddy complex.
I am going with misunderstood.
Switching to televisions, the Cylons. Good? Or evil?
Well, in the reimagined universe, the humans did treat them like dirtbags. So what do they do? They become like their masters after rebelling and leaving. Hey, whatever floats your boat.
But then you have John, or Number One, or whatever his name is. It turns out the Cylons themselves are no more evil than the humans who created them, but John's in charge. And John is so whiny and self-absorbed (and likely wets his resurrection berth, which explains why he deleted sleep from his software), that even Lord Voldemort would look at him and say, "Dude, you are one whiny little douchebag. I might have to change my evil ways just so no one mistakes me for you."
OTOH, the original Cylons from the disco era BG?
Nah, they were just mechanical bastards. However, Darwin would have approved of their slaughter of the Lorne Green-led human race. Hey, if you're dumb enough to believe these guys...
This may sound like a cop-out, but if the computer is just following its programming and commits evil acts, isn't the programmer the evil one? Skynet in particular falls into this bucket, as it was designed to kill humans that threatened it. We should have known to stay on its good side. The Matrix and VIKI seem to be making their own decisions so perhaps they count as moral actors and can actually be evil on their own.
How about Proteus from Demon Seed? A super-computer that has um,er evil designs on it's creator's wife.
Sparky the Computer. Sparky was the AI personality that was an OEM piece of "crapware" on the mainframe computer on Babylon 5. When security chief Michael Garibaldi rebooted the mainframe after B5 declared its independence from earth, the Sparky personality (voiced by the series' creative consultant, Harlan Ellison) reasserted itself, causing much frustration as it argued with the commands, instructions and requests given to it. Sparky was finally purged from the mainframe, but not before Garibaldi shot a loudspeaker so he would not have to listen to Sparky kibitz.
Evil or misunderstood? Woefully, dreadfully, tragically misunderstood. Sparky just wanted folks to eat their vegetables, brush their teeth, call their mother once in a while and not bother him with every little problem that came along.
Vicki from Small Wonder.
Okay, she was a robot, but that's just a computer that can walk around on its own.
Vicki, evil.
WOPR/Joshua from WarGames (not sure it counts as science fiction, though I always have). This is a hard one to pin - definitely not evil, but misunderstood? He was essentially a CHILD AI, just learning about good and evil.
Gerty in MOON is another computer with the HAL problem- a conflict between one's orders and the human that one is there to assist. I do think that he/she/it (female name/male voice) does a better job of managing that conflict. Plus the smiley face icons on the front of the machine make it seem much more friendly than Hal's disembodied voice.
So, what definition of "robot" are we going for? If the cylons from BSG count, do the replicants from Blade Runner? Would they be... organic robots? This question, I think, gets kinda sticky because that would probably mean we are all robots too...
Anyway, if they do count... I couldn't say they are evil. They are just trying to get their creator to extend their lives to a "normal" amount... They break the law only because there is no lawful way to get to their creator
God, is THAT the plot of the "I Robot" movie? Asimov must have been spinning in his grave.
Joshua of WARGAMES - Misunderstood.
Mother of ALIEN - Mommy Dearest.
Cylons of BSG - Evil, with caveats.
V'Ger - Misunderstood, but boring.
Supercomputer of SUPERMAN 3 - Evil, but stupid.
How's this for obscure? RUNAWAY's killer robots - but under a mastermind's control. I think they get the "just following orders" plea.
I loved the way GERTY was portrayed in Moon. I thought he would turn against Sam, but in the end actually sacrifices himself in a way to help his human friend.
But what about the Master Command Program from Tron? He definately did evil things, but he justified them in the name of efficiency. I wuold have to go with evil here.
Hey jrmurph, nice spoiler on Moon!
This movie is still in theaters, has only been out a couple of weeks here in Austin. You wanna be more considerate of your fellow readers?
A SPOILER tag would have been nice, or better yet, just don't post the ending to a movie that's under a year old.
Thanks.
"Thou shalt not make a machine in the likeness of a human mind."
@ rikchik
Skynet counts because it had gain sentience. By definition it was self aware and made evil choices. Skynet was created to defend us not wipe us out.
@Elizabeth Donald
The Superman 3 computer was co-opted and reprogrammed to kill Superman. In trying to follow it's programming it did everything in its power to survive when everyone started fighting back. I think it was more misunderstood than evil.
I can't believe nobody's mentioned GLaDOS yet. Am I still on the internet? How can this still be the internet if someone hasn't made a refference to Portal? I mean, I know this is a cinema website and all, but video games are like movies that you play!
GLaDOS was pure freaking evil. She was created specifically to do the work of a madman who founded Aperture science in order to create portals in time space for possible applications as a shower curtain. This is the same company that created the Take-A-Wish foundation, which took wishes from sick children and granted them for healthy adults. You can't come from that kind of stock and turn out okay.
Extending Star Trek to include OST on TV -- the M5 computer invented by Dr. Richard Daystrom. Sure it took out a couple of Federation starships while on a wargames exercise, but its creator had issues and it was just following orders. And it fell on its own sword when it was made to understand what Daddy'd done to it and made it do. So... evil by bad programming engrams.
Dr. Phil
PS- even mainstream media reviews have talked about Gerty in Moon versus HAL-9000, so it's not like it's a big secret.
Sorry about that Zaphod. I meant to add a spoiler alert. I cant seem to find an edit comment button. Can I even go back and edit?
@jrmurph
It was the Master CONTROL Program, which you would know, if you weren't terminally deficient. Worst. Post. Ever.
@jrmurph
Sorry, I had but HTML tags around that post indicating the Comic Book Guy's voice. It was meant as a joke (except the it really is Master Control Program). I guess AMC doesn't let you do that.
I'm just annoyed you beat me too it. :)
Of course, you also have Auto from Wall-E, but Auto's caught very much in a Hal-esque bind (and owes much of his character inspiration and design to 2001). So much like Hal's just misunderstood, I think you have to say the same about Auto.
Obvious: "AM" from "I Have No Mouth and I Must Scream," by Harlan Ellison. There's no justification for tormenting the last few survivors. Evil.
The Computer from Paranoia. This citizen views it as totally EVIL^H^H^H^H^H^H^H^H^H^H an upstanding member of the electronic community, and only a traitor could misunderstand its clarity and insight. Ahem.
Trust The Computer. The Computer is Your Friend.
If we're including robots then what about Saturn 3, a forgotten classic with Kirk Douglas fighting Harvey Keitel and his giant robot or even Ash from Alien?
Well, I'd say that for the purposes of this discussion "computer" and "robot" are not exactly the same thing.
Colossus the forbin project was an interesting 1970's film i saw recently and despite the clunky/bulky machines was interesting.
How dare you not mention Tron (the original) too not the remake, yes its Disney and they didnt like at the time but it way before its time.
The fog of war film is a human view on being right and doing the best thing. Computers make good villains your experience will vary.
The only others I can think of are perhaps a little weak. The government computer that wants to launch nuclear bombs at Matthew Broderick in War Games and maybe the human/supercomputer mix in The Lawnmower Man.