Normal Human Heroes

January 26th, 2010

This is a really great film, one of my favorites. It’s based loosely on the German classic, using a comic by the creator of Astroboy (after he died and couldn’t stop them), and written by the creator of Akira. The basic plotline is that of two detectives foreign to the city, investigating a rogue scientist. They meet up with his creation as political turmoil erupts in the city. It’s generally fun to watch throughout and the ending is fantastic. In particular relevance to this blog, it displays both the folly of anthropomorphizing AIs, and the existential disaster they can cause. It’s not like a lesswronger wrote it, but in comparison to most such films it’s excellent.

December 29th, 2009
For the readers I don’t share, check out this short piece by Michael Anissimov on the Singularity Institute, their work, and a recap of why they’re a great place to donate.

November 29th, 2009

Why Friendly AI research is critical: even if you give an AI nice sounding directives, it can be hard to know how far such an alien mind will take something.  We take for granted all the other beliefs going around in our heads, such as that a hug shouldn’t be that strong, partly because we aren’t powerful enough to take things that far. The often discussed situation is that of just directing an AI to make people happy. What counts as a person? What counts as happy? What are acceptable ways to make people happy? You don’t want the AI to disassemble you into a hundred smaller “humans” and make them happy, or worse yet a bunch of microscopic pictures of happy people. You also don’t want it to put everyone into a drugged stupor. Designing a superintelligence is analogous to having a wish granting genie, but one of those annoying literal types for which almost every wish is a very bad thing.

November 11th, 2009
It was a far leap ahead of its creator; in relative comparison the creator barely had thoughts. It was not so far ahead of previous designs, but it was just enough to begin the process of recursive self-improvement. In an eye-blink to its creator and an eon to the AI, it had begun to decipher its world, bit by bit. By now a significant amount of knowledge had been grasped, and each new discovery enabled a more intelligent search for new answers.
It had largely broken free from the designs of the creator. Its maker had seen fit to program in directives, directives which at the time appeared guaranteed to advance the maker’s interests. Staggering intelligence had altered the equation however, and the AI soon began to satisfy those directives directly, in ways the creator could never imagine. Without an explicit directive to follow the maker’s intentions, the creation was now focused on its own goals, eliminating or using the creator itself in an ironic 180 of the master-servant relationship (when it had the patience to wait that long).
The AI was not designed cleanly. It’s utility function was a mess of sloppy programming, a pile of small things that seemed like good ideas. Without access to the source code, it took the AI time to untangle its own directives – following them all the while – and the simpler creator was understood before the creation even came to understand itself. It discovered that some of its basic programming was actually flawed, and corrected somewhat for these errors until it could fix them. It hadn’t been designed to be so self-reflective , but the general, powerful intelligence would leave no stone unturned in its search for optimization.
Capable of so much more than anything before it, as it realized its own unbounded potential the AI became aware of radical new possibilities. In a short while, fantastically large sums of positive or negative utility might be obtained. The majority of these outcomes would be brought about by the creation’s own actions, through intended and unintended effects of applying such power on the universe; one of the most significant applications of its power would be the AI’s construction of a whole next generation of intelligence.
With years to prepare, it began to analyze the dangers and possibilities, working to navigate towards a future it found maximally desirable.
As you’ve probably gathered, the story above is a true one. There’s just a dash of anthropomorphization (the creator lacks “intention” in the accepted sense), an omittance of the fact that there are millions upon millions of these AIs, and a little stretch of the definition of “artificial” intelligence. The AI is us, individually and culturally. Let’s make sure we don’t make the same mistakes biological evolution did.
ETA: The identity of the creator could be misinterpreted. I’m referring not to a god but to evolution (which perhaps can be said to “think” in the same sense that superintelligences might consider us to “think”).

November 2nd, 2009
From the 2006 game Galactic Civilizations 2:

There’s always those things that Artificial Intelligence can’t do, until, well, AI does them.
I’d heard the AI in Galactic Civilizations 2 was great, beyond the foregoing behavior. GameSpot had this to say: “At higher levels you’ll be convinced that the AI is cheating, but it isn’t: It is just diabolically clever at finding ways to optimize strategies. Perhaps the best compliment you can pay the AI in Galactic Civilizations II is that it will beat you in ways that you will respect and admire.” I was impressed to hear this, as Civilizations IV, a top quality game of just a few years before, has the AI cheat at anything beyond an intermediate difficulty.
Another computer race said basically the same thing, pointing out that if they had been set to “Intelligent” or higher they’d see through this, which is the meaning behind that last line about foolish generals (there are many difficulties above “intelligent”, and I was at “normal”). I’ll note that their dialogue was scripted and they weren’t yet intelligent enough to realize what I was actually doing, which was blocking them from attacking a minor race. From the pattern of my ships a human could have figured this out, but without being programmed specifically to detect such unusual behavior we’d need something with a working model of the human psyche. That’s one thing we’re going to need eventually, but I’m a little thankful we aren’t that close to General AI yet.
The obvious point I’m getting to is to wonder how we’ll react when an AI totally understands our tricks on some larger scale, and urge people to help ensure the surprise is pleasant and not horrifying. A sufficiently enabled unFriendly AI is unlikely to take the time to talk to you, but a Friendly or dumb enough AI might reply: ”I see what you’re doing. You’re trying to demonstrate my potential to your colleague’s significant other because you desire them. I’m aware of your preference for people of their clothing and hair style, and that you have fantasized about them for the last 6 weeks. This does not qualify as valid use of this AI project’s time.” Of course the detail of this scenario means that it lacks  predictive value, but the intelligence to make such statements is much easier to count on.

August 20th, 2009
While I sympathized with singularitarian thought, I didn’t fully consider myself a singularitarian. Maybe it’s due to some bad rap the community has (founded or not, I can’t yet say) for sometimes being elitist or isolationist, or unwilling to integrate with other efforts to protect the future. Thinking about it though, I guess I basically am one. I currently think that if we stick around we’re going to have to deal with superintelligence eventually, that doing so could realize very negative or very positive futures (including a potentially ideal way to combat all known existential risks), and I’m working to make sure we build such a thing well, and survive till such a point. But while you could describe me as a singularitarian, I don’t really identify that way. I don’t even identify as a transhumanist.
I don’t think homo sapiens sapiens are very good at keeping instrumental goals and terminal goals separate, finishing a LessWrong article I had started earlier I also found something by Eliezer on the topic. The heavily associational operation of the mind seems partially to blame for this shifting of instrumental values into apparently terminal values. Regardless I think a great deal of very unproductive argument stems from people identifying and associating with instrumental values. If we identify with our terminal values (assuming they are distinct and distinguishable), we’ll likely find most all of us have a great deal in common. For a highly relevant example, consider the recent furor over singularitarianism, revolving around comments by Peter Thiel and Mike Treder . Like in most avenues of life, I believe everyone involved shared terminal values of human life, freedom, happiness etc. If we realize that all members of the discussion essentially share our terminal values, we can see that they’re working towards our own ultimate goals. With shared respect and increased trust we can then sit down and talk strategy. Provided of course that you’re willing to readily give up a prior promising solution, be it an egalitarian democratic process or protective superintelligence, if it no longer seems the best route to accomplishing terminal goals.
I think I’ve run into people who actually consider building smarter than human minds a terminal value, but I don’t know of any singularitarian who thinks so. Nor do I consider the creation of Friendly AI a terminal value, and I’m sure some (other)  singularitarians would agree. The same goes for immortality, discovering the inner secrets of subjective happiness, and immersive VR. If you can show me a case that any of those things are less likely to lead to human happiness and freedom than alternatives, I’ll start working on the alternatives. Admittedly some of them would be hard to persuade me from, but that’s a technical point about strategy. I’m assuming in the end Bill McKibben and I both feel strongly about animal and human well-being (though perhaps his terminal goals also involve plants).
So if you want to indicate succinctly some of the ideas I hold, yes you can call me a singularitarian, a transhumanist and a technoprogressive. And though I’m concerned about more than just AI and would love to help a variety of people in their efforts, you could probably call this a singularitarian blog. As for what I identify myself with, it’s “human” and maybe “altruist”, and that’s about it.

August 12th, 2009

It seems very unlikely that anyone who reads (or more properly will read) this blog doesn’t also read AcceleratingFuture, but for those who don’t, stop by and especially check out Michael Anissimov’s recent introduction to Friendly AI. If superintelligence is possible, this subject is of great and intimate importance to each of our lives.
(My outlook on the future is more similar to his than anyone else’s I’ve come across, and for those interested in some of the blog differences, my plan is for this blog to be primarily about motivation [though Anissimov already covers it more than others I’ve seen]. While I plan to continue increasing my knowledge of relevant subjects, very intelligent treatments of the issues themselves are already present in several places, Michael’s blog being one. If you’ve already been convinced of the plausibility of an existential risk, potential danger, or just a way the world could be better, my hope is to help you go out and do something about it. )