quote:

Now here's the moral dilemma. If neither event is going to happen to you personally, but you still had to choose one or the other:

Would you prefer that one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes?

I think the answer is obvious. How about you?

quote:

Wow. The obvious answer is TORTURE, all else equal, and I'm pretty sure this is obvious to Eliezer too. But even though there are 26 comments here, and many of them probably know in their hearts torture is the right choice, no one but me has said so yet. What does that say about our abilities in moral reasoning?

quote:

Torture,

Consider three possibilities:

(a) A dusk speck hits you with probability one, (b) You face an additional probability 1/( 3^^^3) of being tortured for 50 years, (c) You must blink your eyes for a fraction of a second, just long enough to prevent a dusk speck from hitting you in the eye.

Most people would pick (c) over (a). Yet, 1/( 3^^^3) is such a small number that by blinking your eyes one more time than you normally would you increase your chances of being captured by a sadist and tortured for 50 years by more than 1/( 3^^^3). Thus, (b) must be better than (c). Consequently, most people should prefer (b) to (a).

Big Yud posted:

I'll go ahead and reveal my answer now: Robin Hanson was correct, I do think that TORTURE is the obvious option, and I think the main instinct behind SPECKS is scope insensitivity.

Some comments:

While some people tried to appeal to non-linear aggregation, you would have to appeal to a non-linear aggregation which was non-linear enough to reduce 3^^^3 to a small constant. In other words it has to be effectively flat. And I doubt they would have said anything different if I'd said 3^^^^3.

If anything is aggregating nonlinearly it should be the 50 years of torture, to which one person has the opportunity to acclimate; there is no individual acclimatization to the dust specks because each dust speck occurs to a different person. The only person who could be "acclimating" to 3^^^3 is you, a bystander who is insensitive to the inconceivably vast scope.

Scope insensitivity - extremely sublinear aggregation by individuals considering bad events happening to many people - can lead to mass defection in a multiplayer prisoner's dilemma even by altruists who would normally cooperate. Suppose I can go skydiving today but this causes the world to get warmer by 0.000001 degree Celsius. This poses very little annoyance to any individual, and my utility function aggregates sublinearly over individuals, so I conclude that it's best to go skydiving. Then a billion people go skydiving and we all catch on fire. Which exact person in the chain should first refuse?

I may be influenced by having previously dealt with existential risks and people's tendency to ignore them.

SolTerrasa posted:

Effortpost incoming, but the short version is that there are so many (so many) unsolved problems before this even makes sense. It's like arguing what color to paint the bikeshed on the orbital habitation units above Jupiter; sure, we'll probably decide on red eventually, but christ, that just doesn't matter right now. What's worse, he's arguing for a color which doesn't exist.

[...]

Okay, so long version here, from the beginning. I'm an AI guy; I have an actual graduate degree in the subject and now I work for Google. I say this because Yudkowsky does not have a degree in the subject, and because he also does not do any productive work. There's a big belief in CS in "rough consensus and running code", and he's got neither. I also used to be a LessWronger, while I was in undergrad. Yudkowsky is terrified (literally terrified) that we'll accidentally succeed. He unironically calls this his "AI-go-FOOM" theory. I guess the term that the AI community actually uses, "Recursive Self-Improvement", was too loaded a term (wiki). He thinks that we're accidentally going to build an AI which can improve itself, which will then be able to improve itself, which will then be able to improve itself. Here's where the current (and extensive!) research on recursive-self-improvement by some of the smartest people in the world has gotten us: some compilers are able to compile their own code in a more efficient way than the bootstrapped compilers. This is very impressive, but it is not terrifying. Here is the paper which started it all!

So, since we're going to have a big fancy AI which can modify itself, what stops that AI from modifying its own goals? After all, if you give rats the ability to induce pleasure in themselves by pushing a button, they'll waste away because they'll never stop pushing the button. Why would this AI be different? This is what Yudkowsky refers to as a "Paperclip Maximizer". In this, he refers to an AI which has bizarre goals that we don't understand (e.g. maximizing the number of paperclips in the universe). His big quote for this one is "The AI does not love you, nor does it hate you, but you are made of atoms which it could use for something else."

His answer is, in summary, "we're going to build a really smart system which makes decisions which it could never regret in any potential past timeline". He wrote a really long paper on this here, and I really need to digress to explain why this is sad. He invented a new form of decision theory with the intent of using it in an artificial general intelligence, but his theory is literally impossible to implement in silicon. He couldn't even get the mainstream academic press to print it, so he self-published it, in a fake academic journal he invented. He does this a lot, actually. He doesn't really understand the academic mainstream AI researchers, because they don't have the same :sperg: brain as he does, so he'll ape their methods without understanding them. Read a real AI paper, then one of Yudkowsky's. His paper comes off as a pale imitation of what a freshman philosophy student thinks a computer scientist ought to write.

So that's the decision theory side. On the Bayesian side, RationalWiki isn't quite right. We actually already have a version of Yudkowsky's "big improvement", AIs which update their beliefs based on what they experience according to Bayes' Law. This is not unusual or odd in any way, and a lot of people agree that an AI designed that way is useful and interesting. The problem is that it takes a lot of computational power to get an AI like that to do anything. We just don't have computers which are fast enough to do it, and the odds are not good that we ever will. Read about Bayes Nets if you want to know why, but the amount of power you need scales exponentially with the number of facts you want the system to have an opinion about. Current processing can barely play an RTS game. Think about how many opinions you have right now, and compare the sum of your consciousness to how hard it is to keep track of where your opponents probably are in an RTS. Remember that we're trying to build a system that is almost infinitely smarter than you.

Yeah. It's probably not gonna happen.

Arcsquad12 posted:

Is it normal for me to look at that quote of word vomit and say "the gently caress did any of that even mean?"

Scintilla posted:

In other words its a bloated mess about one and a half times longer than a Wheel of Time entry, or ten times the size of an average novel. Fun times are ahead!

petrol blue posted:

Harry is the author's best guess at what the author to a fellow human is like.

Night10194 posted:

I would no poo poo read a story about a magical banker who helps run a wizard bank and goes on banking adventures, though. Exploring what the 'normal' world is like for wizbiz would be lots of fun, because much like school, I imagine their banking is full of hilarious misadventures.

quote:

Again, seems like the author is speaking of himself in this case. Has he claimed to be a child prodigy and/or a genius in his other writings?

quote:

I think that an evil child genius plotting to take over the world may in the right circumstances be entertaining, but this particular smug know-it-all Harry is just so unpleasant to read that it sucks all the potential joy out of the story.

i81icu812 posted:

LotR: Words: 455,125
Atlas Shrugged: Words: 561,996
War and Peace: Words: 587,287
All seven Harry Potter books: 1,084,170 Words

quote:

Adults don't respect me enough to really talk to me. And frankly, even if they did, they wouldn't sound as smart as Richard Feynman, so I might as well read something Richard Feynman wrote instead. I'm isolated, Professor McGonagall. I've been isolated my whole life. Maybe that has some of the same effects as being locked in a cellar. And I'm too intelligent to look up to my parents the way that children are designed to do.

IronClaymore posted:

Maybe Yudkowsky will actually have time to put his money where his mouth is now and actually build an AI that outdoes the Roomba (a device he disparages, but which does far more than he does to advance the cause of artificial intelligence, thanks partially to cats and Youtube). And maybe he'll finally get therapy? I hope he gets therapy.

sarehu posted:

If you think rape's the worst thing people treat casually in the Harry Potter universe then I've got some unicorn blood to sell you.

Legacyspy posted:

But what? How is that relevant. Nessus seems to be saying that Harry is was supposed to be a uber-rationalist, and isn't, and is instead is like Artemis Fowl, so this is a failing of the story. I want to understand whether or not that is actually Nessus is saying, because that seems ridiculous. If harry fails at being an uber-rationalist, then obviously he wasn't intended to be an uber-rationalist. Your comment has no bearing on that.

Legacyspy posted:

Also why is his behavior so upsetting that this term keeps getting used? I want to understand why Harry is so upsetting to people. Repeatability saying he is smug or a douche... just tells me that his behavior is upsetting to you. It doesn't explain why that is the case.

Night10194 posted:

I had a professor once tell me the Enlightenment was the first time in human history anyone had ever had the idea that maybe rulers should be chosen because they're intelligent, virtuous, and skilled. Seriously. I mean, this poo poo is pretty wide-spread. "All past eras were stupid and we are the best forever." is a very comforting delusion.

Legacyspy posted:

Naw. I have two motivations. I wanted to understand why people dislike Harry so much, and initially people were going "He is infuriating because he is irritating" which doesn't explain much, there have been better explanations since then. The second is that I honestly like hpmor & Eliezer. I'd rate it a 7/10 and I do not think the hate against Eliezer is warranted. I think a lot of stems from people not understanding he writes (due to little fault of his). Like the guy on the first page who, when speaking of the torture vs dust specks says "3^^^3 is the same thing as 3^3^3^3" despite the fact that the very post where Eliezer raised the discussion, he explained the notation before even getting to the torture & dust specks.

Legacyspy posted:

I'm just using this as a really simple example of how the people who mock Eliezer's writing don't even understand basic things hes written and are mocking him from a position of ignorance.

Legacyspy posted:

Similarly I don't think "friendly A.I" is some sort of crazy idea. It seems pretty reasonable. It is basically: "Are there solutions to problems where the solutions are so complex we can't understand everything about the solution? If yes, how do we build something that will give solutions to problems that won't provide solutions that will conflict with other things we care about?"

quote:

"- and after we were done giving him all the sweets I'd bought, we were like, 'Let's give him some money! Ha ha ha! Have some Knuts, boy! Have a silver Sickle!' and dancing around him and laughing evilly and so on. I think there were some people in the crowd who wanted to interfere at first, but bystander apathy held them off at least until they saw what we were doing, and then I think they were all too confused to do anything. Finally he said in this tiny little whisper 'go away' so the three of us all screamed and ran off, shrieking something about the light burning us. Hopefully he won't be as scared of being bullied in the future. That's called desensitisation therapy, by the way."

"I think the word you're looking for is enjoyable, and in any case you're asking the wrong question. The question is, did it do more good than harm, or more harm than good? If you have any arguments to contribute to that question I'm glad to hear them, but I won't entertain any other criticisms until that one is settled. I certainly agree that what I did looks all terrible and bullying and mean, since it involves a scared little boy and so on, but that's hardly the key issue now is it? That's called consequentialism, by the way, it means that whether an act is right or wrong isn't determined by whether it looks bad, or mean, or anything like that, the only question is how it will turn out in the end - what are the consequences."

Yudkowsky's OK Cupid posted:

I happened to be in New York City during the annual Union Square pillow fight, so I showed up dual-wielding two pillows, a short maneuverable pillow for blocking incoming blows and a longer pillow in my right hand for striking. These two pillows were respectively inscribed "Probability Theory" and "Decision Theory"; because the list of Eliezer Yudkowsky Facts, which I had no hand in composing, says that all problems can be solved with probability theory and decision theory, and probability theory and decision theory are the names of my fists.

i81icu812 posted:

Tunicate posted:

I like how 75-1=7.

blackmongoose posted:

Based on the recognition code, I'm pretty sure this sequence is dealing with one of the parts of the actual books that actually does deserve some mocking, only the story is using it in an even dumber way somehow. I hope I'm wrong, but all the elements fit so far

anilEhilated posted:

Somehow the whole thing gets stupider as it goes on.