|
Yeah, exactly. Infinite regression seems to be pretty unavoidable to me in any situation where the results of the prediction have any kind of influence on the subject of the prediction. There just isn't such a thing as the specific outcome of only one single decision.
|
#
?
Oct 29, 2014 05:40
|
|
|
|
| # ? Jun 8, 2024 15:51 |
|
|
Spoilers Below posted:It is if you're going to predict how you'd predict the prediction of a prediction that's predicted by you predicting the prediction of a prediction of a... ad infinitum. Not quite the same thing, but if you look at for example OkCupid's analyses of their user data there's all kinds of simple factors that predict what we would imagine to be elaborately complicated situations. Like their three questions (`Do you like horror movies?', `Have you ever traveled around another country alone?', `Wouldn't it be fun to chuck it all and go live on a sailboat?') that are good predictors of the long-term compatibility of couples. That's not to say that they're perfect predictors. Or that I can come up with a similar set of criteria for the Newcomb box off the top of my head. My point's just that even if we imagine that a particular decision is based on an almost impossibly complicated set of contingencies, conditionals, and subtle personal inflections (like whether or not we want a long-term relationship with a particular person), that doesn't mean that the outcome can't be modeled by simpler means. Put in slightly different terms: our internal models for our own decisions appear to contain an awful lot of redundancies.
|
|
#
?
Oct 29, 2014 06:07
|
|
|
http://lesswrong.com/lw/q9/the_failures_of_eld_science/quote:Jeffreyssai repeated. "So, Brennan, how long do you think it should take to solve a major scientific problem, if you are not wasting any time?" So is Yudkowsky suggesting that all major theoretical scientific breakthroughs should take less than a month because if so holy lol.
|
|
#
?
Oct 29, 2014 06:20
|
|
|
SubG posted:Nah. I mean it's possible that this is some sort of special edge-case problem, but in general it seems that people's behaviour in typical game theory tests is something that's amenable to things like factor analysis. People just tend to overestimate the complexity of their deliberations. Sure, sure. I agree with you completely there. You are totally not wrong. The problem is that Yudkowsky is proposing a super computer that can model human behavior with 100% accuracy, including all future activity, based on past behavior. Such that it would always know if it was dealing with a "moral" person before they made the decision to take the money from the box, even if they were to suddenly get blazed right afterwards and act completely out of character and do something that they would ordinarily never do (because it would have somehow known beforehand that you were going to toke up after you got the money, and not put the extra money in the box). It would also somehow account for you less moral friend insisting on tagging along that day even though neither of you had planned on it, and who would take money from both boxes, even though you yourself would only take from one. Which is pretty different from what you're talking about.
|
|
#
?
Oct 29, 2014 06:45
|
|
|
How do you think Yudkowsky feels about the possibility of, i.e., a second supercomputer that's even smarter than the first supercomputer and knows exactly what the first supercomputer predicts it will do, but always does the opposite? So, if the first supercomputer thinks it will take both boxes, it only takes one, and if the first supercomputer thinks it will take one box, it will take both. And suppose the first supercomputer knows that it knows as much as it knows. How does the first supercomputer know what to say? Clearly in all cases it can't absolutely determine what the second one will do. (This is pretty similar to the halting problem.)
|
|
#
?
Oct 29, 2014 06:49
|
|
|
Mort posted:http://lesswrong.com/lw/q9/the_failures_of_eld_science/ He's suggesting that the scientific method is a bad method of inquiry because it is too slow and the vast majority of the work in solving a problem comes from deciding on a hypothesis from the infinity that is "idea space". This is a huge theme throughout everything he writes, because he is a quintessential ideas guy who wants to believe in his own importance.
|
|
#
?
Oct 29, 2014 06:57
|
|
|
Spoilers Below posted:The problem is that Yudkowsky is proposing a super computer that can model human behavior with 100% accuracy, including all future activity, based on past behavior. Such that it would always know if it was dealing with a "moral" person before they made the decision to take the money from the box, even if they were to suddenly get blazed right afterwards and act completely out of character and do something that they would ordinarily never do (because it would have somehow known beforehand that you were going to toke up after you got the money, and not put the extra money in the box). It would also somehow account for you less moral friend insisting on tagging along that day even though neither of you had planned on it, and who would take money from both boxes, even though you yourself would only take from one. And even if we believe that all of a person's moral complexities influence the outcome of a Newcomb box choice, that doesn't mean that we have to model all of those complexities. Unless none of it is redundant...but it almost certainly is. Like if you wanted to predict how a person will decide some theoretically complicated moral, social, or political question you're probably most of the way there if you ask them their zip code, education, and income.
|
|
#
?
Oct 29, 2014 07:41
|
|
|
SubG posted:That's also different from what Cardiovorax was saying, which is what I was responding to. That is, his claim that you have to model some sort of infinitely recursive thing in order to predict a person's A/B decision. I'm confident that this is not true because it does not appear to be necessary for the person making the decision to do some sort of infinite recursion gymnastics in order to make the decision in the first place. Because people don't, as a general matter, start going `error, error' and smoking out the ears when you present them with such a problem, like some kind of robot out of Star Trek. Sure. But none of this can cause the money to magically wink out of existence, per the rules of the game. Once the problem is set up, without the use of trap doors, legerdemain, teleporters, some concealed thermite and a video camera, etc. there is no way for the AI to retroactively change its mind if you do. By definition it cannot be 100% accurate and cannot account for all the various circumstances that could occur which might cause the other box to be taken without the agent actively choosing to take it. Comedy forum answer: Always choose B, and then break the computer if it doesn't contain $1,000,000 because it was wrong and bad at predicting, preferably using the empty box as a warning to any future iterations of the computer. The computer will not commit suicide because it knows the third law of robotics, so box B should always be full. Then take A also, because hey, free extra money. Or wear x-ray goggles and inspect box B beforehand, because seriously, we've got a computer that can model behavior this accurately, and we can't get some x-ray specs? Toph Bei Fong fucked around with this message at 08:05 on Oct 29, 2014 |
|
#
?
Oct 29, 2014 08:03
|
|
|
You seem to have misunderstood me there. I didn't say that a person can't make decisions based on a prediction, I said that you can't make an accurate prediction when the point of the prediction is to influence the outcome of the predicted event. It's computationally impossible. Event E is predicted by prediction P. The actor influences event E and it becomes event E', which makes prediction P invalid and requires a new prediction P'. The actor influences event E' and it becomes event E'', which makes... etc. etc. That's a textbook endless loop. It just can't be resolved.
|
|
#
?
Oct 29, 2014 08:06
|
|
|
Spoilers Below posted:By definition it cannot be 100% accurate and cannot account for all the various circumstances that could occur which might cause the other box to be taken without the agent actively choosing to take it. Actually, by definition the AI is 100% accurate in that thought experiment. That's the whole point. It just so happens that it's impossible to make such an AI in real life.
|
|
#
?
Oct 29, 2014 08:12
|
|
|
RPATDO_LAMD posted:Actually, by definition the AI is 100% accurate in that thought experiment. That's the whole point. Ah yes, real life. Bane of thought experiments, solver of paradoxes, frustrater of "idea guys" 
|
|
#
?
Oct 29, 2014 08:22
|
|
Krotera posted:How do you think Yudkowsky feels about the possibility of, i.e., a second supercomputer that's even smarter than the first supercomputer and knows exactly what the first supercomputer predicts it will do, but always does the opposite? This sort of idea combined with Yudkowsky's belief that 0 probability of any event can't exist kind of wreck the premise of that TDT Problem. The problem is based on the decision process of the tester and subject, if the subject is also capable of these perfect predictions and decides to make his decision making so the situation turns out as: Tester: Put $100 in A if Subject will open only A Subject: Take only A only if Tester put nothing in A This ends up with a paradox where it's impossible for both predictions to be correct.
|
|
|
#
?
Oct 29, 2014 09:13
|
|
|
Mort posted:http://lesswrong.com/lw/q9/the_failures_of_eld_science/
|
|
#
?
Oct 29, 2014 09:26
|
|
|
The idea of a perfect AI taking control of the world by simply communicating with people reminds me of some concepts in tool-assisted speedruns. When a player is limited to only pressing buttons connected to a video game, it seems implausible that you could do much outside of the limitations of the game. But with a careful understanding of the underlying code, it's possible to make a bot that uploads custom code through the controller port by manipulating Mario's actions until it finds a hole in the code. So the idea is that a perfect AI would understand the exact sequence of communication required to convince a person of anything, whatever those inputs are, and exploit that to bend everyone to its will. If you presuppose a perfectly intelligent AI, the idea that people are gullible enough to obey it is not the implausible part. The handwaving all comes earlier, when Yud argues that a perfect AI can arise by accident.
|
|
#
?
Oct 29, 2014 09:46
|
|
|
Cardiovorax posted:You seem to have misunderstood me there. I didn't say that a person can't make decisions based on a prediction, I said that you can't make an accurate prediction when the point of the prediction is to influence the outcome of the predicted event. It's computationally impossible. Event E is predicted by prediction P. The actor influences event E and it becomes event E', which makes prediction P invalid and requires a new prediction P'. The actor influences event E' and it becomes event E'', which makes... etc. etc. That's a textbook endless loop. It just can't be resolved. It's worth pointing out that this is actually a different proposition than your original statement to which I objected: Cardiovorax posted:Predicting yourself accurately would require you to have a perfect mental model of your own mind, which kind of by definition requires you to be larger than yourself. Spoilers Below posted:By definition it cannot be 100% accurate and cannot account for all the various circumstances that could occur which might cause the other box to be taken without the agent actively choosing to take it. And I'm not sure what you mean `by definition' here. There's nothing saying that a person's decision schema can't be as simple as always choosing A if it's Tuesday and B otherwise. I mean I'm not arguing that it's necessarily that simple (and I'm pretty sure it isn't) but the idea isn't incoherent or a contradiction in terms or whatever.
|
|
#
?
Oct 29, 2014 12:11
|
|
|
SubG posted:This is only true of both sides are doing beep boop infinite recursion like Star Trek robots about to explode. But humans aren't like that, so the AI doesn't have to dive down a recursive bunny hole because it just has to predict the human's behaviour, and the human's behaviour isn't derived by an infinite loop of `...but he knows I know he knows...' nonsense. Because it can't be. SubG posted:It's worth pointing out that this is actually a different proposition than your original statement to which I objected:
|
|
#
?
Oct 29, 2014 12:24
|
|
|
Chamale posted:The idea of a perfect AI taking control of the world by simply communicating with people reminds me of some concepts in tool-assisted speedruns. When a player is limited to only pressing buttons connected to a video game, it seems implausible that you could do much outside of the limitations of the game. But with a careful understanding of the underlying code, it's possible to make a bot that uploads custom code through the controller port by manipulating Mario's actions until it finds a hole in the code. So the idea is that a perfect AI would understand the exact sequence of communication required to convince a person of anything, whatever those inputs are, and exploit that to bend everyone to its will. But only if such a sequence exists, which it probably doesn't even if you do suppose a perfect omniscient AI larger than the universe that doesn't give a poo poo about chaos theory. The human brain has a bunch of feedback and control systems and works in really weird ways. To suppose that for any given behaviour Y there is always a sequence of inputs X that causes Y is implausible, in the same way that saying for any given physical phenomenon Y there is always a sequence of body movements X that cause Y, so you too can fly at 500mph and punch tornadoes at people if only you had sufficient perfect mastery of your body chi or something.
|
|
#
?
Oct 29, 2014 13:17
|
|
|
SubG posted:If you mean that the human could just flip a truly random coin or something sure. But that's not even an interesting quibble in the context of the problem. It could be as simple as that, but it could also be infinitely more complex, hence the bringing a friend along examples, which a 100% accurate computer would have to account for to be 100% accurate. Also it would need to account for a sudden shift in personality that day (the person just had a major death in the family as is feeling out of sorts), having a stroke and making a random choice (and thus the computer not putting anything in B) when one otherwise would have ordinarily only taken one box, etc etc. which quickly spirals into the "Holy poo poo, this would need to model a lot more than a simple human mind" to account for. Because 100% accurate means Not Ever Wrong, and Impossible in Real Life Because Computers Can't Model That Much Stuff The Calculation Would Take Forever. The ordinary personality test, "works with 99% accuracy", type stuff I agree you could probably refine to. It's the 100% accurate "retroactively affects the past" that I quibble with. I do think it's very interesting that most of the paradoxes discussers think only in terms of modeling one person's mind, though. I find discussing the limits and problems with the question as presented a lot more interesting than the possible solution, personally v  v
|
|
#
?
Oct 29, 2014 14:40
|
|
|
You don't need recursive super predictors. You just need to let the AI choose what to put in the box after it's received your decision. Or in a military analogy, let the enemy apportion forces between two targets after their spies tell them your orders about which to attack, but before they are carried out. These are logically identical to an AI that predicts your actions with knowledge of your psychology. The whole time travel thing is a red herring that makes the problem seem more mysterious and mindblowing than it is. The future can affect the past! No, just people's anticipations of your actions will affect their response to them. It's difficult to think of a more trivial point.
|
|
#
?
Oct 29, 2014 15:25
|
|
|
Peel posted:You don't need recursive super predictors. You just need to let the AI choose what to put in the box after it's received your decision. Or in a military analogy, let the enemy apportion forces between two targets after their spies tell them your orders about which to attack, but before they are carried out. These are logically identical to an AI that predicts your actions with knowledge of your psychology. But Yud's entire point with the exercise is to reverse cause and effect because he wants his computer god to be infallible and almighty so he is thought of as a unique thinker. Every "unique" idea he's had is just him being (in his mind) contrary to the humanities. And all his big thinking I'd to make himself seem important and enlightened to the masses so he can get off on his status as leader. He reminds me if some relevant passages of "why does he do that?" and my own insecurities as a teenager.
|
|
#
?
Oct 29, 2014 16:13
|
|
|
Peel posted:You don't need recursive super predictors. You just need to let the AI choose what to put in the box after it's received your decision. Or in a military analogy, let the enemy apportion forces between two targets after their spies tell them your orders about which to attack, but before they are carried out. These are logically identical to an AI that predicts your actions with knowledge of your psychology. That's not the question as presented, though. The computer makes a prediction (based on a brain scan or whatever), puts the stuff in boxes, and is then powerless to change anything. You then make you choice. If it gets to alter things after you've made your decision, then the experiment is rather different. But even the military analogy can quite easily grow more complicated than that quickly. Is there any doubt as to the spies loyalty? How many spies were sent? What if they bring back conflicting information? Which spies can be trusted, if there is more than one? Have they verified this kind of information before? How accurate was that information the last time? What if this time the spies have been turned, because Internal Affairs has uncovered subversive literature planted into their rooms by agents from our side, to cast doubt on the spies completely legitimate information? Do you change your orders based on the knowledge that the plans have been leaked (trivial, given the speed to today's communications)? Can you use the homing signal from the little beacon planted inside the plans to execute a drone strike on the enemy central command, rendering their entire attempt to apportion forces null and void? quote:The whole time travel thing is a red herring that makes the problem seem more mysterious and mindblowing than it is. The future can affect the past! No, just people's anticipations of your actions will affect their response to them. It's difficult to think of a more trivial point. This is 100% correct.
|
|
#
?
Oct 29, 2014 16:21
|
|
|
Reminder that Yud invented TDT because he was disappointed with the answer the Bayesian approach gives for this thought experiment.
|
|
#
?
Oct 29, 2014 16:35
|
|
|
Also reminder that Yudkowsky's original paper ('paper') where he explains TDT ran to over 100 pages. It has since been rewritten by one of MIRI's research people into 17 pages.
|
|
#
?
Oct 29, 2014 16:54
|
|
|
Moddington posted:Reminder that Yud invented TDT because he was disappointed with the answer the Bayesian approach gives for this thought experiment.
|
|
#
?
Oct 29, 2014 16:55
|
|
|
Mort posted:http://lesswrong.com/lw/q9/the_failures_of_eld_science/ No wonder he loves to stick to fiction. In fiction, everyone can have already invented a unified field theory or a god AI or whatevs by following his simple trick (THE SCIENTIFIC METHOD HATES THIS!) and he can spend the entire article beating off about his thinking methods. The hard part (actual discovery) is easily written as 'And then the smart people doing what I say discovered the thing way faster than the stupid people who do it wrong' and he can spend the whole story smugging it up about how right he was.
|
|
#
?
Oct 29, 2014 17:48
|
|
|
Night10194 posted:No wonder he loves to stick to fiction. In fiction, everyone can have already invented a unified field theory or a god AI or whatevs by following his simple trick (THE SCIENTIFIC METHOD HATES THIS!) and he can spend the entire article beating off about his thinking methods. The hard part (actual discovery) is easily written as 'And then the smart people doing what I say discovered the thing way faster than the stupid people who do it wrong' and he can spend the whole story smugging it up about how right he was. I love the "suppose you have exactly the amount of exprimental data you need" part. Like experimentation is just some busy work you need to get out of the way of your really important task of
|
|
#
?
Oct 29, 2014 18:01
|
|
|
Political Whores posted:I love the "suppose you have exactly the amount of exprimental data you need" part. Like experimentation is just some busy work you need to get out of the way of your really important task of Tell me about it. I failed as a scientist and have no scientific experience beyond undergrad organic chemistry, but holy poo poo, even just doing titration for those labs and materials analysis for extremely simple experiments designed solely to teach young students how to gather experimental evidence and demonstrate basic principles took a lot of time. I've got friends who work in medical research and the amount of time they spend trying to get the machines and robots working and sort through results is enormous. Yud wants to be the wild Hollywood Scientist who gets The Answer in a short time because the plot demands it and gosh he's so brilliant; he's afraid of doing the real work required.
|
|
#
?
Oct 29, 2014 18:05
|
|
|
Class Project posted:"We know from common sense," Yin said, "that if we stepped outside the universe, we would see time laid out all at once, reality like a crystal. But I once encountered a hint that physics is timeless in a deeper sense than that." Yin's eyes were distant, remembering. "Years ago, I found an abandoned city; it had been uninhabited for eras, I think. And behind a door whose locks were broken, carved into one wall: quote .ua sai .ei mi vimcu ty bu le mekso unquote." This is what Yudkowsky thinks science looks like.
|
|
#
?
Oct 29, 2014 18:36
|
|
|
Hmm yes "common sense" is a phrase you want to see in a scientific treatise.
|
|
#
?
Oct 29, 2014 18:41
|
|
|
Hahaha Lojban loving seriously?
|
|
#
?
Oct 29, 2014 18:42
|
|
|
Cardiovorax posted:Hahaha Lojban loving seriously? The man thinks that monolithic RSI AGI can be developed entirely accidentally, of course he thinks Lojban is the perfect language.
|
|
#
?
Oct 29, 2014 18:45
|
|
|
The only particularly nice thing about lojban is that it's really easy to write a parser for. If they were really looking for a pure logical language they would use Ithkuil obviously  . I'm sure they would be able to tell that its 3^^^3 case inflections make it objectively superior.PS I'm a big conlang nerd.
|
|
#
?
Oct 29, 2014 18:49
|
|
|
Anyway, wasn't one of the conclusions of chaos theory that sufficiently complex systems are effectively impossible to accurately predict for anything more than the shortest term? And isn't the current theory that human cognition relies on processes down to the quantum level, making it essentially impossible to perfectly measure in order to create a model? Either of which would invalidate most of TDT, relying as it does on perfect modeling and prediction, except as a somewhat interesting thought experiment for solipsists.
|
|
#
?
Oct 29, 2014 18:52
|
|
|
One Swell Foop posted:And isn't the current theory that human cognition relies on processes down to the quantum level, making it essentially impossible to perfectly measure in order to create a model?
|
|
#
?
Oct 29, 2014 19:19
|
|
|
One Swell Foop posted:And isn't the current theory that human cognition relies on processes down to the quantum level, making it essentially impossible to perfectly measure in order to create a model? Even statements like that are pure handway speculation. We currently do not have a solid model of how human cognition arises and even basic models of how it functions are deeply contested. So not even going into the AI bullshit even the "humans function and can be modelled in X way" stuff Yudkowski spouts is also built on clouds and fairytales. [edit] Just re-reading this I can see how it could sound a bit harsh. Sorry One Swell Foop I in no way meant to sound aggressive towards you. Yours is a perfectly reasonable comment. Munin fucked around with this message at 19:27 on Oct 29, 2014 |
|
#
?
Oct 29, 2014 19:19
|
|
|
Lottery of Babylon posted:.ua sai .ei mi vimcu ty bu le mekso As a lojban enthusiast, lol at the pointless 'mi' ("I") and the terrible translation that makes a boatload of assumptions as to what the message meant. Dude could've been working on just one equation, and 't' could've been torque or anything really. And personally I love all these people who treat lojban like it's some perfect marriage of science and language. Lojban is just an experiment in how close to predicate logic you can get in human language. And while the native speakers we need to really answer this are still forthcoming, it's fairly clear that a different approach would fare better. Funnily enough, I originally learned about Less Wrong from the lojban chaps. Telarra fucked around with this message at 19:33 on Oct 29, 2014 |
|
#
?
Oct 29, 2014 19:31
|
|
|
I wonder how Yudkowsky would react to the revelation that science and logic only even interact tangentially.
|
|
#
?
Oct 29, 2014 19:36
|
|
|
Cardiovorax posted:I wonder how Yudkowsky would react to the revelation that science and logic only even interact tangentially. He would probably bemoan how "Frequentist" scientists are, before spending 3 blog posts ruminating on how to create a 'Bayesian scientific method' and concluding with a really roundabout way of saying "I dunno".
|
|
#
?
Oct 29, 2014 19:43
|
|
|
Munin posted:Even statements like that are pure handway speculation. No problem, it's many years since I've been current on scientific literature, which is probably why I still talk about Chaos Theory like it's a thing.
|
|
#
?
Oct 29, 2014 20:07
|
|
|
|
| # ? Jun 8, 2024 15:51 |
|
|
Cardiovorax posted:I'm really sceptical about that, personally. The brain is primarily biochemical in its operation, with subatomic particles playing a minor role and only in amounts that are suffiently large that the behaviour of individual particles does not matter except in a statistical sense. Plus, most interesting interactions between quantums break down at higher temperatures, meaning anything hotter than frozen nitrogen in this case. The "theory" seems to be primarily espoused by people who heard about the observer effect and misunderstood it completely. It's a chaotic system, though, so even if quantum dice-rolling isn't statistically significant at any given step, it means that the system as a whole is still ultimately non-deterministic. Easy example would be a C-14 atom decaying and causing a point mutation that alters gene expression, but you don't need anything that direct.
|
|
#
?
Oct 29, 2014 22:12
|
|
