Reminds me of other cartesian boundary holes opened when “trying to explain” good to evil “people”:

* Evil: “If we’re going to die anyway we should at least have a good time”. Me: “I don’t care what happens if just everyone’s going to die anyway!!! Wait, what am I saying, that’s not what I mean!” (I still care about justice even if everyone’s going to die, but I care about that by just doing justice in the first place)

* My use of the term “spite” when trying to justify that people do care about undoing timelines.

And arbitrary implies a semantic stopsign to the endless progression of “why”s, implies trying to make the thought experiment out of cut-off structure.

Acting for arbitrary reasons. But decision theory is all about not being cut-off from a unified UDT stance.

Newcomb’s problem is a flawed prototype for decision theory because it’s about not being able to beat an authority defined by the first ordinal.

Lies about post singularity decision theory which Yudkowsky has told:

lie: Anyone who doesn’t believe in vengeance bankruptcy doesn’t have enough hope:

> in a hundred million years the organic lifeform known as Lord Voldemort probably wouldn’t seem much different from all the other bewildered children of Ancient Earth. Whatever Lord Voldemort had done to himself, whatever Dark rituals seemed so horribly irrevocable on a merely human scale, it wouldn’t be beyond curing with the technology of a hundred million years. Killing him, even if you had to do it to save the lives of others, would be just one more death for future sentient beings to be sad about. How could you look up at the stars, and believe anything else?

lie: Pushing the frame that there are only two futures, extinction or heaven, downplaying the possibility of what Yudkowsky calls “hyperexistential risk” e.g. an AGI that cares about humans but not other species (not that yudkowsky believes in animal rights, he apparently doesn’t). Framing alignment as purely a matter of intelligence and capability rather than justice.

> A dead planet, lifelessly orbiting its star, is also stable. Unlike an intelligence explosion, extinction is not a dynamic attractor—there is a large gap between almost extinct, and extinct. Even so, total extinction is stable. Must not our civilization eventually wander into one mode or the other? As logic, the above argument contains holes. Giant Cheesecake Fallacy, for example: minds do not blindly wander into attractors, they have motives. Even so, I suspect that, pragmatically speaking, our alternatives boil down to becoming smarter or becoming extinct.

http://intelligence.org/files/AIPosNegFactor.pdf

> One seemingly obvious patch to avoid disutility maximization might be to give the AGI a utility function U=V+W where W says that the absolute worst possible thing that can happen is for a piece of paper to have written on it the SHA256 hash of “Nopenopenope” plus 17

https://www.lesswrong.com/posts/hGmFNBXDinfiKJGD6/don-t-even-think-about-hell

lie: Excluding nonhuman animals.

> This means that, e.g., a vegan or animal-rights activist should not need to expect that they must seize control of a CEV algorithm in order for the result of CEV to protect animals. It doesn’t seem like most of humanity would be deriving huge amounts of utility from hurting animals in a post-superintelligence scenario, so even a small part of the population that strongly opposes* this scenario should be decisive in preventing it.

https://arbital.com/p/cev

> To spell it out in more detail, though still using naive and wrong language for lack of anything better: my model says that a pig that grunts in satisfaction is not experiencing simplified qualia of pleasure, it’s lacking most of the reflectivity overhead that makes there be someone to experience that pleasure.

https://rationalconspiracy.com/2015/12/16/a-debate-on-animal-consciousness/

But given that MIRI, inventors of TDT, paid out to blackmail (by a former employee who knew all about TDT and the integrity of these people and therefore had high subjunctive dependence for sure), in contradiction of one of their own basic thought experiments, I think my version of TDT actually is weird for covering the case of collapsing timelines.

The conception of TDT that Anna Salamon pushed at WAISS was strictly about preserving timelines (via submission). One-boxing in Newcomb’s problem can be seen this way.

I think if they had any, I call it “S1 buy-in” in this post (but it’s not a term I’d consider very accurate now) for TDT, it didn’t address the feeling of doing something although the universe screams collapsing the timeline failed, calling the bluff of the reality of a world you experience.

I bet they S1 imagined TDT worked like, they’d say “No, I won’t pay out to blackmail, because I have TDT and we have subjunctive dependence”, and the blackmailer would say, “Zounds! So clever! I’m defeated!” and go away. As opposed to escalating minor harassment like he did, to traumatize them until they accepted his proof that he was just crazy and no reasoning with him.

I think vengeance is basically the natural prototypical instance of using TDT.