(need to apply skill template here properly…)

"If you’re interested in being on the right side of disputes, you will refute your opponents’ arguments. But if you’re interested in producing truth, you will fix your opponents’ arguments for them. To win, you must fight not only the creature you encounter; you must fight the most horrible thing that can be constructed from its corpse."

No one begins to truly search for the Way until their parents have failed them, their gods are dead, and their tools have shattered in their hand.

The Sequences

good starting article:

(checklist to copy-paste goes here…)
(not really sure why someone would have stumbled onto this site without already being a rationalist though!)

"Can this really be true, when it seems so obvious now, and yet none of the people around me believe it?"
Yes. Welcome to the Earth where ethanol is made from corn and environmentalists oppose nuclear power. I'm sorry.

"However, just because Science does not trust scientists to do something, does not mean it is impossible.

But a word of caution here: The reason why history books sometimes record the names of scientists who thought great high-minded thoughts, is not that high-minded thinking is easier, or more reliable. It is a priority bias: Some scientist who successfully reasoned from the smallest amount of experimental evidence got to the truth first. This cannot be a matter of pure random chance: The theory space is too large, and Einstein won several times in a row. But out of all the scientists who tried to unravel a puzzle, or who would have eventually succeeded given enough evidence, history passes down to us the names of the scientists who successfully got there first. Bear that in mind, when you are trying to derive lessons about how to reason prudently.

In everyday life, you want every scrap of evidence you can get. Do not rely on being able to successfully think high-minded thoughts unless experimentation is so costly or dangerous that you have no other choice.

But sometimes experiments are costly, and sometimes we prefer to get there first… so you might consider trying to train yourself in reasoning on scanty evidence, preferably in cases where you will later find out if you were right or wrong. Trying to beat low-capitalization prediction markets might make for good training in this?—though that is only speculation.

As of now, at least, reasoning based on scanty evidence is something that modern-day science cannot reliably train modern-day scientists to do at all. Which may perhaps have something to do with, oh, I don't know, not even trying?

Actually, I take that back. The most sane thinking I have seen in any scientific field comes from the field of evolutionary psychology, possibly because they understand self-deception, but also perhaps because they often (1) have to reason from scanty evidence and (2) do later find out if they were right or wrong. I recommend to all aspiring rationalists that they study evolutionary psychology simply to get a glimpse of what careful reasoning looks like. See particularly Tooby and Cosmides's "The Psychological Foundations of Culture"."

I will not say, "Don't try this at home." I will say, "Don't think this is easy." We are not discussing, here, the victory of casual opinions over professional scientists. We are discussing the sometime historical victories of one kind of professional effort over another. Never forget all the famous historical cases where attempted armchair reasoning lost.

Yudkowsky's errors

It is easy to be blindsided by trust.
There are problems in each of these.
Can you spot them?


The Roko's Basilisk Incident (We care more about its handling, and Yudkowsky does explain himself elsewhere… but you should work out for yourself if the basilisk is worth worrying about. How should he have responded to this?) (Using the RW link because it is decently formatted.)

older stuff


(Compile a list of everything interesting Yudkowsky has been mistaken about, especially pre-2002 stuff. Warning: we don't want to provide material for unscrupulous individuals to attack him with, so this perhaps should be placed at a… higher level of security clearance. Something akin to compiling a list of memetic hazards- you don't hand those things out like candy on the street corner. At the same time, we also don't want to create a Streisand effect either. It should be an unknown unknown for most everyone- they don't realize it is even something they should even look for.) one relevant thread (I swear Yudkowsky did a proper write up somewhere… something about how he was arguing against the general case because the specific case was obviously wrong.)

Random resources

expected utility





Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License