I don’t think Popper said “hard to vary”. But Deutsch presents it like a major, original idea whereas I think it’s just a different wording of existing ideas. Evolutionarily adapted ideas are hard to vary in the sense that they are on peaks (local optima) of Mount Improbable (as Dawkins titled a book about evolution), so all directions are down in the short term. Ideas at the highest peak of the mountain are also hard to vary since all directions are down in the short and long term. In that book, Dawkins presented himself as explaining evolution. Whereas Deutsch did something similar but presented himself as saying something important and new. Adaptation basically means that most random changes make things worse not better, which could be phrased as “adaptation makes things hard to vary” without saying anything significantly new.
I raised this issue with Deutsch before BoI was published, when he was talking with me a lot, but I was unable to get an answer that I thought addressed my argument.
My answer to this is simple. A doesn’t have obvious errors. B might have obvious errors. B is higher risk. This is a criticism of using B now. So, based on criticism, I’ll use A.
That’s if I’m picking one right now – more realistically I’d probably critically analyze B before building a bridge. I’d criticize the idea of using A now by pointing out that bridges are really expensive and take a long time so it’s better to look into B a bit before committing all that money and effort. Of course it depends some on context and details that weren’t provided, like if the bridge is merely putting a single piece of wood over a stream and we’re already there and want to cross now then maybe we don’t analyze B first.
It’s cool that you got some fairly friendly replies trying to explain stuff. I think it’d be good for you to keep talking to LW more until something changes: you start getting ignored, get mainly unfriendly replies, it gets too repetitive, etc. Seems like maybe you can get some amount of debate.
Yeah, I’m glad about that. I think giving them an example that was fairly simple (so even beginners thought they could address it) and being more curious about how they address it (as opposed to a ‘haha gotcha’ tone) both helped. I asked claude for review beforehand too and it pointed out my original title (Are Bayesians Easy to Scam?) was probably too clickbaity and accusatory.
“risk” is interesting to me here. I suspect a bayesian might think risk → probabilities → induction (otherwise how do you calculate the risk %). This seems like an artifact of our modern world trying to quantify risk for everything (insurance premiums, business reports, etc). But we can also treat risk qualitatively, which I guess is what you’re doing.
Risk can be treated as a quantity. One is bigger than the other (which we can tell without measuring either), which is a difference that arguments can refer to.
It’s core epistemology things like argument strength and degree of truth/belief that shouldn’t be seen as quantities.
You get widely varying responses on Less Wrong, from people of varying degrees of Bayesianism adherence or knowledge. I don’t think there’s any one trying to take any responsibility for organizing canonical answers, especially since Yudkowsky mostly stopped writing about rationality years ago.
Yeah, I’m curious to see if there’s many bayesian-bayesian replies correcting each other. There have been a few things so far that might be conflicting.
Also all my replies have this showing to me atm. I’m not sure if that’s normal or was like that yesterday. I don’t see anything that would have caused my account to be flagged besides questioning bayesianism, and no other relevant notifications.
[This comment will not be visible to other users until the moderation team has reviewed it.]
I don’t think P(my epistemology) is a practically useful concept.
Huh? Shouldn’t we evaluate different epistemologies and compare which is better? And P(X) is how you score things in Bayesianism and this guy seems to be a Bayesian. Does he also object to P(Critical Fallibilism)? If so how would he evaluate CF?
Does critical fallibilism have anything to say about itself? If it gives itself a full bill of health, that’s no more useful than the statement “This sentence is true”, which can consistently be assigned any truth value.
But if P(Bayesianism) was low according to Bayesianism then that would be problematic. Seems he thinks of the philosophies more as biased people rather than a framework with rules which lead to unavoidable outcomes which may or may not contradict itself, like math axioms.
Yeah, I thought the same thing. There’s been a bit of that, like Cole Wyeth:
Arguably, using Bayesianism to evaluate schools of epistemology (which may not be Bayesian) is a type error.
I think some of these people are confused about epistemology, or at least have picked up some weird ideas about it from somewhere.
It’s like some of them don’t realize they’re on a site for bayesians and think it’s a site for statisticians instead. A few people have retreated to a bayesians just like bayesian stats type argument, but it’s not like their first reply to the scam scenario was ‘you can’t apply a statistical method to a sample size of 1’ or something – almost everyone answered with something about explanations. Maybe they subconsciously realize the sample size of 1 thing would be bad to say (for bayesianism/ists).
On the note of that quote and CF, here was my reply to it. I tried not to be like pushy and just to explain (very briefly) how CF approaches things. Posting here for criticism. Maybe the debate stuff was a bit of a side track but it does relate to claiming no unanswered criticisms though I wasn’t explicit about how. I also tried not to put words in @Elliot’s mouth but also not say like “I think ET thinks X” or other awkward phrasing. I also think “So maybe CF isn’t as good as it could be if there was more debate” was good to add. It’s not really a concession of anything but it feels like the right kind of humility to have around one’s own school, and it runs counter to a ‘here is the one true epistemology’ vibe. I don’t think I was giving off that vibe, but my impression is that kind of attitude is often common with online philosophy discussions.
following your link from the AGI thread. really interesting article.
i went and read the Paths Forward summary and the debate policies introduction and the discussion trees essay since you mentioned those as CF’s tools for criticism breadth.
a few things jumped out reading them alongside this article. Paths Forward says:
Fallibilists should want a better approach than trusting their own judgment because they know they may be mistaken (including mistaken about their methodology, biases or integrity, not just the topical issue). Paths Forward offers ways to be more objective instead of trusting yourself.
and the debate policies piece compares them to laws - they help limit the power of kings, judges and other authorities and help prevent people from making arbitrary or biased decisions sometimes overruling the intellectual’s personal judgment.
from the timeline it sounds like Paths Forward and debate policies were already in place during the period you were wrong about the things you list here. what is it that the tools didn’t catch, and why? like, was it that:
people didn’t engage with Paths Forward at all (intake problem)
people engaged but the debates didn’t surface the right kind of criticism (the intuitive/experiential stuff you mention)
or something else?
this is basically a case study of the system running in production and not catching certain error types. would be helpful to debug the failure mode precisely.
yeah the problem you describe is really common. i used to look at political tribalism and think people supporting the opposite side were just being irrational. but a lot of it is people never hearing real criticism of their own group’s ideas. it’s social thinking, not truth-seeking. people conform and don’t say what they actually believe.
but i don’t think obscurity is really the issue in your case. you’ve had an active forum, a blog with comments, and Paths Forward has been published for over 10 years. that’s way more opportunity for critics to reach you than most thinkers ever create.
the thing that stuck out to me was what you identified about people who were there didn’t share intuitive, practical, or experience-based criticisms. they were present, they had relevant knowledge, and they didn’t share it. why do you think that is?
yeah the list makes sense. i thought of a couple more: there’s the thing in aviation where copilots under-report problems because the captain is the recognized expert and ‘probably already knows’. if the expertise gap is big enough, people just stop speaking up. and there’s a survivorship bias thing too, where people who disagree with a community’s core framework just quietly leave rather than fight about it, so you never even know they had something worth hearing.
the intuition point stood out most. you said your intuition articles are fairly recent and probably missing something. so during the actual years you’re describing, if someone on the forum had a feeling that something was off but couldn’t turn it into a formal argument, what were they supposed to do? was there guidance for that?