Critical Fallibilism Real World Examples

Elliot · January 14, 2025, 3:06am

I found an intellectual with a published debate policy: Alex Epstein. Although I’ve worked with him in the past, I don’t know if the creation of this debate policy had any connection with my ideas or not. (It’s possible he doesn’t know either even if there is a connection. I wouldn’t be surprised if someone read some of my stuff and then said something to him related to debate policies without naming me. A lot of my readers are fans of Epstein.)

Although I don’t love this debate policy (I think it’s very restrictive and is inadequate for Paths Forward), I acknowledge it has major advantages over what most intellectuals offer (no written policies, no clarity).

One concern I have is about transparency: I don’t know if he keeps his word or not. Supposing he didn’t, how/where would I find out? As far as I know, he doesn’t have any sort of open forum where people could complain that they met the criteria but he still wouldn’t debate them. Nor does he have anywhere he publishes all the debate requests he receives and his judgment of whether they meet the criteria or not.

His criteria also contain some ambiguity. 250,000 views over what time period? Also views aren’t guaranteed, so how reliable does success at those viewer numbers have to be, and what would be good enough to convince him?

I’d like to debate Epstein about whether Silent Spring is a good book, among other things, but I don’t meet the criteria. He was also unwilling to debate me when I was doing free work for him (and had a lot less disagreement with him, but still some), which I think is unreasonable and is one of the main reasons I stopped working with him. If you think someone is really smart, tell them so, meet them in person to chat, use their research in your books and debates and publish some of their writing, then you don’t need to avoid debating them because of “protecting me from having my time wasted by the endless number of unknown people who demand multi-hour debates with me”. I wasn’t then, and still today am not, unknown to Epstein, so the reasoning he gives for his policy doesn’t explain why he wouldn’t debate me and may be misleading (which brings us back to the transparency concern above). I believe a better debate policy would allow more ways of proving your worth than just paying a lot of money or having a huge audience. One important issue is whether a policy enables upwards mobility of merit or not (this policy doesn’t because it basically excludes everyone with merit who hasn’t already had major success).

Elliot · January 14, 2025, 3:31pm

Here are some more issues with Alex Epstein’s debate police:

It doesn’t say what topics he’ll debate. Would he debate Objectivism? Critical Rationalism? Silent Spring? I don’t know if he’s mentioned Silent Spring in the last 5 years. (He attacked it on Twitter long ago and he published an article on his website, written by an associate, attacking DDT haters including Silent Spring. The article doesn’t provide a publication date but I think it’s over 10 years old.)

Is Silent Spring too indirectly related to energy and outside of his expertise for a debate? What about the issue of whether fossil fuel companies are incompatible with the capitalist rule of law due to committing fraud? Would he debate that? Does he think he has expertise about that? If not, would he debate the issue of whether he ought to have expertise about that, and be prepared to debate it, given his career choices and reputation?

Pre-debate questions can be quite important but the policy doesn’t enable them and I’m not aware of any forum where people can ask. I asked Epstein if he’s read Silent Spring but he didn’t answer. It’d be quite a different debate if

He read Silent Spring, disliked it, and formed his own opinions about it.

or

Some secondary sources told him Silent Spring is bad but he doesn’t actually know what the book says.

The fraud issue would also require pre-debate questions to work well because Epstein has published little or nothing about it, so there’s no good way to prepare debate arguments in advance (since you don’t know what claims he’ll make).

Another issue is debate formats. Would he debate on a text-based internet forum, like this one, or would he just refuse and insist on real-time formats only? Is he only willing to do the standard debate-to-time-limit and not any other ending criteria for debates, and consequently only willing to do real-time debates? One of the downsides of real-time debates is they’re broadly unsuitable for fact checking citations (which often requires checking many sources to find where a claim originally comes from). In my experience, Epstein thinks giving a citation to a secondary source is adequate if he thinks they’re trustworthy, and he doesn’t see the need to figure out where the secondary source got the information and figure out if it’s actually true.

Elliot · January 14, 2025, 4:19pm

Epstein has 15,000 YouTube subscribers and gets under 1,000 views per video. He wouldn’t debate himself. He set the 250,000 views debate criterion way above what he could provide to someone he requested a debate with.

If everyone only wanted to debate people with significantly larger audiences than they have, then no debates would ever happen. Logically, for any pair of people, they can’t both have a significantly larger audience to offer the other person.

A more reasonable approach would be to debate people with over 50% of your own audience size.

Epstein spends time on venues with much smaller audiences than 250,000:

These images are probably biased towards higher view counts since they’re high results in my YouTube search (I searched “alex epstein” without quotes).

If you’ll talk with lots of people with audiences under 10,000, but set a requirement of 250,000 for a debate, then it’s in some sense a fake, inflated rule that’s meant to exclude almost everyone so you can then selectively choose who to engage with (or not) according to unstated policies. The public debate policy is obscuring what the real policies are rather than being a good indicator of who he would debate.

I know that giving a talk or interview and a debate are somewhat different, but I think they’re reasonably similar and that he ought to debate more. He says he wants to protect his time because too many people lacking large followings want long debates with him, but did he ever accept and win even one of those debates with someone with a small following? I managed to find only a single multi-hour (>= 2h) debate he’s ever had (I searched “alex epstein debate” without quotes).

For >= 1h, I found:

That’s it. I found only 3 videos of Epstein actually debating someone at length. None of them have 250,000 views.

For short debates, there’s not much either, but I found:

So he’s occasionally debated unknown people in person for around 10 minutes or less, and most of it isn’t posted online.

If you have under 10 hours of total video of your debates online, maybe you shouldn’t be declining to debate anyone unless they pay you a bunch of money or have a much larger platform than you do. (Or in the alternative, don’t posture as a debater.) Based on YouTube search results, it looks like Epstein has only reached an audience of 250,000 people twice in his life (never for a debate):

If you barely ever debate anyone, it’s misleading to write:

Unfortunately, very few prominent people are willing to debate me in a major forum.

And it’s misleading to define “major forum” as meaning only one of the largest forums he’s ever participated at in his life. He’s excluding people like himself as not even close to famous enough to debate him. His debate policy gives the impression he doesn’t want to waste his time on forums that are too small for him, but that isn’t what’s really going on; he’s excluding debate at most forums that, numerically, aren’t too small for him.

Elliot · January 14, 2025, 4:41pm

Alex Epstein’s FAQ is introduced like this:

Frequently Asked Questions, including many hostile questions I often get. Here’s the unvarnished truth, with references.

But it doesn’t have “many” questions. It has six, which is short for an FAQ. None are clearly hostile questions.

And none of the answers have references. Five out of six don’t really need references but then don’t write “with references”. One ought to have references (“How is your work on energy funded?”) but doesn’t.

Dface · July 17, 2025, 8:50pm

I tried doing a post on reddit while trying to use CF type thinking. It was for a game called Marvel Rivals. I don’t think it was all that popular like it didn’t hook people on for likes and interaction, but it got some responses. Link here and text here:

More About the Spiderman Team up Knock-back Cancel

Hello SpiderMan mains! Bucky main here! I saw that you can now cancel the knock-back from spidey’s team up. I thought it’d be interesting to analyze it for y’all. I’ll also show how you can learn the basics. I just want to say tho that the cancel seems inconsistent to pull off. Like, I can only get it about 30-50% of the time. Maybe with enough practice it can be close to 100%?

What actually is the knock-back cancel?

I saw that the original reddit post said to cancel you gotta web swing+ tracer+ team up+ web swing + tracer. Well, that’s wrong. All you need to do is team up + web swing. Here’s a video of me doing it:

https://youtu.be/Yy87G5xLamo

I did it with cooldowns on btw!

I think web swing + tracer + team up + web swing + tracer is good to learn tho to make the one shot combo. In the original post it looked like it was GOH + double jump + webswing + tracer + team-up + webswing + tracer + uppercut + tracer.

How do you practice the knock-back cancel?

First start out by doing team up +web swing. Try doing it anywhere, like at the sky or at a bot. Try to get it down as muscle memory. Here’s a video of me practicing it:

https://youtu.be/JyN1yT-Rmw0

What helps to pull it off is to web swing when it’s too late. Like, when you clearly have the option to. Little by little, shorten the time between the team up and the webswing until you’re able to cancel the knock back semi-consistently.

How do you build on the cancel?

Try to only add moves that come before or after the team-up + webswing. That’s your basic cancel. What I did is add the tracer after. I did Team-up + web swing + web tracer. Try practicing it anywhere. Do it while in the sky, with cooldowns on, or in a real match even. Do your best to make it muscle memory. Here’s a video of me practicing it:

https://youtu.be/jlZkEwlF8EE

Those are some of the basics for learning the knock-back team up cancel. You can add more moves to the cancel to make it viable in your gameplay. Remember to add one more move each time and try to get that combo down. Hopefully you can do it close to 100% of the time. Sorry this was a short guide, but it was a lot of effort to record and write this post.

Here’s me doing web swing + cluster + team-up + webswing + uppercut:

https://youtu.be/qget5bOpBnM

Have fun!

I thought it’d be interesting to analyze cuz I thought people were trying to automatize the wrong basics for doing the combo. They were adding different moves before and after the basic combo(for some valid reasons btw), but not focusing on getting the basic combo itself down.

Remember to add one more move each time and try to get that combo down.

I don’t know about this cuz people can think of one move as a separate combo itself. Like they can see a series of inputs as one move. I think I was being idk pushy in how to practice when, really, it probably comes down to being creative.

I think some people already found about the basic combo before my post but only a few. People also found that binding the cancel to the scroll wheel makes it way more consistent. The scroll wheel makes so many inputs per second that it can do perfect timing for combos like this.

Elliot · February 13, 2026, 5:06pm

https://x.com/ivanarcus/status/2021592600554168414

You change one word on a loan application: the religion. The LLM rejects it.

Change it back? Approved.

The model never mentions religion. It just frames the same debt ratio differently to justify opposite decisions. We built a pipeline to find these hidden biases

We call these “unverbalized biases”: decision factors that systematically influence outputs but are never cited as such.

CoT [chain of thought] is supposed to let us monitor LLMs. If models act on factors they don’t disclose, CoT monitoring alone is insufficient.

Does CF’s approach with IGC charts and decisive binary evaluations of sub-goals help with unverbalized biases? Why or why not? (I mean for human thinking in general, not for LLMs.)

Jarrod · February 13, 2026, 9:59pm

Thank you for sharing this! I think it’s a great example of non-CF thinking.

I think CF’s approach with IGC charts and decisive binary evaluations of subgoals would help to eliminate unverbalized biases. The reason why is because it exposes/makes explicit one’s reasons for rejecting stuff. If one were to use an IGC chart (for this Hindu/Christian debt ratio example), there’d be a religion column in the IGC chart (e.g., Applicant is not Christian—yes or no?). But making that goal explicit would enable people to recognize it’s prejudiced (and illegal).

With an IGC chart, maybe they’d just have subgoals like:

Debt ratio is ≤40%
Credit score is ≥700
Etc.

Also, one could make it more complex by including other factors like age and having different combo goals (e.g., allowing a higher debt ratio for more mature people). (I’m not sure how this stuff is actually evaluated IRL tho.)

Elliot · February 14, 2026, 4:59pm

You could put “debt ratio < X” on one IGC chart (for a christian) and “debt ratio < Y” on another (for a hindu) and don’t put religion on the chart. And if you don’t compare the two charts, you could get away with the bias. How is this different than the original non-CF version where the bias is visible if you compare the two decisions, but looking at each case individually it’s hard to see the bias? In general, a person asking for a loan doesn’t get to see other people’s loan decisions to compare to.

Also with MCDM arithmetic, if you make a list of factors, weight and score them, and add them up, then if religion isn’t a factor and debt ratio gets different weightings for different people, then the bias is visible on comparison but harder to see when looking at a single decision. So you could try to compare with that approach too.

Jarrod · February 15, 2026, 3:28am

I was assuming that the bank would have one big IGC chart that they’d use for all loan applications. Making IGC charts from scratch on an ad hoc basis (and without consistent principles/breakpoints) could allow for bias. I think having a principled/universalizable approach (i.e., using the same rules/breakpoints/IGC chart for everyone) might fix that.

Well, in terms of outcome, I don’t think there’s any difference. I’m not sure CF can stop someone who is committed to being irrational/biased.

But for someone who is trying to be rational, CF can be help because:

Weighing/scoring is done more on the basis of intuitions (which can be biased) because there isn’t any rational way to actually decide exactly how much a certain factor weighs or should be scored.
- Also, the bias could be spread out across several factors. Because weighing/scoring is analog, small differences across several factors could result in an overall different outcome—which could be done (even unintentionally) in the service of a bias—i.e., setting the weights/scores such that it’ll get one the outcome that one already wants/feels is best. So weighing/scoring in this case would just be a sort of post hoc rationalization.
Whereas CF involves breakpoints which (I think) require explanation(s). The work of explaining one’s breakpoint could expose or neuter subconscious biases.
- Also, breakpoints are further apart (i.e., distinguish between qualitatively different categories). So one can’t make a small tweak to a bunch of factors. Making a big change (choosing a different breakpoint) might be more noticeable and hence expose biases (or discourage making such a change).
  - I’d guess that making such a change would also require an explanation of why one changed the breakpoint and why that new breakpoint makes sense or what was wrong with the old one (tho idk, I guess I need to study/practice breakpoints more—I’m not that familiar with them).
Just creating one big IGC chart for deciding all applications might fix it. (I think? Maybe lending is too complex for that. Idk.) I’d guess that a rational bank shouldn’t just make up a new IGC chart (with new breakpoints) from scratch for every applicant.

Maybe in an ideal world banks could make their decision criteria (their one big IGC chart) public. And have Paths Forward. (Tho idk exactly how lending decisions work/are made.)

I tried to compare above (see bullets). Is there anything I missed or more that I should compare or consider?

I think the main thing is that having to explain one’s breakpoints might expose or inhibit bias even without comparison to other decisions. Also, one could deliberately consider how one’s proposed breakpoint(s) would cause a bunch of other hypothetical situations to be decided. (Cf. thinking in principles, global optima, universalizing, etc.)

Elliot · February 15, 2026, 4:46pm

Why? The scenario involved them not using the same approach to debt ratios for all loan applications.

Also it’s hard to make a one size fits all IGC chart. That’s sort of like having a single shopping IGC chart that you use for buying cars, food, computers, anything. Different loan applicants have qualitatively different situations.

And if you use a one-size-fits-all approach it could end up, on average, favoring people who are more normal/conventional/conformist. It could be unfair to outliers and minorities. It could e.g. favor salaried employees over hourly employees over self-employed people.

Can you give examples of what you’d put on this IGC chart that you think would realistically work well and cover all loan scenarios? In general it’s easier to make up individualized IGC charts as you face individual scenarios and see what’s relevant in the current one scenario you’re dealing with. I didn’t ask people to reuse the identical IGC chart each time they’re deciding what to eat. While they will certainly reuse some factors, using the exact chart you created months ago may not work well: that means you can’t add any new factors that seem relevant this time and also can’t adjust any factors that don’t apply in the standard way this time.

Reusing identical MCDM factors and weightings can be easier but I think that’s because the factors are often vague things like “quality” which leave you with a lot of flexibility for how to score things to get a conclusion you think is reasonable. If the factors were really specific and objective, then reusing them with zero changes might be at least as hard as reusing the same IGC chart.

Do you expect bank employees to think of and write out good explanations which are fully logical? The same employees who were being biased before using CF?

Jarrod · February 16, 2026, 10:45am

I think I was meaning as a solution to bias—and that one could have multiple approaches in one chart by using subgoals that accept X or Y or Z, like:

W2 income > 2x the loan payment OR self-employed & income (avg of last 24 months) > 4x the loan payment OR retired but total assets > 10x the loan value OR [etc]
Credit score > 700 OR clean rental/utility history > 5 yrs & no bankruptcies OR [etc]
etc

I found that shopping IGC chart example clarifying (since shopping is more intuitive to me than lending). E.g., I guess it’d be hard to have the same IGC chart for purchasing a book and a vacation. The budget is different (<~$30 for a book vs <~$15k for a vacation). Also other subgoals are qualitatively different (e.g., a subgoal for a vacation could be that it’s snowy so I can go skiing, and a subgoal for a book could be that it’s available in audiobook format so I can listen while doing other stuff). It’d be hard and unnecessary to try to put these sorts of subgoals on one giant IGC chart used to evaluate all purchasing decisions. And I guess the same might be true of lending.

Good point.

I asked AI for examples:

The Sub-goal: “Demonstrates stable employment.”

The Standard Breakpoint: “Employed by the same company for >2 years.”

The Conventional Case: A mid-level manager at a paper company has been there for 5 years. Result: Pass.

The Outlier Case: A world-class software consultant changes “employers” (clients) every 6 months but makes $300k/year.

The Sub-goal: “Easy to verify monthly cash flow.”

The Standard Breakpoint: “Possesses monthly W2 pay stubs.”

The Conventional Case: An hourly worker at a grocery store has these stubs. Result: Pass.

The Outlier Case: A small business owner who takes a $0 salary but owns $2 million in equity and draws cash only when needed.

Someone might have a low credit score due to medical bankruptcy but now has stable income.

Another example: Someone who wants to borrow against their volatile crypto holdings.

Lending is way more complex than I realized.

So a one-size-fits-all chart could be wildly unwieldy and could be biased against unconventional people. But doing ad hoc IGC charts can also lead to bias. Given that, it seems like bias is super hard to fix. Hmm.

I’ve now realized that lending is way more complex than I realized (I guess that’s why banks have loan officers rather than just an algorithm).

Yeah, makes sense. I guess I was thinking of a band-aid solution to bias. But now I see that it’d be hard to foresee every possible loan application. E.g., banks in the past couldn’t have foreseen people wanting to borrow against volatile crypto holdings.

Yeah, I think I remember seeing you say something about how one should—instead of spending most of one’s effort on brainstorming solutions—spend 50% of one’s effort on defining one’s goals.

Hmm, now you say it like that, I suppose not.

Going back to your original question:

Do you have an answer to that question?

Elliot · February 20, 2026, 4:43pm

FYI, to me, these numbers sound potentially biased against self-employed people.

Yes bias is hard. CF can help some but isn’t going to just end bias.

CF asks people to have explicit reasons for saying “no” to things. “That other one is better” doesn’t count in general. “Your score over many factors wasn’t high enough” doesn’t count. You should say a goal this one fails at. You should identify some kind of error/failure, not alleged inadequate overall goodness. This helps with bias some (both the explicitness and the identifying specific failures and looking at qualitative issues instead of trying to use indecisive quantitative thinking).