CF Decision Making

I wrote this yesterday and today. It’s reasonably typical of what I’ve been writing lately (I’ve been averaging over 2k words/day) except it’s less organized than average (which is part of why I shared it – I’m less likely to edit it into a final article). I’d like feedback on the ideas/content (not typos or writing issues) – questions, comments, thoughts, stuff you understand, stuff you don’t understand, stuff you think is useful, stuff you don’t see the point of, comments related to trying to learn the ideas, etc.

Most decisions are pretty easy, so the focus of advice is generally on harder and more complicated decisions. You make thousands of decisions per day. Many people don’t even recognize 90%+ of those as decisions. That’s because they’ve automated most of their decision making. Which is good. But those things are still decisions.

You should automate most of your life. Then you pay conscious attention to a few things (e.g. a new thing you’re learning) and also consciously monitor your automated life. You can sometimes find problems with your automatic stuff and make changes. But most of the time, most of it works OK.

Automation means you made decisions in the past and you’re reusing them without redoing all that decision making work. If a decision is the same or similar, it shouldn’t require as much attention the 10th time as the 1st time.

For most decisions, the aim is a “good enough” choice. It doesn’t need to be perfect. You don’t need to optimize it. Trying to optimize it would actually be inefficient, because you’d get a slightly better outcome but the optimization would take time and effort. That same time and effort would provide more benefit if you used it on something else. You shouldn’t spend your attention trying to get tiny wins. There is a major exception which is mass production. If something will be used by millions of people, then tiny wins are worthwhile because they’ll provide a small benefit millions of times. So it’s worth polishing and optimizing the design of a product that will be mass produced in a factory or the text of a book that will have many readers. Similarly, if you will reuse a skill a lot – e.g. typing – then it’s worth optimizing how you do it.

For most decisions, you’re trying to come up with a solution/answer/plan that meets a pretty low bar that’s easy to meet. The goal isn’t very hard for you to succeed at. So you could succeed in lots of ways and any solution is fine, and it’d be bad to compare the solutions much and try to figure out which is best. Like if you’re picking something for dinner, there are lots of good answers that will feed you, be pleasant to eat, not poison you, and stay within your budget.

What about hard, complex, important decisions where you actually want to compare solutions and figure out the most optimal one? Decisions where many factors are important and it’s worth putting effort into deciding.

CF’s method goes like this.

First, consider your goal/purpose. What do you want your idea/choice/solution/plan to accomplish? This is your big picture or overall goal. You may not know it in detail but you’ll at least need a rough idea. You can’t pursue a goal, and try to succeed, without having a decent idea of what it is.

Break the goal into parts/sub-goals. You can brainstorm these. Don’t try to know exactly what you want – like don’t try to decide on a specific, exact goal in this phase. You have a general idea of what you want, but there is some flexibility or uncertainty. So you can come up with sub-goals that might be part of your goal but might not. Including extras is OK. Some can contradict each other and that’s OK too.

The point here is to split up the overall goal into individual factors. We’re going to evaluate solutions for factors separately. Evaluating an idea for all factors at once is hard, but doing it for an individual factor is much easier. For example, if you’re choosing a pet you could use cuteness, price, species and messiness as factors. Each of those is pretty separate and autonomous, and can be evaluated on its own in isolation, and it’s basically just one thing instead of several things at once.

Write the sub-goals along the top of a chart/grid (like a spreadsheet).

Then brainstorm ideas/solutions/plans/choices. Each one should be good for at least one sub-goal and sound like it could work for your overall goal. You want to come up with things that will work well for many sub-goals or which can be combined. It’s useful to come up with a way to get one sub-goal that can be easily added on to another plan. But if a plan is comprehensive, high effort and contradicts other plans, and it only works for one sub-goal, that that’s bad. Basically if it’s more like a single action or small number of actions, then it’s OK if it only helps for one or a small number of sub-goals. But if it’s more of an overall plan then it better work well for the overall goal.

It’s possible to only use sub-ideas – individual single parts – and then look at combinations later, like we’ll do with goals. That can work well sometimes, but it usually doesn’t work well.

Write the ideas along the left of your chart.

Next, evaluate ideas for sub-goals. Normally what people do is try to figure out, overall, how good an idea is. They’d brainstorm the left side of the chart and then try to judge, rank or score the ideas. They try to give a single evaluation per idea.

Instead of giving an idea one evaluation, we’re giving it one evaluation per sub-goal/factor. These evaluations are much easier to figure out because we broke the problem (the overall goal) down into parts, so we’re only trying to evaluate one thing (issue, factor, sub-goal) at a time.

There’s actually a really fundamental difference here. I can rank pets by which I think are cuter. And I can rank pets by price. But ranking pets by price and cuteness at the same time is much harder. There’s no straightforward way to do that. Combining factors, or taking into account multiple factors at the same time, runs into deep, hard problems. In short, it can only be done approximately, and the approximations are pretty good in special cases but pretty bad in general. One of the major design goals of CF’s decision making method is not needing to do that. It avoids those problems by finding a different approach.

So now we’ve got this chart. And I’ve actually skipped something important. How do you evaluate one idea for one factor? Do you give it a ranking compared to other ideas regarding the same factor? Do you give it a 0-100 score? Do you use words like “pretty good” or “great”? You could do any of those and this method so far would help some. Breaking a goal into parts and using a grid to evaluate factors separately is a useful method on its own. But it’s not my main point.

Before evaluating ideas, you should break the sub-goals/factors into two categories. You can do this before brainstorming the ideas too. The categories key sub-goals and secondary sub-goals. The key sub-goals are the things you really want to pay attention to and optimize. The secondary sub-goals are everything else. Sometimes I call them “low attention goals” (I’ve often called sub-goals “goals” because they are goals – the overall goal is the combination of multiple smaller goals, so we were breaking a goal down into multiple goals. Sub-goals are not a fundamentally different type of thing. Any one sub-goal could be someone’s goal if they didn’t care about the other stuff.).

Our chart is probably too complicated and overwhelming. If we have 20 ideas and 20 sub-goals, then there are 400 places in the chart where we could fill in an evaluation. This is one of the reasons people try to do one evaluation per idea. It’s less stuff to do. (It’s also because they think of “the goal” instead of recognizing flexibility and decision making about what specific goal to have. In general, I think people put 95% of their thought into ideas/solutions, but should allocate around 50% of their thought to getting the goal right and 50% to ideas/solutions. People should brainstorm and compare alternative goals, rather than only brainstorming and comparing alternative ideas/solutions. This leads naturally to a grid because you have to evaluate idea+goal pairs instead of judging how good an idea is at “the goal” on the assumption that there’s only a single goal under consideration.)

The main solution to this complexity is to deal with secondary sub-goals quickly and easily. If we have only 1-7 key sub-goals – and rarely over 3 – then we’ll have a lot less to worry about. We’ll evaluate ideas in detail for just a few key factors, and the rest will be done quickly.

For secondary factors, evaluations should be purely pass/fail. Is it good enough or not? You could also write “maybe” as a third alternative if something is near the borderline, so it’s hard to tell. Instead of actually putting in effort to figure it out, you could just write “maybe” or “undecided” or leave it blank for now, and only try to figure it out if it ends up mattering. Most stuff isn’t borderline and you can just decide on pass or fail really quickly and easily.

There are some other time saving approaches. One is that similar ideas will get similar evaluations. If you take an idea and modify it to a variant idea, you should start with all the same evaluations and then modify a few that changed, rather than starting from scratch. Using a software spreadsheet, you can copy the entire row, then modify the idea a little, and then go think about what factors that would change the result for and change those.

Another big time saver is if you write one “fail” you may be able to stop evaluating the other boxes. It depends though. Some factors are optional – maybe they will end up being part of your goal and maybe they won’t. Don’t stop due to failing at one of those factors. But some factors are mandatory. Like you don’t want to be seriously injured. Whatever final goal you settle on, that’ll be part of it. So if an idea fails at that, you can probably just stop evaluating it further. (And if you find no solutions that avoid a risk of serious injury, then you can go back and reconsider the ones you gave up on earlier. Abandoning an idea doesn’t have to be permanent. So you can abandon them fairly aggressively and then relax your standards, and consider less promising options, if you start getting stuck and it seems like you won’t be able to get a great solution.)

Pass/fail evaluations are the simplest evaluations. There are only two options. You literally can’t get simpler than that. If there was only one option you wouldn’t be evaluating anything because there’d be no choice to make and everything would get the same evaluation and it’d be meaningless. Pass/fail is the minimum that lets some things do better than others.

You can’t and shouldn’t try to optimize everything. Most stuff is of lesser or secondary important or is easy. We only need to optimize stuff that’s both hard and important. If it’s unimportant, then it just needs to be decent and we can move on. If it’s easy, then we just need something that works and we can move on without worrying about it more.

We need to focus our attention on key factors – factors that are both important and kinda hard so there’s more room for success or failure, or better or worse outcomes.

We can only optimize a few things at once whether we like it or not. Focusing on a few key factors isn’t optional. (If something is a really big deal, like building a skyscraper, you can have different people work on different parts of it. That lets more factors get more attention and optimization. My goal here isn’t to explain that kind of collaboration, but basically each person or team focuses on just a few key factors for their area of responsibility, so most people’s experience is similar to individual decision making. There’s also a leader or leadership team that tries to look at the big picture, coordinate the teams, make sure their separate work is compatible, assign responsibilities to different teams, etc.)

For decision making in general, the most common case is zero key factors. That goes back to what I was saying above about you making thousands of decisions per day without much trouble. Most stuff is easy.

For harder decision making that gets conscious attention, the typical and ideal number of key factors is one. You can rank options by how good they are on that one key factor, and also rule out any option that fails a secondary sub-goal, and then make your decision. This doesn’t run into the difficulties of trying to rank by two factors, like pet price and cuteness, at the same time. Why? Because, unlike other factors, pass/fail factors are easy to combine. You just demand only passes and reject anything that fails. That’s simple and easy.

You have to decide on your final goal at some point though. It’ll be a combination of sub-goals. But it may not include every sub-goal. If an idea fails at a sub-goal which you don’t end up including in your final goal, that’s fine. Only the evaluations for the included sub-goals matter.

How do you decide which sub-goals to include in your final goal? Think about what you want. Think about which sub-goals are important to your bigger picture goal and why, and also which are important to your life. Not getting injured may be off-topic to your goal about gardening, but it’s still important. Some sub-goals are background context from your life.

And think about which sub-goals you can meet. How ambitious can you be and succeed? You can usually tailor your goals to what’s actually achievable. Sometimes some aspect of nature or society, or something else outside your control, pushes certain goals on you. But overall you generally have a lot of flexibility and choice about what goals to have. Life is like a sandbox video game where you can decide what to do and what you care about instead of one of the more common video games where the designers decided on the goals for the players (e.g. killing the monsters, solving the puzzles, or jumping over the pits and getting to the end of the level).

You can often leave sub-goals out of your chart when they’re part of the background context of your life. You’re so used to them that you won’t even brainstorm ideas that violate those sub-goals. You’d automatically notice if an idea was going to screw your life up and could add it to the chart at that point so you could give that idea a fail for an explicit, written reason. But dealing with many goals in your life is automatic enough you don’t even need to write about them, so that also helps keep the chart size down.

Related to background context: we’ve been talking about a two-dimensional chart where the dimensions are ideas and (sub-)goals. In other words, you evaluate (idea,goal) pairs. Decision making is actually three dimensional. The third dimension is the context. You really evaluate (idea,goal,context) triples. The same idea can work for a goal in one context and fail at the same goal in a different context. Usually we do evaluations for a single context so this isn’t a problem. Sometimes we might want to evaluate ideas for multiple contexts, e.g. for several hypothetical scenarios. In that case, you can make one chart per context. That’s more work. But if the contexts are similar, then most stuff will stay the same. So you could copy an entire spreadsheet and then just modify what’s different for the second, variant context, and also color the parts that changed to highlight the differences.

Above I criticized people for trying to evaluate ideas as the one single dimension, with the assumption there is a single goal (“the goal”). They also frequently assume a single context (“the context”, which is just your current situation/life – the present day). I agree that the single context assumption is often fine. We often don’t need to worry about that third dimension of different contexts. But we do need to routinely worry about the second dimension of different goals. Taking a single context for granted causes trouble sometimes but it’s usually OK; taking a single goal for granted is OK sometimes but is frequently problematic; it’s a common error.

What do you do if there’s more than one key factor? How can you handle that given the problems of combining factors into an overall evaluation? Key factors can be evaluated in a pass/fail way too. You can do that with all factors (though all but one is adequate). It’s simpler to see how to do this with the secondary factors – and we needed to differentiate secondary factors anyway so we can focus our attention on just a few key issues – but pass/fail judgments can be done with all factors.

The basic strategy is to consider what is good enough? For example, is this pet cute enough to satisfy me? Is it cheap enough to fit in my budget? Those factors can be dealt with using pass/fail.

The more complicated way to approach it is to consider types of factors. Some factors are already binary (e.g. they’re phrased as a yes or no question), some factors involve discrete categories (binary is one example of that), and some factors involve quantities or degrees. Basically factors either deal with separate categories or with a spectrum.

If a factor deals with categories, you need to figure out which categories are the good ones and which are bad. The (sub-)goal needs to define success and failure. Which categories constitute success and which don’t? So you can evaluate the factor by figuring out which category it’s in. But that also leads to a pass/fail evaluation. For example, pet species is discrete categories. Chicken, dog, and turtle are not different points on a spectrum. They’re separate things with no obvious single, inherent ranking (you can pick a factor, e.g. average weight, and then rank them). You can evaluate pets by figuring out which category they’re in (e.g. this one is a cat). And a sub-goal about species can say which species of animals you like. E.g. dogs and cats pass and other species fail. You can define multiple different sub-goals about species, e.g. one includes snakes as success too and one doesn’t. Each idea/option (different animals for sale at the pet store) could then be evaluated for each sub-goal. Then for your final goal, you’d only use one species sub-goal. You’ll have to, at some point, decide. But you can write out multiple options for sub-goals, evaluate them, see how different options do for them, etc., which can help with that decision. That’s part of what figuring out your goal is. Just like you’d consider variants of the same idea and try to figure out which one is better, you can also consider variants of the same sub-goal.

If a factor deals with a spectrum (aka continuum) – quantities, degrees or amounts – then converting it to pass/fail involves breakpoints. A breakpoint is a point on a spectrum where there is a qualitative difference. Every point on a spectrum has a quantitative difference compared to its neighbors. E.g. the point “5” is quantitatively different than “4” or “6” – it’s one more or less than those (the amount, aka quantity, is different). But most points are not qualitatively different than their neighbors. E.g. 4, 5 and 6 might all be small – they’re all functionally similar. A qualitative difference is one where something important changes. It’s where there’s a change of category, type or quality. It’s where there’s an explanation of why there’s a kind of boundary there so something other than the quantity changes when that boundary is crossed. For example, a breakpoint for furniture is whether it fits through my door. If it’s too big to get into the house, that’s qualitatively different than something that I can get into my house. Breakpoint’s like that are very rare compared to non-breakpoints. For almost all small changes in size, you would not cross a breakpoint – e.g. it’d keep fitting or not fitting through my door.

You can find inherent breakpoints that matter to the subject matter or you can generally consider “good enough?”. You can also breakpoint on maximization (the breakpoint is between “most” and “not most”). And if all else fails, you can add arbitrary breakpoints just to have fewer things to think about. You can’t think about every value from 0-100 individually because it’s too much stuff. You can’t meaningfully differentiate them all in your mind. Even 0-10 is a bit much. How much stuff can you meaningfully differentiate at once? Roughly seven things. So if you’re dealing with 0-100 and really stuck on categorizing it (finding separate categories is the same issue as finding breakpoints – breakpoints are the spots where a category change) then just add breakpoints evenly spaced out. In other words, count by 20’s. In other words, round everything to the nearest 20. So there’s the 0 category (0-10), the 20 category (10-30), the 40 category (30-50), etc. This gets it down to 6 categories which you may be able to actually talk and think about and tell the difference between. Then you can judge which categories succeed and which fail. You’ll quickly realize e.g. that 80 and 100 succeed, and 20 and 0 fail, and so the breakpoint for good enough somewhere around 40 or 60 (it’s in the 30-70 range). You can start narrowing it down because you can see a clear difference between the 20 category and the 80 category. You can also narrow down without rounding things into categories. You can just ask like “is 50 definitely enough and not even a close call?” and if the answer is “no” you try 75 (binary search is efficient but you don’t need to know the ideal search strategy – this will work fine even with some rough guesses for what numbers to try – plus we’re just trying to just find the rough region of the breakpoint not hone in on an exact solution).

Way more details could be given about breakpoints and qualitative differences but let’s move on and stick with more of and overview. The point is that all factors can be converted to pass/fail. You can do this by breakpointing the quantitative factors (spectrums) to turn them into discrete categories. Then for category factors (naturally or converted) you define which categories succeed or fail in the goal, and then you evaluate ideas pass/fail by whether they’re in a success category or not.

What if some categories are better than others but both succeed? So we have categories A, B and C. A is failure. B and C both succeed but C is superior to B.

Define two separate goals, a more ambitious one that only counts C as success, and a less ambitious one that counts both B and C as success. We’ve now differentiated our ideas. Some succeed at both of these sub-goals and some at only one. Look for solutions that succeed at more and better sub-goals.

The point here is that, when we choose one idea over another, it should never be because it’s “better” in some vague, amorphous way. What does better mean? How should we be comparing ideas? One idea is better than another because it succeeds at a specific sub-goal, that you care about, which the other fails at. You should be able to identify that sub-goal and use it as the reason you’re picking one thing over another. This is a way of judging one idea as better than another that’s clear and avoids the problems of combining multiple factors into a single ranking.

If many ideas succeed at your sub-goals, then you have two choices. The typical response is to say: awesome, any will work. They all meet my goals. They’re all satisfactory. I don’t need to figure out which is “better” since they all succeed, at in terms of my goals they are all exactly equal.

The other choice is to come up with some more ambitious sub-goal to differentiate them. You can think of a sub-goal you’d like that some of the ideas succeed at and others fail at. This differentiates them and says exactly in what sense the winners are “better”. They don’t “succeed more at my goals” They succeed where other ideas fail – at this specific sub-goal.

There are lots of bad decision making systems where people basically end up saying “this one seems more good” or “this one has a higher score”. They don’t clearly know why it’s winning. “This one succeeds at this sub-goal” is a clear differentiator that lets you know why it’s better.

So don’t evaluate ideas with a single 0-1 credence each that’s a matter of degree for how good they are. Instead, evaluate them with many pass/fail (0 or 1) judgments. We can differentiate ideas in a binary system by giving them multiple evaluations. The error was trying to do a single evaluation per idea. Instead of a single evaluation that tries to capture too much information and can’t do the job (because goodness is not actually a single thing), we need to have more than one evaluation per idea. We need the second dimension of sub-goals. We need a 2-dimensional chart instead of just a list of ideas with their scores. This gets us what we want (nuanced differentiation between how good ideas are) without the major problems that degrees of belief, credences or quantitative/analog thinking in general run into.

Instead of trying to think about how great things are, we should be looking for dealbreakers. Thinking critically means trying to figure out what we can rule out. It means looking for errors, flaws, problems and failure. Once we rule enough out, the remainder is good. That’s the Critical Rationalist approach, instead of the justificationist approach of trying to support our ideas with how great they are, and often scores representing their amount of goodness (though many people shy away from numbers, the fundamental issues are the same with or without numbers).

One of the big ideas here is to have clear goals. Most goals are kinda ambiguous. The main reason is you only get a single goal (as a kinda default assumption in how people think) so all flexibility and complexity has to be built into it. Instead, we should define many specific, clear goals that unambiguously define what is success and what is failure. Instead of being like “I’m not sure exactly how much is good enough” you should just define several goals with different breakpoints for how much might be good enough and consider each of them. Instead of “I’d like X but I don’t know if I can get it” just define two separate goals – one that includes X and one that doesn’t. Then evaluate options for each of those two goals instead of evaluating them vaguely regarding one ambiguous goal. Instead of “I don’t know what I want” brainstorm specific things you might want and use those as sub-goals. Writing down possible sub-goals and evaluating candidate ideas for them will help you figure out what you want. Instead of “I want more of this – more is greater success” define a clear goal like “I want the maximum of this. So I should choose the option with the most. Any option with less than the maximum is a failure.” Or if that’s not what you want, you should know it and be clear (actually wanting to maximize is uncommon – people think in terms of maximizing more than they should). What you really want might be to maximize up to a point which is enough, and also not go over a certain amount that is too much. That is a clear goal that lets you evaluate every idea on a candidate list as either success or failure. If nothing is in the ideal range, then the highest one below that is the only one that succeeds. If one are more in the ideal range then they succeed and anything below fails.

What if you want as much as possible up to a point (first breakpoint), at which point more is still better but it’s not such a high priority? (And maybe there’s a second breakpoint for too much.) What does that mean exactly? If more is better but it’s not a high priority, that sounds like everything else being equal more is better, so (looking at stuff that reached the first breakpoint) anything that ties on all other factors but offers less should be considered a failure. Alternatively, you might decide this factor is not important past the first breakpoint and should not be prioritized – just call that a “good enough?” cutoff, pass anything above it, and focus your attention elsewhere. Basically what you need to do here is make a choice – do you actually care or not? Is it a key factor or not? You can’t optimize everything and shouldn’t try. If it’s not so important, don’t worry about it. If it’s of minor importance but the differences are really big and clear, then maybe that is important after all because a huge amount of a minor thing can be important. You can define categories like maximize up to here, then if you can get up to this next breakpoint that’s so much more that I’ll actually care, but everything between the first and second breakpoints is equal.

Figuring out your goals with this kind of clarity and specificity, and defining what differences actually matter, is much better than having one vague goal and trying to figure out a single complex evaluation of each idea. Here, instead, we have simple evaluations of idea+goal pairs instead of complex evaluations of ideas for a vague goal. It’s the clarity – well-defined, unambiguous goals – that enables the simplicity. (You also need well-defined, unambiguous ideas. If in doubt, specify multiple ideas, including similar ones. Actually defining your ideas/plans/solutions is much more common. People screw it up sometimes but people also get it right; doing it right isn’t unusual. Whereas doing clear sub-goals that enable pass/fail grading is actually unusual.)

Once you write out your options – both ideas and sub-goals – in a chart it’s much easier to decide what you want. You can see that some things are unattainable, others are reasonably attainable so anything that fails at it can be excluded, and what hard choices you might have to make (you can get subgoal A or B but not both). You can also brainstorm more and try to add to the chart if you aren’t satisfied.

If you have to decide between subgoal A and B and it’s hard, what can you do? Look for decisive criticism. Look for failure, error, unacceptability. If it’s fine either way just do what you want – both are OK, you can’t really go wrong, it’s (to an adequate approximation) a matter of taste or preference. If one has some kinda major flaw then don’t do it. If they both have major flaws you need to reconsider something.

Why does this work well? One reason is because criticism and error correction are the key to thinking, not positive arguments and support. And because error correction is fundamentally digital not analog. And there are mathematical reasons about combining factors. And another reason is because binary factors work better because they’re simpler and easier to deal with and also combining non-binary factors from different dimensions is hugely problematic (basically it’s broken and doesn’t work, and people can only make it approximately work sometimes – the more narrow the context and specific and parochial the thinking, then the better approximation you can get, but the more you try to use broad principles and do thinking with wide, general-purpose applicability the more it fully doesn’t work).

And also, good systems have lots of resilience and margin for error. That’s why “Good enough?” is a great question to ask. For most stuff, more isn’t better. It has enough plus a margin for error. When more is better, there is no such thing as a margin of error. You can’t have extra – a safety buffer – when you’re maximizing. But the only way to deal with many factors in a world with variance, statistical fluctuations, mistakes, etc., is to have most of them be reliable. Otherwise stuff will always be going wrong. How do you make factors reliable? Have a margin of error. Have excess capacity. Don’t have them near the border between success and failure. But that means binary factories or at least discrete categories. The only way to have a margin of error away from a breakpoint is to be dealing with breakpoints not continuums. And the only way to have a margin of error for success not failure is to pick a specific breakpoint that differentiates success and failure.

Basically, it’s only by using binary pass/fail that you can have margins of error aka safety buffers aka excess capacity. There has to be a good enough point for it to be possible to have extra or spare stuff. And it’s only by having extra that things can be reliable or resilient. In short, you need extra on all secondary factors, which is why they can be secondary issues that don’t take much attention. Key factors are the ones that don’t have enough margin for error, so they take your attention because you have to watch out for errors yourself.

A system with many factors with low or no margins of error is unstable. Random fluctuations and mistakes will always be causing failures. So it won’t work. You may object that with a maximization goal or “more is better” attitude there can be less and the system still works. A negative fluctuation won’t necessarily cause failure. But when you say that, you are admitting that you know what failure is, that failure is a point you’re above, and that you’re far enough above it not to be too worried. By thinking of any amount as failure and any other amount as not failure, you’re doing binary thinking that differentiates success and failure. Real more-is-better maximization only makes sense when anything less than the max is failure – there is no failure point, failure is defined as not maximizing. And basically if you try to maximize two factors that have any tradeoffs then you must fail – those goals are in conflict. And pretty much any factors have tradeoffs b/c you could put more time/money/attention into either one, and spending those resources on one means less resources available for the other, so you are not in factor maximizing B if you spend any resources at all on A above the minimum to not fail (you can have one maximization factor and many binary factors, but two maximization factors both demand all resources beyond avoiding failure, which is contradictory).

i had a new idea while writing this. the new idea is that only with binary success/fail factors can you have a margin of error or buffer

you must define a good enough point – success breakpoint – to have more than that

if you have some other type of factor, e.g. maximization, and deny a success/failure distinction, then you cannot have extra, and therefore have no margin for error

and the only way to manage many factors is to have margins of error. otherwise variance will constantly fuck you

the point is you need my binary perspective for margins of error to make sense. that concept presumes a binary success/failure viewpoint.

Discussions, debates and thinking about things yourself involve two things arguments and non-arguments.

A non-argument gives useful information, e.g. facts, evidence, ideas, goals, answers to questions, or explanations of how things work.

An argument is a decisive criticism. It says why an idea/plan/solution fails at a purpose/goal/problem. Decisive means it contradicts the thing it criticizes, so they’re incompatible: at least one must be wrong (or part of the background context is wrong – e.g. if you misunderstood logic you could be mistaken that they contradict).

If an argument is not a decisive criticism, then either it can be converted to an equivalent or similar decisive criticism, or it’s mistaken.

In other words, all good arguments say why ideas fail at purposes. If you can’t point out a problem with an idea in terms of failure at some goal, you don’t have a criticism of it.

Partial arguments – arguments that something is good or bad by degree, rather than that it fails or succeeds – misunderstand the hard distinction between success and failure at specific goals. They’re frequently attempts to evaluate an idea for multiple goals at once and blend the answer. Or they’re attempts to have an ambiguous goal which doesn’t clearly distinguish between success and failure (which again is most commonly related to trying to deal with multiple goals at the same time and blending things together).

Positive arguments – arguments in favor of ideas instead of negative arguments that criticize ideas – misunderstand that we must find and correct errors. Positive arguments can never prove anything is correct because there could still be an error. You can’t rule out errors; you can only find errors or look in some places and fail to find errors there (but there are unlimited remaining places that could still have an error). Positive arguments don’t contradict anything so they don’t actually do anything. Decisive, negative arguments contradict something. Contradictions are useful. When things contradict, something is wrong. Pointing out a contradiction means there’s an error there. You still have to figure out which side of the contradiction is wrong but you’ve massively narrowed down the location of an error.

Many positive arguments are pretty reasonable. That’s because they’re equivalent to negative arguments. “X is good because Y” is just the argument “Alternatives are bad because they lack Y”. The negative version is better because it reminds you to look at alternatives. Maybe the alternatives also have Y, in which case it was a bad arguments. Maybe some do and some don’t. You’ve gotta check the alternatives. Positive arguments for X can’t prove X is true. We need to compare X to alternative ideas we know of, and the negative form of the argument guides us to do that. What negative arguments really compare though is ideas to their purpose – does it work at it’s goal? The other issue with “X is good because Y” is that Y might be unnecessary to your goal. Y could be a local optima, a non-constraint, something you already have more than enough of, something irrelevant, etc. You can’t have positive arguments of the form “X will definitely succeed at its goal because…” That’s seeking infallibilist proof. Nor can you have “X will probably succeed at its goal because…” No list of the merits of X can ever mean it’s unlikely that there’s a reason it won’t work. Good traits don’t rule out failure. You can have negative arguments like “X will fail at its goal because…” Basically if one important/relevant thing goes wrong causes failure, but if any list of things go right that can’t prevent or rule out failure. So negative arguments are effective but positive arguments aren’t.

The best status an idea can have is “I can’t find any reason it won’t work; I have no criticism.” Nothing but trying to find reasons it won’t work can tell you it’s likely to work.

If you look for positive merits and don’t consider failure, you don’t know if it’ll fail. If you look for failure cases and find none, now you’re getting somewhere. That’s the most effective (and only effective) thing you can do.

Not having criticism requires trying. If you don’t try to think of criticism, it doesn’t mean anything. If you try and fail – and it was a real try not a biased fake try – then that’s the best you can do.

There are always choices about how much time and effort to put into looking for criticism. In general, if it’s more complicated or important then it needs more time. Most stuff can be quick.

I divide criticism into two basic categories. There’s criticism you already know and new criticism. For new criticism, 30 seconds of brainstorming is a fair amount. Important ideas can get more than that, but most ideas can be 30 seconds or less. Some really important ideas can get an actual science experiment or prototype – some stuff beyond just thinking.

For criticism you already know, mostly you access this through intuition and memory. It should be mostly automatic. You learned about certain kinds of errors in the past. You look at this new thing and see if it seems similar to a past error you know about. If it does seem like there might be a problem, it often takes extra time to remember the issue better and consciously figure out what the criticism is. Sometimes you have to look stuff up to double check details in your notes, in a book you’ve read, with an internet search, etc. But the initial check about whether your existing knowledge indicates you suspect an error, or not, is generally really fast (under 5 seconds – the majority of the time, you hear an idea and know your opinion right away. Then after 30 seconds of conscious critical thought, you might change your mind away from your initial opinion, but usually don’t.).

So if you do this – 5 seconds of intuition and 30 seconds of thought – now you’ve critically considered an idea enough for the majority of ideas. Most ideas aren’t that important and shouldn’t be focused on. Actually, most ideas are so minor they don’t even get focused, conscious thought at all, and aren’t even being counted in these numbers. Of the ideas you notice as something to critically consider, 30 seconds of serious thought is enough for more than half of them.

Eliezer Yudkowsky says to spend five minutes by the clock considering stuff before giving up. Use an actual clock so you don’t trick yourself into thinking you considered it when you didn’t really. This is a good amount for many ideas that are notable and important enough that you think you ought to consider them. Five minutes of actually thinking about something and doing nothing else, as measured by a clock and not your intuition, is actually a significant amount. People often give up on stuff within 10 seconds of conscious effort. Often they give up with zero seconds of conscious effort – they consider stuff in an intuitive way using their automatic existing knowledge and that’s it. They don’t even try to brainstorm any new insight. Or they start to do conscious effort and don’t like it in the first few seconds before they actually get anywhere, so then they stop. That dislike usually has nothing to do with the issue they were trying to think about – it’s about their pre-existing bad experiences with rationality, life, thinking, criticism, problem solving, work, effort, etc. (E.g. for various reasons people dislike thinking or expending mental effort. They’re also scared of failure, and often find it a lot worse to try and fail than to not try and not clearly know whether they could have done it if they had actually tried.)

Anyway, you can find errors in ideas. You can’t find successes in ideas. “It will work” is not a thing you can look for and discover. Something you can and should have is an explanation of how and why an idea will work, what’s good about it. why it’s designed the way it is, what mental model was used to imagine how it’d work and see it working, what relevant evidence exists, etc. Those things are not positive arguments. They are not arguments. They are explanations, reasoning and information. If you don’t have those, and should, then that’s a flaw – you can criticize e.g. “there is no plan for how this would even work” or “this is the kind of thing where we should consider some evidence, but this idea hasn’t brought up any evidence”.

We have knowledge and experience regarding some of the positive attributes ideas need in order to work rather than not work. Checking if they’re missing something important can be confused with positive thinking/arguments, but it is a type of looking for errors.

Another confusion is people think a criticism has to prove something won’t work. But saying “That is arbitrary and I have no reason to expect it to work” is a criticism. If some plan is half-baked, ambiguous or incomplete, you can’t prove it’ll fail but you can reject it anyway. Plans should not only succeed but give people a reasonable understanding in advance instead of just hoping to get lucky. If a plan is poor but might get lucky, that is not good enough to meet your goal (usually – it could be if you were really desperate) so it should be criticized.

Ideas have positive traits but the arguments are negative. Why? Because you can’t evaluate or compare ideas just by listing positive traits they have. You must look for positive traits that are missing or compare positive traits between alternative ideas to find where one idea is missing something another has. If you don’t do that, you have no idea whether there are errors. If you don’t look for problems, you have no information about whether there are any problems. You can never say it’s unlikely there are any flaws if you haven’t looked for them. And if you have looked for errors, there’s nothing else to do. The process of looking for errors considers positive traits indirectly because if you know a positive trait should be present, and it’s not, that’s an error to criticize. And looking at positive traits in a negative way makes sure they’re relevant – it’s only a criticism if lacking that trait means failure at the goal. The only way to know which positive traits matter is to figure out which ones, if missing, will cause failure. Just listing positive traits can’t figure that out. Only thinking about ways of failing can do that.

And you only need one error to reject an idea. How many positive traits do you need, and which ones, to accept an idea? There’s no good answer. Even with a billion positive traits an idea could fail due to also having one error. And even with a billion positive traits, an idea could be inferior to an alternative. As a logical matter, we can always list an infinite number of positive traits of any idea. Why? Because positive traits are not constrained in any clear way. They don’t have to ensure success at the goal. They just have to sound nice or good in some way. There are an infinite number of minor traits we can define and state that sound positive and identify e.g. excess capacity or local optima.

For example, if I’m considering buying a dog, I can point out that it has less than 999,995 fleas. It also has less than 999,994 fleas, 999,993 fleas, 999,992 fleas, and so on. It also has less than 999,996 fleas, 999,997 fleas, etc. I can raise the numbers up to infinity. That’s already infinite, but there’s more. And I can do this with ticks instead of fleas and make an infinite number of positive statements about this particular candidate dog lacking ticks. And I can comment on the absence of an infinite number of hypothetical types of fleas, e.g. fleas with 999,995 molecules, fleas with 999,995 currently living members of their species, fleas weighing 999,995 pounds, fleas with an average lifespan of 999,995 seconds, and so on. It can get really absurd, e.g.: “This dog has fewer than 999,995 fleas manufactured by an alien species with a 999,995 letter long name.”

Because logically there are infinitely many positive traits of any candidate we’re considering, we have to consider which ones matter. How do we do that? We must consider our goal. Can a positive trait ensure success? No. We can’t get guarantees of success. That’d be infallibilism. So we must look for errors. We must think critically. So we can make negative arguments, and we can figure out which positive traits we’d fail without and criticize options which are missing those traits. E.g. I might want to buy a dog with no fleas right now, so then the positive trait “has zero fleas” would be relevant for avoiding failure/criticism.

There are infinitely many negative traits (that are trivial to start listing many of) for any option too, if you include negatives that are compatible with goal success. If you don’t constrain a negative as “a thing that causes failure at the goal” then there are infinitely many. But if goal failure is the only negative then then it’s not trivial to list even one negative, let alone infinitely many. If you do find an error you might be able to state it in infinitely many ways, but it doesn’t particularly matter. The important differentiator is one or more known errors (things contradicting success), or zero known errors. The best we can do is use ideas we don’t know anything (important) wrong with (where “important” flaws are ones that cause failure instead of success at a goal you have).

We screw up a lot but at least we can try to find mistakes mentally before enacting them in real life, and we can avoid acting on stuff that, to the best of our knowledge, won’t work at our goal.

If you think an idea will fail to get result X, and your goal is X, then you should never ever ever ever use that idea. In your own view, it won’t work for what you want.

People get confused because they have other goals. You can still act on that same idea because you have goal Y. Whether to use an idea depends both on criticism of it and on your goal.

An idea can have known criticism for some goals and not others. In fact, all ideas fail at some goals. For any idea, you can easily come up with a goal it will fail at. What you always need, when acting/deciding (my conception of action includes mental actions like reaching a decision, coming to a conclusion, evaluating an idea, or accepting an idea), is to have a goal, have an idea, and not know any reason that idea will fail at that goal.

I think, perhaps incorrectly, that I learn more from this type of article than more organized stuff. If I’m right, perhaps it is because the lack of pre-existing organization causes me to self-organize the information in a way that better fits my existing knowledge.

I had to make a decision related to a car that seemed dificult. A few weeks ago while I was parked in a parking lot and while inside the building, a large tree limb cracked and fell across the hood and fender causing substantial damage to my car. Once they got the limb off, the car was drivable and mechanically fine. But the driver door doesn’t open correctly, parts no longer come together in a way that prevents water from going where it shouldn’t, parts of the front structure are bent, and maybe other problems would be evident at highway speeds (I didn’t need to drive on the highway to get home or other places I’ve taken it since). Owner of the parking lot said they’d only pay my insurance deductible because it’s an “act of God” and thus not their responsibility. I find that argument dubious, but haven’t investigated or asked a lawyer etc. so don’t know.

It’s a fairly small, old car. But it has low miles and has been very well maintained as all my vehicles. With my usage pattern it would last another 5-10 years before needing to be replaced and would easily go double the number of miles currently on it.

Upon getting a couple of repair estimates it became clear that:
(1) Insurance would probably total it rather than fix it because its market value is low due to its age and cost to fix to pre-accident condition (the only way they fix) is likely to be high relative to market value.
(2) Amount I’d get from it being totaled is way less than it’d cost to effectively replace it with something similarly reliable. And I’d want to replace it with something different, but not ready to do that right now + it’s a bad time generally to buy a vehicle due to shortages/supply problems.
(3) The first thing any insurance certified shop will do is take it apart to see what other damage there is/might be, before getting a ruling from insurance on whether to total it.
(4) Once it’s apart, if insurance says total it my only other option is to buy back a salvage title for it & have that shop fix it at my expense - if they even can/will.
(5) I could get my normal mechanic to “hack” a repair, but neither insurance nor the parking lot owner would help with that so it’s probably cost me the same or more but with less quality / not pre-accident condition.
(6) I could pressure the property owner legally with demand letters and/or small claims case. But that’s effort, time, and expense plus I don’t know how it would turn out.

I already knew this was not a high stakes decision for me. Any way I go I’ll be fine. But still, deciding which way to go seemed difficult and was bugging me for several days.

After reading your article, I thought more explicitly about what my goals were related to this car.

One of them was flat-out dumb: I had been worrying about the goal of preventing a “good car” from ending up in a junk yard, or with someone who would not maintain it as well as I have. I dropped that.

Other goals were OK generally, but not really suited to my current context. Stuff I needed a lot more when I originally got this car than I do now. As an example, having a vehicle to get around town that’s good on gas, and easy to maneuver and park. Pre-COVID, I or someone from my family used this vehicle for that kinda thing approximately daily. Now, not so much. And we have other vehicles that work OK for that purpose. And for just what we pay in insurance + license + basic maintenance on this car we could take about an Uber ride a week if our other vehicles are in use. The damage happened 3 weeks ago and we haven’t used this car except to take it for repair estimates since, and haven’t particularly missed it.

Other goals were about possible future scenarios where we’d need a car like that more than we currently do. People changing habits back to leaving the house more, getting different jobs, and that sort of thing. After discussion we figured out that while it could happen, those aren’t scenarios we want to plan and prepare for.

Another goal I had I still think is good: I want to avoid getting into a legal dispute over this car. Whether I’d win or not, I don’t think it’s worth the time/trouble/risk/angst/etc. of some kind of legal dustup.

After re-evaluating my goals it turns out I don’t really need to care about what happens to this car. We don’t actually need it any more, and had just been keeping it due to the inertia of already having it and a bunch of unlikely “what if” scenarios.

So I’m just gonna take it in to the insurance certified shop. If they total it, they total it. If they actually fix it to pre-accident condition I’ll decide then whether to sell it.

That’s good. Sounds like you had inertia to keep using old policies which are issues Goldratt wrote about.

Makes sense.

Yes! this immediately made sense to me. (edit: I haven’t read anything much before or after this part – I was skimming through and it caught my eye b/c of the <hr>)

It’s also useful to me WRT docs I’m writing.

A subtlety around maximization+buffer is that you can use maximization + a binary success criterion to increase buffer. Like, if you have a plumbing system then throughput (water volume / s) might both be the thing you maximize and also the input for the success criterion. Different arrangements of pipes (stuff like the order, total length, diameter, number of paths, angles in those paths, etc) is something you could (roughly) maximize throughput against, but the success criterion might be like output_flow > 5 m^3/s. Maximizing output_flow gives you the most buffer, and it also tells you whether, given possible arrangements, you can meet the success criterion at all.

So maximization is useful if you want buffer, but it’s not useful as a success criterion (e.g. make the system with maximum throughput)

Maximization can be a binary success criterion (you look at all your options, and then refute any that aren’t tied for first place).

If you just say “more is better with no breakpoints, and i might pick an option that gives less if other factors seem nice” then you’re using maximization in a different way without clear success/failure criteria.

Another example re breakpoints and buffer:

my state is in lockdown. The breakpoint set for coming out of lockdown was ‘the first monday after we hit 70% double-dose vax rates’[1]. I’ve been watching the prediction of when that will be: it had to be one of 11th Oct, 18th Oct, or 25th Oct.

The reason I was watching it was because I knew that the prediction based on average rates was like 12th Oct – nearly but not quite good enough. This is what it looks like atm: (20 is days to 9th Oct which is the predicted date)

Just before reading the breakpoints+buffer idea I checked again today and was rly happy that we’d gained a day of buffer (it hit 10th oct a few days ago, and was hovering on 11th oct for a few days too).

I thought this was worth mentioning b/c breakpoints+buffer idea explains a good chunk of what was going WRT my watching the date and being happy about getting some buffer (we had no days of buffer when the prediction was 10th oct)

  1. Not an exact quote; but I wanted to signal that I put the condition precisely and carefully. ↩︎

Ahh, yeah that makes sense, but there’s no buffer, then, right? (you can’t get better than the best possible option except by making new options, and those new options won’t have buffer, then, b/c the goalposts changed).

So breakpoints aren’t sufficient for buffer?

If you have one thing in clear first, then its lead over second place is a buffer – it could lose up to that much without you wanting to change plans.

You could also have maximization within a margin of error instead of strict maximization. E.g. take whatever is in first place, then do ±5% of it, and consider that whole range a tie. something has to drop below that to fail and a new option would have to get over that to make the current option no longer count as maximizing.

1 Like

Ahh, yeah. I was only considering buffer like excess capacity in a system rather than buffer before that system isn’t the best anymore. Those are the same thing, tho, I think. They’re both buffer (in the same units) that indicates how much can change before the idea doesn’t work.

Is comparing capacity basically measuring which idea/system has more buffer?

I think maximization is sort of related. Like to figure out which system has the most buffer, you need to be able to go most of the way to doing maximization (until all but one options are eliminated or there’s a tie)

Like, if you have two plumbing systems A and B, with known throughput-equations, then you can only compare buffer by saying like given some load X, how much larger can X be before system {A,B} fails? Or, at least, something that can answer that question; like maybe those equations (describing the flow rates) aren’t represented in those terms, but there must be some way to transform them into those terms so that the success criterion can be evaluated.

How is comparing capacity different from comparing buffer given some load? I don’t think it is.

I wonder if, like, you have competing systems, is maximization useful beyond showing that a system has buffer in situations where competing systems doesn’t? Is there any benefit to magnitude of the difference? I guess only if there is an additional breakpoint that gets met.

(note: I think this post is a bit disorganized, and also that it was better to post than not)

Had an idea about buffers and variance – it’s about excess capacity in ideas (and the nature of that capacity)

Binary Buffers (or Qualitative Buffers)

I think buffers don’t have to be continuous, and don’t have to be quantitative. Here’s an example to demonstrate:

  • Goal: travel across a sandy desert in a car
  • Context: you have access to experimental tires and an experimental fuel
    • The experimental tires, if they work, will give you a 2x optimization on fuel consumption (they’re good on sand)
    • The experimental fuel, if it works, is 2x as energy dense, so will let you go twice as far on the same amount of fuel.
    • Both are binary – they’ll either work or they won’t
    • The tech is independent of each-other (if both work, you can use both – there’s no interaction between them)

For the sake of the example: if you had both stock tires and standard fuel, then whether you could make it across is within a small margin of error (you might, or you might not). But if either of the experimental technologies work out then you will definitely be able to cross safely.

If you test the experimental gear, you can find out if your plan to cross the desert will work or not.

IGC Chart

idea success according to {G,C}?
safely cross desert w/ both experimental tires and experimental fuel :white_check_mark:
safely cross desert w/ experimental tires and standard fuel :white_check_mark:
safely cross desert w/ stock tires and experimental fuel :white_check_mark:
safely cross desert w/ stock tires and standard fuel :x:

So, WRT crossing the desert, the experimental tech is excess capacity in the plan. Either of those experimental ideas could not-work-out and you can still cross safely. But if both don’t work out then you can’t cross safely. (“Safely” in this case means that your car+tires+fuel won’t be the bottleneck)

I think that demonstrates the idea. Can write more if it’s not clear.

Yeah, resources don’t have to come in smooth amounts. This comes up a lot. E.g. I can take full time job and get paid $100k or decline it and get paid $0 with no options in between. Another example of non-smooth resources is buying a lotto ticket or gambling in general.

This undermines ET by suggesting his most polished articles are worse for learning from. It discourages people from reading and studying them.

I also don’t think AD knows what he was confessing. In short, he turns his brain off when reading good articles. But when the article is disorganized in ways that his autopilots can’t deal with, he turns his brain back on. So he pays selective attention to worse articles and deals with better ones only with autopilots that do not connect what he reads to his existing knowledge.

Does this work? No. AD has never take appropriate steps to objectively check whether his learning actually worked. He’s assuming and bragging that he learns “more” – meaning some from both, and more from this type – but there is no evidence that he learns anything significant from either. AD is also bragging about his ability to self-organize information and customize it for himself – again with no evidence. When articles are less clear, it’s easier to have larger misunderstandings and still think you got it right.

The basic objection here is that maybe some articles are too easy so less learning happens because too much is handed to the reader so he doesn’t do enough work – like spoiling a child. This is wrong. The best articles require a lot of work to learn much from. There is no writing that makes philosophy too easy. There are other problems with that perspective, but regardless philosophy is something where people need all the help they can get. One of the things AD is admitting is that he can’t or doesn’t tell when he’s not getting stuff and needs to think more. That will ruin all his learning, not just his learning from more polished articles, because it means his standards for success stop short of actual success.

AD gave no introduction or conclusion for his story about his car. The only thing relating it to ET’s article was this sentence in the middle.

This is not substantive engagement with ET’s article or CF ideas. Thinking about your goals is not a unique insight of CF. A significant portion of the more effective and smart people in the world already think about their goals. AD didn’t respect CF enough to try using its methods like making an IGC chart or specifying goals with binary success/fail criteria

AD’s post is meant to give an impression of engaging with CF when he isn’t actually. There’s no framing about how he isn’t. Nor does he claim to. He keeps his options open to make claims later about what his goals with the post were. By leaving out any introduction or conclusion, he can claim he was trying to do whatever he succeeded at if he gets a positive response, and claim he was not trying to accomplish whatever he failed at if he gets a negative response.

Leaving out goals is a standard way people prevent criticism. As CF discusses, criticism relates ideas to goals. Criticism is relative to goals. Hide your goals and be ready to move the goalposts and to deny any goals people guess you had, and you prevent criticism. Or at least reduce it to the lowest common denominator criticism that only deals with the most generic, universal goals in our society.

AD’s story contains a bunch of brags about how wealthy he is. It also has indicators about him being reasonable. It’s designed to make him seem smart, rational and good at decision making.