Multi-Factor Decision Making Math [CF Article]

I did some editing to try to make that part easier to understand:

What do people do about that? They multiply by weighting factors that get results they think are reasonable. But that isn’t actually a way of making decisions. How do they know what’s reasonable? They must be using their intuition, common sense or something else other than weighted factor summing. So the weighted factor summing method doesn’t work as a self-contained solution to decision making. It relies on pre-existing opinions reached some other way. People often make the math (or non-numerical estimate) come out to fit what they already think (without realizing they’re doing it). For example, college rankings often start with the pre-existing idea that Harvard is good and then give high weightings to whatever factors Harvard is good at so that Harvard-like schools come out on top, which seems like a reasonable conclusion to people who already believed that Harvard is one of the best schools.

Any method involving arbitrary choices (like what unit conversions or weights to make up) runs into a major problem: You have no good way to make an arbitrary choice unless you have pre-existing knowledge of what a good answer is.

This reminds me of the point expressed here (and I’m sure other places) about how induction doesn’t work as an explanation of how people think.

So we have this graph and we’re connecting the dots. Induction says: connect the dots and what you get is supported, it’s a good theory. How do I connect them? It doesn’t say. How do people do it? They will draw a straight line, or something close to that, or make it so you get a picture of a cow, or whatever else seems intuitive or obvious to them. They will use common sense or something – and never figure out the details of how that works and whether they are philosophically defensible and so on.

People will just draw using unstated theories about which types of lines to prefer. That’s not a method of thinking, it’s a method of not thinking.

They will rationalize it. They may say they drew the most “simple” line and that’s Occam’s razor. When confronted with the fact that other people have different intuitions about what lines look simple, they will evade or attack those people. But they’ve forgotten that we’re trying to explain how to think in the first place. If understanding Occam’s razor and simplicity and stuff is a part of induction and thinking, then it has to be done without induction. So all this understanding and stuff has to come prior to induction. So really the conclusion is we don’t think by induction, we have a whole method of thinking which works and is a prerequisite for induction. Induction wouldn’t solve epistemology, it’d presuppose epistemology.

So the connection I have in mind is this: weighing factors according to reasonableness doesn’t work as a self-contained general purpose solution to making decisions, since it’s relying on some pre-existing opinions reached by means outside the weighing factors decision-making method. Induction doesn’t work as a complete/general purpose explanation of how knowledge is created, since it apparently has to rely on ideas about simplicity and Occam’s razor and other ideas that need to be created without induction.

1 Like

The arguments in this article seem to refute judicial balancing tests!

One balancing test from American administrative procedure law applies to the question of due process of law, a consideration arising from the Fifth Amendment and Fourteenth Amendments to the constitution. Due process questions concern what type of procedures are appropriate when the government takes away property or a privilege from an individual; the individual would argue that the government should have, for example, given them a hearing before taking away their driver’s license or cutting off their Social Security benefits. This balancing test, of which it weighs considerations:

  1. Private interest affected by an official action taken by a government agency, official or non-governmental entity (company) acting as a governmental agency. (i.e., how important is the property or privilege that is being withheld or confiscated?)
  2. The risk of some deprivation being erroneously inflicted on the respondent through the process used or if no process is used. (i.e., does giving the person a hearing or whatever else they asked for actually make it less likely that the government will make some sort of error by giving the individual an opportunity to point out the government’s mistake?)
  3. The government’s interest in a specific outcome (for example, the government may say that giving a hearing is too expensive).

There’s no way to convert Importance of Private Interest Affected and Risk of Deprivation and Importance of Government’s Interest into some common factor. So the result IRL is that judges are just gonna go according to their intuition of what’s fair and right - so the supposed “test” isn’t doing much except maybe serving as a reminder list of things to consider when using their intuition.

If for factor 1 you had “Affects an important private interest?” and important private interest were defined in some reasonably objective and clear way, and then you did the same things with 2 and 3 (basically convert them to binary factors), then i think you could maybe have something inspired by the above test that could perhaps be applied in some reasonable and consistent way. But not as it stands.

Yes. That is actually reasonable. There’s nothing wrong with “here are a few tips about stuff to keep in mind”. That does help people make better decisions and judgments. People sometimes forget key stuff or don’t know which factors merit major emphasis. The balancing test helps guide that.

The issue is when the tips are presented as an actual decision making method. It’s problematic when it’s like 30% complete – some guidance but the judge has to figure out the majority of the matter himself – but it’s seen as 90% complete.

1 Like

I like the new version.

I’ve thought about this most days since then, and I get a bit stuck. I have some ideas, though. I’m intending to make a linked post soon. (This reply is brief b/c of that getting-stuck-ness. It’s been a barrier to posting, tho, so I’m saying this in part to get past that)

I had an idea today about a convergence between a common thinking technique and the OP.

One of the ways I make ideas more manageable is to use a deliberately limited context – like say that 2 factors are equal, or some factor has no effect. That way, it’s easier to reason about fundamentals (and if you find a problem at that point then it’s probs generalizable). Like someone might say all else being equal, if x goes up then y goes down. I think this is a pretty common method.

If you do this with MFDMM (mutli-factor decision making math – I guess abbreviation-pending), there are two important changes that can happen:

  1. Two factors can be equal, i.e., their conversion constant is 1.[1] This is important when factors are multiplied or conversion usually matters. (nb: this includes ‘canceling out’ via division)
  2. A factor could be set to zero (i.e. has no effect), which makes a difference when things are being summed

Both cases can make some previously unanalyzable situation analyzable.

Anyway, this seemed like a notable convergence between an existing traditional method and MFDMM (which, AFAIK, didn’t have a good rule of thumb about when to do what, but MFDMM does).


  1. or they could be set to a fixed ratio, which is thus equal to the conversion constant. ↩︎

Seems like either:

  1. You’re evading. You don’t want to talk about it.

and/or

  1. You’re trying to jump straight to a conclusion instead of make incremental progress. You wanted to figure out a good answer by yourself instead of talk about it. If your goal for a reply was just to make one step forward you would have been able to say something, e.g. a piece of relevant info, a thought, or a question/issue you don’t know the answer to.

@Max bump

Thanks (I’d forgotten about this)

I think I’ve been doing both – using (2) as an excuse to do (1).

WRT

If your goal for a reply was just to make one step forward you would have been able to say something, e.g. a piece of relevant info, a thought, or a question/issue you don’t know the answer to.

I think @ingracke’s analysis was right. I don’t like the way I wrote this post – I think there’s a problem there, particularly:

So I was hostile and dishonest (particularly about the nature of the post and my motivations). I don’t like that, but I don’t know what to do – besides like reflecting on it and trying to figure out what ideas motivated that vs what ideas I want to motivate my posts/replies. That reflecting mostly seems like “[wanting] to figure out a good answer by [myself] instead of talk about it.” So while reflection is still important (or I think it will be, like to understand/align my ideas/priorities/etc), doing it the way I usually do isn’t the best way. (Note: I do think I make some progress this way, but how could it be anywhere close to efficient? IDK)

To sum up, I feel sorta like my post was me playing against my team or something. Like who’s side am I on? Why am I hostile? Why would I attack you – something that’s inconsistent with a lot of things I say!

I feel a bit lost trying to answer those questions.

Try using the bookmarks feature for setting reminders. (described under Forum Features->Bookmarks)

Yeah.
I’ve started to use this – but didn’t in this case (the MFDMM link at the top of screenshot is to Elliot’s bump).

image

I’ve been advising people about learning activities they can do, things to study, how to incrementally build up starting with small successes, etc. You and others don’t do it, and also don’t disagree/debate/criticize what I’m saying. Suggesting things you can work on and practice more was also one of the major themes of our tutoring sessions.

My serious, considered advice: Hire ingracke to help you. Something like 2 calls a month ongoing. You need both the advice and the consistency/regularity. Your intermittent CF posting isn’t effective.

I’ve been considering this. I haven’t asked her about it (yet), though.

Yeah.

I have a brief comment. When there are methods that combine dimensions that are relevant to a decision, like maximising the benefit per unit cost, you can use that measurement as input to the binary factor multiplication approach. You put breakpoints on the benefit/cost and then use those breakpoints to produce a binary value for your binary factor multiplication.

Yes, you can treat the combined dimensions as a single conceptual unit, and then deal with it using the normal methods for dealing with one factor. That it works as one conceptual unit is why combining in that way makes sense.

Integrating Conceptual Units and Combining Dimensions

Dimensions or units are a type of idea or concept. Two or more of them can be integrated together into a single conceptual unit. This follows the general rules of integration of ideas.

Any group of ideas can be integrated together, but most groupings and combination methods result in nonsense. Also, the usefulness of concepts is contextual. So context determines which ideas are useful to combine in what ways. You need to start with, and conclude with, concepts that fit your context.

A concept that seems like nonsense could make sense in some other logically possible context. That works for all possible finite results of integration. Speaking more broadly, the result of an integration is information (and it’s logically possible to get the same information by another method without integrating it). So the question is whether any arbitrary information would be meaningful in the right context. The answer is “yes” for finite information because you can imagine e.g. that it’s the password to decrypt a computer file.

With integration, we’re always considering which results are relevant to our goals. We try combining many ideas in many ways to try to find some combinations that are useful for us (that are adapted to our goals and context). You can integrate any ideas, but the large majority of results are junk.

Combining dimensions is the same way. You can combine any dimensions but most results are junk – they don’t mean something meaningful to you.

Integration is a generic idea about combining multiple parts into a greater whole. It doesn’t specify a specific way of combining. There are many ways to combine ideas and we don’t know what they all are. When we integrate ideas, we often don’t conscious know what method we used. We’re bad at verbalizing how ideas combine.

Dimensions, being ideas, could be integrated with any general purpose method. But we generally focus on a few combination methods which are not general purpose: they only apply to numbers. As long as we only define dimensions for things we know how to put a number on, then we can combine dimensions using numerical operations (math), like multiplication or division. (In general, we cannot combine them and get a reasonable result with addition or subtraction, as I’ve [[written about]].)

Can we put numbers on all dimensions? Not very well. Some dimensions, like length or mass, are measurable physical quantities. Others, like cuteness or deliciousness, are things we don’t know how to measure accurately. We don’t know, clearly enough, what they are. And we don’t know the right units. However, people make up very loose, approximate numbers for dimensions like those, often on scales like 1-100 (including 1-10, 1-5, percentages, and real numbers from 0-1). Scales like those are often used in a comparative way. You imagine some super cute or uncute stuff, and some middle of the road stuff, and then consider how close the thing you’re judging seems to those points of reference. These comparisons are not made in a reliable, accurate way like measuring, so doing math with these dimensions, at all, is questionable. Happily, Critical Fallibilism does not do math with those dimensions.

CF uses yes-or-no questions to convert to related binary dimensions. You can still use 1-100 scales for informational purposes when you find them useful, but no math on problematic numbers is needed. By switching to a 0 to 1 scale, with no fractions allowed, CF allows for clear, accurate answers instead of intuitively trying to guess the right number (and never getting it perfect, and not having any good way to figure out how much you’re wrong by or how consistent your evaluations are for different things – e.g. did you really estimate the cuteness of three different cats on the same scale so that the numbers are all directly comparable?).

CF also allows for non-binary numbers when they represent distinct categories. It’s problematic to use a 1-5 scale for how good something is. But it’s OK to have 5 discrete, qualitatively-different categories with breakpoints differentiating them, and to rank them by how good they are and number them. Not all categories have clear rankings (in your context, for your goals) but some do. The reason this works is that category membership is a clear, decisive thing – in fact whether something is a member of a category is a binary judgment (it is or it isn’t). So you aren’t trying to measure an intellectual quantity. The numbers do not represent different amounts of the same thing. Only physical quantities can be measured; using measurement of amounts intellectually is only an approximation. (Note that counting is a type of measurement, even though we can count without a tool like a ruler or scale.)

With binary judgments, the numbers also are not different amounts of the same thing. When you say something like “how good is this on a 1-10 scale?” you’re looking at an amount or degree of one thing. When you ask, “Is X the case, yes or no?” you are not considering an amount or degree. It’s not a spectrum. In binary thinking, we can use the numbers 0 and 1, but they represent qualitative not quantitative differences, like yes/no, true/false or pass/fail.

Suppose you’re considering pets and use numbers for different species of animal. You rank them by length and assign dog=4, cat=3, hamster=2, goldfish=1. These numbers are not measurements of anything, including length. They are not degrees or quantities. Quantities would be: dog = 36 inches, cat = 20 inches, hamster = 8 inches, goldfish = 4 inches. Numbers are more normally useful for quantities/amounts/degrees/spectrums, but they can be used for qualities/categories (like using 1 and 0 for true and false is common).

In the example, I ranked the animals from 1 to 4. It’s customary for the lowest number to be either 0 or 1, and for the numbers to be consecutive integers. But you could use other numbers, e.g. dog=5743, cat=994, hamster=22, goldfish=-5. And you don’t have to rank things and put the numbers in any particular ordering. You could just have five things and give them five different numbers in no particular order. Then the numbers are essentially just new names for the things, which can be useful if they didn’t have names already or the names were long. When using numbers that way, small positive numbers are good because they are short to say or write.

There’s a related technique to help figure out if numbers are being used in a quantitative or qualitative way. You can try changing the numbers around. Can you add 10 to all the numbers, or triple them, without screwing things up? If the specific numbers used seem to matter and changing them causes problems, that’s an indication that you’re dealing with quantities. If other numbers would work fine, that’s an indication you’re dealing with qualities/categories.

People sometimes talk about cardinal numbers (counting numbers – one, two, three) and ordinal numbers (rankings in a sequence like first, second, third). Cardinal numbers are normally used in a quantitative way – to say how many of one thing there are. Whenever there is more than one number for different amounts of one thing, it’s quantitative. When each number represents a different thing, then it’s qualitative and suitable for some important uses in Critical Fallibilism. Ordinal numbers are technically qualitative. First, second and third are different categories. Third is not some amount of firstness or fifthness. You can’t do math with ordinal numbers in the regular ways that you do with cardinal numbers. And ordinal numbers do not directly represent measurements. However, people general think of ordinal numbers in connection with quantities – e.g. people run a race and are ranked by their time (a quantity), and first place corresponds to a particular measured time.

You can add non-negative integer dimensions under mod 2 and get useful results. With some tweaks you could work with something else like non-negative reals. You find out whether every dimension is 0, or at least one is non-zero (or is a 1, if you convert them all to 0 or 1 first). In other words, with addition you can OR together all the dimensions. Whereas with multiplication, you can AND them together.

The mod 2 part isn’t necessary. Alternatively, you can interpret the result as either zero or non-zero.

Which is useful, ORing or ANDing dimensions together? Generally ANDing. It tells us if there are any failures. Whereas ORing tells us if there are any successes. Success at one sub-part isn’t good enough.

1 Like

You could label success 0, failure 1, and then you’d want to use OR/addition rather than AND/multiplication. If you swap the labels on the values, then it swaps which operation is typically useful.

Is 1=success, 0=failure an arbitrary convention, or are there fundamental reasons for it? Are there important, fundamental mathematical reasons for using 1=true, 0=false or could you develop a math system that’s just as good with them swapped?

Regardless, the current advocates of addition are not using 1=failure. They are not trying to check for any failures. They’re trying to add up degrees of goodness because they don’t do decisive thinking.

Is 1=success, 0=failure an arbitrary convention, or are there fundamental reasons for it? Are there important, fundamental mathematical reasons for using 1=true, 0=false or could you develop a math system that’s just as good with them swapped?

You can swap the roles of 1 and 0 just by applying the not operation to all of the inputs and outputs of whatever operation you want to do.