Yeah, true, good point. But that doesn’t sound “just as good” since it adds extra steps. It looks like tacked on complexity and less elegance.
Why has no one really tried to engage with and learn the major ideas in this article?
For me: it’s too hard for me. I’m doing simpler stuff.
That’s fine. But what stuff? I don’t recall seeing it.
I’m doing grammar. Slowly (~1h / week). I haven’t posted anything yet. Not sure what to post about it at the moment. Also, I don’t want to post and not follow up (edit: quitting is my problem) - a problem I have had here / learning philosophy stuff.
Re grammar I’m still only on the first 3 steps of simple sentence analysis (verb, subject, object or complement).
Do you think people should prioritise learning the material in this article in particular?
Yes. While it depends on one’s life situation, goals, values, etc., I think this is generally a good article to prioritize unless it’s too hard, in which case something easier would be better. Doing other stuff also makes sense if that is urgent, e.g. a crisis.
I think some people don’t believe it’s too hard. And some didn’t clearly judge that even in their own mind, and that ambiguity is problematic.
If it is too hard, it’d be good to consider why it’s too hard. What specifically is hard about it? What skills are you missing? What problems do you run into? That could itself be too hard, too, though.
If doing easier stuff (or anything), it’s also important to have some kinda specific (tentative) plan with things like milestones and ways to judge if you’re getting anywhere, not to vaguely, non-specifically try to work on stuff. You need some way to tell if you’re making progress. You don’t want to try to do easier stuff indefinitely and not know if it’s working and never come back to the harder stuff. It’s also problematic if someone doesn’t post let’s say once every two weeks minimum to try to check in on their progress on their plan in some way. (Checking in doesn’t have to be meta comments about progress itself or sharing metrics. It could also be posting something topical like some practice work, such as a grammar analysis, a worked math problem, a short writing passage, or a tree.) Plans can be hard because people often make bad plans (e.g. rationalistic, impractical plans) that don’t correspond well to their actual goals, but just giving up on planning is a bad response to that problem. (BTW DD/TCS had a lot of anti-planning, anti-goals, anti-scheduling and anti-organization type ideas. I don’t think those came from CR and I think they were mostly incorrect and dangerous/harmful.)
1 Like
This is a summary of the multi factor decision making maths article.
when you make a decision you usually have to take many factors into account. taking multiple factors into account leads to problems.
if a particular option is better than all of its competitors on every factor then iyou should pick it, but this is not usually the case. so you need to combine multiple factors.
two factors can be added up if they have the same dimensions, e.g. - they are all lengths or masses or whatever. this might sometimes require converting units.
you shouldn’t always add things of the same dimension. if you have a desk that is one metre by two metres, you don’t have a three metre desk.
For example if you have two chocolate bars, one of which is 100g and the other is 1kg, the total amount of chocolate is 1100g. this requires knowing that 1kg = 1000g. more generally to convert amounts in different units you have to know the rule for converting between these units.
but often different factors have different dimensions so adding them up isn’t an option.
you can sometimes multiply units with different dimensions. for example, you can multiply how many cans of tuna there are in your apartment by the length of your bed, but this quantity, like most quantities you get by multiplying two things together, isn’t useful. so in general multiplying quantities together won’t help. dividing is like multiplying by 1/quantity you want to divide by so it has similar problems.
averaging uses addition, so it only works in cases where addition would work
multiplying things together may also require unit conversion. For example, if your desk has length 6 feet and width one metre, then you should convert either the length or the width if you want to multiply them together to get the area.
people often try to use pro-con lists and give different options with different dimensions goodness points. and then you pick the option with the most goodness points. but in general there’s no good way to calculate the number of goodness points. you are effectively making up arbitrary conversion factors between quantities of different dimensions. how do you convert the number of potato waffles in you freezer so that it’s comparable to the amount of ram your computer has?
also what is the significance of option 1 having 5 goodness points versus option 2 having 200 goodness points? in what sense is option 2 40 times better than option 1?
how do you convert binary factors like the answer to the question “will this option result in me having a limb amputated?” into goodness points?
and what about category based factors, like red objects versus green objects?
conversion to goodness points requires making arbitrary choices about how many goodness points you should reward.
CF’s solution is that we should always use binary factors: every factor should be 1 or 0, pass or fail.
binary factors often ask whether a particular option is good enough to meet a particular goal. does this toaster have wide enough slots to fit a muffin? is this knife sharp enough to cut an orange?
binary factors can then be combined with logical operations such as “and”. “does this vegetable have vitamin c in it?” and “does this vegetable taste nice?” if the answer to both questions is yes, then the result is yes and you might buy the vegetable, otherwise you don’t. if you have more factors and only produces a yes if all of the factors used as input are yeses.
to use this method we need a way of converting factors with other dimensions into binary factors.
converting to a binary dimension simplifies the factor being converted into a 1 or 0. the way to do this is to consider whether a particular value for a factor would fix a problem you want to solve or not.
factors should be relevant to subgoals of your overall goal, e.g. one subgoal might be to have some food you can store for a long time without it going bad, another subgoal might be to have food that tastes good.
this also comes up in psychology. some people try to be maximisers, they try to improve their decisions by adding up everything into an overall evaluation, which leads to problems like perfectionism and having evaluations that change a lot. satisficers just try to some up with options that are good enough, small changes needn’t change your choices if the differences don’t matter for solving actual problems.
if any idea passes for every dimension, then there are no known criticisms of it. we shouldn’t assume more of a thing is always better. we should have breakpoints for when more of something actually solves a new problem. this is like focusing your attention on bottlenecks in the theory of constraints. it also uses the yes-no philosophy idea that every idea should be judged as refuted or unrefuted.
1 Like
Cool. Do you have any opinions about the article?
The article is a good explanation of how to deal with making decisions involving multiple factors and I haven’t found any problems with it. I am not aware of comparable material elsewhere that takes this problem seriously and proposes ways to solve it that stand up to criticism.
Did you brainstorm any criticisms of the article that you then came up with a refutation of?
Yes.
From the article:
If you multiply twenty factors, you’ll get twenty-dimensional units. The problem is usually even worse than in my example with three factors.
There are quantities with units such that if you multiply them together you get a unitless quantity. For example, the wave number of a wave is 2*pi/wavelength and has units of 1/length and this is often multiplied by the distance a wave has propagated d, which has units of length to get a quantity that is just a number. But in general the units of different quantities don’t cancel out like this so I don’t think this is an issue that matters in real problems.
Another issue is what happens if a quantity is in a range where you don’t know if it will produce some known problem. Suppose a pipe bursts in your plumbing if the pressure in the pipes is too high, but you don’t know what pressure will cause the pipes to burst, but you know a range where the pipes won’t burst. In that case, you have to decide about whether you should gain better knowledge about the safe range by going outside the range where you know a problem won’t arise. But this is just another factor in the multifactor decision making. If you want to prevent the pipe from bursting in your house you shouldn’t go outside the safe range in your house. Coming up with a new 1 factor for learning more about your plumbing system is irrelevant because the pipes bursting is a 0.
I’m not seeing an issue even in this case. I’m not very familiar with wavelengths but is the following right?
The wavelength is saying how far the wave has to travel to repeat. Let’s align the wave along the x-axis in a 2d plot and start it at the origin. So it’s traveling horizontally while oscillating vertically. The vertical oscillations are a repeating pattern and get back to where they started every e.g. 7 units the wave travels in the +x direction.
So one piece of information we have is that something is happening at a rate per amount distance travelled along the x-axis. And the other piece of information that we multiply with is the distance the wave has propagated along the x-axis.
So in other words, omitting the constants on every term, the rate is r/d and we’re multiplying that with d and we get just r. r is the number of times the wave pattern repeated, which could be seen as unitless since it’s just a number (count), though we could also (I think more precisely) see it as having a particular units since it’s a number of something.
Anyway the main point is I think the distance units cancelling out here is pretty ordinary. This looks to me like just how rates work in general. A rate is e.g. “5 X per 2 Y” and then you can multiply that by the number of Y’s in a particular case to get the X’s in that case (the Y’s cancel out). It’s like multiplying 53 miles per hour by 3 hours to find out the number of miles driven.
Also, two things is not a lot of potential criticisms. Can anyone here brainstorm more?
I found standard academic terms for multi-factor decision making math:
Multiple-criteria decision analysis - Wikipedia
Multiple-criteria decision-making ( MCDM ) or multiple-criteria decision analysis ( MCDA )
These terms are used in many papers. It’s often “multi” rather than “multiple”.
I think these terms are errors. It should be “criterion” (singular) not “criteria” (plural). It’s easier to see by considering “multi-factor decision making math” vs “multi-factors decision making math”. Using “criteria” as a singular word is a common error so the issue stands out much more when switching to another word like factor(s) where our intuitions automatically recognize singular and plural correctly. Similarly, it’s a “multi-car garage” not a “multi-cars garage” and objects are “multicolor” not “multicolors” and it’s “multi-level marketing” not “multi-levels marketing” (it’s spelled with or without a hyphen; I don’t know which is better but here are some general guidelines).
Using criteria as a singular word has become common enough to be partially normalized, but I don’t think academic experts screwed it up on purpose. The wikipedia article uses “criterion” 29 times and is less formal than the papers, and it’s also more recent than the papers. Criteria and Criterion: Can 'Criteria' Be Singular? | Merriam-Webster
A paper from 1972 used multi-criterion https://pubsonline.informs.org/doi/abs/10.1287/mnsc.19.4.357 but later papers seem to have switched to criteria.
This article gives a readable overview of Multiple Criteria Decision Making (MCDM) techniques. They’re all, from my point of view, attempts to optimize weighted factor math that is fundamentally broken at a deeper level (they’re trying to combine qualitatively different dimensions). It’s all a bunch of local optima work that neglects the bigger picture. They need to check their premises better and rethink the foundations of the field.
Nothing discussed resembles my binary multiplication solution to MCDM, nor did I see anything Goldratt-type stuff about focusing on key factors, bottlenecks, constraints or limiting factors. They seem to have no idea that most factors have excess capacity, just that some factors are more important than others (which is handled by weighting).
https://pubsonline.informs.org/doi/epdf/10.1287/ited.2013.0124
Most real-life decisions involve multiple criteria, yet many business and management university courses, MBA included, do not deal explicitly with this topic within any quantitative module. It is a remarkable fact that a number of current operations research and management sciences textbooks do not cover the topic of multiple criteria decision analysis (e.g., Hillier and Lieberman 2010, Powell and Baker 2011).
(In the original, the two books mentioned are links to where they appear in the References section at the end.)
This was written in 2013. I wonder if he means that these books actually don’t talk about multi-criterion decision making at all, or that they just don’t give a (flawed) mathematical method for doing it like AHP.
This paper basically says that adding weighted factors is highly flawed but everyone is doing it anyway. It says we need non-linear weightings and multiplication instead. It says multiplying factors is not a new idea by has been ignored for too long.
The paper says people routinely weight factors twice, first in a normalization step and then in a weighting step. This leads to some problems. Different choices of normalization, with the same weights, can lead to dramatically different rankings for options.
Some systems people use for combining weighted factors change the ranks between options A and B based on whether option C is included in the analysis or not. Adding or removing other factors can cause A to go from worse than B to superior to B. That is bad!
He also points out that simple additive weighted systems are unable to ever select (4.5, 4.5) as the best option when the alternatives are (1,9) and (9,1), even though a well-rounded option may be better despite having a slightly lower unweighted total. Weighting either the first or second factor more will never fix this – it’ll just make one of the polarized options win by more. This was called the “linearity trap” in a 1982 paper; it’s not new. Broadly, the paper says weighted sum addition systems have a tendency to select unbalanced options over balanced ones, contrary to how human preferences often are (the suggested solutions are extra decision rules or non-linear weighting functions). In a real life example in Poland:
Organizers of the tenders soon discovered that they were forced to select the offer that is cheapest and worst in quality, or the best in quality but most expensive [the decision making system didn’t let them pick moderate options]
Another example is diamonds. Their price depends primarily on four factors: carat (weight), cut, colour, and clarity. An additive weighted model works poorly because to be really valuable a diamond can’t be bad at any factor. A multi-factor weighted multiplicative model does better (according to research using 257 real diamond prices).
When multiplying criteria, simple weightings of factors can be done with exponents. And, contrary to addition, multiplication tends to favor balanced options (e.g. for multiplication without weightings, (4.5,4.5) wins significantly vs (1,9) or (9,1)). Note that this sort of favoring doesn’t come up with CF’s multiplication of only 1s and 0s.
The paper uses the term “units of goodness”, in quotations, to refer to how multiple judges on a panel may score options in essentially different units even if using the same scale (like 1-10). This is like how some teachers will give 100% grades and others think nothing is perfect and won’t give 100% – those teachers are effectively grading with different units or on different scales, despite both appearing to be grading in the same way, from 0% to 100%. Grades from those two teachers are not directly comparable but many people would incorrectly directly compare them.
The paper independently overlaps with my position by favoring multiplication over addition for combining factors. It does not consider avoiding weighting or combining of factors, as my approach does, nor does it consider issues like binary factors, focus, breakpoints, bottlenecks and excess capacity. Non-linear weightings (aka conversions between dimensions) don’t address the fundamental problems I raised with factors being in different dimensions. It doesn’t matter what complex conversion aka weighting function you use; apples aren’t oranges. I do agree that non-linear weightings make more sense than linear weightings in many scenarios (the paper points out that many things have diminishing returns as you get more, and that weightings can be curves. I think spectrums often have breakpoints/discontinuities rather than being smooth curves).
Multiplying non-linearly weighted factors is problematic because the weighting part is essentially converting their units to a generic goodness dimension. If you assume that’s fine, then whether it’s better to add multiple different goodness factors, or multiply them, depends. The paper does have some good points about the advantages of multiplication for many real-world scenarios (ignoring the qualitative difference aka dimension conversion problem), however multiplication is also imperfect and there is no single perfect answer for that (how best to combine numeric factors, even if they can be combined, depends on your goals and context).
https://www.sciencedirect.com/science/article/abs/pii/S0360835207001891
the commensurability problem which arises when trying to add together criteria measured in different units
This is by Tofallis; I talked about another paper by him in Multi-Factor Decision Making Math [CF Article] - #56 by Elliot
He mentions one of the main problems with adding factors that I also talked about. Incommensurable = in different dimensions = can’t be added. It’s also explained some at Dimensional analysis - Wikipedia
Commensurable physical quantities are of the same kind and have the same dimension, and can be directly compared to each other, even if they are originally expressed in differing units of measure, e.g. yards and metres, pounds (mass) and kilograms, seconds and years. Incommensurable physical quantities are of different kinds and have different dimensions, and can not be directly compared to each other, no matter what units they are originally expressed in, e.g. meters and kilograms, seconds and kilograms, meters and seconds. For example, asking whether a kilogram is larger than an hour is meaningless.
GIS (Geographic Information Systems) MCDA sometimes uses binary factors:
https://storymaps.arcgis.com/stories/b60b7399f6944bca86d1be6616c178cf
- a GIS-MCDA is a alternative to a typical binary or ‘coincidence’ analysis, providing a more robust criteria based methodology. It can therefore support multiple values of multiple criteria at once. This allows for more in-depth decision making.
They briefly list using binary factors as an option but they basically think it’s simpler and inferior.
Their context is basically you take a map and tile it, then you can have a binary evaluation of each tile, and you can have several layers on the map for different factors, and then you can find the tiles which succeed on every factor which are the best areas of land for your factors.
http://innovativegis.com/basis/mapanalysis/Topic23/Topic23.htm
The individual binary habit maps are shown in 3D and 2D displays on the right side of figure 2. The dark red portions identify unacceptable areas that are analogous to McHarg’s opaque colored areas delineated on otherwise clear transparencies.
A Binary Suitability map of Hugag habitat is generated by multiplying the three individual binary preference maps (left side of figure 3). If a zero is encountered on any of the map layers, the solution is sent to zero (bad habitat). For the example location on the right side of the figure, the preference string of values is 1 * 1 * 0 = 0 (Bad). Only locations with 1 * 1 * 1 = 1 (Good) identify areas with out any limiting factors—good elevations, good slopes and good orientation. These areas are analogous to clear areas showing through the stack of transparencies.
This uses some identical images to the previous article. I guess one is copying from the other or they are by the same people.
Again they think binary factors are a poor method:
While this procedure mimics manual map processing, it is limited in the information it generates. The solution is binary and only differentiates acceptable and unacceptable locations. But isn’t an area that is totally bad (0 * 0 * 0 = 0) different from one that is just limited by one factor (1 * 1 * 0 = 0)? Two factors are acceptable thus making it “nearly good.”
They talk about two more ways to use binary factors to generate maps (you can add them to get a simple count of successful factors, and you can number them with powers of 2. Given that they assign colors to numbers, the powers of 2 method results in unique colors for every different combination of factors passing.) The point of these methods is basically to make it more like using non-binary factors.
Then in the section “Breaking Away from Breakpoints”(!) they say basically they’d rather have smooth curves, not use the breakpoints involved in binary factors.
I think they don’t know the epistemological reasons to favor binary factors (e.g. its connections to error correction and refuted/non-refuted) nor the fundamental problems with combining non-binary factors from different dimensions.
I think the basic reason they were willing to use binary factors is they are generating maps with many tiles, so they get complexity from having lots of tiles. I think people basically want to have a lot of complexity somewhere or think they need it. When people are trying to make just one decision instead of evaluate hundreds of of tiles on a map, then they look to more complicated decision making procedures. Having to evaluate hundreds of tiles is what got people to try binary since it was easier to get it working at all.
They even say:
A rating model is the most powerful approach. It breaks the good/bad dichotomy into a gradient of preference most often expressed as 1= very bad to 9= very good.
This reads like non-linear weightings and conversions between dimensions are the same thing (the terms are interchangeable). But not all conversions between dimensions are non-linear weightings, e.g., if speed is a constant, then you can multiply by any distance to get duration; that’s converting between dimensions but is also linear.
Do you mean something more like: non-linear weightings are always conversions between dimensions (but not necessarily the other way around)? That’s what I’d expect.
You can make a map of a single factor like land-or-water, elevation or average annual high temperature. Making one map that presents information about multiple factors in an understandable, useful visualization is desirable. So people have looked into that. It’s apparently called geographic information system (GIS).
If you tile maps you can evaluate factors for specific tiles which can be easier than trying to combine factors in continuous ways.