I don’t remember exactly, so I’m going to read Postmortems Help Address Causes of Errors and Errors Merit Post-Mortems first, then try.
relevant quotes from PMHACoE:
If you just make a straightforward correction – correct the error you made in this one case, but not in any broader category or pattern of cases – then you’re missing out on a potential opportunity for significant improvement.
Some people are too eager to update and move on without substantial investigation or discussion of their error. They just “fix” it in one minute and then are done. These fixes are usually errors too.
and EMPMs:
How did that error happen? What caused it? What thinking processes were used and how did they fail? Try to ask “Why?” several times to get to deeper issues than your initial answers.
In this case, I think there were a few errors.
- thinking probabilistically and in terms of systems not agents/units. (e.g., what probability does this guy have of a left or right swipe) – this is a pattern of mine.
- not recognizing decisiveness correctly. (treating something called non-decisive as non-decisive, instead of judging how it is applied)
- thinking there was prioritization (this seems like a ‘fix’ for the above point)
I also think I got one thing right:
- I realized that factors which were not called decisive were in some way decisive because they were applied (though I was wrong about the nature of the application)
E1:
I think this is two errors.
The first is that I often default to thinking about how I’d simulate a system as a method of understanding it. Analyzing or running such a system often involves probabilities (e.g., choosing what proportion of people apply a factor X). However, we’re not talking about a big simulation, we are talking about a single person. There is no probability distribution of outcomes, there are just two answers: 0 or 1; swipe left or not swipe left. In some sense I confused the simulation for the test that the simulation would run each trial.
The second is resorting to probabilities and distributions as a shortcut instead of thinking through the whole process. This (I think) often yields approximately correct conclusions, at least for some range. The problem is that I’m using tools built around probability to think about boolean intersections.
Another error (that I thought of now) around this kind of thing is that I’m trying to go from probability stuff → boolean stuff in a way that doesn’t work. Partly I think this is due to the other way kind of working. That is, if you have a lot of boolean processes in parallel that are a bit random in some way, you can get some kind of distribution or cumulative distribution graph out. Then, I’m taking the idea of a CDF and working backwards to apply it to the original thing, instead of going back to first principles and applying the original boolean logic directly. (I’m not sure this played a role in my error, but it seems related to the other two)
E2:
I think this one is a bit simpler, I recognized that some kind of decisive factor was being applied, but didn’t properly identify that it was a decisive factor. This is related to E3.
E3:
Because I didn’t label the first factor as decisive, I needed a way for it to become decisive, hence introducing prioritization.
While, often, when physically evaluating something for meeting goals we will evaluate them in an order, the order itself doesn’t matter for passing all the goals (this is just basic boolean logic; commutativity).
This error is two fold: first, it results from E2 as a necessity for the construction, and second, it was allowed to occur because I did not detecting and address E2 (and thus made something up instead).
Okay, so maybe E2 isn’t as simple as I thought.
Why didn’t I recognize it as decisive? In part because it was called not decisive. I did recognize something about it though, because earlier I was talking about looking at what people do not what they say. I knew it mattered for the outcome.
I’m not sure if this is just simple or I’m stuck.
Hmm, I’m not sure, but I would now change how I’d respond to anon105’s quote:
Instead, I might now say: whether they are decisive factors or not depends on whether the woman in question would reject someone (swipe left) based on them. Even if she is doing this for load management (too many possible matches), they become decisive because she is applying them decisively (swiping left).
Though this is kinda how I corrected it before.
Why talk about probabilities and order at all?
One thing I was thinking about is strategies to apply at various levels of filtering. So say you start with 1000 candidates and have some filter X, after which there are 200 candidates left. Since 200 is too many, and if X is expensive or slow, then you might introduce a second filter Y that is faster. I was thinking there was a difference between applying Y before or after X. (There might be an overall time difference, but the set of resulting candidates will be the same.)
The reason I thought there was a difference was because I actually had a different Y' that I was using in one of the situations. Like if Y is facial attractiveness, one might set a high bar Y when it is applied first (to thin out candidates quickly), and use a lower bar Y' when it is applied afterwards (most of the hard work is done, so you can use a more generous filter to end up with more potential matches). So it looks like order matters if I confuse Y and Y', but actually there were just 2 different conditions.
So in summary:
- I incorrectly generalized the situation and prematurely optimized (confusing
Y and Y' in the process)
- I incorrectly reduced the generalization back down to (what should have been) a boolean case, failing to recognize
Y and Y' were different, and that the ‘non-decisive’ factor was applied decisively (since it resulted in rejection).
- I incorrectly kept some ideas around probabilities / statistical distributions from the generalization, which made them a ‘natural’ example, and overcomplicated things. Instead I should have recognized that it’s simpler to talk about boolean intersections in terms of boolean operators, not via a complex chain of functions on probability distributions.
It’s a bit taxing to think about so I’ll leave it there for now. I think I made some progress understanding how I went wrong.