Single Responsibility Principle for the Human Mind
This is about an engineering order for human minds, known elsewhere as the single responsibility principle.
Double purposes of the same module of a person’s mind lead to portions of their efforts canceling the other effort out.
Imagine you’re a startup CEO and you want to understand economic feasibility to make good decisions, but you also want to make investors believe that you are destined for success so you can get their money whether or not you are, so you want to put enthusiasm into your voice…
…so you’ve got to believe that your thing is a very very good idea…
When you are deciding to set the direction of product development, you might be more in contact with the “track-reality” purpose for your beliefs, and therefore optimize your beliefs for that, and optimize your belief-producers to produce beliefs that track reality.
When you are pitching to investors, you might be more in contact with the “project enthusiasm” goal, and therefore optimize your beliefs for that, and optimize your belief producers to produce beliefs that project enthusiasm.
In each case, you’ll be undoing the work you did before.
In a well-ordered mind, different “oh I made a mistake there, better adjust to avoid it again”s don’t just keep colliding and canceling each other out. But that is what happens if they are not feeding into a structure that has different spaces for the things that are needed to be different for each of the goals.
Self-deception for the purpose of other-deception is the loudest but not the only example of double purposes breaking things.
Buckets errors cause a similar clashing-learning thing, too.
Maybe you can notice the feeling of clashing learning? Or just the state of having gone back and forth on an issue several times (how much you like someone, for instance) for what don’t seem like surprising reasons in retrospect.
Part of my account of why never in practice self-deceive in order to get the best consequences, is: better to optimize separately having true beliefs and doing good things with beliefs than just doing-good-things with what the intermediate beliefs are as a free parameter. Because of the usual reason to separate modules. So that there is a place to put all the learning. So that you can finish off in less time a sub-part of the problem of designing the system and reduce the state space you’re exploring while still leaving yourself enough freedom to finish optimally.