We all know what it’s like to make choices from a first-person perspective, but to recognize unconventional intelligences in unconventional embodiments, we need a third-person theory of choice. A third-person theory of choice is one we could apply even to systems that we find very challenging to understand what it might be like to see the world from their perspective.
In economics, a choice is a selection of an element from a set of alternatives. Rational choice refers to the ability to consistently rank the alternatives by preference and then select the most preferred alternative. The search for (rational) choice in unconventional intelligences is the search for this process.
Even humans are unconventional intelligences, so we should try to find this process in humans. Evidence that humans undergo this process is found by studying motor behavior. I previously described a hypothesis where rational choice as a selection of the best of several alternatives is achieved by constructing several alternatives and having them compete with each other until one wins out. In this model, a motor behavior such as an act of reaching is constructed by having several different possible ways of constructing that reach compete with each other internaly, with the winner propagating across the brain and body while the losers are inhibited.
But another possibility is that the winning choice isn’t selected for among the present alternatives but is instead constructed by them. In this model, the various alternatives are making a social choice about how to proceed as a group. They do this by constructing a governing “dictator” whose choices constitute the group choices. The result is not a simple aggregation of the present alternatives but a novel construction that they build together through their interactions with each other. This novel governing construction is what you experience as your choice.
Just like thoughts are thinkers, this model suggests that choices are themselves choosers. The neural signals and so on that seem to constitute our choices are agents that are trying to optimize for their own self-interest, and the plans of the other signals are both obstacles and affordances. Conflict is very costly, so the choices try to work together to achieve a group decision, and that group decision, if it is to be rational, requires constructing a greater regulating entity whose decisions are shapings of the energy landscapes that allow the member choices to pursue their own goals without coming into conflict with the goals of other choices, and indeed to be able to exploit each other’s activities so as to scale up goals and competencies. Perhaps this scaling up is necessary to explain how simple neural signals become complex, context-sensitive, dynamic motor behaviors that change intelligently over time to achieve existing goals and adapt to new ones on the fly.
Which if either of these models is true I do not know. Possibly each happens some of the time. It will be interesting to find out.
What do you think of the preference reversal phenomenon in economics? I take it that you don’t see it as accurate to think of us as having static, predefined preferences that we simply draw upon when making a decision. But then, what’s the alternative? I’m still trying to grasp what you mean here.
Is the basic idea from embodied cognition that we come to know what we want based on our interactions with the environment? That is, we only form a "preference" as we move to complete a goal and, in the process, discover what our preference is? So, instead of having a stored, static preference, our preferences emerge dynamically as we engage with a particular context or task?