There's no such thing as terminal values

Claude’s simplification can be found here.

Values can presumably be decomposed into terminal values, which are the things you really want, and instrumental values, which are the things that help you get the things that you really want. For example, you might have a terminal value of survival and corresponding instrumental values of finding food and shelter.

Despite this intuitive framework, there is no evidence for the existence of terminal values, and no theoretical need for them to exist. Instead, all values are instrumental, serving to preserve preferences, which are distinct from terminal values, and which themselves have no core or essence analogous to terminal values.

Briefly, regarding the evidence for terminal values, no one has seen a terminal value or knows how to identify one in principle. As a practical matter, it may be helpful in some circumstances to conceive of a value such as survival as being terminal, but this proves nothing: As a practical matter, it may be helpful to think of the Earth as flat when traveling short distances. No empirical test for the terminal nature of a value has ever been proposed or could be conceived of.

The latter claim can be inferred from economics. In economic theory, people act to achieve their values. However, these values are not terminal. No definition of a terminal value in economics has ever been offered. This is because the concept of terminal value would not be useful to economics in any way. Since economics is the science of the implications of the existence of values, and there are no implications for the existence of terminal values, it follows that terminal values are as useless to science as the idea of a fairy that makes everything obey the laws of physics; i.e., terminal values have no testable implications.

So what are values, if they’re never terminal? Why do people do what they do? What do we really want or care about?

I prefer to think of values as measurement. A measurement is an interaction between two structures, a measuring structure M and a measurable object O, such that M transforms in a way that predicts structure about O. (I.e., M transforms in an efficient, structure-preserving way.) For example, a mercury thermometer works by having a quantity of mercury, which is our M in this case, exchange energy with, say, a room in a house, O, such that the way amount that the mercury expands or contracts reveals the structure of the average kinetic energy of the particles in the room. The resulting measurement, e.g., 22 degrees Celsius, is the value of the measurement, or the value that was measured. We normally use the word “value” in this context, so it’s reasonable to see if it generalizes to other contexts such as people’s desires.

(Note the inherently collective nature of measurement—it requires a system of parts interacting in efficient, structure-preserving ways to produce measurements/values.)

Based on the idea of values as measurements, values can be thought of as transformations: ways that a system (which we called M in the context of measurement) efficiently transforms to preserve some of the structure of the system it interacts with (which we called O previously). A system would preserve the structure of the system it interacts with in order to preserve some of its own structure, in a manner exactly analogous to how an economic agent would preserve some of the structure of the other economic agent they are bargaining so as to preserve some of their own structure—e.g., they would pay a price that allows the other economic agent to preserve their plan to sell some of their apples so as to preserve their own plan to buy some of their apples. In this context, the values of the system M are its measurements of the system(s) it interacts with, which in equilibrium are the parameters that determine how M will transform in the future given efficient, structure-preserving interactions. I.e., values are Bayesian priors.

Associating values with Bayesian priors might be counterintuitive, but it lets us bring the powerful lens of the free energy principle to bear on economics. According to the free energy principle, people behave by optimizing how their priors change as they interact with their environment, trying to learn as much as possible while losing as little internal structure as possible. By substituting the word “values” for the word “priors”, we have the familiar economic claim that people act to get as much as they can while giving up as little as they need to, optimizing the tradeoff between the values they gain and the values they lose.

In the ordinary sense of the word “value”, it’s strange to think about giving values up, particularly if we believe that some of those values are terminal, like survival. However, people routinely choose to trade off seemingly terminal values such as survival by doing dangerous things like driving, playing sports, or eating unhealthy foods. We can easily explain this by substituting the word “prior” for the word “value”, in which case we have the claim that priors should not be fixed. Keeping one’s priors constant means being unable to learn, which would be very bad, so priors, and equivalently values, must be able to change, which means giving up some beliefs (values) to gain others.

Values are traded off against each other to preserve internal structure. “Preserve” in this context means that values trade off to produce something that is mathematically a structure-preserving transformation, and does not rule out, e.g., the substitution of artificial organs for organic ones. Internal structure refers to preferences, i.e., the system that constructs the preference order in a situation that elicits it, that is, internal structure or preferences means the internal system of relationships that determines optimal tradeoffs between values in a given situation. “Preferences” is the appropriate term because in any case, there are conceivably many ways a system can transform to be optimal; the internal factors that bias the transformation to a particular outcome or class or range of outcomes are the system’s preferences. To preserve preferences, therefore, is to optimize tradeoffs between values such that the internal structure of which the values are parameters—i.e., the measurements the internal structure makes of the environment it interacts with—updates, changes, or transforms in a structure-preserving manner.

What in particular is preserved of the internal structure? The answer, perhaps surprisingly, is nothing: There is no core or essence that a person, organism, or other complex optimizing system tries to preserve at all costs. Just as there are no terminal values, there is no core or essential properties that must be preserved no matter what transformations take place. For example, evolution transforms species over time. Any particular mutation changes only a small part of the species. Over a long enough period of time, however, nothing is sacred: anything and everything about the species can be transformed.

Examples of core preferences do exist in artificial contexts. In chess, for example, the king has infinite value; it is always worth sacrificing every other piece if necessary to defend ones own king or capture the opponent’s king. However, this is only true within the limitations of a formal description of chess that does not describe the reality of the empirical system. A chess player usually does not physically assault their opponent in order to protect their king because doing so would mean giving up other things, and they do not really place infinite value on their king. Also, a chess player who plays fun or interesting moves on the board rather than the ones that they genuinely expect to do the best job of preserving their king is not necessarily playing chess incorrectly, showing that the king does not have infinite value even in terms of making chess moves.

It may seem odd to think that although you try to maintain consistency over time, you do so by giving up aspects of yourself, and there is no core part that makes you truly “you” or anything along those lines, such that preserving that part makes you behavior correct somehow. However, if there was something about your internal structure that you could never surrender under any circumstances, it would be very difficult to survive and thrive. More to the point, it is very weird mathematically to envision such a core that cannot be allowed changed by a succession of mathematical transformations, except for the “core” of logical consistency/scarcity.

Relatedly, the closest thing to a “core” that is preserved is the efficient allocation of scarce internal resources. The brain’s job is to maintain an efficient internal balance with respect to the environment so as to regulate the body’s use of energy. Just like a business needs to regulate its budget so that it spends enough to achieve its goals without spending so much that it runs out of money, the brain needs to regulate the internal “body budget” to achieve an efficient form with respect to the environment by optimally allocating internal resources.

This is all quite abstract, so consider a simple example of eating a meal when hungry. Naively, we might consider this as involving values such as “survival”, “satiating hunger”, “tastiness”, etc. However, from an economic point of view, values should predict behavior, and we can’t use these vague values to predict anything beyond general classes of outcomes like “they will eat food they like to eat.” To make more granular and precise predictions, we can associate values with every measurable parameter of the person. Under the rationality assumption, these values are the results of transformations that brought the person into an optimal relationshp with their enviornment. As the internal state of the person and their external environment continue to change, the parameters therefore change as well to become optimal under the anticipated internal and external transformations, which to a human observer appear as movements such as a mouth opening, a hand moving, etc. As long as the person remains rational, acting to “achieve their values”, i.e., undergoing changes to make their parameters optimal, will successfully map their internal structure onto a goal state. This internal structure is a model, and the efficient, structure-preserving motion of this model is what we call the choice made by the person. In this view, values are embodied and not inherently mental.

The idea that there are no terminal values or core preferences resolves age-old questions as to what people truly care about or desire by revealing that those questions contain a false premise. For example, when we consider transformations of ourselves and our environments brought about by technological change, there is no answer the question, “What do we truly want?” or even “What does that particular person truly want?” beyond the general goal of allostasis, or internal economic efficiency, which in principle can take any form, sacrificing any value or values in the process.