Strong perception

One way of thinking about perception is based on weak anticipation. The idea is that you take in data about the world around you, feed that into your internal model, and use that to produce some picture of the outside world. Call this “weak perception”.

Another way is based on strong anticipation. This kind of perception is about the organism coupling to the environment so that their interactions with it directly perceive information. When a baseball fielder chases a fly ball, they anticipate its trajectory by maintaining a visual relationship with it, not by computing a physics model. There’s no internal model, just direct coupling. As Karl Friston observed, “An agent does not have a model of its world – it is a model.” Call this “strong perception”.

In the ecological approach to perception and action, strong perception is called direct perception. Direct perception is about lawful interaction. When you sit in a chair, your body couples to the chair because of how lawful local interactions within the body, within the chair, and between the body and the chair, as well as other things like gravity, scale across the body-chair system. Sight as direct perception is enabled by the lawful interactions between light and the surfaces it passes over, meaning that the light that hits your eyes is structured in a way that precisely specifies the things you’re trying to look at. Without this structure, the fielder wouldn’t be able to reliably catch the baseball by maintaining a visual relationship with it.

Strong perception can be considered a generalization of direct perception if we extend it to economically lawful interactions. An example comes from the price system. Prices are perceptions in the sense that they are information you can use to form plans about how to navigate the world. But they’re organized by economic regularities rather than not physical laws. Prices emerge from the interactions of supply and demand. When you use prices to perceive the world, you’re peering into an economic problem space rather than a physical one—how to spend your budget optimally.

Indeed, strong perception is all about action-cognition, specifically how to reorganize your body in response to novel information. This reorganization is what makes the perception actual perception—strong perception naturally ignores all information consistent with its “prior”, or current dynamical trajectory. (I.e., such information produces no reorganization signal; the system’s dynamics effectively decide that the “information” isn’t informative.)

This process of reorganization via perception (really, reorganization as perception) is enabled by a cognitive glue. It transforms novelty into signals that the parts of the system are capable of dealing with, such as how the price system turns novel events into price updates that humans know how to deal with by updating their shopping decisions. Novelty that a cognitive glue can’t absorb is an externality.

So strong perception signals don’t end up telling you what’s going on in the world—that’s not the point. Instead, strong perception signals, like prices, end up tying a bow between you and the rest of the world so that you can use them to coordinate. These bow ties encapsulate the dynamical history of the relevant part of the world in a way that is suitable for being exploited to determine the optimal reorganization. For example, traffic lights are strong perceptions because you know based on your experience in a traffic-based society that it’s optimal for you to stop on red and go on green. This allows you to coordinate with a huge mass of other drivers just by looking at some lights and making decisions based on your own local circumstances. The meaning traffic lights have for action aren’t lawful so much as regular—another society could have red lights mean go and green lights mean stop—which is still sufficient for strong perception.

This kind of collective coordination based on strong perception doesn’t even require shared goals. Another kind of strong perception is marginal value. Sports offenses and defenses coordinate around points where the marginal value of scoring is highest—the offense wants to go there to score, and the defense wants to go there to stop them. The result is that teams with directly opposed goals coordinate around a shared value, the marginal value of a scoring attempt from that spot.