paraafPeter A. van der Helm Demo link

About Teaching Research Publications Presentations

Structural information theory and its applications


In the 1960s, Emanuel "Maan" Leeuwenberg initiated structural information theory (SIT). SIT began as a quantitative coding model of visual pattern classification (demo), which, in interaction with empirical research, developed into a competitive theory of perceptual organization. In object perception, SIT proposes a Bayesian integration of viewpoint-independent and viewpoint-dependent factors quantified in terms of descriptive complexities (demo). Furthermore, nowadays, SIT includes empirically successful quantitative models of amodal completion (demo) and symmetry perception (demo), and a neurally plausible model of neuro-cognitive processing and human cognitive architecture (demo).

Central to SIT is the simplicity principle, which holds that the simplest interpretation of a stimulus is the one most likely to be perceived by humans. To make predictions, candidate interpretations are represented by symbol strings, and the string with the overall simplest code is taken to specify the preferred interpretation. A simplest code is a symbolic representation that enables the reconstruction of the stimulus using a minimum number of descriptive parameters. It is obtained by capturing a maximum amount of regularity and yields a predicted hierarchical organization of the stimulus in terms of wholes and parts.

Emanuel Leeuwenberg
Emanuel Leeuwenberg

See Acta Psychologica 2003 for a special issue in his honor, on the occasion of his 65th birthday.

Historically, SIT's simplicity principle is a descendant of Hochberg and McAlister's (1953) minimum principle. Both reflect Occam's razor and both are modern information-theoretic translations of the law of Prńgnanz. This law had been proposed by the godfathers of Gestalt psychology — Wertheimer (1912, 1923), K÷hler (1920), and Koffka (1935) — and refers to the natural tendency of physical systems to settle into relatively stable states defined by a minimum of free-energy. In the case of the human visual system in the brain, the idea then is that preferred interpretations correspond to such states. Hochberg and McAlister formulated this Gestalt idea in terms of representational compactness by replacing minimum free-energy by minimum descriptive complexity. SIT added a concrete coding language specifying the so-called transparent holographic regularities to be captured in descriptive codes. For the mathematical foundation of SIT's coding language, see Journal of Mathematical Psychology 1991, and for the psychological foundation of the transparent holographic nature of visual regularity, see Psychological Review 1996 and Psychological Review 2004.

SIT's simplicity principle concurs with the minimum description length principle in the mathematical domain of algorithmic information theory (AIT), which, also in the 1960s, was initiated by Solomonoff (1964ab) and Kolmogorov (1965). Until the 1990s, SIT and AIT evolved independently, and indeed, there are differences between SIT and AIT:
For the rest, however, SIT and AIT share many starting points and objectives. SIT's and AIT's modern information-theoretic approaches can be said to present viable alternatives to Shannon's (1948) classical information-theoretic approach and to the classical Helmholtzian likelihood principle which, in vision, assumes that the preferred interpretation of a stimulus is the one most likely to be true in the world, that is, the one with the highest probability of specifying the actual distal stimulus. Just as Shannon's approach, the Helmholtzian likelihood principle presupposes knowledge about probabilities in terms of, for instance, frequencies of occurrence of things in the world. However, such probabilities are often hardly quantifiable, if at all (demo). In SIT and AIT, this problem is circumvented by turning to precisals, that is, artificial probabilities derived from the length of shortest descriptive codes under the motto: simpler things get higher precisals. AIT showed that, in inductive inference, precisals might well be reliable alternatives to the often unknown real probabilities.

In vision, the latter paved the way for a more detailed Bayesian comparison of SIT's simplicity principle and the Helmholtzian likelihood principle (demo). This comparison revealed that precisals and real probabilities might be far apart for viewpoint-independent factors (Bayesian priors) but seem close for viewpoint-dependent factors (Bayesian conditionals). The latter factors are decisive in everyday perception by moving observers, which implies that both the simplicity principle and the likelihood principle may have guided the evolution of the human visual system — the difference being that the likelihood principle assumes that the human visual system is a special-purpose system in that it is highly adapted to one specific world, whereas the simplicity principle assumes it is a general-purpose system in that, by way of emergent property of its preference for simplest interpretations, it is fairly adaptive to many different worlds. For an extensive discussion on these issues, see Psychological Bulletin 2000, and for an updated brief discussion, see Acta Psychologica 2011.

Furthermore, in Marr's (1982/2010) terms, SIT began as a theory at the computational level of description (demo). Just as Bayesian implementations of the likelihood principle, for instance, SIT modeled vision as if it, somehow, considers all possible interpretations before selecting a preferred interpretation. In both cases, this involved modeling competence (what is a system's output?) rather than performance (how does a system arrive at its output?). Nowadays, however, SIT also includes process models. For instance, see Psychological Review 1999 for the so-called holographic bootstrapping mechanism in the detection of visual regularity (demo), and see Proceedings of the National Academy of Sciences USA 2004 for the so-called transparallel processing mechanism in the selection of simplest codes of strings. This transparallel processing mechanism relies on so-called hyperstrings (demo), which are special distributed representations that enable hierarchical recoding of up to an exponential number of strings as if only one string were concerned (demo). This differs from standard parallel distributed processing (PDP, demo) in two crucial ways:
These computational findings suggest an intriguing connection between transparallel processing (a form of cognitive processing) and neuronal synchronization (a form of neural processing that differs from standard PDP). That is, hyperstrings might well correspond to transient neural assemblies which, like hyperstrings, are involved in binding similar features in the input and which signal their presence by synchronous firing of the neurons involved. This connection has been explored in Cognitive Processing 2012, leading to a concrete picture of flexible cognitive architecture (constituted by hyperstring-like "gnosons" or "fundamental particles of cognition") between the relatively rigid level of neurons and the still elusive level of consciousness (demo). This picture, or the transparallel mind hypothesis, provides a neurally plausible alternative to Penrose's (1989) quantum mind hypothesis (see Artificial Intelligence Review 2015).

For books on these ideas, see Structural Information Theory and Simplicity in Vision
For more details on methodological principles guiding these ideas, see Marr's levels, Research cycles, and Metaphors of cognition


The conglomerate of ideas within SIT has found societal applications in art science and visual ergonomics. This led, for instance, to traffic reconstructions yielding safer roads, bridges, and tunnels (demo). Furthermore, during the past decades, it has been applied to a wide range of topics in vision science, including: