Flemming Topsoe (University of Copenhagen)

Title:
On the generation of measures of entropy, divergence and complexity
 
Abstract:
 
Often, there is a focus on concepts of entropy. There is nothing wrong
with that, but one must be aware that normally entropy should not stand
alone but be considered alongside with other concepts, especially
divergence and, what I perhaps a bit misleadingly, shall refer to as
complexity, simply the sum of entropy and divergence.
 
A primary goal for the introduction of said concepts is, typically, the
wish to solve specific tasks of optimization, such as they are known from
information theory proper, statistics or statistical physics, to name but
a few relevant areas of application.
 
Appropriate concepts should, ideally, reflect sensible interpretations and
have operational technical properties.  Various methods for the generation
of desirable concepts will be mentioned with a focus on two interrelated
methods, the presently popular method going back to Bregman and then a new
method, which reflects an interaction between "truth" and "belief".
 
Whereas the two mentioned methods relate directly to modelling based on
probability distributions, a more abstract axiomatic approach will also be
discussed. This approach will, for instance, allow the treatment of
geometric problems. As an example, this includes Sylvester's problem, to
determine the point with the smallest maximal distance to a given set of
points in the plane.
 
There are various open problems related to ongoing research in the area.
Some of these will be discussed, possibly limited to just two problems:
The problem of coding interpretations of non-Shannon measures of entropy
and the problem of singling out appropriate topologies in the abstract
setting (this is related to the study of Jensen-Shannon type quantities).