The goal of this workshop is twofold. First, it aims at acquainting
the audience with the role of realization theory in
system identification of linear systems. This is done by presenting a
number of results, both classical and recent, on realization theory of
linear systems and its
applications to system identification. Second, it aims at presenting
realization theory and its applications for other classes of
systems: finite-state stochastic systems, hybrid systems, LPV systems
and some classes of non-linear systems.
The workshop is thus intended to be tutorial in nature: its goal is to
present the basic ideas in a clear and accessible fashion, and
to give a glimpse of various application domains.
The motivation for the workshop is the following. Realization theory
plays an important role in system identification, but this role is a
technical one and for this reason
it is not always visible. As a result, young researchers are often not
very familiar with the topic. In turn, this makes it difficult for them
to use
realization theory to analyze system identification methods or to come
up with new one, based on realization theory. In addition, when attempts
are made to
develop system identification methods for systems which are not linear,
for example LPV or hybrid systems, many of the standard facts, which
are taken for granted for
linear systems, need not remain true.
Hence, in order to be able to decide which assumptions are
meaningful and to be able to come up with a theoretical analysis of
system identification algorithms, realization theory for these new
classes of systems is needed.
The workshop addresses both issues, by providing a tutorial overview of
classical linear realization theory and by giving an overview of
realization theory for other system classes.
full-day.
The workshop targets researchers in control theory, especially in
system identification, who would like to get acquainted with
realization theory and its application to system identification. The
workshop could especially be useful for young researchers, who
might not have been exposed to the topic before.
The workshop assumes no more than the standard knowledge of classical linear control theory. However, a certain level of
mathematical maturity and some degree of familiarity with system identification is an advantage.
The topic of the workshop is realization theory of various classes of systems and its role in system identification.
The workshop will start by a brief review of basics of realization theory of linear systems, both in deterministic and
stochastic settings. Most of the members of the audience are likely to be at least partially
familiar with these results. This short introduction will be followed by more specialized lectures.
The lecture by Anders Lindquist will deal with stochastic partial
realization theory and its relationship with subspace
identification methods. Subspace identification methods are usually
based on partial realization theory, however, they
often ignore the condition of positive realness imposed by the
stochastic realization problem. Instead, they treat the
problem as a deterministic partial realization problem. As a result,
these algorithms may perform very poorly for certain systems. The talk
will explain
this problem in detail and discuss examples when subspace
identification fail due to this problem.
This talk demonstrates the importance of realization theory for
subspace identification algorithms.
The lecture by Bernard Hanzon will present the application of
realization theory to parametric system identification. It will present
the
smooth Riemannian manifold structure of equivalence classes of minimal
linear systems, their local charts and their relationship with local
canonical form, and the relationship
between parametric system identification and this manifold structure.
Parameterization and parametric identification of lossless systems will
also be discussed. The lecture
will conclude by presenting future research directions and the
relationship with related disciplines such as machine learning and data
analytics.
This talk demonstrates the importance of realization theory for
parametric identification of state-space representations.
The lecture of Alessandro Chiuso will deal with application of
realization theory to system identification methods based on
regularization. Regularization is a standard approach in machine
learning. The talk will explain how to combine realization theory
in general, and the concept of Hankel matrices in particular, with
regularization, in order to derive new
identification algorithms for MIMO systems. The latter algorithms turn
out to be competitive with the existing algorithms,
and sometimes they are better than the state-of-the-art. This talk
demonstrates the relevance of realization
theory for new developments in system identification.
The lecture by Jan H. van Schuppen provides an overview of realization
theory of finite-valued and countably valued
stochastic systems, i.e. stochastic systems the state and output
process of which take values in a finite (respectively countable)
set.
The talk will cover both the discrete-time and the continuous-time
case.
Such stochastic systems include finite-state hidden Markov models and
counting processes, and they are widely used
in information theory, communication theory, signal processing,
queueing theory, and control engineering. Such models are also
useful for approximating highly complex non-linear systems, which are
difficult to handle with other techniques.
Mihaly Petreczky will present an overview of realization theory of
linear switched, bilinear and LPV systems. The relevance of these
results for
system identification will also be discussed. The rationale for
grouping these topics together is that
realization theory of bilinear, switched and LPV systems are closely
related and use essentially the same mathematical tools.
Finally, the talk of Jana Nemcova will provide an overview of
realization theory of nonlinear systems described by
polynomial/rational equations. These systems appear naturally in a
number of domains, for example, they are standard tools for modelling
biochemical
reaction networks. While this theory is not yet complete, it is a
subject of intensive research, and the
existing results are already useful for system identification of these
systems, in particular, for identifiability analysis.
- 09:00 - 10:30 Realization theory of linear systems: a brief review of the classical results (Mihaly Petreczky).
- 10:30 - 11:00 Partial realization theory and subspace identification for time series (Anders Lindquist).
- 11:00 - 11:30 Break.
- 11:30 - 12:30 Realization theory meets Regularisation: an overview of some recent results (Alessandro Chiuso).
- 12:30 - 13:30 Lunch break.
- 13:30 - 15:00 Geometric aspects of system identification (Bernard Hanzon).
- 15:00 - 15:15 Break.
- 15:15 - 16:15 Stochastic Realization of Finite-Valued and of Countably-Valued Stochastic Processes (Jan H. van Schuppen).
- 16:15 - 16:30 Break.
- 16:30 - 17:30 Realization theory of switched linear, bilinear and LPV systems (Mihaly Petreczky).
- 17:30 - 18:00 Realization theory of rational and polynomial systems (Jana Nemcova).
- Title: Realization theory of linear systems: review of the classical results.
Speaker: Mihaly Petreczky (Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL), UMR CNRS 9189)
Abstract:
The goal of this talk is to review some basic notions of realization
theory of time-invariant linear (LTI) systems, thus
preparing the ground for the other lectures. We briefly review such
topics as minimality of LTI systems, Kalman decomposition,
the notion of Markov parameters and Hankel-matrices, Kalman-Ho
realization algorithm.
We will also briefly mention some core notions of stochastic
realization theory, such as the notions of weak and strong realizations,
deterministic realizations of covariance sequences, matrix Riccati
equations and forward innovation representations.
- Title: Partial realization theory and subspace identification for time series.
Speaker: Anders Lindquist (Shanghai Jiao Tong University, Royal Institute of Technology (KTH), Stockholm)
Abstract: This is a tutorial talk reviewing
the connection between subspace identification for time series and
partial realization theory. Most subspace identification procedures are
directly or, more often, indirectly based on partial realization theory.
The appropriate realization problem for system identification is the
rational covariance extension problem. However, most basic subspace
algorithms actually solves a deterministic partial realization problem
where the basic condition of positive realness has been removed. This
may lead to failure for theoretical rather than numerical reasons. In
fact, a sequence of estimated covariance lags has both an algebraic and a
positive degree, and failure may occur when the degrees do not
coincide. Thus we need the condition that the algebraic and the
positive degrees are the same. One can generate data for which this
degree condition does not hold and for which several subspace
identification algorithms exhibit massive failure.
- Title: Realization theory meets Regularisation: an overview of some recent results
Speaker: Alessandro Chiuso (Department of Information Engineering - University of Padova), joint work with Giulia Prando.
Abstract:
In this talk we shall discuss some recent developments in the area of
system identification where regularisation ideas have been used, in
different ways, in conjunction with realisation theory. In particular
the low rank structure of Hankel matrices has been exploited, thus
allowing to apply, and actually extend, recent work on low rank matrix
estimation. This has led to new algorithms for MIMO system
identification which are competitive, and sometimes outperform,
state-of-the-art identification algorithms.
- Title: Geometric aspects of System Identification.
Speaker: Bernard Hanzon (University College Cork), joint work with Martine Olivi and Ralf Peeters.
Abstract:
In this presentation we want to give an overview of the geometric
approach to system identification and related problems such as model
reduction. We want to use Maximum likelihood estimation and ;)
("least squares") model reduction as leading examples. The starting
point is the result (due to JMC Clark) that families of linear
time-invariant systems of fixed finite order are differentiable
manifolds. It can be shown that in most cases they cannot be described
by a unique Euclidean chart of the dimension of the manifold. Therefore
it is natural to use local canonical forms and corresponding local
coordinate charts. In case one employs search algorithms such as
gradient algorithms to find local optima of the criterion function (such
as the likelihood function) one needs to be able to change between
coordinate charts. In order to keep consistency between the gradient
steps taken in the various charts (ideally the steps taken are wholly
coordinate free) one can use Riemannian gradients. These are based on
the concept of a Riemannian metric on the manifold. This will be
explained and some examples will be treated, including the illustrative
example of the case of stable SISO systems of order one (viewed as
submanifold of the space ),
in which the intrinsic geodesic metric is fully understood.
The example also shows some intrinsic difficulties as it turns out there
is no Euclidean parametrization that can describe the neighbourhood of
the origin.
In answer to the perceived problems one can use the idea of Separable
Least Squares and Separable System Idenfication, in which one splits the
optimization problem involved into a family of quadratic, or at least
convex, problems and a "concentrated" optimization problem over the set
of (stable) systems in input-normal form. Actually the optimization can
be viewed as optimization over a associated family of lossless systems.
The geometry of that family is much better behaved.
The family of lossless systems has interesting balanced (this concept
will be explained) local canonical forms that can be parametrized using
only orthogonal matrices and transformations. These are well-behaved
numerically and have a lot of mathematical structure that can be
exploited for various purposes. A very interesting set of local
canonical forms is obtained by imposing a so-called subdiagonal pivot
structure on the matrices involved. This is a very flexible set of local
canonical forms which allow easy decision rules to shift between
charts. Also some remarks will be made about a much larger family of
charts that could be used, and which are based on interpolation theory.
They are especially useful in case one is interested in systems that
satisfy certain interpolation conditions.
If time permits some further applications will be mentioned as well as
related recent theoretical developments. Also we plan to give some
concluding remarks about how the theory may develop further given the
recent advances and interest in related areas such as machine learning
and data analytics.
- Title: Stochastic Realization of Finite-Valued and of Countably-Valued Stochastic Processes
Speaker: Jan H. van Schuppen (Delft University of Technology and Van Schuppen Control Research)
Abstract
The aim of the lecture is to present to the audience of the workshop
a tutorial lecture on stochastic realization
of finite-valued and of countably-valued stochastic processes.
The motivation of the lecture is the use in control and in filtering
of stochastic control systems
with finite-valued and with countably-valued output processes.
Such systems are known as
finite stochastic systems, hidden Markov models,
and counting process systems.
Research areas in which such systems are used include:
information theory, communication theory, signal processing,
queueing theory, control engineering, etc.
In addition, such systems are used as approximations
of continuous-space stochastic systems.
Such an approximation has the advantage
that algorithms for filtering and for control
of finite stochastic systems are available.
The concept of a finite stochastic system with a finite-valued output process
will be defined
both for discrete-time and for continuous-time systems.
The associated concept of a countable-valued stochastic system is defined
in which the set of gamma probability distributions appears.
It will be shown that all these stochastic systems
have both a forward and a backward representation.
The weak and the strong stochastic realization problems
for finite-valued and for countably-valued processes
will be formulated and embedded in a larger class of problems.
The weak problem asks when considered an observed process
for the existence of a stochastic realization as a finite system,
for the minimality of a realization, and
for the classification of all minimal stochastic realizations.
The existence problem has been solved for finite-valued processes
during the 1960's.
It is formulated in terms of the probability distributions of the output process
which have to belong to a polyhedral cone with a finite number of vertices.
The minimality of stochastic realizations is not solved satisfactorily yet
and the existing approaches will be discussed.
For this the concepts of stochastic observability and
of stochastic co-observability are formulated.
It will be shown how to reduce a stochastic system
to a stochastic realization of its output process
which is both stochastically observable and stochastically co-observable.
For the classification subproblem,
the decomposition of positive matrices
into a direct sum of irreducible positive matrices will be used
followed by an investigation of the class of irreducible positive matrices.
For finite stochastic control systems the concepts
of stochastic controllability and
of stochastic co-controllability
will be defined and
its characterizations are discussed.
The weak stochastic realization problem
for countable-output stochastic systems
will be discussed separately towards the end of the lecture.
This case can be related to that for finite-valued processes.
- Title: Realization theory of switched linear systems, bilinear and LPV systems
Speaker: Mihaly Petreczky (Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL), UMR CNRS 9189)
Abstract:
In this lecture we will present an overview of recent results on
realization theory of linear switched, bilinear systems and of linear
parameter-varying
(LPV) systems. These three system classes are closely related:
bilinear systems can be viewed as a subclass of LPV systems, if the
scheduling parameter is viewed as an input, and
linear switched systems can be viewed as LPV systems if the
scheduling parameters take values in a finite set. We will present
results on minimality, with reachability and observability,
Hankel-matrices, realization algorithms and partial realization.
We will explain the relationship with bilinear systems too. We will
mention the consequences of these results for system identification
of linear switched, bilinear and LPV systems
- Title: Realization theory of rational and polynomial systems.
Speaker: Jana Nemcova (University of chemistry and technology, Prague)
Abstract:
The talk will give an overview of realization theory of non-linear
systems described by polynomial and rational equations. Such systems
appear in a wide variety of
different domains, for example, in models of biochemical reaction
networks. Conditions for minimality in terms of
reachability and observability will be presented, along with
conditions for existence of a realization. Uniqueness modulo isomorphism
of minimal realizations will
also be discussed, together with the application of these results to
identifiability analysis of rational/polynomial systems.
Finally, if time permits, recent results on realization theory of the
so-called Nash systems (non-linear systems defined by analytic
semi-algebraic functions)
will also be presented.
The organizer of this workshop Mihaly Petreczky.
Mihaly Petreczky received the Ph.D. degree from Vrije Universiteit
in Amsterdam, The Netherlands in 2006.
In the past, he was a postdoc at Johns Hopkins University, USA (2006 -
2007),
Eindhoven University of Technology, The Netherlands (2007-2009) and
assistant professor at Maastricht University, The Netherlands (2009 -
2011) and at Ecole des Mines de Douai, France (2011 - 2015).
He is currently a CNRS researcher at Centre de Recherche en
Informatique, Signal et Automatique de Lille (CRIStAL), UMR CNRS 9189,
France. His research interests include realization theory, system
identification and model reduction of hybrid and LPV systems.