Companion code to the paper

Title: Practical Bayesian System Identification using Hamiltonian Monte Carlo

Authors: Johannes N. Hendriks, Adrian G. Wills, Brett Ninness and Johan Dahlin

This paper considers Bayesian parameter estimation of dynamic systems using a Markov Chain Monte Carlo (MCMC) approach. The Metroplis–Hastings (MH) algorithm is employed, and the main contribution of the paper is to examine and illustrate the efficacy of a particular proposal density based on energy preserving Hamiltonian dynamics, which results in what is known in the statistics literature as “Hamiltonian Monte–Carlo” (HMC). The very significant utility of this approach is that, as will be illustrated, it greatly reduces (almost to the point of elimination) the typically very high correlation in the Metropolis– Hastings chain which has been observed by several authors to restrict the application of the MH approach to only very low dimension model structures. The paper illustrates how the HMC approach may be applied to both significant dimension linear and nonlinear model structures, even when the system order is unknown, and using both simulated and real data.

This git repository contains code to run the examples included in the paper as well as several additional examples. Examples included in the paper:

• Matlab code for the pedagogical illustration of sampling from a donut shaped target density (paper section 4.2)
• Estimation of an ARX model of known order from simulated data, comparing HMC with other MCMC and ML approaches (paper section 7.1)
• Estimation of an ARX model of unknown order from simulated data, comparing HMC with different priors to ML (paper section 7.2)
• Estimation of an OE model of unknown order from simulated data using HMC and comparing to ML approach (paper section 7.3)
• Estimation of an ARX model of unknown order from simulated data with significant outliers and comparison to an ML approach (paper section 7.4)
• Estimation of a non-linear inverted pendulum model from real data (paper section 7.5)

Additional examples not included in the paper:

• Pedagogical first-order state space example implemented in matlab
• Finite impulse response example comparing different prior choices and the ML appraoch
• Example of estimating a 4th order linear state space model from simulated data and comparison to ML approach
• An example of estimating a 6th order Linear state space model from simulated data and using the posterior samples to determine the distribution of a fast and slow controllers phase margin

• Python 3.7 or greater
• Pystan 2.19 or 2.21
• numpy
• matplotlib
• seaborn
• pandas
• pickle
• scipy

A recent version of matlab with the system identification toolbox

Matlab code to sample from this distribution using HMC is given in

matlab/HMC_donut_example.m

Plots of the results shown below

k=1	k=2	k=3	k=1,...,10

Matlab code to sample from this distribution using MH with a random walk proposal is given in

matlab/MH_donut_example.m

Plots of the results shown below

k=1	k=2	k=3	k=1,...,10

Plots from this code

Estimation of an ARX model of known order from simulated data, comparing HMC with other MCMC and ML approaches (paper section 7.1)

ARX model structure

The following parameters are sampled/estimated

Running the following code performs the HMC sampling and generates corresponding plots using saved data

python example1_arx_part1.py

Alternatively new data and ML results can be generated by running the matlab code

matlab/example1_arx_part1.m

The following diagnostic of the different sampling methods performance is given showing the trace of the a_1 parameter, the autocorrelation of the samples for this parameter and the resulting distributions

Estimation of an ARX model of unknown order from simulated data, comparing HMC with different priors to ML (paper section 7.2)

The same model structure and parameters to be estimated as in the last example

Running the following code performs the HMC sampling and generates corresponding plots using saved data

python example1_arx_part2.py

Alternatively new data and ML results can be generated by running the matlab code

matlab/example1_arx_part2.m

The impact of the different priors on the estimated parameters is compared and we can see that the horseshoe prior has the greatest impact on setting unnecesary parameters to zero

Output regressor coefficients	Input regressor coefficients

The HMC with horseshoe prior estimated systems frequency response is compared to the true system and the ML estimate given using Matlab's arx function

Estimation of an output error model of unknown order from simulated data and comparison to ML approach with regularisation (paper section 7.3)

Output model structure

The following parameters are sampled/estimated

Running the following code performs the HMC sampling and generates corresponding plots using saved data

python example2_oe.py

Alternatively new data and ML results can be generated by running the matlab code

matlab/example2_oe.m

The following diagnostic of the different sampling methods performance is given showing the trace of the a_1 parameter, the autocorrelation of the samples for this parameter and the resulting distributions

Estimation of an ARX model of unknown order from simulated data with significant outliers and comparison to an ML approach with regularisation (paper section 7.4)

The same model structure as in ARX example 1 is used

Running the following code performs the HMC sampling and generates corresponding plots using saved data

python example3_outliers.py

Alternatively new data and ML results can be generated by running the matlab code

matlab/example3_outlier.m

The estimated nyquist plot for the hmc samples and the ML approach are shown below

Estimation of the parameters of a non-linear rotary inverted pendulum from real data collected using the QUBE-Servo 2 of QUANSAR. Refer to the paper for the modelling of this system.

The HMC sampling of this system is computationally intensive and can take several hours to run, hence saved results are included which can be plotted using

python example4_plot.py

The sampling can be rerun using

example4_pendulum.py

Joint state and parameter estimation is performed for this problem. Marginalising over the parameters gives the smoothed state estimates shown below

The marginal parameter estimates are shown below

Some joint parameter and state estimates are given by

A pedagogical example of jointly sampling the parameters and states of a first-order state space model from simulated data implemented in matlab

The system is modelled by

The results are shown below

states	theta

Finite impulse response (FIR) example comparing different prior choices and the ML appraoch

Model structure given by

Paremeters to be estimated

The estimation can be run using

python additional_example1_fir.py

Alternatively, new simulated data can be generated using the matlab script

matlab/additional_example1_fir.m

The impulse of the estimated systems given using

• ML assuming n_b = 13
• ML with TC regularisation assuming n_b = 33
• HMC with horseshoe prior assuming n_b = 33
• HMC with TC prior assuming n_b = 33

Example of estimating a 4th order linear state space model from simulated data and comparison to ML approach

The model structure is given by

Since this sampling can take several hours, saved results have been included and the results can be plotted using

python additional_example2_lssm_plot.py

The HMC sampling can be run using

python additional_example2_lssm.py

New simulated data can be generated using the matlab script

matlab/additional_example2_lssm.m

The parameters to be estimated are A_d, B_d, C, D, Q, R and x_1,...,x_T. An LKJ prior is placed on Q and R

The bode response of the HMC samples, true system and ML estimate are shown below

An example of estimating a 6th order Linear state space model from simulated data and using the posterior samples to determine the distribution of a fast and slow controllers phase margin

The same model structure is used as in the previous example

The HMC sampling can be run using

python additional_example3_sysid.py

The following script plots the results and also prepares them for the matlab script that computes the phase margins of the controllers

python additional_example3_plotsysid.py

The distribution of the fast and slow controller phase margins can be computed and plotted using the matlab script

matlab/additional_example3_controller_phase_margin.m

Alternatively, new simulated data can be generated using

matlab/additional_example3_generate_data.m

The results are shown below for a fast controller with a nominal phase margin of 13 degrees and a slow controller with a nominal phase margin of 26 degrees. Both controllers were designed base on the conditional mean estimate.

Estimated system	Controller phase margin