Continuing the learning journey
With this article, we are reaching the end of this course. We tried to teach you concepts we consider fundamental and which will help you to continue the learning journey by yourself. In this article, we want to show you some resources that can help you to deepen your understanding of shape modelling as well as to put to practical use the things you have learned.
Gaussian Processes
We used Gaussian Processes as our fundamental modelling technique but could only show you a small part of what Gaussian Processes can offer. You can find a wealth of literature about Gaussian Processes in the field of statistics. In this course, we have used Gaussian Processes to represent spatial data. This type of Gaussian Processes is usually referred to as Gaussian random field in the statistics literature. An introduction to Gaussian random fields, with additional pointers, is given in this review article:
Abrahamsen, Petter. A review of Gaussian random fields and correlation functions, 1997.
Another area where Gaussian Processes proved to be very useful is machine learning. An excellent introduction to Gaussian Processes and their use in machine learning is given in this free online book:
If you would like to acquire an understanding of the mathematical structures associated with Gaussian Processes and their connection to kernel methods and differential equations, we strongly recommend the following article:
Steinke, Florian and Schölkopf, Bernhard. Kernels, regularization and differential equations, 2008.
Shape modelling
The approach to shape modelling that we have presented in this course is summarised in our paper
Lüthi, Marcel, et al. Gaussian process morphable models, 2017.
We have applied our approach to many modelling problems. The most complex example is the construction of the Basel Face Model. We have open sourced the complete model construction pipeline as well as our code for analysing 2D face images. This pipeline, together with the accompanying paper, should serve as a good example of how the methods you learned in this course can be applied to challenging modelling problems.
Our approach is by no means the only possible way to model shapes. There are many other approaches, which differ in the way the shapes are represented or use different mathematical tools for modelling.
Many modern shape modelling methods are described in this book:
Zheng, G. et al. Statistical Shape and Deformation Analysis, 2017.
The excellent review article by Tobias Heimann et al. also provides a good summary of shape modelling methods with many pointers to the relevant literature:
Long before statistical shape modelling became a popular tool for image analysis, statistical shape models were studied as a branch of statistics. A good introduction to the statistical aspects of shape modelling can be found in the following classical book:
Dryden, Ian L., and Kanti, V. Mardia. Statistical shape analysis, 1998.
Model-based image analysis and pattern theory
In this course, we have shown you how you can perform model fitting using a simple iterative algorithm. The problem of fitting would deserve much more attention and although these simple algorithms can get you started, more sophisticated methods are needed to successfully apply model fitting to image analysis. We refer again to the paper by Heimann and Meinzer for a overview of different methods used in model-based image segmentation:
The problem of model-based image analysis can be seen as a special case of the pattern analysis problem. Pattern theory provides a very general framework for the description of patterns based on probabilistic models that are then fitted to the data using an analysis-by-synthesis approach. We recommend the following book by D. Mumford and A. Desolneux, but remark that the mathematical prerequisites are much higher than what has been required in this course:
In our research group we are following this general analysis-by-synthesis approach. Specifically, we have implemented a fully probabilistic fitting framework using Markov Chain Monte Carlo methods, which we also made available as part of Scalismo. You can find teaching materials on this part, including theory-videos and software tutorials on our website.
Scala
If you continue working with Scalismo, it is essential that you learn more about the programming language Scala, as you would eventually like to use Scalismo as a library that can be used in your own applications. There are many good books on Scala available. We particularly like the book:
Welsh, Noel and Gurnell, Dave. Essential scala, 2011.
Another good introduction to Scala:
Haoyi, Li. Hands-on Scala, 2020.
whose first chapters covering the basics of the language are freely available on the book website.
Software packages
Shape modelling is very much an applied discipline and software plays an extremely important role. I hope we could convince you that a system like Scalismo, which can directly visualise what is going on, is of great help.
Note that the Scalismo version used for this course (v0.10) is not the latest version. This is due to the fact that version release speed of software is rather high, which would require shooting new tutorial videos frequently. On the Scalismo webpage you will find the tutorial articles that you worked on during this course translated to the latest version of Scalismo. The tutorials will also help you to switch to Scalismo as a library, which we recommend when you are working on real projects.
We have also released the software library scalismo-faces, which specifically targets the application of face image analysis. It contains a complete computer graphics pipeline and complements scalismo with functionality for handling color mehes and 2D color images. It features an interactive tutorial, to get you started developing your own computer vision applications.
If you prefer to use R instead of Scala, you can use the package Morpho together with RvtkStatismo. These packages together offer all the functionality that we covered in this course as well as great visualisation capabilities.
Data
Maybe the most important part in shape modelling is a set of representative example data from which the shape variations can be learned: An ideal data set for shape modelling is a set of faces. Besides being great fun, modelling faces also lets us understand and visually assess the plausibility of the modelled shape variations. The Basel Face Model is a face model built from a set of 200 face scans. Besides the statistical model for shape, the Basel Face Model also contains a color and an expressions model. Among many other applications, you can use it for generating realistic face examples for your own shape modelling experiments. It can be downloaded and used freely for academic and non-commercial purposes.