Unsupervised hyperspectral image segmentation with the Mumford-Shah model and motif extraction from crystalline images

Benjamin Berkels
RWTH Aachen University

This talk consists of two parts. The first part is concerned with the segmentation, i.e. decomposing an image into disjoint regions that are roughly homogeneous in a suitable sense, of hyperspectral images in an unsupervised manner. In such images, a full spectral band of data is collected at each pixel by sampling the spectral space with a very high resolution. A particular challenge of hyperspectral data is a high spectral variability that causes the spectral signatures of pixels of the same constituent to vary strongly in some channels.
The basis for our hyperspectral segmentation approach is the famous Mumford-Shah functional, a variational approach to the problem. One of its core components is the chosen indicator function that quantifies how well a pixel fits into a segment. We propose a new indicator function based on estimates of the means and the covariance matrices of the segments, resulting in an anisotropic non-squared version of the Euclidean norm. A regularization of the estimates of the covariance matrices ensures the feasibility of our method. Moreover, the high dimensionality and the noise in the data are tackled by applying the minimum noise fraction transform as a preprocessing step.

The second part of the talk considers the analysis of atomic resolution crystalline images, typically acquired using transmission and scanning transmission electron microscopes ([S]TEM). Here, we propose a novel method for the automatic motif extraction in real space from such images based on a variational approach involving the unit cell projection operator. Due to the non-convex nature of the resulting minimization problem, a multi-stage algorithm is used. First, we determine the primitive unit cell in form of two lattice vectors. Second, a motif image is estimated using the unit cell information. Finally, the motif is determined in terms of atom positions inside the unit cell.


Back to Workshop II: Mathematical Advances for Multi-Dimensional Microscopy