publications
publications by categories in reversed chronological order.
2023
-
Advanced Inference and Representation Learning Methods in Variational AutoencodersIgnacio PeisPhD Thesis Dissertation, 2023Deep Generative Models have gained significant popularity in the Machine Learning research community since the early 2010s. These models allow to generate realistic data by leveraging the power of Deep Neural Networks. The field experienced a significant breakthrough when Variational Autoencoders (VAEs) were introduced. VAEs revolutionized Deep Generative Modeling by providing a scalable and flexible framework that enables the generation of complex data distributions and the learning of potentially interpretable latent representations. They have proven to be a powerful tool in numerous applications, from image, sound and video generation to natural language processing or drug discovery, among others. At their core, VAEs encode natural information into a reduced latent space and decode the learned latent space into new synthetic data. Advanced versions of VAEs have been developed to handle challenges such as handling heterogeneous incomplete data, encoding into hierarchical latent spaces for representing abstract and richer concepts, or modeling sequential data, among others. These advances have expanded the capabilities of VAEs and made them a valuable tool in a wide range of fields. Despite the significant progress made in VAE research, there is still ample room for improvement in their current state-of-the-art. One of the major challenges is improving their approximate inference. VAEs typically assume Gaussian approximations of the posterior distribution of the latent variables in order to make the training objective tractable. The parameters of this approximation are provided by encoder networks. However, this approximation leads to a lower bounded objective, which can degrade the performance of any task that requires samples from the approximate posterior, due to the implicit bias. The second major challenge addressed in this thesis is related to achieving meaningful latent representations, or more broadly, how the latent space disentangles generative factors of variation. Ideally, the latent space would modulate meaningful properties separately within each dimension. However, Maximum Likelihood optimizations require the marginalization of latent variables, leading to non-unique solutions that may or may not achieve this desired disentanglement. Additionally, properties learned at the observation level in VAEs assume that every observation is generated independently, which may not be the case in some scenarios. To address these limitations, more robust VAEs have been developed to learn disentangled properties at the supervised group (also referred to as global) level. These models are capable of generating groups of data with shared properties. The work presented in this doctoral thesis focuses on the development of novel methods for improving the state-of-the-art in VAEs. Specifically, three fundamental challenges are addressed: achieving meaningful global latent representations, obtaining highly-flexible priors for learning more expressive models, and improving current approximate inference methods. As a first main contribution, an innovative technique named UG-VAE from Unsupervised-Global VAE, aims to enhance the ability of VAEs in capturing factors of variations at data (local) and group (global) level. By carefully desigining the encoder and the decoder, and throughout conductive experiments, it is demonstrated that UG-VAE is effective in capturing unsupervised global factors from images. Second, a non-trivial combination of highly-expressive Hierarchical VAEs with robust Markov Chain Monte Carlo inference (specifically Hamiltonian Monte Carlo), for which important issues are successfully resolved, is presented. The resulting model, referred to as the Hierarchical Hamiltonian VAE model for Mixed-type incomplete data (HH-VAEM), addresses the challenges associated with imputing and acquiring heterogeneous missing data. Throughout extensive experiments, it is demonstrated that HH-VAEM outperforms existing one-layered and Gaussian baselines in the tasks of missing data imputation and supervised learning with missing features, thanks to its improved inference and expressivity. Furthermore, another relevant contribution is presented, namely a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM. This approach leverages the advantages of HH-VAEM and is demonstrated to be effective in the same tasks.
-
Variational Mixture of HyperGenerators for Learning Distributions Over FunctionsIn Proceedings of the 40th International Conference on Machine Learning, 2023Recent approaches build on implicit neural representations (INRs) to propose generative models over function spaces. However, they are computationally intensive when dealing with inference tasks, such as missing data imputation, or directly cannot tackle them. In this work, we propose a novel deep generative model, named VAMoH. VAMoH combines the capabilities of modeling continuous functions using INRs and the inference capabilities of Variational Autoencoders (VAEs). In addition, VAMoH relies on a normalizing flow to define the prior, and a mixture of hypernetworks to parametrize the data log-likelihood. This gives VAMoH a high expressive capability and interpretability. Through experiments on a diverse range of data types, such as images, voxels, and climate data, we show that VAMoH can effectively learn rich distributions over continuous functions. Furthermore, it can perform inference-related tasks, such as conditional super-resolution generation and in-painting, as well or better than previous approaches, while being less computationally demanding.
2022
-
Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte CarloIgnacio Peis, Chao Ma, and José Miguel Hernández-LobatoIn Advances in Neural Information Processing Systems 35, 2022Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data. However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations. To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference. Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features. Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM. Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.
-
Unsupervised learning of global factors in deep generative modelsIgnacio Peis, Pablo M. Olmos, and Antonio Artés-RodríguezPattern Recognition, 2022We present a novel deep generative model based on non i.i.d. variational autoencoders that captures global dependencies among observations in a fully unsupervised fashion. In contrast to the recent semi-supervised alternatives for global modeling in deep generative models, our approach combines a mixture model in the local or data-dependent space and a global Gaussian latent variable, which lead us to obtain three particular insights. First, the induced latent global space captures interpretable disentangled representations with no user-defined regularization in the evidence lower bound (as in β-VAE and its generalizations). Second, we show that the model performs domain alignment to find correlations and interpolate between different databases. Finally, we study the ability of the global space to discriminate between groups of observations with non-trivial underlying structures, such as face images with shared attributes or defined sequences of digits images.
2020
- Nature SCActigraphic recording of motor activity in depressed inpatients: a novel computational approach to prediction of clinical course and hospital dischargeIgnacio Peis, Javier-David López-Morı́ñigo, M Mercedes Pérez-Rodrı́guez, and 4 more authorsScientific reports, 2020
Depressed patients present with motor activity abnormalities, which can be easily recorded using actigraphy. The extent to which actigraphically recorded motor activity may predict inpatient clinical course and hospital discharge remains unknown. Participants were recruited from the acute psychiatric inpatient ward at Hospital Rey Juan Carlos (Madrid, Spain). They wore miniature wrist wireless inertial sensors (actigraphs) throughout the admission. We modeled activity levels against the normalized length of admission—‘Progress Towards Discharge’ (PTD)—using a Hierarchical Generalized Linear Regression Model. The estimated date of hospital discharge based on early measures of motor activity and the actual hospital discharge date were compared by a Hierarchical Gaussian Process model. Twenty-three depressed patients (14 females, age: 50.17 ± 12.72 years) were recruited. Activity levels increased during the admission (mean slope of the linear function: 0.12 ± 0.13). For n = 18 inpatients (78.26%) hospitalised for at least 7 days, the mean error of Prediction of Hospital Discharge Date at day 7 was 0.231 ± 22.98 days (95% CI 14.222–14.684). These n = 18 patients were predicted to need, on average, 7 more days in hospital (for a total length of stay of 14 days) (PTD = 0.53). Motor activity increased during the admission in this sample of depressed patients and early patterns of actigraphically recorded activity allowed for accurate prediction of hospital discharge date.
2019
- JBHIDeep sequential models for suicidal ideation from multiple source dataIgnacio Peis, Pablo M. Olmos, Constanza Vera-Varela, and 4 more authorsIEEE Journal of Biomedical and Health Informatics, 2019
This paper presents a novel method for predicting suicidal ideation from electronic health records (EHR) and ecological momentary assessment (EMA) data using deep sequential models. Both EHR longitudinal data and EMA question forms are defined by asynchronous, variable length, randomly sampled data sequences. In our method, we model each of them with a recurrent neural network, and both sequences are aligned by concatenating the hidden state of each of them using temporal marks. Furthermore, we incorporate attention schemes to improve performance in long sequences and time-independent pre-trained schemes to cope with very short sequences. Using a database of 1023 patients, our experimental results show that the addition of EMA records boosts the system recall to predict the suicidal ideation diagnosis from 48.13% obtained exclusively from EHR-based state-of-the-art methods to 67.78%. Additionally, our method provides interpretability through the t-distributed stochastic neighbor embedding (t-SNE) representation of the latent space. Furthermore, the most relevant input features are identified and interpreted medically.
2017
- FrontiersA heavy tailed expectation maximization hidden markov random field model with applications to segmentation of MRIDiego Castillo-Barnes, Ignacio Peis, Francisco J Martı́nez-Murcia, and 5 more authorsFrontiers in Neuroinformatics, 2017
A wide range of segmentation approaches assumes that intensity histograms extracted from magnetic resonance images (MRI) have a distribution for each brain tissue that can be modeled by a Gaussian distribution or a mixture of them. Nevertheless, intensity histograms of White Matter and Gray Matter are not symmetric and they exhibit heavy tails. In this work, we present a hidden Markov random field model with expectation maximization (EM-HMRF) modeling the components using the α-stable distribution. The proposed model is a generalization of the widely used EM-HMRF algorithm with Gaussian distributions. We test the α-stable EM-HMRF model in synthetic data and brain MRI data. The proposed methodology presents two main advantages: Firstly, it is more robust to outliers. Secondly, we obtain similar results than using Gaussian when the Gaussian assumption holds. This approach is able to model the spatial dependence between neighboring voxels in tomographic brain MRI.
2016
- NSS/MICMRI brain segmentation using hidden Markov random fields with alpha-stable distributionsIgnacio Peis, Francisco J. Martı́nez-Murcia, Fermín Segovia, and 4 more authorsIn 2016 IEEE Nuclear Science Symposium, Medical Imaging Conference and Room-Temperature Semiconductor Detector Workshop (NSS/MIC/RTSD), 2016
A MRI brain image segmentation method using a hidden Markov random fields with heavy-tailed alpha-stable distributions is presented. Each brain tissue is modelled using an alpha-stable distribution. Then, a HMRF is used to include spatial information in the classification model. The Gaussian distribution has been widely used for the modelization of the cerebrospinal fluid, white matter and gray matter. Nevertheless, the alpha-stable distribution has been recently proposed as a more accurate alternative for this task. The alpha-stable distribution is more impulsive and is also able to model the asymmetry and heavy-tails of the histogram of the brain tissues. We have tested the proposed methodology in 18 MR images from the Internet Brain Segmentation Repository. The proposed methodology outperforms the segmentation results obtained when a Gaussian model for the histogram of the brain tissues is considered. Furthermore, as the Normal distribution is a particulaar case of alpha-stable distribution. Therefore, the proposed approach is also a generalization of the hidden Markov random field segmentation method with Gaussian distributions.