Abstract
We introduce VAE-HyperNetFusion, an innovative framework that integrates variational autoencoders (VAEs) with interpolation-based data augmentation to enhance hypernetwork training on small tabular datasets. Using VAE-HyperNetFusion we initially focused on a prostate cancer dataset to address the critical challenge of limited sample size by generating augmented data that substantially enriches the training environment for hypernetwork integration models. The hypernetwork adopts a multi-head design and generates the parameters of an ensemble of differentiable soft decision trees, improving training stability and reducing sensitivity to random initialization. Beyond the prostate cancer dataset study, we validate the effectiveness of the proposed approach on several published tabular datasets. It demonstrates that VAE-HyperNetFusion can be generalized and is robust across different domains. By incorporating VAE for data augmentation within the VAE-HyperNetFusion framework, the risk of overfitting, a common pitfall when handling small datasets, is substantially reduced through the introduction of a more diverse and enriched pool of training samples. This innovation extends the applicability of deep learning methods to problems traditionally dominated by shallow learning techniques, particularly in fields requiring precise and reliable analysis of limited data, such as medical diagnostics and personalized healthcare. Experimental results show that VAE-HyperNetFusion consistently outperforms existing state-of-the-art models in small dataset settings highlighting its potential as an important advance in applying deep learning to tabular data with severe sample constraints. This work provides the deep learning community with a new approach for analyzing small tabular datasets and establishes a strong benchmark for future research. © 2026 IEEE.