Branislav Pecher
PhD at Kempelen Institute of Intelligent Technologies, member of Slovak.AI
Abstract: Learning with limited labelled data, such as meta-learning, transfer learning or in-context learning, aims to effectively train a model using only a small amount of labelled samples. However, there is still limited understanding of the required settings or characteristics for these approaches to effectively generalise and achieve high performance with limited data. The choices of data, both systematic at the dataset level, which affect robustness, and non-deterministic at the sample level, which affect stability, significantly influence the success of transfer and the overall performance. Due to the different choices leading to unpredictable behaviour of the models, with performance varying significantly across different training runs or settings, the trustworthiness of these models is compromised. Despite these negative effects, the impact of data choice remains under-researched and poorly understood. Therefore, the goal of my research visit at the Eindhoven University of Technology is to enhance the stability and robustness of approaches for learning with limited labelled data. As a first step, I aim to investigate and analyse the impact of data choice by exploring various sampling strategies. This will provide a better understanding of how to effectively learn with limited data. Subsequently, I intend to leverage this insight to enhance the stability and robustness of existing methods by proposing improved ways to choose samples for the adaptation stage of training and for the entire model pre-training, based on the investigated sampling strategies.
Keywords: Learning with Limited Labelled Data, Few-shot Learning, Stability, Robustness
Scientific area: Artificial Intelligence – Learning with Limited Labelled Data
Branislav Pecher is a second year PhD student at the Kempelen Institute of Intelligent Technologies in Bratislava (member of Slovak.AI). His research focuses on learning with limited labelled data, such as meta-learning, fine-tuning or few-shot learning. In his research he mainly focuses on studying the stability and robustness of these approaches to allow for their more effective use.