TAILOR WP7 SEMINAR SERIES

The fifth event of WP7 seminar series will be on Tuesday June 21st, from 15.30 to 17.00 (CET). The seminar will consist of two parts: a scientific and a social part. The seminar talk is hosted by Manh Hung Nguyen (Chalearn/Inria) and its on Meta-learn 22.

Meta-learning aims to leverage experience from previous tasks to solve new tasks using only little training data, train faster and/or get better performance. TAILOR has been sponsoring two challenges in meta-learning, described in this presentation: “Meta-learning from Learning Curves” (AutoML-Conf’22, ending July 4) and “Cross-Domain Meta-learning” (NeurIPS’22, ending Sept 1). For both challenges, we will summarize the results of past rounds, explain lessons learned and the new settings, then present our baseline results for the new versions. There is no need for you to have entered past editions, and the code of the winners has been open-sourced!

The first challenge aims to push the state-of-the-art in meta-learning from one of the most essential model evaluation data, the learning curve. The setting is similar to that of algorithm recommendation, in a black-box setting, i.e. using as information only performances of models (no access to the training algorithm). However, use is made of performances of partially trained models. The goal is to rapidly find the best solution to a given classification problem, without training all algorithms/models to convergence. This problem can be treated with conventional black-box optimization methods or with reinforcement learning. The first round was held for IJCNN’22 and the second round will be part of AutoML-Conf’22, the first conference on automated machine learning https://automl.cc/, Baltimore, USA, July 25-27, 2022.

The second challenge focuses on cross-domain meta-learning for few-shot image classification using a novel “any-way” and “any-shot” setting. The setting is similar to the classical N-way k-shot meta-learning setting that has attracted much attention recently, with the appearance of episodic algorithms such as MAML and Prototypical Networks. In this setting, tasks consist of small classification problems with N classes (ways) and k training examples (shots). During meta-training, the algorithms are exposed to many pairs of training and test sets (called support and query sets) for such tasks. Then, during meta-testing, new tasks are provided, for which test labels are withdrawn, and the goal is to make predictions on such test examples. While in our previous few-shot challenge, meta-training and meta-test tasks were drawn from the same domain, in this new challenge they are drawn from 10 different domains, making the problem more challenging and realistic. Also in the previous challenge, we addressed only 5-way 5-shot problems. In the new challenge, we vary the number of ways and shots in the ranges [2-20] and [1-20]. The previous round was held for NeurIPS’21 and the second round will be part of NeurIPS’22 https://neurips.cc/Conferences/2022/CompetitionTrack, New Orleans, USA, Nov 28-Dec 9 2022.

If you want to participate to the seminar, just contact the TAILOR WP7 Seminar Organising Committee on Mattermost or by sending an email to Annelot Bosman (a.w.bosman@liacs.leidenuniv.nl).