TAILOR WP7 Seminar Series

The fourth event of WP7 seminar series will be on 10th May 2022, from 15.30 to 17.00 (CET). The seminar will consist of two parts: a scientific and a social part.

Scientific part:

• 15:30-16:00 —Talk by Pieter Gijsbers— AMLB: An AutoML Benchmark

Abstract: Comparing different AutoML frameworks is notoriously challenging and often done incorrectly. In this talk, we will present an open and extensible benchmark that follows best practices and avoids common mistakes when comparing AutoML frameworks on tabular dataset problems. The benchmark comes with an open-source tool that integrates with many AutoML frameworks and automates the empirical evaluation process end-to-end: from framework installation and resource allocation to in-depth evaluation. We will discuss its design and limitations and show brief examples of how to use the benchmarking tool. Additionally, we present the results of a thorough comparison of 9 well-known AutoML frameworks across classification and regression tasks. The differences between the AutoML frameworks are explored with a multi-faceted analysis, evaluating model accuracy, its trade-offs with inference time, and framework failures.

• 16:00-16:30 — Round table updates for all WP7 tasks

Social Part

• 16:30-17:00 —Gathertown

There will be the possibility for: giving suggestions for improving the seminar format; informal chats about possible collaborations; brainstorm/discuss ideas and mechanism for making our WP7 more impactful, more efficient and/or more enjoyable; chat about anything else you feel worthwhile!

If you want to participate to the seminar, just contact the TAILOR WP7 Seminar Organising Committee on Mattermost or by sending an email to Annelot Bosman (a.w.bosman@liacs.leidenuniv.nl).