NeSy2023 conference (https://sites.google.com/view/nesy2023/home?authuser=0) will host a TAILOR WP4 workshop on July 3rd, 11:00-13:00, about “Benchmarks for Neural-Symbolic AI”.
The workshop will be in hybrid format, however in-person participation is strongly suggested, especially for NeSy researchers and PhD students. For more information, we recommend to visit the webpage https://sailab.diism.unisi.it/tailor-wp4-workshop-at-nesy/.
Abstract
The study of Neural-Symbolic (NeSy) approaches has been a longstanding goal in the field of AI. However, the evaluation of novel ideas and frameworks in this context poses a significant challenge due to the lack of appropriate benchmarks. Existing tasks and datasets tend to exhibit a myopic approach, tailored exclusively to specific algorithms or methodologies, and limited to narrow scenarios and application domains. This limitation hinders the ability to obtain meaningful experimental comparisons and constrains the overall progress of NeSy AI. Consequently, there is an urgent need for more comprehensive and inclusive benchmarks that facilitate the testing of diverse approaches across a wide range of tasks and under varying conditions. The workshop will present some prominent examples of existing benchmarks in different sub-fields, like continual learning, NLP, knowledge graphs, and so forth, from the perspective of NeSy-based approaches. In addition, we will discuss the importance of developing new benchmarks that address some shortcomings of existing ones.
Program
- 11:00-11:05 – Opening – Luc De Raedt
- 11:05-11:15 – Intro about Taskforce activity – Luc De Raedt
- 11:15-11:25 – Neuro-Symbolic Continual Learning: Harnessing Knowledge for Drifting NeSy Scenarios – Emanuele Marconato
- 11:25-11:35 – Entity Linking Benchmark in Computational Food: Combining Text Mining, Logical Reasoning, and LLM Prompting – Agnieszka Lawrynowicz
- 11:35-11:45 – Challenges for Neuro-Symbolic Approaches: Case Study on Legal Analytics and Argument Mining – Federico Ruggeri
- 11:45-11:55 – A neuro symbolic setup for link prediction in KGEs – Michelangelo Diligenti
- 11:55-12:05 – Three New Benchmarks for Neuro-Symbolic Systems – Fabrizio Riguzzi
- 12:05-12:15 – V-LoL: A Diagnostic Dataset for Visual Logical Learning – Devendra Singh Dhami
- 12:15-12:25 – A common experimental framework for NeSy systems – Robin Manhaeve
- 12:25-13:00 – Round Table Discussion – Marco Gori