Logo image
0x.Yuan at SemEval-2024 Task 2: Agents Debating can reach consensus and produce better outcomes in Medical NLI task
Conference paper   Open access

0x.Yuan at SemEval-2024 Task 2: Agents Debating can reach consensus and produce better outcomes in Medical NLI task

Yu-An Lu and Hung-Yu Kao
SemEval 2024 - 18th International Workshop on Semantic Evaluation, Proceedings of the Workshop, pp.305-310
2024

Abstract

Computational Theory and Mathematics Computer Science Applications Theoretical Computer Science
In this paper, we introduce a multi-agent debating framework, experimenting on SemEval 2024 Task 2. This innovative system employs a collaborative approach involving expert agents from various medical fields to analyze Clinical Trial Reports (CTRs). Our methodology emphasizes nuanced and comprehensive analysis by leveraging the diverse expertise of agents like Biostatisticians and Medical Linguists. Results indicate that our collaborative model surpasses the performance of individual agents in terms of Macro F1-score. Additionally, our analysis suggests that while initial debates often mirror majority decisions, the debating process refines these outcomes, demonstrating the system's capability for in-depth analysis beyond simple majority rule. This research highlights the potential of AI collaboration in specialized domains, particularly in medical text interpretation.
url
https://doi.org/10.18653/v1/2024.semeval-1.47View
Published (Version of record) Open

Related links

Metrics

1 Record Views

Details

Logo image