With the advancement of information retrieval (IR) technologies, robustness is increasingly attracting attention. When deploying technology into practice, we consider not only its average performance under normal conditions but, more importantly, its ability to maintain functionality across a variety of exceptional situations. In recent years, the research on IR robustness covers theory, evaluation, methodology, and application, and all of them show a growing trend. The purpose of this workshop is to systematize the latest results of each research aspect, to foster comprehensive communication within this niche domain while also bridging robust IR research with the broader community, and to promote further future development of robust IR.
Theory
• Game theory: Modeling strategic interactions as games, designing mechanisms to mitigate adversarial behaviors, and understanding the implications of these strategies for robust system design.
• Competitive search: Analyzing how competition influences the ecosystem and exploring mechanisms to promote desired properties in these contexts such as robustness and fairness.
• Probability ranking principle: Investigating the assumptions of PRP and their adaptation to different adversarial scenarios.
Evaluation method
• Specific evaluation: Robustness evaluation for specific robustness types (e.g., adversarial, OOD robustness), scenarios (e.g., corpus updating, queries with typos), and IR models (e.g., sparse, dense, and generative retrieval models).
• General evaluation: Using an evaluation metric to comprehensively cover as many robustness scenarios as possible.
• Diverse evaluation tools: Developing new evaluation forms, including new evaluation functions, LLMs, and human evaluations, for comprehensive robustness comparisons.
• Benchmarks of robustness: Discussion of existing robustness datasets and proposing new benchmark tasks to address diverse robustness categories and requirements.
Method
• Adversarial attack & defense: Investigating adversarial vulnerabilities in IR models and developing defenses against malicious attacks like data poisoning and adversarial document generation.
• Zero/few shot IR: Using zero-shot or few-shot learning, transfer learning, and large-scale pre-trained models to improve cross-domain and task generalization.
• Balancing robustness and effectiveness: Exploring enhancing robustness without compromising effectiveness.
• Long-term learning: Explores continual learning in IR to enhance stability.
• Noise Resistance: Making IR systems resistant to noise in queries, documents, or training data, such as typo handling, semantic noise filtering, and processing incomplete or corrupted inputs.
• Enhancing RAG Robustness: Improving RAG pipeline robustness by reducing error propagation, ensuring consistency between retrieved documents and generated content, and enhancing output reliability under uncertain conditions.
Application
• Robust search engines: Deploying robustness enhancement methods in resource- and condition-constrained search engines.
• Robust recommendation systems: Robustness for recommendation systems, addressing sparse user data, cold-start issues, adversarial manipulation, and dynamic user preferences.
• Data-specific scenarios: Robustness in specialized data retrieval, including scientific literature, medical documents, legal data, and long-form or multi-modal documents.
• Federated and distributed IR systems: Robustness for distributed IR using federated learning involves tackling inconsistent local data, implementing privacy-preserving strategies, and improving communication efficiency across distributed nodes.
Society Impact
• Human behaviors that affect robustness: Understand how user behaviors like query biases, click feedback loops, and manipulated web content affect IR system robustness.
• Robustness and Ethics: Addressing ethical concerns by ensuring fairness, minimizing algorithmic biases, and maintaining transparency. Ensuring robust models do not disproportionately affect certain user groups or perpetuate societal inequalities.
• Explainability and truthfulness: Discuss the impact of IR model explainability and information truthfulness on users.
Dates and Deadlines | |
---|---|
Workshop paper submission | 24 April, 2025 |
Workshop paper notification | 21 May, 2025 |
Workshop paper camera - ready | 28 May, 2025 |
Workshops | 18 July, 2025 |
July 17th 2025
8:30 AM - 8:45 AM | Welcome and opening remarks |
---|---|
8:45 AM - 9:15 AM | Keynote 1 |
9:15 AM - 9:40 AM | Invited talk 1 |
9:40 AM - 10:00 AM | Coffee break |
10:00 AM - 11:30 AM | Research paper presentations |
11:30 AM - 12:00 AM | Panel discussion |
TBD.
![]() ICT, CAS |
![]() Technion, IIT |
![]() ICT, CAS |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() Technion, IIT |
![]() ICT, CAS |
![]() Technion, IIT |