Robust IR @ SIGIR 2025

The First Workshop on Robust Information Retrieval

July 17th 2025, Padua, Italy

Held in conjunction with the SIGIR 2025

More





About

With the advancement of information retrieval (IR) technologies, robustness is increasingly attracting attention. When deploying technology into practice, we consider not only its average performance under normal conditions but, more importantly, its ability to maintain functionality across a variety of exceptional situations. In recent years, the research on IR robustness covers theory, evaluation, methodology, and application, and all of them show a growing trend. The purpose of this workshop is to systematize the latest results of each research aspect, to foster comprehensive communication within this niche domain while also bridging robust IR research with the broader community, and to promote further future development of robust IR.





Invited speakers

TBD.

Call for papers

We invite submissions related to Robust IR, including (but not limited to):
  • Theory
    • Game theory: Modeling strategic interactions as games, designing mechanisms to mitigate adversarial behaviors, and understanding the implications of these strategies for robust system design.
    • Competitive search: Analyzing how competition influences the ecosystem and exploring mechanisms to promote desired properties in these contexts such as robustness and fairness.
    • Probability ranking principle: Investigating the assumptions of PRP and their adaptation to different adversarial scenarios.

  • Evaluation method
    • Specific evaluation: Robustness evaluation for specific robustness types (e.g., adversarial, OOD robustness), scenarios (e.g., corpus updating, queries with typos), and IR models (e.g., sparse, dense, and generative retrieval models).
    • General evaluation: Using an evaluation metric to comprehensively cover as many robustness scenarios as possible.
    • Diverse evaluation tools: Developing new evaluation forms, including new evaluation functions, LLMs, and human evaluations, for comprehensive robustness comparisons.
    • Benchmarks of robustness: Discussion of existing robustness datasets and proposing new benchmark tasks to address diverse robustness categories and requirements.

  • Method
    • Adversarial attack & defense: Investigating adversarial vulnerabilities in IR models and developing defenses against malicious attacks like data poisoning and adversarial document generation.
    • Zero/few shot IR: Using zero-shot or few-shot learning, transfer learning, and large-scale pre-trained models to improve cross-domain and task generalization.
    • Balancing robustness and effectiveness: Exploring enhancing robustness without compromising effectiveness.
    • Long-term learning: Explores continual learning in IR to enhance stability.
    • Noise Resistance: Making IR systems resistant to noise in queries, documents, or training data, such as typo handling, semantic noise filtering, and processing incomplete or corrupted inputs.
    • Enhancing RAG Robustness: Improving RAG pipeline robustness by reducing error propagation, ensuring consistency between retrieved documents and generated content, and enhancing output reliability under uncertain conditions.

  • Application
    • Robust search engines: Deploying robustness enhancement methods in resource- and condition-constrained search engines.
    • Robust recommendation systems: Robustness for recommendation systems, addressing sparse user data, cold-start issues, adversarial manipulation, and dynamic user preferences.
    • Data-specific scenarios: Robustness in specialized data retrieval, including scientific literature, medical documents, legal data, and long-form or multi-modal documents.
    • Federated and distributed IR systems: Robustness for distributed IR using federated learning involves tackling inconsistent local data, implementing privacy-preserving strategies, and improving communication efficiency across distributed nodes.

  • Society Impact
    • Human behaviors that affect robustness: Understand how user behaviors like query biases, click feedback loops, and manipulated web content affect IR system robustness.
    • Robustness and Ethics: Addressing ethical concerns by ensuring fairness, minimizing algorithmic biases, and maintaining transparency. Ensuring robust models do not disproportionately affect certain user groups or perpetuate societal inequalities.
    • Explainability and truthfulness: Discuss the impact of IR model explainability and information truthfulness on users.

Submission Site: Robust IR @ SIGIR 2025.
Author Kit: Overleaf. LaTeX, Word.

All submissions will be peer reviewed (double-blind) by the program committee and judged by their relevance to the workshop, especially to the main themes identified above, and their potential to generate discussion. All submission must be written in English and formatted according to the latest ACM SIG proceedings template.
We accept submissions that were previously on arXiv or that got rejected from the main SIGIR conference.
The workshop follows a double-blind reviewing process. Please note that at least one of the authors of each accepted paper must register for the workshop and present the paper either remote or on location (strongly preferred).
We invite research contributions, position, demo and opinion papers. Submissions must either be short (at most 4 pages) or full papers (at most 9 pages). References do not count against the page limit. We also allow for an unlimited number of pages for appendices in the same PDF.
We encourage but do not require authors to release any code and/or datasets associated with their paper.


All deadlines are at 11:59 PM UTC-12:00 (“anywhere on Earth”).
Dates and Deadlines
Workshop paper submission 24 April, 2025
Workshop paper notification 21 May, 2025
Workshop paper camera - ready 28 May, 2025
Workshops 18 July, 2025

Schedule

July 17th 2025


8:30 AM - 8:45 AM Welcome and opening remarks
8:45 AM - 9:15 AM Keynote 1
9:15 AM - 9:40 AM Invited talk 1
9:40 AM - 10:00 AM Coffee break
10:00 AM - 11:30 AM Research paper presentations
11:30 AM - 12:00 AM Panel discussion

Accepted Papers

TBD.

Organizers

Yu-An Liu
ICT, CAS
Haya Nachimovsky
Technion, IIT
Ruqing Zhang
ICT, CAS
Oren Kurland
Technion, IIT
Jiafeng Guo
ICT, CAS
Moshe Tennenholtz
Technion, IIT