This post is a brief summary about the paper that I read for my study and curiosity, so I shortly arrange the content of the paper, titled Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (Li et al., ACL-findings 2024), that I read and studied.

Li et al. ACL-findings 2024

They proposed the solution to debiasing In-context Learning (ICL) they call demonstration bias, which is from the selection and order of demonstrations.

To deal with demonstration bias, they propose two de-biasing strategies for in-context learning,

1) Instance-Free Demonstration Reordering, which progressively selects demonstrations by maximizing the semantic ambiguity reduction of in-context demonstrations.

2) Self-Explanatory In-Context Learing framework, which generates explicit explanatory guidelines for each instantce and then instructs LLMs to select appropriate semantic modes by following these guidelines.

For detailed experiment and explanation, refer to the paper, titled Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (Li et al., ACL-findings 2024)

Note(Abstract): In-context learning(ICL) has gained considerable attention due to its data efficiency and task adaptability. Unfortunately, ICL suffers from the demonstration bias, i.e., its performance and robustness are severely affected by the selection and ordering of demonstrations. In this paper, we identify that such demonstration bias may primarily stem from the semantic ambiguity induced by demonstrations, i.e., a demonstration may indicate multiple input-to-label mappings and its mapping can be interpreted differently in different contexts by LLMs. Such semantic ambiguity disrupts task comprehension during ICL and results in performance fluctuations. To resolve the semantic ambiguity problem, this paper further proposes two de-biasing strategies to mitigate demonstration bias in in-context learning. Experiments on six datasets show that our methods can effectively alleviate demonstration bias and significantly improve task performance.

Download URL:
The paper: Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (Li et al., ACL-findings 2024)

Reference

Paper
- ACL Version: Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (Li et al., ACL-findings 2024)
For Your Information
- Lil’Log’s Prompt Engineering
How to use html for alert
- how to use icon
How to use MathJax
- MathJax basic tutorial and quick reference in StackExchange