Microsoft’s new Pareto Optimal Self-Supervision Framework Automatically Corrects Language Models to Boost GPT SOTA Records

Large language models (LLMs) have undoubtedly displayed immense potential in the field of natural language processing and understanding. However, the issue of hallucination remains a prominent challenge, particularly in real-world applications where a high degree of accuracy and reliability is expected.

Addressing this issue, a research team from Microsoft proposed an innovative solution in a new paper Automatic Calibration and Error Correction for Large Language Models via Pareto Optimal Self-Supervision. The paper introduces a novel, flexible framework -Pareto optimal self-supervision, which utilizes programmatic supervision to calibrate and rectify errors inherent in LLMS, thereby eliminating the necessity for additional manual intervention.

The team considers the problem as a general prompt-based LLM work flow: given input text with specific prompt setting, the LLM first generates a text response and then projects this response onto the desired output space via an operator, and at the same time an abstain option is added to decide whether the state of the generated text response is clear or unsure. If the model confidently states and the generated answer is wrong, the problem of hallucination remains.

The goal of LLM calibration is to design a risk indicator function to estimate the true probability of hallucination. In this work, the researchers use self-supervision to derive the risk indicator function, in which setting enables them to access to unlabeled training examples with ground truth outputs completely unavailable.

The key component of the proposed pareto optimal learning is a harmonizer model that fits in the semantic space to simultaneously consistent with both the LLM response and the specified supervision function. And to ensure the harmonizer model to provide high-quality calibration, the team proposes two dynamic prompting strategies: dynamic self-examination and dynamic self-supervision that leverage POLAR score to facilitate error correction for more certain LLM responses.

In their empirical study, the team implemented Pareto optimal learning on four challenging natural language processing datasets: CDR, ChemProt, SemEval, and SMS. The proposed framework demonstrates consistent high real error calibration rate of LLMs, boosting both state-of-the-art (SOTA) GPT-3 and GPT-4 results with additional human-labeled training data.

The paper Automatic Calibration and Error Correction for Large Language Models via Pareto Optimal Self-Supervision on arXiv.

---

Author: Hecate He | Editor: Chain Zhang

---

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.