In Short:
Researchers from MIT and the MIT-IBM Watson AI Lab have introduced a calibration method called Thermometer that improves the accuracy of large language models (LLMs). Thermometer involves building an auxiliary model on top of the LLM to calibrate it efficiently and with less computation. This method helps users determine when a model’s response is accurate or inaccurate, ultimately preventing them from deploying unreliable models.
Large language models are widely used for various tasks, but they can sometimes provide inaccurate responses. Additionally, these models may exhibit overconfidence in incorrect answers or lack confidence in correct ones, which can make it challenging for users to trust the model.
Introducing Thermometer
Researchers from MIT and the MIT-IBM Watson AI Lab have developed a new calibration method specifically designed for large language models. This method, known as Thermometer, involves creating a smaller auxiliary model that works alongside the large language model to calibrate it.
Thermometer is more efficient than traditional methods, requiring less computational power while maintaining the model’s accuracy and improving its calibration for unseen tasks.
Benefits of Thermometer
By effectively calibrating a large language model using Thermometer, users can better identify situations where the model may be too confident in incorrect predictions. This can help prevent the deployment of a model in scenarios where it may fail.
Maohao Shen, an EECS graduate student at MIT and lead author of the paper on Thermometer, emphasizes the importance of providing users with a clear signal to determine the accuracy of a model’s response.
Universal Calibration
Unlike traditional models designed for specific tasks, large language models require a universal calibration method to handle various tasks without sacrificing performance. Thermometer leverages temperature scaling to efficiently calibrate a large language model for new tasks.
The researchers train an auxiliary model with labeled datasets from representative tasks to predict the appropriate temperature for calibrating the large language model. This approach eliminates the need for additional labeled data for new tasks within a similar category.
An Efficient Approach
Thermometer does not require multiple training runs and only marginally slows down the large language model. It maintains the model’s accuracy while consistently producing better calibrated uncertainty measures with reduced computational costs compared to baseline methods.
Future efforts will focus on adapting Thermometer for more complex text generation tasks and applying the technique to larger large language models. The researchers aim to determine the diversity and amount of labeled data needed to train a Thermometer model for generalizing to new tasks.
This research received partial funding from the MIT-IBM Watson AI Lab.
