Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Today’s paper introduces PROMETHEUS 2, an open-source language model specialized in evaluating the outputs of other language models. It aims to address the limitations of existing open evaluator models which usually cannot effectively handle both direct assessment (scoring responses) and pairwise ranking (comparing responses) evaluation formats or not being able to evaluate based on custom evaluation criteria.
Method Overview
PROMETHEUS 2 uses weight merging. So, the authors train two separate evaluator models, one on direct assessment data and one on pairwise ranking data, and then merging their weights to obtain a unified model that can perform both tasks well.
Specifically, the authors construct two new datasets - the Feedback Collection for direct assessment containing 100K responses with scores and rubrics, and the Preference Collection for pairwise ranking with 200K response pairs, verbal feedback comparing them, and evaluation criteria.
They train evaluator models separately on each dataset using Mistral-7B and also Mixtral-8x7B as base models. Then they linearly merge the weights of the two trained models to obtain PROMETHEUS 2-7B and PROMETHEUS 2-8x7B.
Results
On four direct assessment benchmarks, PROMETHEUS 2 models achieve the highest correlation (>0.6 Pearson) with human and GPT-4 scores among open evaluators, reducing the gap with proprietary models by over 0.2.
The authors find that this weight merging approach outperforms jointly training a single model on both tasks, as well as ensembling multiple models trained on the same task. Merging allows positive transfer between the two tasks.
Conclusion
PROMETHEUS 2 sets a new state-of-the-art for open evaluator language models, being able to handle both direct assessment and pairwise ranking while maintaining high consistency across formats. For more information please consult the full paper.
Congrats to the authors for their work!
Code: https://github.com/prometheus-eval/prometheus-eval
Kim, Seungone, et al. "PROMETHEUS 2: An Open Source Language Model Specialized in Evaluating Other Language Models." ArXiv, 2 May 2024, arxiv.org/abs/2405.01535v1.