Search results
Results From The WOW.Com Content Network
BLEU (bilingual evaluation understudy) is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another. Quality is considered to be the correspondence between a machine's output and that of a human: "the closer a machine translation is to a professional human translation, the better it is" – this is the central idea behind BLEU.
The quality of a translation is inherently subjective, there is no objective or quantifiable "good." Therefore, any metric must assign quality scores so they correlate with the human judgment of quality. That is, a metric should score highly translations that humans score highly, and give low scores to those humans give low scores.
ROUGE, or Recall-Oriented Understudy for Gisting Evaluation, [1] is a set of metrics and a software package used for evaluating automatic summarization and machine translation software in natural language processing.
To calculate the recall for a given class, we divide the number of true positives by the prevalence of this class (number of times that the class occurs in the data sample). The class-wise precision and recall values can then be combined into an overall multi-class evaluation score, e.g., using the macro F1 metric. [21]
This differs from the BLEU metric in that BLEU seeks correlation at the corpus level. Example alignment (a). Results have been presented which give correlation of up to 0.964 with human judgement at the corpus level, compared to BLEU's achievement of 0.817 on the same data set. At the sentence level, the maximum correlation with human judgement ...
Get AOL Mail for FREE! Manage your email like never before with travel, photo & document views. Personalize your inbox with themes & tabs. You've Got Mail!
It is based on the BLEU metric, but with some alterations. Where BLEU simply calculates n-gram precision adding equal weight to each one, NIST also calculates how informative a particular n-gram is. That is to say when a correct n-gram is found, the rarer that n-gram is, the more weight it will be given. [1]
LEPOR [4] is designed with the factors of enhanced length penalty, precision, n-gram word order penalty, and recall.The enhanced length penalty ensures that the hypothesis translation, which is usually translated by machine translation systems, is punished if it is longer or shorter than the reference translation.