Classification Metrics
Definitions
Single Label Classification measures the performance of a classification module.
Example Usage
Required data items: predicted_class
, ground_truth_class
from continuous_eval.metrics.classification import SingleLabelClassification
datum = { "predicted_class": "quantitative_question", "ground_truth_class": "qualitative_question",}
metric = SingleLabelClassification(classes={"qualitative_question", "quantitative_question"})print(metric(**datum))
Example Output
{ 'classification_prediction': 'quantitative_question', 'classification_ground_truth': 'qualitative_question', 'classification_correct': False}
Aggregate Results
y_pred = ["A", "A", "B", "A", "B"]y_true = ["A", "B", "B", "A", "B"]
metric = SingleLabelClassification(classes={"A", "B"})results = [metric(y, y_gt) for y, y_gt in zip(y_pred, y_true)]print(metric.aggregate(results))
Example Output
{ 'accuracy': 0.8, 'balanced_accuracy': 0.83, 'precision': 0.83, 'recall': 0.83, 'f1': 0.8}