Skip to content

LLM-based Context Precision


Context Precision is used to measure information density.

Example Usage

Required data items: question, retrieved_context

from continuous_eval.metrics.retrieval import LLMBasedContextPrecision
from continuous_eval.llm_factory import LLMFactory
datum = {
"question": "What is the capital of France?",
"retrieved_context": [
"Paris is the capital of France and also the largest city in the country.",
"Lyon is a major city in France.",
metric = LLMBasedContextPrecision(LLMFactory("gpt-4-1106-preview"), log_relevance_by_context=True)

Note: optional variable log_relevance_by_context outputs LLM_based_context_relevance_by_context - the LLM judgement of relevance to answer the question per context retrieved.

Sample Output

'LLM_based_context_precision': 0.5,
'LLM_based_context_average_precision': 1.0,
'LLM_based_context_relevance_by_context': [True, False]