Compute Mean Average Precision (mAP)
Compute Mean Average Precision (mAP)
For a set of retrieval queries, compute Average Precision (AP) per query and Mean Average Precision (mAP) across all queries. Each query has items with binary relevance labels and real-valued scores.
Mean Average Precision is a standard evaluation metric in information retrieval and object detection. It measures the quality of ranked retrieval results by considering both precision and the ranking order of relevant items.
mAP Formulation:
Average Precision for a single query:
AP=R1k=1∑nP(k)rel(k)Mean Average Precision across queries:
mAP=Q1q=1∑QAPqWhere P(k) is precision at rank k, rel(k) is relevance at rank k, R is total relevant items, and Q is number of queries.
Function Arguments
y_true_list: list of arrays- Binary relevance labels {0,1} for each queryy_score_list: list of arrays- Real-valued scores for each query (same lengths)k: optional int- Cutoff rank; if None, use full length
Examples
Input: y_true_list = [[1, 0, 1, 0]], y_score_list = [[0.9, 0.8, 0.7, 0.1]]
Output: (0.8333, [0.8333])
Sorted by score: labels become [1, 0, 1, 0]. P(1)=1.0, P(3)=2/3. AP = (1.0 + 0.667)/2 = 0.8333
Input: y_true_list = [[1, 0, 1], [1, 1, 0]], y_score_list = [[0.9, 0.8, 0.7], [0.9, 0.8, 0.7]]
Output: (0.9167, [0.8333, 1.0])
Query 1: AP = 0.8333. Query 2: both relevant items at top, AP = 1.0. mAP = (0.8333 + 1.0)/2 = 0.9167
Hint 1
Sort indices by descending score using np.argsort() for each query.
Hint 2
Use np.cumsum() to get cumulative relevant items and compute precision at each rank.
Hint 3
Average precision is the mean of precisions at positions where items are relevant.
Requirements
- Return tuple: (map_value, ap_per_query)
- Sort items by score descending for each query
- AP = average of P@i over ranks i where item i is relevant (up to k)
- Handle queries with zero relevant items → AP = 0
- Support optional cutoff k (e.g., mAP@10)
- Vectorized within each query where possible
Constraints
- Total items across queries ≤ 1e6; NumPy only
- Time limit: 800ms; Memory: 512MB
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Accepts: array
Accepts: any
Compute Mean Average Precision (mAP)
Compute Mean Average Precision (mAP)
For a set of retrieval queries, compute Average Precision (AP) per query and Mean Average Precision (mAP) across all queries. Each query has items with binary relevance labels and real-valued scores.
Mean Average Precision is a standard evaluation metric in information retrieval and object detection. It measures the quality of ranked retrieval results by considering both precision and the ranking order of relevant items.
mAP Formulation:
Average Precision for a single query:
AP=R1k=1∑nP(k)rel(k)Mean Average Precision across queries:
mAP=Q1q=1∑QAPqWhere P(k) is precision at rank k, rel(k) is relevance at rank k, R is total relevant items, and Q is number of queries.
Function Arguments
y_true_list: list of arrays- Binary relevance labels {0,1} for each queryy_score_list: list of arrays- Real-valued scores for each query (same lengths)k: optional int- Cutoff rank; if None, use full length
Examples
Input: y_true_list = [[1, 0, 1, 0]], y_score_list = [[0.9, 0.8, 0.7, 0.1]]
Output: (0.8333, [0.8333])
Sorted by score: labels become [1, 0, 1, 0]. P(1)=1.0, P(3)=2/3. AP = (1.0 + 0.667)/2 = 0.8333
Input: y_true_list = [[1, 0, 1], [1, 1, 0]], y_score_list = [[0.9, 0.8, 0.7], [0.9, 0.8, 0.7]]
Output: (0.9167, [0.8333, 1.0])
Query 1: AP = 0.8333. Query 2: both relevant items at top, AP = 1.0. mAP = (0.8333 + 1.0)/2 = 0.9167
Hint 1
Sort indices by descending score using np.argsort() for each query.
Hint 2
Use np.cumsum() to get cumulative relevant items and compute precision at each rank.
Hint 3
Average precision is the mean of precisions at positions where items are relevant.
Requirements
- Return tuple: (map_value, ap_per_query)
- Sort items by score descending for each query
- AP = average of P@i over ranks i where item i is relevant (up to k)
- Handle queries with zero relevant items → AP = 0
- Support optional cutoff k (e.g., mAP@10)
- Vectorized within each query where possible
Constraints
- Total items across queries ≤ 1e6; NumPy only
- Time limit: 800ms; Memory: 512MB
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Accepts: array
Accepts: any