Sensitivity and Specificity


Posted by Pierre-Edouard Guerin · 4 min read · Published on April 6, 2015

How do we know if a test or classifier is reliable? In both medicine and machine learning, classifier are used to attribute a class to an input data. Therefore, performance metrics are necessary to assess the reliability of the resulting classification. They help answer questions like:

On the Origin of Sensitivity and Specificity

Originally, sensitivity and specificity were qualities of a biological test used in chemistry, serology and immunology.

Later, in clinical epidemiology, Jacob Yerushalmy in 1947 introduced sensitivity and specificity as probabilities to assess reliability of diagnoses in radiology.

Today, sensitivy and specificity are used as metrics beyond medecine, in data science and maching learning. They evaluate the performance of binary classifiers.

Binary Classifier

One can consider the medical diagnoses as binary classifier. A classifier is a process that takes input data and assigns it to a class. Such classifier is binary when it can only choose between two possible classes.

For instance, the diagnosis based on X-ray data only produce two possible outputs: Positive or Negative.

input X-ray datadiagnosis
case 1Positive
case 2Negative
case 3Positive
case 4Positive
case 5Negative

The Confusion Matrix

The confusion matrix is a table that compares classifier's prediction with the observed class.

Predicted PositivePredicted Negative
Observed Positive✅ True Positive (TP)❌ False Negative (FN)
Observed Negative❌ False Positive (FP)✅ True Negative (TN)

Sensitivity

The sensitivity is the ratio of observed positive cases, correctly predicted by the test.

$$ \text{Sensitivity} = \frac{TP}{TP + FN} $$

Specificity

The specificity is the ratio of observed negative cases, correctly predicted by the test.

$$ \text{Specificity} = \frac{TN}{TN + FP} $$

Type I Error

The Type I Error or False Positive Rate impacts the precision of a test. It is the ratio of observed negative cases that the test failed to detect.

$$ \text{False Positive Rate} = \frac{FP}{FP + TN} $$

False Positive Rate can be expressed with specificity:

$$ \text{False Positive Rate} = {1 - Specificity} $$

Type II error

The Type II Error or False Negative Rate impacts the sensitivity of a test. It is the ratio of observed positive cases that the test failed to detect.

$$ \text{False Negative Rate} = \frac{FN}{FN + TP} $$

False Negative Rate can be expressed with sensitivity:

$$ \text{False Negative Rate} = {1 - Sensitivity} $$

Accuracy

The accuracy is how often the test correctly predicted the outcome for both positive and negative cases.

$$ \text{Accuracy} = \frac{TP + TN }{TN + TN + FP + FN} $$

Accuracy can be expressed as the weighted mean of sensitivity and specificity, weighted by the number of observed positive and negative cases.

$$ \text{Accuracy} = \frac{P ⋅ Sensitivity + N ⋅ Specificity }{P + N} $$

References

Statistical Problems in Assessing Methods of Medical Diagnosis, with Special Reference to X-Ray Techniques

Jacob Yerushalmy

Public health reports. 1947 October 03. DOI: 10.2307/4586294

On the Origin of Sensitivity and Specificity

Nicholas Binney, Christopher Hyde, and Patrick M. Bossuyt

Annals of Internal Medecine. 2021 March 16. DOI: 10.7326/M20-5028

ORION : a web server for protein fold recognition and structure prediction using evolutionary hybrid profiles

Yassine Ghouzam, Guillaume Postic, Pierre-Edouard Guerin, Alexandre G. de Brevern & Jean-Christophe Gelly

Scientific Reports. 2016 Jun 20. DOI: 10.1038/srep28268







Relevant Tags

About the Author

Portrait of Pierre-Edouard Guerin Bioinformatician Data Scientist
Pierre-Edouard Guerin
Bioinformatician Research Scientist

Latest Articles