Thursday, November 15, 2018

Validity of Tests and Measurements

Validity of a test or measurement relates to the accuracy of a method to measure what it is intended to measure.  Because measurement error is related to the reliability of a test, one could argue that a test must be reliable in order to be valid.  Yet, an invalid test can be reliable.  For instance, the results of a testing methodology may be reproducible, but not accurately measuring the variable of interest. 

To illustrate the link between reliability and validity of a measurement, consider the analogy of shooting an arrow at a target.  Shooting the arrow at the bulls-eye of the target would represent performing the test.  The bulls-eye represents the characteristic of the variable being measured.

A tight shot pattern suggests that the test produces reproducible results (high reliability).


A shot pattern that is spread "wide" suggests that the test produces results that are not consistently reproducible (low reliability).


A shot that hits the bulls-eye suggests that the test produces results that reflect the true characteristic of the variable being measured (high validity).


A shot that misses the bulls-eye indicates that the test produces results that do not reflect the true characteristic of the variable being measured (low validity).




A tight shot pattern that consistently hits the bulls-eye indicates that the results of a test are reliable and are measuring the true characteristic of the variable being measure (high reliability and high validity).

A test with acceptable accuracy demonstrates an acceptable degree of reliability and validity.  One can now understand that "acceptable" should be defined, meaning what is an acceptable degree of reliability and validity.  The definition of "acceptable" is important since testing methods rarely display perfect reliability and validity.  Acceptable reliability and validity of a test is sometimes debatable, but should be based on evidence-informed decisions.

Validity is based on the categories of evidence that can be used to support the validity of a test.  Below are descriptions of various types of validity with links to example studies.

Face Validity
Indicates that a testing methodology appears to measure what it is supposed to measure.

Content Validity
Indicates that the various components of the testing methodology include the content of the variable being measured. 

Criterion-related Validity
Indicates that the results of a test can be used as a substitute for the findings of reference-standard test.

Concurrent Validity
Indicates that the results of a test are valid when compared to reference-standard test; established when both tests are performed at relatively the same time (concurrently); a form of criterion-related validity.

Predictive Validity
Indicates that the results of a test are valid since the test results can predict a future outcome; another form of criterion-related validity.

Construct Validity
Indicates that the results of a test reflect an abstract construct.

No comments:

Post a Comment