Reliability and validity sound alike and mean different things. Reliability is consistency. Validity is whether the measure captures what it claims to.
Reliability
Reliability, named in the BASK's basic statistics and measurement key concept, concerns the consistency of measurement.
An engagement survey re-administered to the same stable group two weeks later that produces very different scores each time has a reliability problem - not a validity problem. Unstable repeat scores are the classic reliability signal, and the three validity types are the distractors.
The three validity types
The Uniform Guidelines on Employee Selection Procedures (41 CFR 60-3) recognize exactly three types of validity evidence: criterion-related, content and construct.
- Criterion-related validity: demonstrated by empirical data showing that a selection procedure is predictive of or significantly correlated with important elements of work behavior.
- Content validity: demonstrated by data showing the content of the procedure is representative of important aspects of job performance.
- Construct validity: demonstrated by data showing the procedure measures identifiable characteristics determined to be important for successful job performance.
The traps
- Face validity is not a recognized type under the Guidelines. Any set containing it is wrong.
- Concurrent, convergent and discriminant are research-design terms, not the Guidelines' three categories.
- Internal, external and statistical validity come from experimental design, not selection-procedure validity.
- A procedure can be perfectly reliable and still not be valid: consistently measuring the wrong thing is consistency without relevance. Reliability is never the answer when the stem asks about validity evidence, and it is always the answer when the stem describes unstable repeated measurement.
Carry this in: the phrase "predictive of or significantly correlated with important elements of work behavior" always points to criterion-related validity. "Representative of important aspects of job performance" points to content validity.