Examining the reliability and validity of the data

Reliability and validity are two key concepts in measurement theory, referring to the psychometric properties of the measurement techniques and the data obtained by them.

• The reliability of a psychometric instrument refers to the extent to which scores on the instrument are free from errors of measurement. For example, bathroom scales are not reliable if they show different figures depending on how steamy the air in the bathroom is, and neither are proficiency test raters if their evaluation varies according to how tired they are.

• Validity is the extent to which a psychometric instrument measures what it has been designed to measure. For example, if a test that is claimed to assess overall language proficiency measures only grammatical knowledge, the test is not valid in terms of evaluating communicative competence, although it may be perfectly valid with regard to the appraisal of grammar (in which case it should be called a grammar test).

Because of the salience of these terms in educational and psychological measurement, tens of thousands of pages have been written about them and every research manual will provide a detailed discussion about how to compute reliability/validity indices.

Questionnaires are measurement instruments and, accordingly, they too must possess adequate reliability and validity. Standardized questionnaires need to undergo rigorous validation procedures and the manuals usually present a variety of reliability and validity coefficients. For made-to-measure research instruments that we develop for our specific purpose, however, it is not always feasible to provide indices of every aspect of validity and reliability. Yet, even in cases where there are no resources and opportunities for elaborate validation exercises, we should strive for a questionnaire that has appropriate and well-documented reliability in at least one aspect: internal consistency. This attribute refers to the homogeneity of the items making up the various multi-item scales within the questionnaire. If your instrument has it, you can feel fairly safe.

Was this article helpful?

0 0
Online Survey Champion

Online Survey Champion

There are people all over the world trying to find ways to make money online. From stay at home moms looking to make a few extra dollars to college students and entrepreneurs, the allure of making your own hours and working from home or from the local coffee shop is very appealing.

Get My Free Ebook


Post a comment