Any student of research methods learns that the quality of a survey, or any other measurement tool, is determined by its validity (the ability to reflect the subject being measured) and reliability (the ability to be consistent in a manner where two employees with a similar stance regarding a specific question would score it the same way every time).
While validity is difficult to evaluate, since it requires additional, “objective” measures by which the survey’s results are examined, the subject of reliability is relatively more convenient to handle since it mainly depends on the approach used to build the questionnaire and analyze the data.
An unreliable survey might be not only worthless, but also have negative consequences since it creates a distorted view of the subject being measured and may lead to erroneous and harmful actions being taken. It is possible that of all the factors which determine the quality of a survey, the subject of reliability is the most critical.
Despite its academic aura, the subject of reliability is completely practical and is an essential part of the survey and not an esoteric aspect thereof.
Therefore, it is astounding that the subject does not receive the appropriate attention which it deserves from those engaging in this field, to say the least. Generally, it can be said that the study and awareness of methodological issues in the field of organization attitude surveys is very sparse and contrary to the popularity of surveys over the past few years.
There are many reasons for this being the case, but we believe the most significant one lies in the difficulty to distinguish the difference between a reliable survey and one which is not upon first glance. The emergence of “automatic” internet systems has worsened the situation since emphasis has shifted towards processing and presentation rather than content and analysis, and the survey’s “soul” has been slightly lost along the way.
For a detailed review of reliability aspects in our article here:-