Although it was argued in Sections 1.3 and 2.5 that wide-open, essaylike questions do not work well in questionnaires and therefore should be avoided, questions that are slightly 'less open' can have some merits and are well worth experimenting with as long as this does not exist at the expense of the closed questions (in terms of response time or willingness). Because open-ended questions do not have precoded response options, their processing is less straightforward than that of closed items.
'Specific open questions' (cf. Section 2.5.1) usually ask about factual information that is easy to summarize. With an adequate coding frame (cf. Section 4.1.2), the responses to these items can be coded into distinct categories and then treated as nominal, or possibly ordinal, data (cf. Section 4.3.4).
With clarification questions, sentence completion tasks, and short-answer questions (cf. Sections 2.5.2 to 2.5.4), the categorization process involves more potentially subjective elements on the part of the coder. In order to avoid the harmful effects of such rater subjectivity, these items are to be processed by means of some systematic 'content analysis,' whereby the pool of diverse responses is reduced to a handful of key issues in a reliable manner. This is usually achieved through a stepwise process that involves two broad phases (for a detailed discussion, see Brown, 2001):
1. Taking each person's response in turn and marking in them any distinct content elements, substantive statements, or key points.
2. Based on the ideas and concepts highlighted in the texts (cf. Phase 1), forming broader categories to describe the content of the response in a way that allows for comparisons with other responses.
The categories obtained in Phase 2 can be numerically coded and then entered into the data file to be treated as quantitative data. Some of the key points highlighted in Phase 1 can also be quoted verbatim for the purpose of illustration and exemplification, or to retain some of the original flavor of the response.
Finally, although often omitted, qualitative data can also be checked for reliability, for example by computing intercoder agreement coefficients that describe to what extent two raters agree on assigning categories to the responses (see Brown, 2001, pp. 231-240).
Was this article helpful?
You won’t want to miss this. Get Paid Taking Surveys In Your Spare Time. I know what you’re thinking, ‘Oh great, another get rich quick scheme’. WRONG! This is your guide to making money from home by participating in paid surveys on the internet.