Writing a data collection instrument: A practical guide

Version 2 - 5 September 2014

A. Checklist for self or external assessment 

Points

Tick  

Item

1.

 

Items on the instrument match the objectives of the study and the analysis plan: All items are relevant and no item is missing.

2.

 

Items do not suggest any specific answer through including information or other elements that could influence respondents.

3.

 

The instrument does not collect identifiers or surrogate identifiers (e.g., dates of birth). These are collected separately.

4.

 

For questionnaires, items are worded as full questions that can be asked directly to participants.

5.

 

The instrument includes a skip pattern that anticipates the need to jump specific items according to some answer.

6.

 

When multiple answers are possible, the instrument specifies whether one or multiple answers are acceptable.

7.

 

Questionnaires are ‘participant friendly’ (introduction, thank you statement, logic flow, clear wording, respect and tact).

8.

 

The instrument specifies how to collect information (e.g., structured observation, interview, record review)

9.

 

Items are well formulated (Clear, avoid negatives, specific, focused. Aim at collecting information about one topic and does not confuse different issues that should be the focus of two different questions)

10

 

The instrument is field worker friendly (Instruction for administration, guidance, auto-coding and numbering).

B. 10 common errors seen in data collection

   1. Data collection instrument not matching the objective of the study and / or the analysis plan

Description of the error

Rationale to change

The data collection instrument is not developed on the basis of the objectives of the study. It does not follow the analysis plan. Some items may be missing and some may be unnecessary.

The data collection instrument is a logical deduction of the analysis plan, not the reverse. All the items on the instrument must be written in anticipation of the analysis.

Examples of the error

Correction needed

  • Missing items (for example: For a study on factors associated with receiving antenatal care among pregnant women, no items on the geographical and financial access to antenatal care)
  • Presence of items that do not match the objectives of the study (for example: For a study on factors associated with receiving antenatal care among pregnant women, list of items about the kind of care that was given during antenatal care)
  • Excessive length, unnecessary questions
  • Analyze the problem, conduct a pilot qualitative study and make sure you make a good inventory of the items that need to be covered as per the study objectives.
  • Stick to items that match the study objective (for example: Stick to items on the knowledge, attitude and practices that may increase or decrease the probability of receiving antenatal care)
  • Shorten, focus on the objectives of the study
  

   2. Items influencing the participants

Description of the error

Rationale to change

The item is worded in a way that influences the participant to provide a specific answer.

For objective data collection, no items should suggest any answer.

Examples of the error

Correction needed

  • Do you know that the routine antenatal check up includes three visit?
  • Split in two questions: “What should a woman do for her health when she expect a baby? (Do not suggest answers but have an option ready for “antenatal visit” if mentioned)”. If antenatal visits are mentioned, ask “How many antenatal visits should take place?”
   

   3. Data collection instrument collecting identifiers or surrogate identifiers

Description of the error

Rationale to change

The data collection instrument contains information that can identify the study participant directly (Name, address, phone number) or indirectly (Date of birth).

All identifiers and surrogate identifiers must be eliminated from the instrument that may be accessible to many people: Data entry clerk, other staff.

Examples of the error

Correction needed

  • Name and address on the questionnaire
  • Date of birth on the questionnaire
  • Replace by ID code and keep track of identifier in separate identifier sheet kept under lock and key by the primary investigator.
  • Stick to year of birth if possible.
   

   4.    Questionnaire items not fully worded as questions

Description of the error

Rationale to change

Some questions on a questionnaire are not fully worded as questions ready to be asked to study participants.

For quality assurance purposes, each participant needs to hear the question in the exact same way. Thus. The exact wording should be proposed by the primary investigator and not left for the field worker to decide.

Examples of the error

Correction needed

  • “Education of woman”
  • “Did you attend school?” If yes: “What is the highest class that you attended?”
   

   5. Absence or inappropriate skip pattern

Description of the error

Rationale to change

The items follow each other on the instrument without any anticipation that some items may not apply to some people given their response to a previous question.

“Skip patterns” plan the use or non-use of specific questions according to the answer given to previous questions. These allow smooth administration of the questionnaire and avoid asking questions that do not apply a particular person (which could generate confusing answers).

Examples of the error

Correction needed

  • “Have you heard about tetanus vaccination during pregnancy?”, then “Did you receive tetanus vaccination during your pregnancy?” and “How many doses did you receive?”
  • Use: “What care should a woman receive during pregnancy?”, then skip other questions if tetanus is not mentioned. If mentioned, ask “Did you receive tetanus vaccination during your pregnancy” then skip other questions if answer is “No”. Then ask: “How many doses did you receive?” if answer was yes.
   

    6.    Unclear multiple answer options

Description of the error

Rationale to change

Items have multiple answer options but do not specify whether one of more than one answer options is acceptable.

From an analysis plan point of view, the number of acceptable answer options changes the nature of the question. If more than one answer option is acceptable, logically, that is equivalent to as many items as there are answer options with a dichotomous yes / no answer. If only one answer option is acceptable, then it is one item with a categorical variable.

Examples of the error

Correction needed

What are the reasons why you did not register your pregnancy at the health centre?

a. Services were not available

What is the main reason why you did not register your pregnancy at the health centre? (Choose one)

a. Services were not available

b. It was too far

b. It was too far

c. I had no time

d. I did not know it was needed

c. I had no time

Among these factors, which ones contributed to the fact you did not register your pregnancy at the health centre?

a. Services were not available Y/N

b. It was too far Y/N

d. I did not know it was needed

c. I had no time Y/N

d. Didn’t know it was needed Y/N

   

   7.  The questionnaire is not participant-friendly

Description of the error

Rationale to change

The questionnaire is thought as a good data collection tool but it has not been polished for use with study participants. Common errors include the absence of an introduction, the absence of thank you statement, the absence of logical sequence, jargon and inappropriate tone.

While the initial stage of the development of the questionnaire need to be centred around the analysis plan, the tool then need to be adapted for use with study participants.

Examples of the error

Correction needed

  1. Absence of introduction, transition sentences and thank you note.
  2. Poor logical sequence
  3. Have you had multiple sex partners in the last 12 months?
  4. Did you receive iron and folic acid tablets?
  1. Add introduction, transitions (“I will now ask you about your past pregnancies”) and thank you statement.
  2. Re-order sequentially, logically, from the general to the specific, from the public to the private.
  3. How many sexual partners have you had in the last 12 months?
  4. Did you receive the tablets against anemia?


   8. Lack of clarity about the data collection procedure

Description of the error

Rationale to change

The instrument is unclear as to whether the information needs to be collected through interview, review of document or observations.

To standardize the data collection procedure, specify exactly how the information needs to be collected if it is not through interviews.

Examples of the error

Correction needed

  • Type of house: a. Brick    b. Mud
  • Observe the house and write the type: a. Brick      b. Mud
  • How many doses of tetanus toxoid did you receive?
  1. Review the vaccination card. Number of tetanus toxoid doses received according to the card: _____
  2. If no vaccination record, ask the participant: How many tetanus vaccine doses did you receive?

   9. Poorly formulated questionnaire items

Description of the error

Rationale to change

Items are poorly formulated: They are unclear (e.g., excessive us of negative), unspecific or unfocused.

Poorly formulated questionnaire items will confuse the field workers and the study participants.

Examples of the error

Correction needed

  • Do you think it is important to attend antenatal clinic?
  • What are the benefit that antenatal clinic provide?
  • Did you miss vaccination because you thought you did not need it?
  1. How many vaccination doses did you receive?
  2. Do you think the vaccination was needed in your case?
  • Did you miss any of the antenatal visits?
  • How many antenatal visits did you attend?

   10. The questionnaire is not friendly to the field workers

Description of the error

Rationale to change

The instrument is not field worker friendly: It contains no instruction for administration; no guidance, no auto-coding features and no items are numbered.

A questionnaire that is easy to use for field workers will be better filled.

Examples of the error

Correction needed

  1. Absence of instruction for administration.
  2. No auto-coding
  3. No numbering of the questions
  1. Insert guidance for administration in italic or in a different font so that field workers know if the text is for their guidance or a question to read.
  2. Insert a column for auto-coding.
  3. Number each questionnaire item.