Version 2 - 5 September 2014
A. Checklist for self or external assessment
Items on the instrument match the objectives of the study and the analysis plan: All items are relevant and no item is missing.
Items do not suggest any specific answer through including information or other elements that could influence respondents.
The instrument does not collect identifiers or surrogate identifiers (e.g., dates of birth). These are collected separately.
For questionnaires, items are worded as full questions that can be asked directly to participants.
The instrument includes a skip pattern that anticipates the need to jump specific items according to some answer.
When multiple answers are possible, the instrument specifies whether one or multiple answers are acceptable.
Questionnaires are ‘participant friendly’ (introduction, thank you statement, logic flow, clear wording, respect and tact).
The instrument specifies how to collect information (e.g., structured observation, interview, record review)
Items are well formulated (Clear, avoid negatives, specific, focused. Aim at collecting information about one topic and does not confuse different issues that should be the focus of two different questions)
The instrument is field worker friendly (Instruction for administration, guidance, auto-coding and numbering).
B. 10 common errors seen in data collection
1. Data collection instrument not matching the objective of the study and / or the analysis plan
Description of the error
Rationale to change
The data collection instrument is not developed on the basis of the objectives of the study. It does not follow the analysis plan. Some items may be missing and some may be unnecessary.
The data collection instrument is a logical deduction of the analysis plan, not the reverse. All the items on the instrument must be written in anticipation of the analysis.
Examples of the error
2. Items influencing the participants
The item is worded in a way that influences the participant to provide a specific answer.
For objective data collection, no items should suggest any answer.
3. Data collection instrument collecting identifiers or surrogate identifiers
The data collection instrument contains information that can identify the study participant directly (Name, address, phone number) or indirectly (Date of birth).
All identifiers and surrogate identifiers must be eliminated from the instrument that may be accessible to many people: Data entry clerk, other staff.
4. Questionnaire items not fully worded as questions
Some questions on a questionnaire are not fully worded as questions ready to be asked to study participants.
For quality assurance purposes, each participant needs to hear the question in the exact same way. Thus. The exact wording should be proposed by the primary investigator and not left for the field worker to decide.
5. Absence or inappropriate skip pattern
The items follow each other on the instrument without any anticipation that some items may not apply to some people given their response to a previous question.
“Skip patterns” plan the use or non-use of specific questions according to the answer given to previous questions. These allow smooth administration of the questionnaire and avoid asking questions that do not apply a particular person (which could generate confusing answers).
6. Unclear multiple answer options
Items have multiple answer options but do not specify whether one of more than one answer options is acceptable.
From an analysis plan point of view, the number of acceptable answer options changes the nature of the question. If more than one answer option is acceptable, logically, that is equivalent to as many items as there are answer options with a dichotomous yes / no answer. If only one answer option is acceptable, then it is one item with a categorical variable.
What are the reasons why you did not register your pregnancy at the health centre?
a. Services were not available
What is the main reason why you did not register your pregnancy at the health centre? (Choose one)
b. It was too far
c. I had no time
d. I did not know it was needed
Among these factors, which ones contributed to the fact you did not register your pregnancy at the health centre?
a. Services were not available Y/N
b. It was too far Y/N
c. I had no time Y/N
d. Didn’t know it was needed Y/N
7. The questionnaire is not participant-friendly
The questionnaire is thought as a good data collection tool but it has not been polished for use with study participants. Common errors include the absence of an introduction, the absence of thank you statement, the absence of logical sequence, jargon and inappropriate tone.
While the initial stage of the development of the questionnaire need to be centred around the analysis plan, the tool then need to be adapted for use with study participants.
8. Lack of clarity about the data collection procedure
The instrument is unclear as to whether the information needs to be collected through interview, review of document or observations.
To standardize the data collection procedure, specify exactly how the information needs to be collected if it is not through interviews.
9. Poorly formulated questionnaire items
Items are poorly formulated: They are unclear (e.g., excessive us of negative), unspecific or unfocused.
Poorly formulated questionnaire items will confuse the field workers and the study participants.
10. The questionnaire is not friendly to the field workers
The instrument is not field worker friendly: It contains no instruction for administration; no guidance, no auto-coding features and no items are numbered.
A questionnaire that is easy to use for field workers will be better filled.