How To Pretest and Pilot A Survey Questionnaire PDF

Practical tools for international development
tools4dev HOW-TO GUIDE:
How to pretest and pilot

a survey questionnaire
www.tools4dev.org
About tools4dev
tools4dev provides templates, reviews and how-to guides for

international development professionals and aid workers. Find out more
at www.tools4dev.org
Copyright © tools4dev, 2014. All rights reserved.

How to pretest and pilot a survey questionnaire
How to pretest and pilot a survey

questionnaire
It’s important to test your survey questionnaire before using it to collect
data. Pretesting and piloting can help you identify questions that don’t
make sense to participants, or problems with the questionnaire that might
lead to biased answers. This guide explains how to conduct basic pretesting
and piloting for a survey.
This advice is for:
 Basic quantitative surveys such as feedback forms, needs assessments,

simple baseline and endline surveys, etc.
This advice is NOT for:
 Complex baseline and endline surveys or research studies.
 Developing new measurement instruments for use in research (e.g.

psychological instruments for measuring concepts such as confidence,
motivation, etc.).
 Qualitative focus groups or interviews.
Any testing is better than no testing

People often think that testing a survey takes a long time. They think they
don’t have the time or resources for it, and so they end up just running the
survey without any testing. This is a big mistake. Even testing with one
person is better than no testing at all. So if you don’t have the time or
resources to do everything in this guide, just do as much as you can with
what you have available.
Copyright © tools4dev, 2014. All rights reserved. 2

As a general rule, you should aim to pretest all your surveys and forms with
at least 5 people. Even with this small number of people you’ll be surprised
how many improvements you can make. Piloting is only really needed for
large or complex surveys, and it takes significantly more time and effort.
Pretesting
Find 5-10 people from your target group
Once you’ve finished designing your survey questionnaire, find 5-10 people
from your target group to pretest it. If you can’t get people from your exact
target group then find people who are as close as possible. I once designed a
survey that was going to be completed by garment factory workers in
another province. There wasn’t enough budget available for us to travel to
that province to pretest it, so we found some garment factory workers in our
own province to test it.
Try to get a range of different people who are representative of your target
group. For example, if your target group is young people aged 15-25, try to
include some who are younger, some who are older, boys and girls with
different socioeconomic backgrounds.
Photo by WhyOhGee

Although 5-10 people might not sound like many, you will usually find that
most of them have the same problems with the survey. So even with this
small number of people you should be able to identify most of the major
issues. Adding more people might identify some additional smaller issues,
but it also makes pretesting more time consuming and costly.
Ask them to complete the survey while thinking out loud
Once you’ve found your testers, ask them to complete the survey one at a
time (they shouldn’t be able to watch each other complete it). The testers
should complete the survey the same way that it will be completed in the
actual project. So if it’s an online survey they should complete it online, if
it’s a verbal survey you should have a trained interviewer ask them the
questions.
While they are completing the survey ask them to think out loud. Each time
they read and answer a question they should tell you exactly what comes
into their mind. Take notes on everything they say.
Observe how they complete the survey
You should also observe them completing the survey. Look for places where
they hesitate or make mistakes, such as the example below. This is an
indication that the survey questions and layout are not clear enough and
need to be improved. Keep notes on what you observe.

Make improvements based on the results
Once all the testers have completed the survey review your notes from each
session. At this point it’s normally clear what the major problems are so you
can go about improving the survey to address those problems. Normally this
is all that’s needed. However, if major changes are needed to the questions
or structure it might be necessary to repeat the pretesting exercise with
different people before starting the survey.
Piloting
Select the pilot sample
For large or complex surveys it’s a good idea to do a full pilot before starting
actual data collection. To do a pilot you need to test all the survey steps from
start to finish with a reasonably large sample. The size of the pilot sample
depends on how big your actual sample is, and how many data collectors
you have. For a typical baseline or endline survey a sample of around 30-50
people is usually enough to identify any major bugs in the system.

Implement all the steps from start to finish
Start by training your data collectors, if you have them. Then distribute and
collect the survey exactly as you would in practice. Enter the completed
surveys into the database that you plan to use and then test the analysis that
you plan to perform.
Make improvements
Assuming that the survey was pretested, piloting will normally identify
practical problems with implementation, rather than problems with the
survey design. For example, lack of staff training, challenges with the
logistics of distributing and collecting the survey, or errors in data entry.
These can then be fixed before you do the actual survey.

More from tools4dev

You can find more templates, how-to guides and reviews at the
tools4dev website www.tools4dev.org
Email Updates
You can also sign-up for tools4dev email updates from

www.tools4dev.org/signup/

How To Pretest and Pilot A Survey Questionnaire PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

How To Pretest and Pilot A Survey Questionnaire PDF

Uploaded by

Copyright:

Available Formats

Practical tools for international development

tools4dev HOW-TO GUIDE:

How to pretest and pilot

tools4dev provides templates, reviews and how-to guides for

Copyright © tools4dev, 2014. All rights reserved.

How to pretest and pilot a survey

This advice is for:

 Basic quantitative surveys such as feedback forms, needs assessments,

This advice is NOT for:

 Complex baseline and endline surveys or research studies.

 Developing new measurement instruments for use in research (e.g.

 Qualitative focus groups or interviews.

Any testing is better than no testing

Copyright © tools4dev, 2014. All rights reserved. 2

Copyright © tools4dev, 2014. All rights reserved. 3

Ask them to complete the survey while thinking out loud

Observe how they complete the survey

Copyright © tools4dev, 2014. All rights reserved. 4

Make improvements based on the results

Copyright © tools4dev, 2014. All rights reserved. 5

Implement all the steps from start to finish

Copyright © tools4dev, 2014. All rights reserved. 6

More from tools4dev

You can also sign-up for tools4dev email updates from

Copyright © tools4dev, 2014. All rights reserved. 7

You might also like