New Technologies for Survey Research I

Home | Examples | Exercise | Evaluation

Creation date: 11/01/99
Author: Karl Ho

This course introduces researchers to new technologies for conducting and administering surveys. This course focuses on creating Web surveys using FrontPage and SPSS or SAS for data analysis. Topics include:

1. How to achieve the best response rate
2. How to minimize invalid entries and missing values
3. Designing single- and multiple-response questions
4. Developing routines for reading data.

1. What is web survey?

Web surveys encompass a wide range of data collection processes using the World Wide Web.  They can be quick polls like the CNN Quick Vote or a formal survey collecting data from HMO patients.  Web surveys serve a variety of causes including collecting data, promotion, garnering support, and gathering opinion.  These surveys however do not necessarily serve the purpose of conducting scientific research and most of the polls conducted and administered on the web are for fun or other business goals (such as advertisement, gathering user information, customer data mining, etc.)  In this class, we will focus on the scientific, formal survey that serves the sole purpose of collecting data for scientific research.   There are plenty of resources and links on the internet that provide tools for conducting quizzes, quick polls and unofficial voting.  However, we will not cover these materials in the current course.

2.  Why survey on the web? What are the advantages and disadvantages of web surveys?

Web surveys become increasingly popular since they provide quick results and cost much less than the formal face-to-face interviews or traditional mailing questionnaire surveys.   Cost-effective as they are, web surveys may not necessarily generate "good data" for empirical analysis, let alone surveys lack of planning and close administration. The following table lists the advantages and disadvantages of web surveys:  

Table 1.  Advantages and Disadvantages of web survey

The ubiquity of web surveys has reached a point web users would not take these surveys seriously.  In some cases, users would lose trust on the surveys and the surveyors and become dubious whether it is worth the time and effort in filling out the forms.   However, web surveys still provide an excellent tools given the modern technology and change of living styles of respondents.  It is more likely for an office worker to fill out a web form in two minutes than dragging him or her to literally write on questionnaire and mail it back to the right address.  Administration cost is much lower on managing the questionnaire and responses but it shifts to the web management in terms of staff hours and server maintenance fees.

Good planning and strategy are warranted in order to achieve good data for analysis.   In the following, we proceed step by step in planning, implementing and administering a web survey:

1. Planning

It is imperative to plan ahead and set budget for conducting web survey.    Picture in mind what information you want to get from whom, when and how.   Sampling ranges from strict random sampled respondents to all who accesses the page.  It is recommended a pre-set sample be made instead of relying those who happen to find the page by chance.  Web surveys are usually plagued by surfers who just do it for fun.  Cleaning data from these ill-purposed entries can sometimes prevent the data from serving any research goals.  To circumvent this, the following procedures are advised:

1. Select your sample in advance via appropriate sampling method (e.g. random sampling, systematic sampling or stratified sampling)
2. Record these potential respondents in database
3. Send out letters to potential respondents with assigned identification numbers for authentication
4. Send out email to invite and remind potential respondents to participate
5. Ready reserved ID numbers for uninvited respondents

Pilot survey is very important to test out the technical difficulties and server reliability.  It also provides sample data for post-processing, i.e. analyzing and presenting the data.

2.  Statement

A statement for the web survey is must and should be identical to the one listed on the letter.  Stress:

• purpose of the survey
• funding of the project
• confidentiality
• availability of report upon request
• names and contact addresses of principal investigator(s)

3. Authentication

Once the sample is selected, assign identification numbers to each potential respondents.  Sample size for web survey is difficult to estimate.  It is recommended to play conservative and use the mail questionnaire rule of thumb: 10 percent response rate.  So, if you expect a data set of 300 cases, send out 3000 letters and/or emails.  In your questionnaire, assign a field for IDs.  This will be very useful for checking the integrity of incoming data.  One caveat about authentication is the respondent may be sensitive in giving particulars like name, addresses, phone number and even social security number.   It is recommended putting a statement prior to these field suggesting they are optional.  Also, leave an email link for those who are interested in participating but lose or do not have the identication number.  Send via email the non-assigned, reserved IDs.  Do this manually instead of using an automated ID creation system.  It will demonstrate the sincerity of surveyor and prevent abuse of automated system.

4. Creating Web survey forms

In Frontpage, most tools for a web survey are available.  They are under the Form Fields as follows:

It is extremely important to configure the form correctly.  Bear in mind the data entry process and the data analysis using the data collected.

Once you create a form (by selecting to create any form field), you can see the form is wrapped up with broken lines.  Put all the form fields and elements in one form, i.e. within one broken-line parameter.  Submit will then process all the filled in response in one batch, send the output to email or append results to the database.

Right click the form and select Form Properties. You can configure the properties of the form submission method, target and database type of the data being collected.

Choose Options at the Form Properties box to specify the type of database you want to create.  You can select a variety of formats but it is recommended to use Text database using tab as a separator:

You can also specify email as alternate reporting channel but you will have to handle tons of incoming messages and email messages are not good for data manipulation.

Before we proceed with the question choice design, it is important to bear in mind the whole data collection process.  When designing your forms, keep a log book that records what question represents what variable(s).  Assign name as variable name following the eight-character rule (good for SPSS and SAS handling) and assign numeric values instead of physical name values (e.g. assign 1 for Male and 2 for Female instead of "Male" and "Female").

In the following, we will cover the form fields one by one.

a. Text box

Text box is particularly good for respondents to enter alphanumeric values like ID, names and addresses. 

Remember that text box will return the alphanumeric values as is.  It is recommended using it as little as possible since processing the data will cause big problem.

Check box

Check box is good for multiple response questions, i.e. questions that allow respondents to select multiple answers.  The key of handling this kind of questions is to assign each box one single name and value.

Radio button

Radio button is good for single-response questions, i.e. there is only one response for the question:

Your gender is

Make sure you leave all radio buttons as not selected so bias could be prevented.

Drop-down menu

Drop-down menu serves similar purpose as radio buttons but saves space and allow multiple selection.  However, it is recommended check boxes be used for multiple response items because of ease of manipulation:

Once you create all questions, specify the submit button to perform the submission task and every hit of the button represents one single case.  Since the IDs are used, we can easily identify the duplicate cases.

Examples:

The UNT Musician Health Survey

Use of media in disseminating news and/or public service
messages

Computer use Survey

North Texan Survey

RSS surveys:

Evaluations
Administration web

Evaluation

MAIN

Last updated: 01/18/06 by Karl Ho