4: Error

What are the differences between the population and the sample data?

Sampling Error

Sampling error is inherent in the process of taking a sample. Every random sample will be different, and we can only hope that the sample will be representative of the sample frame.

Sampling error is the difference between an estimate of a population parameter (e.g. a proportion, median, mean) and the unknown and unknowable population parameter.

An estimate of a population parameter can only come from the sample parameter.

Taking a bigger sample can reduce sampling error (see 5: Sample Size).

It is hard to give a Merit- or Excellence-level discussion about sampling error. It's always there, and there's nothing we can do about it other than take a census. It is more interesting to discuss...

Non-Sampling Error

Non-sampling error is caused by other factors (not the sample selection). It makes the results of the survey biased in some way. This bias is usually not intentional.

Taking a bigger sample cannot reduce non-sampling error.

A very common example is that people in a sample may choose to not respond, or not respond truthfully, because the question (or their honest answer) is embarrassing. This is a cause of non-sampling error.

It is hard to know for sure whether there is non-sampling error from a specific cause. Some possible causes of non-sampling error are discussed below.

Coverage Error

The sampling method could be a source of coverage error. This is when a section of the population is under-represented due in the sample, and their responses differ from others in the population.

For example, a school sending an email home to all parents about homework would probably get proportionately less responses from one-parent families, and would not get responses from families without internet connection at all.

Non-response Error

The design of the survey could be a cause of non-response error. This is when some people in the sample do not answer.

If some sections of the population are less likely (or unable) to respond, then the results will be biased against their responses.

Reasons for non-response could include:

unable to contact
unwilling to participate
language barriers

Response Error

The design of the survey could be a cause of response error. This is when responses are unreliable in some way. This could be because of:

difficult to understand questions
unreliable recall from respondents
giving false, socially desirable responses
not giving true but embarrassing responses

For example, people might not admit to being a regular smoker in a health survey.

Interviewer Error

Conducting the survey could be a source of interviewer error. This is when the interviewer is not neutral and tries to influence responses, or assumes responses based on their own observations or assumptions.

Question Error

The design of the survey could be a cause of question error.

If questions are poorly worded, then responses might not be reliable. A leading question could encourage a desirable response.

If a question asks for self-reporting, then it relies on an accurate self-assessment. If the question is complicated, self-reporting may be unreliable.

Processing Error

The data analysis process could be a source of processing error. If the data entry has errors in it, this will affect the results.

If the data is entered by untrained staff who do not understand the data, they may enter similar but incorrect responses.

For example, if an interviewer was recording the electorate a voter lived in, they might get Rangitata and Rangitīkei mixed up.

The Psychology Behind Why People Don't Recycle

3 Aug 2016, Erin Shumaker, huffpost.com (article link)

The benefits of recycling seem straightforward. The practice reduces waste sent to landfills, conserves natural resources, reduces pollution and creates jobs. And the majority of Americans do recycle... sometimes.

Far fewer, however, do it consistently.

Indeed, according to a 2011 Ipsos Public Affairs [online] survey, only half of adults recycle daily. Another third of respondents said they recycle less frequently than that, and a full 13 percent revealed that they never recycle.

The primary excuse people gave for not recycling [25%] was that recycling wasn’t convenient or accessible to them.

According to The Economist, about a quarter of Americans don’t have access to curbside recycling.

While doing an informal survey of college students in Arkansas, Jessica Nolan [an associate psychology professor at the University of Scranton] noticed that individuals from the same town sometimes answered differently about whether or not a recycling program existed in their hometown.

“If you’re not interested, you might think you have no recycling program, but in fact you do,” she said. “If there’s a drop off center, you wouldn’t see it unless you went looking for it.”

199 words

Focus Questions

What are the sources of bias in the survey?
Could the survey questions be misinterpreted or not give the data needed?
What has been done (or should have been done) to mitigate sources of non-sampling error?

Possible Answers

The participants in the survey might be over-reporting the amount that they recycle, either to impress the interviewer or feel less embarrassed about not doing enough to recycle.

When asking "whether or not a recycling program existed in their hometown", respondents might give conflicting answers about the same town if they have different definitions of a "recycle program". Some might think this only means curbside recycling, while others include drop off centers. This question may need more refining.

By conducting the survey online, responses should be more honest than having to admit to an interviewer in person to never recycling.

Task 5

Answer the Non-sampling Errors focus questions about the Rotorua report:

What are the sources of bias in the survey?
Could the survey questions be misinterpreted or not give the data needed?
What has been done (or should have been done) to mitigate sources of non-sampling error?

Report-Rotorua.pdf

Google Sites

Report abuse