STAT-C 225 (SURVEY SAMPLING)

Course Packs on Survey Sampling

Course : STAT-C 225 Survey Sampling

Program : BS Statistics

Year Level : 2

Semester : Second

Lesson 1. Survey Sampling Concepts and Definition

What is survey sampling, and how is it used for research?

Survey sampling is a statistical process that involves selecting and surveying individuals from a particular population. The population you choose to survey could be based on a range of attributes. The target audience could be the United States general population or perhaps a more specific group, like 18-25-year-olds, voters in Ohio, or female pet owners from the Midwest. Your sample options are endless, really.

By asking survey questions and collecting data on a subset of your target population (a “sample”), you can make inferences about the whole population. Suppose 48% of those in your sample say they support a particular political candidate. In that case, you can assume that 48% of the target population feels the same way, within a certain margin of error. Survey sampling is used in statistical research, economics, marketing, clinical and academic studies, and political polling.

How does survey sampling work, exactly? At the most basic level, it involves three steps:

• Sample selection:

First, you must decide who will be in your sample. Many sampling methods can help you select survey respondents who are representative of your target population. This first step is crucial because a quality sample ensures your results’ validity.

• Data collection:

Next, you can use online surveys, interviews, or other methods to ask open- and closed-ended questions from your sample. Their answers give you raw data.

• Estimation:

After collecting data, you can make estimates about your general population using precise statistical calculations.

Conducting sampling via online surveys can be an intricate process. You must ensure accuracy and validity at every step to get reliable results. First, it is important to select a sample that truly represents the population you are researching. Your survey must prompt users with clearly-worded, unbiased questions. Then, you must use the correct statistical equations to make sense of your data and determine your margin of error, confidence intervals, and other statistics. To wade through these complexities, let’s talk more about how survey sampling is applied to research.

Why Is Survey Sampling Important?

In an ideal research world, you would be able to gather data from everyone within a target population. However, most of us do not have the budget or resources to procure that kind of data. Perhaps if your target population is small enough — say, all the residents of a small town or all the students at a particular university — that might be possible. A survey that includes 100% of your target population is called a census. However, if your population is much larger — like all of the small business owners in the U.S. — it’s harder to survey everyone. There’s a reason the U.S. Census only takes place once a decade.

That’s why survey sampling is so important. It lets you gather data on a representative portion of your target population. The operative word here is “representative.” For a survey to accurately reflect the target population, you must carefully select your sample. It must be balanced to include the same demographics as your target population and cannot be too big or too small. A well-designed sampling method will give you reliable insights into your target market, research subjects, or political constituency.

[1] Advantages of Survey Sampling

Survey sampling is often the only practical way to gather data on a large population. Thanks to its many advantages, it’s usually the preferred method, even when other data collection strategies work. Sample survey advantages include:

• It’s Relatively Low Cost

Where conducting a census requires immense resources, a survey is much more cost-effective. You can find survey participants easily from across the web and pay just a set amount per survey taker.

• It’s Fast and Convenient

It’s much easier to survey a select sample of the population than to take a census. You can ask detailed questions and receive your data quickly. A survey is pretty quick and easy for respondents, too. When you field your survey online, it’s easy for survey-takers and for you to view and analyze your data.

• You Can Conduct Research on a Broad Scale

When your sample is accurate and representative, you can make inferences about large populations. It’s challenging to study such a vast scope with a census. When working with large populations, their compositions will change by the time you finish collecting data. Others are so large that censusing them is impossible.

• The Results Are Accurate

If you set the right quality checks in place for your sample, it’s possible to achieve highly accurate results. By working with fewer participants, you can take more care analyzing the data to reach reliable conclusions.

• You Can Gather Intensive and Exhaustive Data

By surveying a small group, you can ask your participants more questions. If asking open-ended questions, you must work with a sample to make sense of the responses people give. You can also ask a larger quantity and variety of questions, including multiple-choice and numerical scale ratings. It’s only possible to gather and analyze such exhaustive data when working with a sample.

[2] Disadvantages of Survey Sampling

While survey sampling offers many benefits, it has a few downsides as well. Those disadvantages include:

• There’s Potential for Errors

Collecting high-quality sample results requires thoughtful survey design and setting the right qualifying criteria for respondents. While population subgroups can provide responses with incredible accuracy, some sampling methods create bias and errors. A sampling error occurs when the mean values in the sample are different from the mean values in the entire population. It’s challenging to calculate your sampling error because you won’t have access to the whole population’s mean values. However, when you use randomization during sample selection, you can estimate sampling error more accurately.

Other errors can affect data quality, too. If the researchers don’t understand their target population, they may survey the wrong people. If they select a sample from the wrong population subset, the results won’t be accurate. If specific individuals are more likely to respond to the survey than others, this will also introduce inaccuracies.

• Developing a Representative Sample Takes Careful Consideration

When you’re studying complex phenomena and types of people, it’s hard to develop a sample that truly represents the larger population. A thoughtful sample balance is crucial.

• It Requires Thorough Statistical Knowledge

The researchers conducting the survey need to understand sampling well to cultivate quality results. The person doing statistical calculations based on your data needs to understand the mathematics thoroughly. Without this specialized knowledge, your survey research and conclusions can be misleading.

• It Can Be More Challenging for Very Niche or Diverse Populations

If you are targeting a niche population, creating a representative sample can be complicated. The same is true when there are many variations within the target population. In these cases, a census may provide more reliable results.

• Some Survey Questions Are Less Likely to Yield Accurate Answers

Questionnaire design is also integral to the survey sampling process. For example, asking about highly controversial topics may mean participants misremember facts or let their emotions cloud their judgment. If questions are confusing, people may interpret them differently. Sometimes, the question wording may elicit specific responses, creating bias. It’s also important to be mindful of respondent experience when designing a survey. If your survey is too long or has too many open-ended questions, respondents may get fatigued and provide less thoughtful responses.

[3] Sample Selection

Sample selection is arguably the most crucial element of sampling for survey research. This step is how you decide who participates in your survey. Choosing a sample that reflects your target population is critical to getting accurate results. That’s because the statistical processes you apply later will assume that your sample is representative of that population. If it is not, you could generate erroneous data.

Many different techniques can produce a sample of your chosen population, and each one will result in a slightly different sample set. Even within each sampling method, there’s potential for variation. If you sample 500 people this week and an additional 500 people the next week, you’ll probably see slightly different results. Sample quality is integral to sound survey research on which you can draw valid conclusions.

[4] Sampling Bias

Through your sample selection, you want to eliminate as much sampling bias as possible. Sampling bias is when certain population members are systematically more likely to be selected for the sample than others. This factor limits your findings.

For example, if you’re researching consumer opinions about cable providers, you might select your sample from a list of home phone numbers. However, people who have home phone numbers are more likely to be cable subscribers. You’re less likely to get opinions from people who are less satisfied with the service.

Sampling bias is usually curbed by random sampling methods, although not eliminated. That’s because there can be bias within the sample frame. A sample frame is the entire set of units you can draw samples from. In the example above, the sample frame is the list of home phone numbers. Even if you choose the phone numbers included in the survey randomly, you’ll still have an inherent bias.

Any sampling introduces bias. If you wait outside a grocery store and ask people to fill out a survey, your sample frame will only include people who visit the grocery store. When surveying males in their early 20s, you may have an overrepresentation of a particular latent characteristic, such as media usage. In other words, the individuals in your sample use media more readily than the general population of early 20s males. If you use this sample, your results will contain biases, especially if the survey is about media.

[5] Population Parameter vs. Sample Statistic

Population parameters and sample statistics are closely related concepts in survey sampling. Because they’re so similar, people often confuse the two. It’s crucial to understand the difference.

[5.1] Population Parameter

A parameter is a fixed value that describes an entire population. It can only be discovered through a census. Say you ask all 100 employees at your office what their favorite food is, and 50 people say pizza. It’s a population parameter that 50% of your office likes pizza.

[5.2] Sample Statistic

In most cases, it’s impossible to determine a population parameter. If you wanted to know the percentage of people living in the U.S. who have pizza as their favorite food, you couldn’t easily find a population parameter. Instead, you would use survey sampling to produce a sample statistic. It’s similar to a population parameter, except that it describes a sample. The sample statistic’s goal is to estimate the population parameter. If you survey several samples of the same population, you’ll get slightly different sample statistics each time.

[6] Things to Keep in Mind When Creating a Sample:

When sampling, you must consider many factors to ensure a valid, representative population subset. Some things to keep in mind include:

• Diversity

A sample must be as diverse as the target population. It must represent all the variations between people naturally occurring within your population in similar proportions. If your target population is the general Philippine population, your sample must represent the many regions, income levels, ethnicities, education levels, and other demographic variations present in the Philippines.

• Transparency

As a researcher, you must know where your samples come from.

• Consistency

When comparing data over time, such as in a brand tracker study, inconsistencies in the sample frame are a particular concern. Survey panels change over time, meaning different people have the chance to be included in your survey. Meanwhile, each survey panel has a different composition. They come from various sources, even if they contain similar populations. Inconsistencies affect your sample’s quality significantly.

Let’s say you work with a particular survey distributor to provide sample for your study. You distribute the same survey year after year using the same distributor for five years. However, over the five years, the survey panel’s users change. Some people stop using the panel altogether, and new users have signed up. Meanwhile, the users who took the survey five years ago have now aged. Even small shifts in this sample frame could create year-over-year trends in your data that don’t reflect the real trends within your target population.

[7] Sample Size

One of the most important decisions related to your sampling is the sample size. It must be large enough to represent the population and all the potential variables at play. However, making the sample size too large can increase the survey’s cost and logistics. Also, a sample size that’s too large can be just as inaccurate as a sample that’s too small. If there’s any bias or flaws in the study design, a sample size that’s too big will magnify the inaccuracies. You can determine your sample size using the sample size formula.

[We will have a separate discussion on how solve for the sample size using different sample size formula using MS Excel]

[8] Survey Sampling Methods

There are many ways to create a sample for your survey. The sampling methods divide into probability sampling, sometimes called probabilistic sampling, and nonprobability sampling, or nonprobabilistic sampling. These two sampling strategies break down even further into several sampling methods.

[Probability Sampling is more reliable because it uses randomization to choose survey participants]

Probability vs. Nonprobability Samples

When selecting respondents for your sample, you have the choice between probability sampling and nonprobability sampling. Probability sampling is more reliable because it uses randomization to choose survey participants. The Office of Management and Budget’s Standards and Guidelines for Statistical Surveys for government agencies requires generally accepted probabilistic methods unless another method can be statistically justified.

Nonprobability sampling doesn’t use random selection. So, participants don’t have an equal chance of being included in the sample. It is often easier to undertake. Nonprobability sampling could even be the preferred method, usually in qualitative research. In some studies, researchers purposely choose certain participants because they can offer unique insights into a topic. However, for quantitative analysis, probability sampling will always be the preferred method, and statistical researchers will use nonprobability selection only for its practicality.

[8.1] Probability Sampling Methods

In probability sampling, everyone in the sample frame has the same chance to be included in the sample. You can also calculate any member’s probability of being included in the survey. These sampling techniques are more reliable and allow researchers to make more accurate inferences about a population.

You can use several probability sampling methods, and each has distinct advantages and disadvantages. Those methods include:

[8.1.1] Random Sampling

Simple random sampling is the most basic form of probability sampling. It involves just one step, and each survey subject is selected independently from the other members of the population or sample frame. A standard method of random sampling is assigning every individual in the sample frame a unique number. Then, a random number generator determines who will be in the sample.

The benefit of random sampling is that each member of the population has an equal chance of being chosen for the survey. This characteristic makes a simple random sample highly representative of the target population.

However, with larger populations, it’s hard to include every individual in the random selection process. Instead, you would draw participants from a sample frame, such as a list of email addresses or phone numbers. Any sample frame will eliminate some members of the population since it’s impossible to contact everyone. It’s time-consuming to conduct random sampling with larger populations and larger sample sizes. A biased sample frame can also skew your results.

Example:

You want to select a simple random sample of 100 employees of Company X. You assign a number to every employee in the company database from 1 to 1000, and use a random number generator to select 100 numbers.

[8.1.2] Systematic Sampling

Systematic sampling is a little easier than random sampling and is similar in reliability. In this method, you assign everyone in the target population or sample frame with a number. Instead of using a random generator, you systematically select candidates at regular intervals. For example, you could select every fifth number or every 20th number.

A systematic sample is highly representative. However, it’s not quite as random as using a random number generator. There’s also a chance that the list’s organization could compromise randomness. When sampling systematically, it’s essential that the list doesn’t have any hidden patterns. If you’re surveying people at a company, the list could divide employees by department and sequence them by rank. It’s a good idea to shuffle a list of names in alphabetical order or otherwise organized.

Example:

All employees of the company are listed in alphabetical order. From the first 10 numbers, you randomly select a starting point: number 6. From number 6 onwards, every 10th person on the list is selected (6, 16, 26, 36, and so on), and you end up with a sample of 100 people.

[8.1.3] Stratified Sampling

Stratified sampling attempts to account for the demographics and traits of the larger population. It attempts to recreate the elements in the sample. For example, if you’re surveying college history majors, and you already know that 40% of history majors are female and 60% are male, you might want your sample to have the same proportions.

Before generating a sample, the researchers first decide what traits or dimensions are significant. They may want to account for gender, social class, age, religion, education level, or other characteristics. Then, they randomly sample within each chosen category. If the researchers know that 70% of their target population is not college educated, they’ll ensure 70% of their survey participants are also not college educated.

This sampling method can better replicate the demographics of the target population. It’s especially useful when one category is a small minority compared to the others. In simple random sampling, this demographic could be underrepresented or even nonexistent in the sample. Stratified sampling is complex and time consuming. It could be challenging to find participants in the target population that meet the other demographic criteria.

Example:

The company has 800 female employees and 200 male employees. You want to ensure that the sample reflects the gender balance of the company, so you sort the population into two strata based on gender. Then you use random sampling on each group, selecting 80 women and 20 men, which gives you a representative sample of 100 people.

[8.1.4] Cluster sampling

Cluster sampling also involves dividing the population into subgroups, but each subgroup should have similar characteristics to the whole sample. Instead of sampling individuals from each subgroup, you randomly select entire subgroups.

If it is practically possible, you might include every individual from each sampled cluster. If the clusters themselves are large, you can also sample individuals from within each cluster using one of the techniques above. This is called multistage sampling.

This method is good for dealing with large and dispersed populations, but there is more risk of error in the sample, as there could be substantial differences between clusters. It’s difficult to guarantee that the sampled clusters are really representative of the whole population.

Example:

The company has offices in 10 cities across the country (all with roughly the same number of employees in similar roles). You don’t have the capacity to travel to every office to collect your data, so you use random sampling to select 3 offices – these are your clusters.

[8.2] Nonprobability Sampling Methods

Nonprobability sampling methods do not use any randomization to select survey participants. Therefore, population members do not have an equal chance of being included. Some members may have no chance of being in the sample. Others may have a much higher chance, and they will have disproportionate representation in the sample. Nonprobability sampling has limited applications for quantitative researchers who need quality samples.

Some nonprobability sampling techniques include:

[8.2.1] Convenience Sampling

Convenience sampling includes participants based on their availability and accessibility. Essentially, it includes people who are easy to reach. If you’re an academic researcher, it’s easy to sample people from your own institution. It’s even more convenient to survey your own students or classmates.

One way to conduct convenience sampling is to wait in a crowded location and approach people to participate in a survey. When enough people agree to meet your sample size, you stop the survey.

Convenience sampling is beneficial because it lets you collect data quickly. It’s usually inexpensive to sample these participants because you can take advantage of a readily available sample. You also don’t need to follow strict rules to ensure randomization. The downside is that the survey won’t be representative of the target population.

Example:

You are researching opinions about student support services in your university, so after each of your classes, you ask your fellow students to complete a survey on the topic. This is a convenient way to gather data, but as you only surveyed students taking the same classes as you at the same level, the sample is not representative of all the students at your university.

[8.2.2] Voluntary response sampling

Similar to a convenience sample, a voluntary response sample is mainly based on ease of access. Instead of the researcher choosing participants and directly contacting them, people volunteer themselves (e.g. by responding to a public online survey).

Voluntary response samples are always at least somewhat biased, as some people will inherently be more likely to volunteer than others.

Example:

You send out the survey to all students at your university and a lot of students decide to complete it. This can certainly give you some insight into the topic, but the people who responded are more likely to be those who have strong opinions about the student support services, so you can’t be sure that their opinions are representative of all students

[8.2.3] Snowball Sampling

Snowball sampling relies on the first survey participants to refer you to the next ones, and so on. Once you’ve found enough people to meet your required sample size, you stop the survey.

One advantage of snowball sampling is it allows surveyors to find people who are generally hard to reach. If people in the target population don’t want to be found, like those involved in illegal activity, their contact information is not readily available. The snowball method creates a kind of word-of-mouth marketing that makes it easier to find participants.

The downside is that it’s impossible to know how representative your sample is. These surveys generally create a very homogeneous sample.

Example:

You are researching experiences of homelessness in your city. Since there is no list of all homeless people in the city, probability sampling isn’t possible. You meet one person who agrees to participate in the research, and she puts you in contact with other homeless people that she knows in the area.

[8.2.4] Purposive sampling

This type of sampling, also known as judgement sampling, involves the researcher using their expertise to select a sample that is most useful to the purposes of the research.

It is often used in qualitative research, where the researcher wants to gain detailed knowledge about a specific phenomenon rather than make statistical inferences, or where the population is very small and specific. An effective purposive sample must have clear criteria and rationale for inclusion.

Example

You want to know more about the opinions and experiences of disabled students at your university, so you purposefully select a number of students with different support needs in order to gather a varied range of data on their experiences with student services.

[8.2.5] Quota Sampling

Quota sampling is similar to stratified sampling. The difference is that this method doesn’t randomly select participants. As with stratified sampling, the researchers first define categories they want to represent in their sample and choose appropriate proportions for each group. These could be equal quotas, like 100 men and 100 women, or they could seek to replicate a target population’s demographics.

Instead of randomly selected participants, the surveyors will use some form of convenience sampling. When they’ve hit the right quotas for each category, they stop the survey.

One benefit of quota sampling is that it can represent the target population more accurately than convenience or snowball sampling. This survey method can cover many different characteristics and handle a lot of complexity. While it is the most accurate of the nonprobability techniques, it’s still not as representative as probability methods.

[9] How to Determine What Type of Survey Sampling Is Best for Your Application

Each type of survey sampling has a place in research. Nonprobability sampling, while less accurate, can be a practical way to begin the research process. Probabilistic sampling can provide more accurate results, which can be used to generalize about the target population. Here’s when to use each type of sampling:

• Random sampling:

As a highly representative method, random sampling has the widest applications. It’s particularly useful for large populations. It’s not ideal for studying an uncommon characteristic in a large group, as you may not randomly select enough participants with the trait.

• Systematic sampling:

If your population is relatively homogenous, you can use the faster and more convenient systematic sampling method.

• Stratified sampling:

When you need to represent many individual characteristics from the target population, choose stratified sampling.

• Convenience sampling:

Since convenience sampling is fast, it’s suitable for preliminary research. You can get a general idea of a more precise study’s results without the upfront time or cost of a probabilistic sample.

• Snowball sampling:

When your target population is homogenous and generally hard to reach, snowball sampling can help. It’s helpful for surveying illicit drug users or those involved in illegal activity.

• Quota sampling:

When a more rigid and costlier study would use stratified sampling, preliminary research can instead use quota sampling. Like stratified sampling, it’s used when the researchers can identify specific traits within the target population that they want to study. Market researchers often use this method.

Watch Video: https://www.youtube.com/watch?v=pTuj57uXWlk

References:

[1] https://luc.id/knowledgehub/what-is-survey-sampling-and-how-is-it-used-for-research/

[2] https://www.statisticshowto.com/probability-and-statistics/find-sample-size/

[3] https://www.scribbr.com/methodology/sampling-methods/

[4] https://www.youtube.com/watch?v=pTuj57uXWlk