Randomness of SamplE

Does Random Sampling ReaLly Work?

Krause (2019)

Random chance is a mathematically elegant notion that logically is inapplicable for social science empirical research purposes. It nevertheless has been widely used to evade, rather than solve, some critical problems of current social science research methodology.

Such problems must either be solved or admitted to be insoluble rather than merely statistically finessed by making unjustifiable randomness assumptions if social science is to soundly progress.

The only obvious way to achieve random sampling requires the population to be fully listed or assembled and every one of its members to be accessible for sampling, because only then could all of its members have an equal probability of being drawn in a sample from that population.

This would be a rare circumstance for many of the populations of interest to the social sciences because of there being persons who are unlocatable, unavailable, or unwilling to participate.“

Hulland et al. (2017)

When the primary aim of the research is to test the veracity of proposed theoretical effects, the use of a convenience sample may suffice.

If the goal is to test theoretical hypotheses of interest, as is usually the case, the most important consideration is to select measurement objects and a research context in which the hypotheses can be meaningfully tested, using sources of data (both primary and secondary) that yield accurate information about the units studied.

.... However, if the researcher is interested in a particular target population to which generalizations are to be made, an explicit sampling frame has to be specified and the manner in which the sample was drawn needs to be clearly and explicitly described.

Based on our review of published survey research, we classified 111 studies (55.0%) as using an explicit sampling frame, and 86 studies (42.6%) as being based on convenience samples.

In many cases we found the descriptions provided for the more complex sampling procedures to be confusing and incomplete.

..., survey researchers follow certain established survey practices ritualistically, even when they are not directly relevant to the research in question or when they are not particularly meaningful. For example, a focus on coverage, sampling, and non-response error seems misplaced when there is no real target population to which the researcher wants to generalize the findings.

However, in typical academic marketing surveys there is often no obvious target population to which the researcher wants to generalize the findings, the sample studied is arbitrary (e.g., chosen based on ease of access), and it is difficult to talk about selection bias when the sample is one of convenience (although nonresponse will lead to loss of power).

Are you sure that your research is generalizable?

Generalizability

Generalizability refers to the applicability of research in one organizational setting to other settings.

Findings that participation in decision making enhances organizational commitment are found to be true in a variety of manufacturing, industrial, and service organizations, and not merely in the particular organization studied by the researcher, then the generalizability of the findings to other organizational settings is enhanced.

The more generalizable the research, the greater its usefulness and value. However, not many research findings can be generalized to all other settings, situations, or organizations.

Two distinct types of generalizability are identified in consumer research.

One entails the application of specific effects, whereas
The other entails the application of general scientific theory.

Effects application and Theory application rest on different philosophical assumptions and have different methodological implications.

Effects Application/Effects Generalizability

The first type of generalizability, which we term effects application, maps observed data directly into events beyond the research setting.

That is, the specific effects obtained are expected to mirror findings that would be observed if data were collected for other populations and settings in the real world.

Theory Application/Theory Generalizability

The second type, which we term theory application, uses only scientific theory to explain events beyond the research setting.

Effects observed in the research are employed to assess the status of theory.

But, it is the theoretical explanation that is expected to be generalizable and not the particular effects obtained.