random sampling eliminates bias in choosing a sample and allows control of variability. So once we see the magic words “randomly selected” and “margin of error,” do we know we have trustworthy information before us? It certainly beats voluntary response, but not always by as much as we might hope. Sampling in the real world is more complex and less reliable than choosing a Simple Random Sample (SRS) from a list of names in an exercise. Confidence statements do not reflect all of the sources of error that are present in practical sampling.
Most sample surveys are afflicted by errors other than random sampling errors. These errors can introduce bias that makes a confidence statement meaningless. Good sampling technique includes the art of reducing all sources of error. Part of this art is the science of statistics, with its random samples and confidence statements. In practice, however, good statistics isn’t all there is to good sampling. Let’s look at sources of errors in sample surveys and at how samplers combat them.
•Sampling errors Random sampling error is one kind of sampling error. The margin of error tells us how serious random sampling error is, and we can control it by choosing the size of our random sample. Another source of sampling error is the use of bad sampling methods, such as voluntary response. We can avoid bad methods. Other sampling errors are not so easy to handle. Sampling begins with a list of individuals from which we will draw our sample. This list is called the sampling frame.
The Dissertation on Sampling Probability
... sampling In quota sampling, the sample has not been chosen using random selection, which makes it impossible to determine the possible sampling error. ... restrictions on the decision processes involved in choosing groups, multi-stage sampling has a level of subjectivity. Thus, there ... quotes are important, surveys to convenience samples can be a great source of rich verbatim comments on specific ...
Ideally, the sampling frame should list every individual in the population. Because a list of the entire population is rarely available, most samples suffer from some degree of under-coverage. If the sampling frame leaves out certain classes of people, even random samples from that frame will be biased. Using telephone directories as the frame for a telephone survey, for example, would miss everyone with an unlisted telephone number.
More than half the households in many large cities have unlisted numbers, so massive under coverage and bias against urban areas would result. In fact, telephone surveys use random digit dialing equipment that dials telephone numbers in selected regions at random. In effect, the sampling frame contains all residential telephone numbers.
•Non-sampling errors Non-sampling errors are those that can plague even a census. They include processing errors mistakes in mechanical tasks such as doing arithmetic or entering responses into a computer. The spread of computers has made processing errors less common than in the past. Another type of non-sampling error is response error, which occurs when a subject gives an incorrect response.
A subject may lie about her age or income or about whether she has used illegal drugs. She may remember incorrectly when asked how many packs of cigarettes she smoked last week. A subject who does not understand a question may guess at an answer rather than appear ignorant. Questions that ask subjects about their behavior during a fixed time period are notoriously prone to response errors due to faulty memory.
