Writing clear survey questions is incredibly important, complicated, and difficult. There are entire books on the topic of survey question design. The wording of the question I showed you here is far from ideal. But what is the right way to ask how many kids are in someone's family? You have to write the question such that the participant understands how to answer, so that the participants understand the question the same way as each other, and so that the participants' understanding is the same as your own. 

You can't write, "How many siblings do you have?" without adding qualifications about step and half siblings. You can't write, "How many kids lived in your house when you were growing up?" because this is not the same as asking about siblings, and siblings don't necessarily live with you. You can't write, "How many kids do your parents have?" because the answer might be different for each parent, and the definition of "having a kid" can be complicated. Etc. So, the question I wrote on the survey was an imperfect attempt to generate useful information and also give me a chance to talk about question design a little!

Nope! For one thing, it is silly to assume that each of the late-joining students would have the same family size as each other and as the overall average. Also, don't we have additional information about these students that would help us make a better guess?

We would add 10 more people to the first bar in the bar chart, and 1 would become the most common answer.

Missing completely at random is the best we can hope for! If there is no relationship between whether the values are missing and what the values would have been, then missingness completely at random just means that we have a smaller sample size, but our data set will still be representative of the target population (if it started out that way!).

Job category or education come to mind. It would be great if you had both. There is very likely a correlation between education level and income. People tend to refuse to provide their incomes when their incomes are on the high end or on the low end. So, you can't just look at the people who provided income information and pretend you have a good summary of income. But you can rely on the correlation between income and education and job to help you make good guesses at the missing incomes.

Assuming missing at random in this case is like pretending that there is a bucket for each education level, and people were randomly selected from certain education levels to refuse to provide their incomes. Perhaps only 1% of people who finished college refuse to give their incomes, but perhaps 40% of high school graduates and 50% of medical school graduates refuse to provide income (making up those percentages).

I don't think that most of the people who refuse to answer are cocaine users. However, I do think that the proportion of cocaine users among those who refuse to answer will be much higher than the proportion of cocaine among those who do answer the question. There is a correlation between whether you answer the question and what your answer would be.

When you are collecting the data! That is your chance to include multiple variable that are correlated with each other, so that you can hope for missingness at random instead of not at random.

The actual value is 2. This isn't the point, though. I was hoping that you would start to understand what the goal is when we are handling missing data and what strategies might be available. As you watch the rest of this lecture, see if you recognize any of the strategies that you considered as you answered this question.

The main one that I think of is that we don't have to make guesses at what the missing values should have been, as we do in most of the remaining strategies for handling missing data.

When data is MNAR, there is a correlation between the values themselves and whether they are missing. To create MNAR data for this example, I threw out any value of education above 13. So, the range - and therefore the variance - of education decreased.

Zero. The software will exclude any person who is missing at least one of the variables used in the model. So, we need a better strategy than dropping missing values.

The variance is the mean of the squared differences from the mean. Sloppily, it is the average distance from the middle. If you insert a bunch of new data points that are right at the mean, the average distance from the mean decreases.

A histogram of the variable after imputing will look a lot like a histogram of the observed values. Mean, variance, and shape of distribution are preserved. This is desirable only if the distribution of observed values was representative of the population distribution - so, only when data is missing completely at random.

Nope! Imputing based on the observed values of another variable only works if you correctly guess which other variables are related to the missingness of the variable of interest; if these other variables are actually observed in the data set; and if you correctly model the relationship between the variables. Obviously, you never guess all of this perfectly. But it is often possible to do a fairly good job if you know the context well or are working closely with collaborators who understand the context.

No. We use our knowledge of the context to figure out the type of missingness. There is some information in the data. For example, you may be able to tell that the rows where a variable is missing are different from the rest of the data on some other variable - this implies that you don't have missingness completely at random, at least.

If you randomly impute the missing values in your data set one time and then analyze the data set as if there was never any missingness, you are underestimating the uncertainty in your analyses. You are ignoring the fact that if you randomly imputed imputed the missing values again, you'd obtain different imputations. You are pretending that your imputations are the actual observed values and that there is no uncertainty in your imputations. Any confidence intervals you end up calculating will be too narrow, underestimating the uncertainty. The goal of multiple imputation is to take into account both the uncertainty about your estimator based on any particular imputed data set and the fact that the imputed values are random and differ between possible imputed data sets.