Likelihood Based Inference with Incomplete Data (Nonignorable)

Apr 27, 2016 rubric

In many cases, analysis methods for missing data are based on the ignorable likelihood

$L_{i g n} (θ ∣ Y_{0}, X) \propto f (Y_{0} ∣ X, θ),$

regarded as a function of the parameters $θ$ for fixed observed data $Y_{0}$ and some fully observed covariates $X$ . The density $f (Y_{0} ∣ X, θ)$ is obtained by integrating out the missing data $Y_{1}$ from the joint density $f (Y ∣ X, θ) = f (Y_{0}, Y_{1} ∣ X, θ)$ . Sufficient conditions for basing inference about $θ$ on the ignorbale likelihood are that the missingness mechanism is Missing At Random(MAR) and the parameters of the model of analysis $θ$ and those of the missingness mechanism $ψ$ are distinct. Here we focus our attention on the situations where the missingness mechanism is Missing Not At Random(MNAR) and valid Maximum Likelihood(ML), Bayesian and Multiple Imputation(MI) inferences generally need to be based on the full likelihood

$L_{f u l l} (θ, ψ ∣ Y_{0}, X, M) \propto f (Y_{0}, M ∣ X, θ, ψ),$

regarded as a function of $(θ, ψ)$ for fixed $(Y_{0}, M)$ . Here, $f (Y_{0}, M ∣ θ, ψ)$ is obtained by integrating out $Y_{1}$ from the joint density $f (Y, M ∣ X, θ, ψ)$ . Two main approaches for formulating MNAR models can be distinguished, namely selection models(SM) and pattern mixture models(PMM).

Selection and Pattern Mixture Models

SMs factor the joint distribution of $m_{i}$ and $y_{i}$ as

$f (m_{i}, y_{i} ∣ x_{i}, θ, ψ) = f (y_{i} ∣ x_{i}, θ) f (m_{i} ∣ x_{i}, y_{i}, ψ),$

where the first factor is the distribution of $y_{i}$ in the population while the second factor is the missingness mechanism, with $θ$ and $ψ$ which are assumed to be distinct. Alternatively, PMMs factor the joint distribution as

$f (m_{i}, y_{i} ∣ x_{i}, θ, ψ) = f (y_{i} ∣ x_{i}, m_{i}, ξ) f (m_{i} ∣ x_{i}),$

where the first factor is the distribution of $y_{i}$ in the strata defined by different patterns of missingness $m_{i}$ while the second factor models the probabilities of the different patterns, with $ξ$ which are assumed to be distinct (Little (1993),Little and Rubin (2019)). The distinction between the two factorisations becomes clearer when considering a specific example.

Suppose thta missing values are confined to a single variable and let $y_{i} = (y_{i, 1}, y_{i 2})$ be a bivariate response outcome where $y_{i 1}$ is fully observed and $y_{i 2}$ is observed for $i = 1, \dots, n_{c c}$ but missing for $i = n_{c c} + 1, \dots, n$ . Let $m_{i 2}$ be the missingness indicator for $y_{i 2}$ , then a PMM factors the denisty of $Y_{0}$ and $M$ given $X$ as

$f (y_{0}, M ∣ X, ξ) = \prod_{i = 1}^{n_{c c}} f (y_{i 1}, y_{i 2} ∣ x_{i}, m_{i 2} = 0, ξ) P r (m_{i 2} = 0 ∣ x_{i}, ω) \times \prod_{i = n_{c c} + 1}^{n} f (y_{i 1} ∣ x_{i}, m_{i 2} = 1, ξ) P r (m_{i 2} = 1 ∣ x_{i}, ω) .$

This expression shows that there are no data with which to estimate directly the distribution $f (y_{i 2} ∣ x_{i}, m_{i 2} = 1, ξ)$ , because all units with $m_{i 2} = 1$ have $y_{i 2}$ missing. Under MAR, this is identified using the distribution of the observed data $f (y_{i 2} ∣ x_{i}, m_{i 2} = 1, ξ) = f (y_{i 2} ∣ x_{i}, m_{i 2} = 0, ξ)$ , while under MNAR it must be identified using other assumptions. The SM formulation is

$f (y_{i}, m_{i 2} ∣ θ, ψ) = f (y_{i 1} ∣ x_{i}, θ) f (y_{i 2} ∣ x_{i}, y_{i 1}, θ) f (m_{i 2} ∣ x_{i}, y_{i 1}, y_{i 2}, ψ) .$

Typically, the missingness mechanism $f (m_{i 2} ∣ x_{i}, y_{i 1}, y_{i 2}, ψ)$ is modelled using some additive probit or logit regression of $m_{i 2}$ on $x_{i}$ , $y_{i 1}$ and $y_{i 2}$ . However, the coefficient of $y_{i 2}$ in this regression is not directly estimable from the data and hence the model cannot be fully estimated without extra assumptions.

Normal Models for MNAR data

Assume we have a complete sample $(y_{i}, x_{i})$ on a continuous variable $Y$ and a set of fully observed covariates $X$ , for $i = 1, \dots, n$ . Suppose that $i = 1, \dots, n_{c c}$ units are observed while the remaining $i = n_{c c} + 1, \dots, n$ units are missing, with $m_{i}$ being the corresponding missingness indicator. Heckman (Heckman (1976)) proposed the following selection model to handle missingness:

$y_{i} ∣ x_{i}, θ, ψ \sim N (β_{0} + β_{1} x_{i}, σ^{2}) and m_{i} ∣ x_{i}, y_{i}, θ, ψ \sim B e r n (Φ (ψ_{0} + ψ_{1} x_{i} + ψ_{2} y_{i})),$

where $θ = (β_{0}, β_{1}, σ^{2})$ and $Φ$ denotes the probit (cumulative normal) distribution function. Note that if $ψ_{2} = 0$ , the missing data are MAR, while if $ψ_{2} \neq 0$ the missing data are MNAR since missingness in $Y$ depends on the unobserved value of $Y$ . This model can be estimated using either a two-step least squares method, ML in combination with an EM algorithm, or a Bayesian approach. The main issue is the lack of information about $ψ_{2}$ , which can be partly identified through the specific assumptions about the distribution of the observed data of $Y$ . This, however, makes the implicit assumption that the assumed distribution can well described the distribution of the complete (observed and missing) data which can never be tested or checked. An alternative approach is to use a PMM factorisation and model:

$y_{i} ∣ m_{i} = m, x_{i}, ξ, ω \sim N (β_{0}^{m} + β_{1}^{m} x_{i}, σ^{2 m}) and m_{i} ∣ x_{i}, ξ, ω \sim B e r n (Φ (ω_{0} + ω_{1} x_{i})),$

where $ξ = (β_{0}^{m}, β_{1}^{m}, σ^{2 m}, m = 0, 1)$ . This model implies that the distribution of $y_{i}$ given $x_{i}$ in the population is a mixture of two normal distributions with mean

$[1 - Φ (ω_{0} + ω_{1} x_{i})] [β_{0}^{0} + β_{1}^{0} x_{i}] + [Φ (ω_{0} + ω_{1} x_{i})] [β_{0}^{1} + β_{1}^{1} x_{i}] .$

The parameters $(β_{0}^{0}, β_{1}^{0}, σ^{20}, ω)$ can be estimated from the data but the parameters $(β_{0}^{1}, β_{1}^{1}, σ^{21})$ are not estimable because $y_{i}$ is missing when $m_{i} = 1$ . Under MAR, the distribution of $Y$ given $X$ is the same for units with $Y$ observed and missing, such that $β_{0}^{0} = β_{0}^{1} = β_{0}$ (as well as for $β_{1}$ and $σ^{2}$ ). Under MNAR, other assumptions are needed to esitmate the parameters indexed by $m = 1$ .

Some final considerations:

Both SM and PMM model the joint distribution of $Y$ and $M$ .
The SM formulation is more natural when the substantive interest concerns the relationship between $Y$ and $X$ in the population. However, these parameters can also be derived in PMM by averaging the patterns specific parameters over the missingness patterns.
The PMM factorisation is more transparent in terms of the underlying assumptions about the unidentified parameters of the model, while SM tends to impose some obscure constraints in order to identify these parameters, which are also difficult to interpret.
Given specific assumptions to identify all the parameters in the model, PMMs are often easier to fit than SMs. In addition, imputations of the missing values are based on the predictive distribution of $Y$ given $X$ and $M = 0$ .

These considerations seem to favour PMM over SM as MNAR approaches, especially when considering sensitivity analysis. Bayesian approaches can also be used to identify these models, by assigning prior distributions which can be used to identify those parameters which cannot be estimated from the data. Justifications for the choice of these priors are therefore necessary to ensure the plausibility of the assumptions assessed and the impact of these assumptions on the posterior inference.

References

Heckman, James J. 1976. “The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models.” In Annals of Economic and Social Measurement, Volume 5, Number 4, 475–92. NBER.

Little, Roderick JA. 1993. “Pattern-Mixture Models for Multivariate Incomplete Data.” Journal of the American Statistical Association 88 (421): 125–34.

Little, Roderick JA, and Donald B Rubin. 2019. Statistical Analysis with Missing Data. Vol. 793. John Wiley & Sons.

Edit this page

Maximum Likelihood Estimation Bayesian Inference Likelihood Based Methods Nonignorable