GEE and Generalised Linear Mixed Models in Clinical Trials

Introduction

Repeated or clustered clinical-trial outcomes are not always continuous.

Examples include:

• response versus non-response;
• adverse event occurrence;
• hospital admission count;
• seizure count;
• ordinal severity categories.

For these outcomes, two important modelling frameworks are:

• Generalised Estimating Equations (GEE);
• Generalised Linear Mixed Models (GLMM).

The key distinction is usually not which model is more advanced, but which treatment effect is being targeted.

Learning objectives

By the end of this guide, the reader should be able to:

1. Explain the population-average interpretation of GEE.
2. Explain the conditional interpretation of GLMM.
3. Describe common working correlation structures in GEE.
4. Recognise why GEE and GLMM coefficients differ for nonlinear links.
5. Choose a modelling framework according to the inferential target.

Generalised Estimating Equations

GEE extends the generalised linear model to correlated observations.[1]

A general form is:

$$g\{E(Y_{ij})\} = X_{ij}\beta$$

where $g(\cdot)$ is a link function.

Continuous outcome

Identity link:

$$g(\mu)=\mu$$

Binary outcome

Logit link:

$$g(\mu) = \log\left(\frac{\mu}{1-\mu}\right)$$

Count outcome

Log link:

$$g(\mu)=\log(\mu)$$

Working correlation

GEE specifies a working correlation structure.

Independence

Repeated measurements are treated as uncorrelated in the working model.

Exchangeable

All within-cluster correlations are assumed equal:

$$R= \begin{pmatrix} 1&\rho&\rho\\ \rho&1&\rho\\ \rho&\rho&1 \end{pmatrix}$$

AR(1)

$$Corr(Y_j,Y_k)=\rho^{|j-k|}$$

Unstructured

Each pair of repeated observations can have a different correlation.

Robust variance

GEE commonly uses a sandwich or robust variance estimator.

Under suitable regularity conditions and with sufficiently many independent clusters, this can provide protection against misspecification of the working correlation.[1]

However:

Robust does not mean assumption-free.

Conventional sandwich standard errors may perform poorly when the number of independent clusters is small.

Population-average interpretation

A binary GEE model might be:

$$logit[P(Y_{ij}=1)] = \beta_0+ \beta_1Treatment$$

The treatment coefficient is interpreted marginally across the study population.

Conceptually:

What is the average treatment effect in the population?

This is the major interpretive feature of GEE.

Generalised Linear Mixed Models

GLMM combines a generalised linear model with random effects.

For a binary outcome:

$$logit[P(Y_{ij}=1|b_i)] = X_{ij}\beta+b_i$$

For a count outcome:

$$\log[E(Y_{ij}|b_i)] = X_{ij}\beta+b_i$$

The random effect $b_i$ may represent:

• participant;
• centre;
• hospital;
• physician;
• cluster.

Conditional interpretation

A GLMM coefficient is typically interpreted conditional on the random effects.

For example:

Among patients with the same value of the random effect, how does treatment affect the odds of response?

This differs from the marginal population-average interpretation of GEE.

Why the coefficients differ

With a nonlinear link such as the logit link:

$$logit[P(Y=1)]$$

a population-average odds ratio and a conditional odds ratio are generally not numerically equal.

Therefore:

$$OR_{GEE} \neq OR_{GLMM}$$

in general.

This difference is not an error. It reflects different estimands.

GEE versus GLMM

Repeated binary example

Suppose response is measured at Weeks 4, 8, 12 and 24:

$$Y_{ij}= \begin{cases} 1 & \text{response}\\ 0 & \text{no response} \end{cases}$$

Logistic GEE

Appropriate when the target is a population-average treatment effect.

Logistic GLMM

Appropriate when a conditional effect and subject- or cluster-specific heterogeneity are scientifically relevant.

Count outcomes

Examples include:

• exacerbation count;
• seizure frequency;
• hospital admission count.

A Poisson model may be written as:

$$\log(E[Y]) = X\beta$$

If participants have different follow-up times:

$$\log(E[Y]) = X\beta+\log(PersonTime)$$

where $\log(PersonTime)$ is used as an offset.

When variance substantially exceeds the mean, negative-binomial models may be preferable to simple Poisson models.

Repeated count outcomes may be analysed using:

• GEE;
• Poisson GLMM;
• negative-binomial GLMM.

Cluster-randomised trials

GEE and GLMM are also important for cluster-randomised trials.

If patients are nested within hospitals:

$$Patient \subset Hospital$$

then outcomes within the same hospital are correlated.

GEE

Targets a marginal population-average effect.

GLMM

Can include a hospital-level random effect:

$$Y_{ij}=X_{ij}\beta+b_j+\epsilon_{ij}$$

or an appropriate non-Gaussian equivalent.

Small numbers of clusters

A trial with:

• 8 hospitals;
• 1,000 patients;

does not provide 1,000 independent clusters.

Inference for cluster-level treatment effects depends heavily on the number of independent clusters.

This may require:

• small-sample corrections;
• cluster-level analyses;
• appropriately specified mixed models;
• randomisation-based approaches.

Common mistakes

”GEE is better because it is robust”

Robust variance estimation does not eliminate all assumptions.

”GLMM and GEE estimate the same odds ratio”

They generally do not for nonlinear link functions.

”The number of patients determines whether GEE asymptotics are adequate”

For cluster trials, the number of independent clusters is often more relevant than the number of individual observations.

”Use exchangeable correlation because it is simple”

The working correlation should be chosen with scientific and design considerations in mind.

Practical checklist

• Is the target effect marginal or conditional?
• What creates correlation?
• How many independent clusters are available?
• Is the outcome binary, ordinal or count?
• Is the chosen link function appropriate?
• Is overdispersion present for count outcomes?
• Is the working correlation or random-effects structure defensible?
• Are small-sample corrections required?

Key takeaways

1. GEE is primarily a marginal, population-average modelling framework.
2. GLMM usually provides conditional effects given random effects.
3. GEE and GLMM coefficients can differ substantially for binary and other nonlinear outcomes.
4. The number of independent clusters matters greatly for GEE inference.
5. Model choice should follow the estimand and correlation structure.

References

1. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13-22. doi:10.1093/biomet/73.1.13