`openstatsware`

and the R Package `{mmrm}`

China-R Conference

Liming Li

2023-11-28

Authors:

- Brian Matthew Lang (MSD)
- Christian Stock (BI)
- Craig Gower-Page (Roche)
- Dan James (AstraZeneca)
- Daniel Leibovitz (Roche)
- Daniel Sabanes Bove (Roche, lead)

- Daniel Sjoberg (Roche)
- Doug Kelkhoff (Roche)
- Julia Dedic (Roche)
- Jonathan Sidi (PinPoint Strategies)
- Kevin Kunzmann (BI)
- Ya Wang (Gilead)

Thanks to:

- Ben Bolker (McMaster University)
- Davide Garolini (Roche)
- Dinakar Kulkarni (Roche)
- Gonzalo Duran Pacheco (Roche)
- Software Engineering working group (
`openstatsware`

)

- Introduction and Overview of
`openstatsware`

- Mixed Models for Repeated Measures - Why is it a Problem?
- MMRM Package
- Why this is not “yet another package”
- Comparing MMRM R Package to other Implementations

- Get Started with
`{mmrm}`

!

- Open-source software has gained increasing popularity in Biostatistics
- Pros: rapid uptake of novel statistical methods, unprecedented opportunities for collaboration
- Cons: variability in software quality

- Developing high-quality software is critical

`openstatsware`

)- Official working group of the American Statistical Association (ASA) Biopharmaceutical Section
- Formed on 19 August 2022
- Cross-industry collaboration (58 members from 37 organizations)
- Full name: Software Engineering Working Group
- Short name:
`openstatsware`

- Home page at rconsortium.github.io/asa-biop-swe-wg
- We welcome new members to join!

- Primary
- Engineer R packages that implement important statistical methods
- to fill in gaps in the open-source statistical software landscape
- focusing on what is needed for biopharmaceutical applications

- Engineer R packages that implement important statistical methods
- Secondary
- Develop and disseminate best practices for engineering high-quality open-source statistical software
- By actively doing the statistical engineering work together, we align on best practices and can communicate these to others

- Develop and disseminate best practices for engineering high-quality open-source statistical software

`openstatsware`

: `mmrm`

- MMRM is a popular choice for analyzing longitudinal continuous outcomes in randomized clinical trials
- No great R package
- Initially thought that the MMRM problem was solved by using
`lme4`

with`lmerTest`

, learned that this approach failed on large data sets (slow, did not converge) `nlme`

does not give Satterthwaite adjusted degrees of freedom, has convergence issues, and with`emmeans`

it is only approximate- Next we tried to extend
`glmmTMB`

to calculate Satterthwaite adjusted degrees of freedom, but it did not work

- Initially thought that the MMRM problem was solved by using

- We only want to fit a fixed effects model with a structured covariance matrix for each subject
- The idea is then to use the Template Model Builder (
`TMB`

) directly- as it is also underlying
`glmmTMB`

- but code the exact model we want

- as it is also underlying
- We do this by implementing the log-likelihood in
`C++`

using the`TMB`

provided libraries - Provide an R solution that
- has fast convergence times
- generates estimates closest to
`SAS`

`TMB`

- Fast
`C++`

framework for defining objective functions (`Rcpp`

would have been alternative interface) - Automatic differentiation of the log-likelihood as a function of the variance parameters
- We get the gradient and Hessian exactly and without additional coding
- Syntactic sugars to allow simple matrix calculations or operations like R
- This can be used from the R side with the
`TMB`

interface and plugged into optimizers

- Ongoing maintenance and support from the pharmaceutical industry
- 5 companies being involved in the development, on track to become standard package

- Development using best practices as show case for high quality package
- Thorough unit and integration tests (also comparing with SAS results) to ensure accurate results

`mmrm`

**Covariance structures**for the dependent observations:- Unstructured, Toeplitz, AR1, compound symmetry, ante-dependence, spatial exponential
- Allows group specific covariance estimates and weights

**Hypothesis Testing**:`emmeans`

interface for least square means- Satterthwaite and Kenward-Roger adjustments
- Robust sandwich estimator for covariance

**Integrations and extentions**`tidymodels`

builtin parsnip engine and recipes for streamlined model fitting workflows`teal`

,`tern`

,`rtables`

integration for post processing and reporting- Support conditional mean prediction and simulation
- Also used in
`rbmi`

for conditional mean imputation!

To run an MMRM model in SAS it is recommended to use either the `PROC MIXED`

or `PROC GLMMIX`

procedures.

Less model assumptions are applied in

`PROC MIXED`

, primarily how one treats missingness.We will compares the

`PROC MIXED`

procedure to the`{mmrm}`

package in the following attributes:

- Documentation
- Unit Testing
- Covariance structures

- Covariance Estimators
- Degrees of Freedom
- Contrasts

Both languages have online documentation of the technical details of the estimation and degrees of freedom methods and the different covariance structures available.

`{mmrm}`

`PROC MIXED`

- Unit tests in
`{mmrm}`

are transparent, compared to`PROC MIXED`

. - Uses the
`testthat`

framework with`covr`

to communicate the testing coverage. - Unit tests can be found in the GitHub repository under ./tests.

**Note**

The integration tests in `{mmrm}`

are set to a tolerance of 10e^-3 when compared to SAS outputs.

- SAS has 23 non-spatial covariance structures, while mmrm has 10.
- 9 structures intersect with SAS
- Ante-dependence (homogeneous) is only in the mmrm package.

- SAS has 14 spatial covariance structures compared to the spatial exponential one available in mmrm.

**Tip**

For users that need more structures, `{mmrm}`

is easily extensible - let us know via feature requests in the GitHub repository.

Covariance structures | `{mmrm}` |
`PROC MIXED` |
---|---|---|

Unstructured (Unweighted/Weighted) | X/X | X/X |

Toeplitz (hetero/homo) | X/X | X/X |

Compound symmetry (hetero/homo) | X/X | X/X |

Auto-regressive (hetero/homo) | X/X | X/X |

Ante-dependence (hetero/homo) | X/X | X |

Spatial exponential | X | X |

Covariance estimators | `{mmrm}` |
`PROC MIXED` |
---|---|---|

Asymptotic | X | X |

Kenward-Roger | X | X |

Empirical | X | X |

Jackknife | X | |

Bias-Reduced Linearization | X |

Method | `{mmrm}` |
`PROC MIXED` |
---|---|---|

Contain | X* | X |

Between/Within | X* | X |

Residual | X* | X |

Satterthwaite | X | X |

Kenward-Roger ** | X | X |

Kenward-Roger (Linear) | X | X |

*Available through `emmeans`

, and ongoing work in `mmrm`

.

** Unstructured has differences compared to `PROC MIXED`

because of different parameterization.

Contrasts and least square (LS) means estimates are available in `mmrm`

using

`mmrm::df_1d`

,`mmrm::df_md`

- S3 method that is compatible with
`emmeans`

- LS means difference can be produced through emmeans (
`pairs`

method) - Degrees of freedom method is passed from
`mmrm`

to`emmeans`

- By default
`PROC MIXED`

and`mmrm`

do not adjust for multiplicity, whereas`emmeans`

does.

**Note**

These are comparable to the LSMEANS statement in `PROC MIXED`

.

Other R packages, like `nlme`

, `lme4`

and `glmmTMB`

can support MMRM analysis. However:

- Covariance structure supported is limited
- Computational efficiency is not satisfying
- Convergence issues(especially for
`lme4`

) - Need other(multiple) packages to support
`Satterthwaite`

and`Kenward-Roger`

degrees of freedom, or Sandwich covariance estimator. - Detailed results at the online comparison vignette

`mmrm`

not only supports multiple covariance structure, it also has good efficiency (due to fast implementations in C++)

`{mmrm}`

has small difference from SAS

`mmrm`

is on CRAN - use this as a starting point:

- Visit openpharma.github.io/mmrm for detailed docs including vignettes
- Consider tern.mmrm for high-level clinical reporting interface, incl. standard tables and graphs