Power - Two-sample Independent T-test

library(MESS)
library(dfmtbx)
library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.5
✔ forcats   1.0.0     ✔ stringr   1.5.1
✔ ggplot2   4.0.0     ✔ tibble    3.2.1
✔ lubridate 1.9.4     ✔ tidyr     1.3.1
✔ purrr     1.0.4     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

library(longpower)

Loading required package: lme4
Loading required package: Matrix

Attaching package: 'Matrix'

The following objects are masked from 'package:tidyr':

    expand, pack, unpack

Loading required package: nlme

Attaching package: 'nlme'

The following object is masked from 'package:lme4':

    lmList

The following object is masked from 'package:dplyr':

    collapse

Sample size estimation for a continuous outcome (legit way)

\[ N = \frac{2 \cdot (Z_{\alpha} + Z_{\beta})^2 \cdot \sigma^2}{(\mu_1 - \mu_2)^2} = \frac{2 \cdot (Z_{\alpha} + Z_{\beta})^2}{\left( \frac{\mu_1 - \mu_2}{\sigma} \right)^2} \]

Where:

• \(N\) is the sample size per group
• \(Z_{\alpha}\) the z score corresponding to 0.05/2 for a two-tailed test
• \(Z_{\beta}\) the z score corresponding to .80 power
• \(\sigma^2\) assumed common variance in two groups
• \({\mu_1 - \mu_2}\) is the difference in means between two groups

effect_size	N_per_group
0.1	1571
0.2	394
0.3	176
0.4	100
0.5	64
0.6	45
0.7	34
0.8	26
0.9	21
1.0	17

Sample size estimation for a continuous outcome (ball-park)

The rule of thumb for estimating sample size is: \[ N \approx \left( \frac{4}{\delta} \right)^2 \]

Where:

• (N) is the sample size per group
  
• (\(\delta\)) is the standardized effect size (e.g., Cohen’s (d))

This approximation assumes a two-tailed test with significance level (= 0.05) and power (1 - \(\beta\) = 0.80).

d <- seq(.1:1, by = .1)

df <- data.frame(effect_size = numeric(), N_per_group =numeric())

for (i in d) {
    n <- (4 / i )^2

    row <- data.frame(effect_size = i, N_per_group = ceiling(n))

    df <- bind_rows(df, row)
}

df %>%
  style_gt_tbl()

effect_size	N_per_group
0.1	1600
0.2	400
0.3	178
0.4	100
0.5	64
0.6	45
0.7	33
0.8	25
0.9	20
1.0	16

Comparison of the two approaches

left_join(
  df,
  df_legit %>% rename(N_per_group_legit = N_per_group ),
  by = "effect_size")

   effect_size N_per_group N_per_group_legit
1          0.1        1600              1571
2          0.2         400               394
3          0.3         178               176
4          0.4         100               100
5          0.5          64                64
6          0.6          45                45
7          0.7          33                34
8          0.8          25                26
9          0.9          20                21
10         1.0          16                17

Comparison of two groups across time ( 2 timepoints)

\[ N = \frac{2 \cdot (Z_{\alpha} + Z_{\beta})^2 \cdot (1 + (n - 1)\rho)}{n \cdot \left( \frac{\mu_1 - \mu_2}{\sigma} \right)^2} \]

calc_sample_size <- function(alpha = 0.05, power = 0.8, n_timepoints = 2,
                             rho = 0.6, mean_diff = 0.5, sd = 1) {
  # Z-scores
  z_alpha <- qnorm(1 - alpha / 2)
  z_beta <- qnorm(power)
  
  # Effect size
  delta <- mean_diff / sd
  
  # Sample size formula
  N <- (2 * (z_alpha + z_beta)^2 * (1 + (n_timepoints - 1) * rho)) / 
       (n_timepoints * delta^2)
  
  return(ceiling(N))
}

calc_sample_size()

[1] 51

One group two time points

\[ N = \frac{(Z_{\alpha/2} + Z_{\beta})^2 \cdot \sigma_d^2}{\Delta^2} = \frac{(Z_{\alpha/2} + Z_{\beta})^2}{d^2} \]

\[ N = \frac{(Z_{\alpha/2} + Z_{\beta})^2}{d^2} \]

Where:

• 𝑁 number of subjects
• 𝑍𝛼/2: Z-score for the significance level
• 𝑍𝛽: Z-score for desired power 𝜎𝑑: standard deviation of the difference scores

Δ: expected mean difference between time points

𝑑=Δ/𝜎𝑑: standardized effect size (Cohen’s d for paired samples)

d <- seq(0.1, 1.0, by = 0.1)

df_legit <- data.frame(effect_size = numeric(), N_per_group =numeric())

for (i in d) {
    result <- power_t_test(
        delta = i,
        ratio = 1,
        power = .8,
        sd = 1, # Set this to one and calculate delta (effect size)
        type = "paired",
        sig.level = .05)

    row <- data.frame(effect_size = i, N_per_group = ceiling(result[["n"]]))

    df_legit <- bind_rows(df_legit, row)
}

df_legit %>%
  style_gt_tbl()

effect_size	N_per_group
0.1	787
0.2	199
0.3	90
0.4	52
0.5	34
0.6	24
0.7	19
0.8	15
0.9	12
1.0	10