We review the difference between balance of covariates in expectation versus balance in a single run of the experiment (sometimes called ‘random confounding’ or ‘realized confounding’ in epidemiology, see for instance Greenland and Mansournia (2015) or Vander Weele (2012)) and the related distinction between precision and unbiasedness. These distinctions should be well known wherever RCTs are conducted or RCT results are used, though much of the discussion is, if not confused, unhelpfully imprecise. Even less recognized are problems with statistical inference, and especially the threat to significance testing posed when there is an asymmetric distribution of individual treatment effects in the study population. good information
We review the difference between balance of covariates in expectation versus balance in a single run of the experiment (sometimes called ‘random confounding’ or ‘realized confounding’ in epidemiology, see for instance Greenland and Mansournia (2015) or Vander Weele (2012)) and the related distinction between precision and unbiasedness. These distinctions should be well known wherever RCTs are conducted or RCT results are used, though much of the discussion is, if not confused, unhelpfully imprecise. Even less recognized are problems with statistical inference, and especially the threat to significance testing posed when there is an asymmetric distribution of individual treatment effects in the study population. good information