Suppose we run an RCT with two groups, treatment and control, and a binary outcome of whether participants recover or not.
There are two potential outcomes: recovery following treatment \((R_t)\) and recovery following control \((R_c)\), \(1\) if recovered and \(0\) if not recovered. Only one of these two potential outcomes is realised, depending on what group someone is assigned to. Let \(W = t\) if a participant was assigned to treatment and \(W = c\) if they were assigned to control.
Suppose, following an RCT, we learn the following (somehow with perfect precision):
\(P(R_t = 1 | W = t) = 0.8\);
\(P(R_c = 1 | W = c) = 0.3\).
Given the two probabilities above, it turns out (I asked Check-Coherence to do the arithmetic) the best we can say is that \(P(R_t = 1) \in [0, 1]\) and \(P(R_c = 1) \in [0, 1]\). So, it seems that we aren’t yet able to infer anything about the potential outcomes beyond those that were realised.
Add to our premises that participants were assigned to treatment or control by coin flip:
\(P(W = t) = P(W = c) = 0.5\).
Now \(P(R_t = 1) \in [0.4 , 0.9]\) and \(P(R_c = 1) \in [0.15 , 0.65]\). These intervals are clearly better that \([0,1]\); however, can we do better?
The key ingredient we need to add is that treatment assignment is independent of the potential outcomes; that is
\(P(W | R_t, R_c) = P(W)\).
Now, given all this information, we obtain point probabilities: \(P(R_t = 1) = 0.8\) and \(P(R_c = 1) = 0.3\). These are equal to the probabilities that were conditional on what group a participant was assigned to.
Another curiosity is what we can infer about the joint distribution, \(P(R_t, R_c)\). The results are probability intervals:
| \(R_t = 0\) | \(R_t = 1\) | ||
|---|---|---|---|
| \(R_c = 0\) | \([0, 0.2]\) | \([0.5, 0.7]\) | \(0.7\) |
| \(R_c = 1\) | \([0, 0.2]\) | \([0.1, 0.3]\) | \(0.3\) |
| \(0.2\) | \(0.8\) |
This illustrates, in a toy example, the more general problem that the joint distribution of potential outcomes typically cannot be obtained from an RCT. However, the joint probabilities are constrained by the marginals.