Visualising programme theories

Lovely collection of examples of the folllowing ways of visualising a programme theory:

  1. Logic model
  2. Logical framework
  3. Theory of change
  4. Context-mechanism-outcome configuration
  5. Causal loop diagram
  6. Stock and flow diagram
  7. Concept map
  8. Network map
  9. Path model
  10. Nested/Hybrid model

Also includes links to tools for reasoning about the representations (where they have some genre of formal semantics).

Examples

References

Lemire, S., Porowski, A., & Mumma, K. (2023). How We Model Matters: Visualizing Program Theories. Abt Associates.

“Use the difficulty”

Michael Caine talking about rehearsing a play where other actors improvising had got carried away.

“They started throwing things, and he threw a chair, and it lodged in the doorway. And I went to open the door, and I got my head around, and I went [to the director], ‘I’m sorry, sir, I can’t get in.’

“He said, ‘What do you mean?’

“I said, ‘There’s a chair there.’

“He said to me, ‘Use the difficulty.’

“I said, ‘What do you mean?’

“He said, ‘Well, if it’s a comedy, fall over it. If it’s a drama, pick it up and smash it. Use the difficulty.’”

“… it is time to bring the subject of reparations firmly to the fore”

“It is time to acknowledge openly that much of Europe and the United States have been built from the vast wealth harvested from the sweat, tears, blood and horrors of the trans-Atlantic Slave Trade and the centuries of colonial exploitation. Maybe we should also admit that it cannot be easy to build confident and prosperous societies from nations that, for centuries, had their natural resources looted and their peoples traded as commodities.

“For centuries, the world has been unwilling and unable to confront the realities of the consequences of the slave trade, but gradually this is changing, and it is time to bring the subject of reparations firmly to the fore. Granted that current generations are not the ones that engaged in the slave trade, but that grand inhuman enterprise was state-sponsored and deliberate; and its benefits are clearly interwoven with the present-day economic architecture of the nations that designed and executed it. Reparations must be paid for the slave trade. No amount of money will ever make up for the horrors, but it would make the point that evil was perpetrated, that millions of productive Africans were snatched from the embrace of our continent, and put to work in the Americas and the Caribbean without compensation for their labour.

“If there are any hesitations in some minds about the paying of reparations, it is worth considering the fact that, when slavery was abolished, the slave owners were compensated for the loss of the slaves, because the human beings were labelled as property, deemed to be commodities. Surely, this is a matter that the world must confront, and can no longer ignore.”

Speech by President of the Republic of Ghana, Nana Addo Dankwa Akufo-Addo, at the 78th Session of the United Nations General Assembly, UN Headquarters, New York, Wednesday 20th September 2023

Actual causes: two examples using the updated Halpern-Pearl definition

Halpern (2015) provides three variants of the Halpern-Pearl definitions of actual causation. I’m trying to get my head around the formalism, which is elegant, concise, and precise, but tedious to use in practice so I wrote an R script to do the sums. This blog post shows two worked examples, primarily for my own benefit; however, they may help you too if you’re also struggling with the examples in the paper!

The second (“updated”) definition of an actual cause asserts that \(\vec{A} = \vec{a}\) is a cause of \(\varphi\) in \((M,\vec{u})\) iff the following conditions hold:

AC1 \((M,\vec{u}) \models (\vec{A} =\vec{a}) \land \varphi\).

This says, if \(\vec{A} = \vec{a}\) is an actual cause of \(\varphi\) then they both hold in the actual world, \((M,\vec{u})\). Note, for this condition, we are just having a look at the model and not doing anything to it.

AC2 There is a partition of the endogenous variables in \(M\) into \(\vec{Z} \supseteq \vec{X}\) and \(\vec{W}\) and there are settings \(\vec{x’}\) and \(\vec{w}\) such that

(a) \((M,\vec{u}) \models [ \vec{X} \leftarrow \vec{x’}, \vec{W} \leftarrow \vec{w}] \neg \varphi\).

So, we’re trying to show that undoing the cause, i.e., setting \(\vec{X}\) to \(\vec{x’} \ne \vec{x}\), prevents the effect. We are allowed to modify \(\vec{W}\) however we want to show this, whilst leaving \(\vec{Z}-\vec{X}\) free to do whatever the model tells these variables to do.

(b) If \((M,\vec{u}) \models \vec{Z} = \vec{z^{\star}}\), for some \(\vec{z^{\star}}\), then for all \(\vec{W’} \subseteq \vec{W}\) and \(\vec{Z’} \subseteq \vec{Z}-\vec{X}\),
\((M,\vec{u}) \models [ \vec{X} \leftarrow \vec{x}, \vec{W’} \leftarrow \vec{w’}, \vec{Z’} \leftarrow \vec{z^{\star}}] \varphi\).

This says, trigger the cause (unlike AC1, we aren’t just looking to see if it holds) and check whether it leads to the effect under all subsets of \(\vec{Z}\) (as per actual world) that aren’t \(\vec{X}\) and all subsets of the modified \(\vec{W}\) that we found for AC2(a). Note how we are setting \(\vec{Z}\) for those subsets, rather than just observing it.

AC3 There is no \(\vec{A’} \subset \vec{A}\) such that \(\vec{A’} = \vec{a’}\) satisfies AC1 and AC2.

This says, there’s no superfluous stuff in \(\vec{A}\). You taking a painkiller and waving a magic wand doesn’t cause your headache to disappear, under AC3, if the painkiller works without the wand.

Example 1: an (actual) actual cause

Let’s give it a go with an overdetermined scenario (lightly edited from Halpern) that Alice and Bob both lob bricks at a glasshouse and smash the glass. Define

\(\mathit{AliceThrow} = 1\)
\(\mathit{BobThrow} = 1\)
\(\mathit{GlassBreaks} = \mathit{max}(\mathit{AliceThrow},\mathit{BobThrow})\)

So, if either Alice or Bob (or both) hit the glasshouse, then the glass breaks. Strictly speaking, I should have setup one or more exogenous variables, \(\vec{u}\), that define the context and then defined \(\mathit{AliceThrow}\) and \(\mathit{BobThrow}\) in terms of \(\vec{u}\), but it works fine to skip that step as I have here since I’m holding \(\vec{u}\) constant anyway.

Is \(\mathit{AliceThrow} = 1\) an actual cause of \(\mathit{GlassBreaks} = 1\)?

AC1 holds since \((M,\vec{u}) \models \mathit{AliceThrow} = 1 \land \mathit{GlassBreaks} = 1\). The first conjunct comes directly from one of the model equations. Spelling out the second conjunct,

\(\mathit{GlassBreaks} = \mathit{max}(\mathit{AliceThrow},\mathit{BobThrow})\)
\(= \mathit{max}(1, 1)\)
\(= 1\)

For AC2, we need to find a partition of the endogenous variables such that AC2(a) and AC2(b) hold. Try \(\vec{Z} = \{ \mathit{AliceThrow}, \mathit{GlassBreaks} \}\) and \(\vec{W}= \{ \mathit{BobThrow} \}\).

AC2(a) holds since \((M,\vec{u}) \models [ \mathit{AliceThrow} \leftarrow 0, \mathit{BobThrow} \leftarrow 0] \mathit{GlassBreaks} = 0\).

For AC2(b), we begin with \(\vec{Z} = \{ \mathit{AliceThrow}, \mathit{GlassBreaks} \}\) and the settings as per the unchanged model, so

\((M,\vec{u}) \models \mathit{AliceThrow} = 1 \land \mathit{GlassBreaks} = 1\).

We need to check that for all \(\vec{W’} \subseteq \vec{W}\) and \(\vec{Z’} \subseteq \vec{Z}-\vec{X}\),
\((M,\vec{u}) \models [ \vec{X} \leftarrow \vec{x}, \vec{W’} \leftarrow \vec{w’}, \vec{Z’} \leftarrow \vec{z^{\star}}] \varphi\).

Here are the combinations and \(\varphi \equiv \mathit{GlassBreaks} = 1\) holds for all of them:

\((M,\vec{u}) \models [ \mathit{AliceThrow} \leftarrow 1, \mathit{GlassBreaks} \leftarrow 1, \mathit{BobThrow} \leftarrow 0 ] \varphi\)
\((M,\vec{u}) \models [ \mathit{AliceThrow} \leftarrow 1, \mathit{BobThrow} \leftarrow 0 ] \varphi\)
\((M,\vec{u}) \models [ \mathit{AliceThrow} \leftarrow 1, \mathit{GlassBreaks} \leftarrow 1 ] \varphi\)
\((M,\vec{u}) \models [ \mathit{AliceThrow} \leftarrow 1 ] \varphi \)

AC3 is easy since the cause only has one variable, so there’s nothing superfluous.

Example 2: not an actual cause

Now let’s try an example that isn’t an actual cause: the glass breaking causes Alice to throw the brick. It’s obviously false; however, it wasn’t clear to me exactly where it would fail until I worked through this…

AC1 holds since in the actual world, \(\mathit{GlassBreaks} = 1\) and \(\mathit{AliceThrow} = 1\) hold.

Examining the function defintions, they don’t provide a way to link \(\mathit{AliceThrow}\) to a change in \(\mathit{GlassBreaks}\), so the only apparent way to do so is through \(\vec{W}\). Therefore, use the partition \(\vec{W} = \{\mathit{AliceThrow}\}\) and \(\vec{Z} = \{\mathit{GlassBreaks}, \mathit{BobThrow}\}\).

Now for AC2(a), we can easily get \(\mathit{AliceThrow} = 0\) as required, since we can do what we like with \(\vec{W}\). It doesn’t help when we move onto AC2(b) since we have to hold \(\mathit{AliceThrow} = 0\), which is the negation of what we want. The same is the case for the other partition including \(\mathit{AliceThrow}\) in \(\vec{W}\), i.e., \(\vec{W} = \{ \mathit{AliceThrow}, \mathit{BobThrow} \}\).

So, the broken glass does not cause Alice to throw a brick. The setup we needed to get through AC2(a) set us up to fail AC2(b).

References

Halpern, J. Y. (2015). A Modification of the Halpern-Pearl Definition of Causality. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), 3022–3033.

How theory-infused forms of evaluation proliferate

In case you’re wondering why we’re blessed with a multitude of terms for evaluations that use theory in some shape or fashion – theory-oriented evaluation, theory-based evaluation, theory-driven evaluation, program theory evaluation, intervening mechanism evaluation, theoretically relevant evaluation research, and program theory-driven evaluation science (Donaldson, 2022, p. 9) – the answer is in an XKCD comic:

References

Donaldson, S. I. (2022). Introduction to Theory-Driven Program Evaluation (2nd ed.). Routledge.

 

Publishing on Amazon

Inspired by nonsense-ridden ChatGPT-authored books appearing on Amazon, I wondered how hard it would be to get my own algorithmic nonsense on there.

So, I’m delighted to announce my new book, Seven Factorial Theatrical Despair, that is 120 pages of all 7! = 5,040 permutations of words in the sentence, “pounds head on table theatrically in despair”, enumerated using the programming language Haskell.

A bargain at only £3.42.

 

Neat guide to {tidyverse} updates in R, by Mine Çetinkaya-Rundel

This guide is intended for people teaching tidyverse, but it’s relevant to anyone (e.g., me!) trying to keep up with developments.

Everytime I load {tidyverse}, it advises me to use {conflicted}. I didn’t realise it would be this useful:

library(conflicted)
    
penguins |>
  filter(species == "Adelie")
#> Error:
#> ! [conflicted] filter found in 2 packages.
#> Either pick the one you want with `::`:
#> • dplyr::filter
#> • stats::filter
#> Or declare a preference with `conflicts_prefer()`:
#> • `conflicts_prefer(dplyr::filter)`
#> • `conflicts_prefer(stats::filter)`

Another neat update: case_when now doesn’t need type-specific NAs like NA_character_ for default cases.

# now, optionally
df |>
  mutate(
    x = case_when(
       ~ "value 1",
       ~ "value 2",
       ~ "value 3",
      .default = NA
    )
  )

Lots of other tips.

Interesting tour of Akaike’s information criterion (AIC)

Interesting tour of Akaike’s information criterion (AIC) and its relationship with (log-)likelihood ratio tests, by Chris Sutherland et al. (in press).

My fav bit is where the authors show that an improved AIC for a model comparison where the bigger model has just one extra parameter is equivalent to p < .157 on the LR-test – because simple arithmetic.

References

Sutherland, C., Hare, D., Johnson, P. J., Linden, D. W., Montgomery, R. A., & Droge, E. (in press). Practical advice on variable selection and reporting using AIC. Proceedings of the Royal Society B: Biological Sciences.

Inside every matching study

A potentially useful one-sentence(!) intervention for making a case to run a statistical matching evaluation rather than a randomised controlled trial:

“Matching can be thought of as a technique for finding approximately ideal experimental data hidden within an observational data set.”

– King, G., & Nielsen, R. (2019, p. 442) [Why Propensity Scores Should Not Be Used for Matching. Political Analysis, 27(4), 435–454]