PHQ-9 “over-diagnosis” paper shows that arithmetic works

A recent paper by Levis et al. (2020) systematically reviews studies looking at depression prevalence in two ways: one using a structured assessment completed by a professional (SCID) and the other using a questionnaire completed by study participants (PHQ-9). The authors conclude that “PHQ-9 ≥10 substantially overestimates depression prevalence.” But this was entirely predictable.

Mean SCID-prevalence was 12.1%.

Mean PHQ-9 prevalence (using a score of 10 or above to decide that someone has depression) was 24.6%.

This is almost exactly what arithmetic predicts; my back-of-envelope estimate of what PHQ-9 would say (see below) gives 23.8%, using estimates of PHQ’s sensitivity and sensitivity from a meta-analysis (88% and 85%, respectively) and the SCID-prevalence found in the review (12.1%).

So the paper’s results are unsurprising.

PHQ-9 (and any other screening questionnaire) gives better predictions in groups with higher rates of depression, such as people who have asked for a GP appointment because they are worried about their mental health.

No clinical decisions – such as whether to accept someone for treatment – should be made on the basis of nine tick-box answers alone. Questionnaires can also miss people who need treatment.

Screening questionnaires are often designed to over-diagnose rather than risk missing people who need treatment, under the assumption that a proper follow-up assessment will be carried out.

When reporting condition prevalence, the psychometric properties of measures should be provided, including what “gold standard” they have been validated against, and the chosen clinical threshold.

Explore PPV and NPV using this app.

 

Back of envelope

P(SCID) = .121
P(PHQ | SCID) = .88
P(not-PHQ | not-SCID) = .85
P(PHQ | not-SCID) = 1 – P(not-PHQ | not-SCID) = .15

P(PHQ & SCID) = P(PHQ | SCID) * P(SCID)
= .88 * .121
= .10648

P(PHQ & not-SCID) = P(PHQ | not-SCID) * P(not-SCID)
= (1 – .85) * (1 – .121)
= .13185

P(PHQ) = P(PHQ & SCID) + P(PHQ & not-SCID)
= .10648 + .13185
= 0.23833

 

Thanks Chris, for pointing out the typo!

Qual and quant – subjective and objective?

“… tensions between quantitative and qualitative methods can reflect more on academic politics than on epistemology. Qualitative approaches are generally associated with an interpretivist position, and quantitative approaches with a positivist one, but the methods are not uniquely tied to the epistemologies. An interpretivist need not eschew all numbers, and positivists can and do carry out qualitative studies (Lin, 1998). ‘Quantitative’ need not mean ‘objective’. Subjective approaches to statistics, for instance Bayesian approaches, assume that probabilities are mental constructions and do not exist independently of minds (De Finetti, 1989). Statistical models are seen as inhabiting a theoretical world which is separate to the ‘real’ world though related to it in some way (Kass, 2011). Physics, often seen as the shining beacon of quantitative science, has important examples of qualitative demonstrations in its history that were crucial to the development of theory (Kuhn, 1961).”

Fugard and Potts (2015, pp. 671-672)

Cyber digital interactive argh – moving teaching online

The courses I teach at Birkbeck use Moodle for uploading slides and other material and I use online forums a bit for follow-up and discussion between sessions, but the programmes are very much face-to-face. Birkbeck is “London’s evening university” and a strength of this is that students who work or have other commitments during the day can physically come to class. Students do readings, discuss these in class, write essays and other projects. In a stats course I teach, students analyse data using R so it’s a bit more techie but still primarily taught face-to-face in a computer lab.

So the coronavirus pandemic has been a huge shock – especially as it came just after a fortnight of jury service and UCU strike action. ARGH.

Here’s where I’m up to, a little over a week into work-from-home. I’m sharing here in case helpful!

Overall “strategy”

Firstly, I accepted that I’m overwhelmed and a bit terrified. Can I do any of this digitally? How will it feel? Oh god I hate the sound of my voice.

I’m working under the assumption that there’s a chance the academic year 2020/21 will be online-only; that this is a marathon not a sprint, so not to burn out!

Luckily I don’t have any more group teaching this academic year so most contact is individual, e.g., project supervision and personal tutor support. However, it quickly became apparent from a questionnaire (see below) that I will need to make an effort to support all students to continue to feel part of the programme and to facilitate students building support networks now they don’t see each other in class. I will also need to support a group with revision prep for a (now take-home) exam.

First steps

I started with the familiar – an open source audio recorder and editor called Audacity which I used to play with years ago. I used this to record myself reading a message to students (which was also sent as text) and to make it slightly less painful (for me anyway) mixed this with some gentle ambient by Brian Eno.

Another familiar: questionnaires. I made a short anonymous “check in” questionnaire with the following questions:

  1. What app do you prefer to use for individual tutorials/other one-to-one meetings (including phone!)
  2. Do you prefer audio, video, or text-only chat?
  3. How well do you think you’re keeping up with deadlines?
  4. Open text comment – how are things generally?

In the group I asked, there was about a 50-50 split between a preference for audio-only and video chat, so I am sure to ask about this and to reinforce that people have different preferences and that’s okay. Nobody wanted live text-chat, but I assume somebody somewhere will, so it’s worth bearing this in mind as an option.

The open text responses were most helpful – and moving: students shared how the pandemic is affecting them, changes to caring responsibilities and employment workload, their worries, how they feel disconnected from university.

Experiments with the digital

Here is where I am up to:

Whole programme check in using Blackboard Collaborate video conferencing

This initial session was unstructured and badged as such: I’m flailing around; let’s see what happens. I made some slides with key results from the survey – mostly as an excuse to see how to share them using Collaborate.

It turned out to be helpful and amusing to share the online whiteboard which anyone could write to. Some students started to write notes there about what was being discussed (alongside noughts and crosses). A text chat thread started with someone sharing an article.

Incidentally, I was pleasantly surprised that it was possible to dial in using a telephone – this seemed to work fine.

The next whole-programme meeting is in a fortnight (following a student’s suggestion) and will have the following structure:

  1. Whole group chat – how are people doing?
  2. Whole group Q&A – if you have any questions for me about programme stuff
  3. Breakout groups to use however students want; the magic of randomness will determine with whom students chat

I have allocated 1.5 hours for all this but was clear that students can leave whenever they want.

Support group roulette

The idea for this emerged from the whole-group chat with all (taught postgrad) students. Those who want to join a group complete a form with their name, email, year of study (if part-time), and can share any other comment to constrain the randomisation (free-text box).

I’ll randomise to groups and share contact details to each group – group members will then decide themselves how to communicate.

Interactive revision session

Next Monday I will try a more formal revision session, including PowerPoint, for the stats course and plan to record this so others can view it later.

Queering anarchism

Image result for queering anarchism

If you’re looking for something to read over the next few weeks as we deal with the coronavirus (COVID-19) pandemic, try Queering Anarchism.

Anarchists are a varied bunch but share a desire for social organisation in the absence of imposed centralised authority. There are anarchist ideas floating around in current society which become apparent when it feels like something is self-organising or when people take it in turns to lead across formal leadership hierarchies. “Queer” is often used as a synonym of LGBT+ but it can also mean a stance of opposition to dominant societal norms – usually concerning sexuality, relationships, and gender. It’s deliberately vague, encouraging a continuous critical stance. It’s also used as a verb, “queering”, which can be thought of as playful mode of theorising which critiques all we take for granted. This book takes these two contested terms of “queering” and “anarchism” and uses them to explore a range of topics including love, feminism, migration, non-monogamy, gender identity, disability. Give it a go for ideas to try to out or to inspire heated arguments with your pandemic cohabitants.

Power Analysis for Parameter Estimation in Structural Equation Modeling

This app by Andre Wang and Mijke Rhemtulla makes it easy to compute power for SEMs by simulation. Includes options for easily estimating power to detect particular associations between latent variables.

pwrSEM: Power Analysis for Parameter Estimation in Structural Equation Modeling

Truth

“What, then, is truth? A mobile army of metaphors, metonyms, and anthropomorphisms – in short, a sum of human relations, which have been enhanced, transposed, and embellished poetically and rhetorically, and which after long use seem firm, canonical, and obligatory to a people: truths are illusions about which one has forgotten that this is what they are; metaphors which are worn out and without sensuous power; coins which have lost their pictures and now matter only as metal, no longer as coins.”

Friedrich Nietzsche (1873), On Truth and Lies in a Nonmoral Sense

An infinite trolley problem

Remember the trolley problem?

There is a runaway trolley barreling down the railway tracks. Ahead, on the tracks, there are five people tied up and unable to move. The trolley is headed straight for them. You are standing some distance off in the train yard, next to a lever. If you pull this lever, the trolley will switch to a different set of tracks. However, you notice that there is one person on the side track. You have two options:

    1. Do nothing and allow the trolley to kill the five people on the main track.
    2. Pull the lever, diverting the trolley onto the side track where it will kill one person.

Which is the more ethical option? Or, more simply: What is the right thing to do?

There are now thousands of variants of this. Today I saw this one:

My first thought was, aha, finally a chance to apply Cantor’s diagonal argument to something useful.

Let’s start with an easier version. Suppose the lower rail is bounded in length, say to 99.9999… metres and there are as many people tied to the track as there as real numbers in the bounded set [0,100). That’s infinitely many reals. Each person lies at a position somewhere in [0,100).

Give each person in the heap a number 0,1, 2, … Since there are infinitely many reals in [0,100), there must be infinitely many people stacked up in some way within that almost-100 metre stretch.

Now work along that infinite philosophically imagined mound of people and construct a new real number as follows from where they are lying along the rail (in metres). (You have a very precise measuring tape.)

From person 0, take the number to the left of the decimal point on their measurement and compute 99 minus that number. From person 1, take 9 minus the 1st number to the right of the decimal point. From person 2 take 9 minus the 2nd digit, and so on. So from person i, take 9 minus the ith decimal digit (pad out the digits with zeros where necessary).

So now we have a new position on the track where by definition there is nobody tied to the track. Our original assumption is false: it was not possible to stack infinitely many people in infinitely many real-numbered positions along an almost 100 metre long stretch of track. Contradiction.

We can’t do it for [0,100). That means there’s no hope for doing it for all real numbers since [0,100) is a subset.

So it is not possible to tie heaps of people to a track so that there are as many people as there are real numbers.

It is (sort of) possible to do it for countably infinitely many people—the top track. But see the distinction between potential and actual infinity.

A more daring approach to writing theory

“What if we took a more daring, modernist, defamiliarizing approach to writing theory? What if we asked of theory as a genre that it be as interesting, as strange, as poetically or narratively rich as we ask our other kinds of literature to be? What if we treated it not as high theory, with pretentions to legislate or interpret other genres, but as low theory, as something vulgar, common, even a bit rude—having no greater or lesser claim to speak of the world than any other? It might be more fun to read. It might tell us something strange about the world. It might, just might, enable us to act in the world otherwise. A world in which the old faith in History is no more, but where there are histories that still might be made—in a pinch.”

—McKenzie Wark (2019). Capital is dead.