Group versus individual statistical predictions

Lovely article in the Guardian by Christine Evans-Pughe on how making statistical predictions about individuals is exceedingly tricky. Reports work by Hart, Michie, and Cooke (2007).  Abstract of the latter:

BACKGROUND: Actuarial risk assessment instruments (ARAIs) estimate the probability that individuals will engage in future violence. AIMS: To evaluate the ‘margins of error’ at the group and individual level for risk estimates made using ARAIs. METHOD: An established statistical method was used to construct 95% CI for group and individual risk estimates made using two popular ARAIs. RESULTS: The 95% CI were large for risk estimates at the group level; at the individual level, they were so high as to render risk estimates virtually meaningless. CONCLUSIONS: The ARAIs cannot be used to estimate an individual’s risk for future violence with any reasonable degree of certainty and should be used with great caution or not at all. In theory, reasonably precise group estimates could be made using ARAIs if developers used very large construction samples and if the tests included few score categories with extreme risk estimates.

Hart, S.D., Michie, C., & Cooke D.J. (2007) Precision of actuarial risk assessment instruments: Evaluating the ‘margins of error’ of group v. individual predictions of violence. British Journal of Psychiatry, 190, s60-s65.

Sónar 2007

Sónar was ace. Some people I’ll probably look out for (or avoid…):

A lovely vocalist in the Red Bull room. No idea who she is. She used one of those delay loop pedal thingies to sing with herself.

Les Anciens: experimental baldy blokes with very tight and clever percussion, bass, random sounding, jazzy but not with the right kind of chords to be jazz if you get what I mean.

Littleprettyautomatique: nice dubby trendy housey DJ set.

Kazumasa Hashimoto: LOVELY. live piano and piano style playing of electronica sounds with live drums.

FM3 with the Buddha machine: BIZARRE wee machines each of which plays 9 loops. There’s a button on the side to decide which loop plays. The performance looked like two people playing a game, moving the machines around, stacking them, knocking them over, sounds picked up with microphones. Looked impressive but we couldn’t really hear what was going on!

Piana: BEAUTIFUL Japanese Sigur Rós-like sounds.

FM3 feat. Blixa Bargeld: WEIRD. Yer man from Einstürzende Neubauten. Lots of delay-loop pedal madness with instrument after instrument fed into it all. Blixa’s voice was put through some kind of scary harmonising layered thing which made him sound like Satan.

Burbuja: PERFECTION. One of my favorite acts from Sónar. (I wish I’d seen Night of the Brain as well—also featured Merche Blasco and Cristian Vogel.) Here, have a listen to Roped.

Beastie Boys: after the initial “WOO it’s the Beastie Boys!” effect had worn off I just thought, whatever :-). Nothing particularly memorable happened.

Nettle: KINDA COOL! electronic drumming action with live Arabic instrumentation.

Black Affair: vocals, drum machine and sequencer, and bass. Dull to watch but sounded like a male-led Ladytron.

Clark: looked like the drummer fell out with the electronic dude at one point. Looked boring but sounded KINDA COOL. Big fat electronica with huge drumming.

Richie Hawtin: bopped away to him for a while! Very serious—wish it had been less gurny!

[Um, some other bands and DJs whose names I’m not quite sure of. I was just kinda wandering around!]

Wolf Eyes: crap, loud, noisy, angry. People with fingers in ears for this, my shorts were flapping with the bass.

Andy Stott: I like his stuff a lot. After Burbuja, though, I really wasn’t impressed by people just nodding behind a laptop—and that’s all he did.

Mira Calix: more nodding. I like her music but visually it was totally boring.

Blackout Concert: this was KINDA FUNKY, down at the Centre d’Art Santa Mónica. Gist: they dumped us all in the basement of an art gallery and turned off everything that makes any noise (water heating, air conditioning, whatever else there is in a building), step-by-step, then the lights, bolted all the doors shut. The silence interrupted only by a bloke issuing orders on a walkie talkie, “Step 1… *tissssst* turn the air conditioning to level… 2 *tissssst*” (presumably—was in Spanish).

Christian Vogel: AMAZING. Up until this bloke’s set I’d decided that techno was no longer worth the misery. Bleepy jerky techno DJ set. The Burbuja crew [I’m not obsessed, honest, I didn’t even mention that I noticed one of the Burbuja dancer people in the Blackout Concert…] were merrily bopping away! Vogel looks a bit like a PhD student trying and failing to explain something mathsy at an undergrad seminar. He sort of stumbled around behind the decks, half-jerking, but still managed to mix seamlessly.

Mogwai: alright, but, MEH!, by this point I was all technoed up so ventured across to another room.

Black Devil Disco Club: AMAZING–just what I needed. Sort of Donna Summery style basslines, bongos, random singing, bloke with long hair and beard. Bouncy discotek, I guess. Audience was a happy audience—much bopping.

Milky Globe: AMAZING—still Lo Recordings. More bouncy happy stuff.

Cursor Miner: AMAZING. I think I like Lo Recordings (also home of The Chap whom I love). “There’s a battle there’s a war / Killing used to be a chore / Now there’s just a switch to flick / Oh so easy oh so quick.” This dude was wearing a ski mask, had flashing lights on his head, wearing a fluroscent vest and Y-fronts. Loved it. Mostly banging trance-techno but I’m sure I heard him sing sonething about philosophical theories of free will.

Altern8: FANTASTIC! They played a DJ set featuring rave classics and the likes of Voodoo Ray, Inner City—hmmm what’s it called—you know that Detroit tune with the vocals.

Much fun had.

What’s the difference between fixed and random effects?

Gelman (2005, p. 21) to the rescue.

We prefer to sidestep the overloaded terms “fixed” and “random” with a cleaner distinction […]. We define effects (or coefficients) in a multilevel model as constant if they are identical for all groups in a population and varying if they are allowed to differ from group to group.

Gelman A. (2005). Analysis of variance—why it is more important than ever. Annals of Statistics, 33(1), 1–53

Logic is the language of love

“Life, love and romance are all about surprise: the surprise of an unexpected bouquet of flowers, a mysterious stranger met by chance at a party, the unannounced return of an old lover at your doorstep, the confusion of finding the unknown side of an old friend or acquaintance who becomes your partner. The freshness of a relationship kept new even after years.

“Alive equals surprise.

“The logical world … seems to be a world of no surprise, no serendipity, no romance. Axioms generate propositions, one deduction blindly follows another. The game is over almost as soon as it is begun. Where is the romance in that? No chance for the heart to overrule the head, contradiction to conquer cogitation, desperation and devotion to deliver you from derivation.

“Or is there?

“In the 1920s, a mathematician named Goedel proved that in any consistent logical system [the result is a bit weaker than that… AF], there will always be statements whose truth or falsity can’t be proved by the simple mechanical rules of logic. Even if these “undecidable” statements are appended to the list of axioms, as long as this enlarged system remains consistent, there will still be other statements whose proof or refutation lies outside the power of formal reasoning.

“The unknowable and the unpredictable is embedded in even the most simple of things. Romantically speaking, I like to think of this as saying that if we have a guarantee of truth, and thus the possibility of honesty, then from this we must necessarily have surprise, even mystery—and maybe then, just maybe, with a little bit of luck, we must have love. Q.E.D.”

From Logic is the language of love by Daniel Rockmore

How to get someone’s g

“Intelligence”, “IQ”, “g” (due to Spearman), are terms that are bandied around.

The following may be helpful: the gist of how to calculate someone’s g score, which is often used as the measure of someone’s “intelligence”.

For example, that’s the “IQ”/”intelligence” referred to in the recentish BBC article on research linking childhood intelligence and adult vegetarianism (clever children grow into clever vegetarian adults).

  1. Give hundreds or thousands of people a dozen tests of ability.
  2. Zap everyone’s scores with PCA or factor analysis.
  3. g is the first component and usually explains around half the variance.  Here’s an example genre of analysis of g with other facets to psychometric intelligence.
  4. Use the component to calculate a score.  For factor analysis there are many ways to do this, e.g. Thompson’s scores, Bartlett’s weighted least-squares.  The gist is that for each person you compute a weighted sum of their scores, where the weights are a function of how loaded the particular test score was on g.
  5. To get something resembling an IQ score, scale it so it has a mean of 100 and an SD of 15.
  6. Talk about it as if it were a substantive psychological construct, rather than a statistical artefact 😉

What is this mysterious g thing?

Florence Nightingale

A bio of Florence Nightingale, statistician and nurse. Excerpt:

“Nightingale helped to promote what was then a revolutionary idea (and a religious one for her) that social phenomena could be objectively measured and subjected to mathematical analysis. Her work with medical statistics was so impressive that she was elected (in 1858) to membership in the Statistical Society of England. One of the pioneers in the graphic method of presentation of data, she invented colorful polar-area diagrams to dramatize medical data. Although other methods of persuasion had failed, her statistical approach convinced military authorities, Parliament, and Queen Victoria to carry out her proposed hospital reforms.”

[Photo of her Polar Area Diagram (“coxcomb”) from over here.]


More and more I found myself at a loss for words and didn’t want to hear other people talking either. Their conversations seemed false and empty. I preferred to look at the sea, which said nothing and never made you feel alone.

– Paula McLain, The Paris Wife


Love, the warmth of bodies in contact, is the only mercy shown to us in the darkness. But the only union is that of the organs, and it can’t bridge over the cleavage made by speech. Yet they unite in order to produce beings to stand by them in their hopeless isolation. And the generations look coldly into each other’s eyes. If you cram a ship full to bursting with human bodies, they all freeze with loneliness.

– Bertold Brecht, In the Jungle of Cities


… of course natural language is meaningless. Even in my most optimistic moments I can only conclude that conversation merely exists to disguise the fact that we are all going to die… No doubt following this lecture you will all exist mostly in silence.

– Maurice Clint, 2001 or so, Belfast, in a maths lecture


Language only lives in and through human culture, which on the one hand needs mutual understanding but on the other hand makes direct communication impossible. […] People who use language lose their primitive desires which, however sinful, remain close to the self. Frightened by solitude, their only home, they become automata, slaves of the monster-machine of public relations.

– Brouwer, Life, Art, and Mysticism


The more the words,
the less the meaning,
and how does that profit anyone?

– Ecclesiastes 6:11


Out beyond ideas of wrongdoing and rightdoing,
There is a field. I’ll meet you there.

When the soul lies down in that grass,
The world is too full to talk about.
Ideas, language, even the phrase ‘each other’
Doesn’t make any sense.

– Rumi


Vows are spoken
To be broken
Feelings are intense
Words are trivial
Pleasures remain
So does the pain
Words are meaningless
And forgettable

– Depeche Mode, Enjoy the Silence


What can be said at all can be said clearly; and whereof one cannot speak thereof one must be silent.

– Ludwig Wittgenstein, Tractatus Logico-Philosophicus


Silence is sexy
Silence is sexy
So sexy
So silence
Silence is sexy
Silence is sexy
So sexy
So sexy

-From Silence is Sexy by Blixa Bargeld of Einstürzende Neubauten


Woke up this morning and my head was in a daze,
Brave new world had dawned upon the human race,
Words are meaningless and everything’s surreal,
Gonna have to reach my friends to find out how I feel,
And if I taste the honey, is it really sweet?
And do I eat it with my hands or with my feet?
Does anybody really listen when I speak or will I have to say it all again next week?

– Shakespears Sister, Hello (Turn Your Radio On)


And it’s not enough
To tell me that you care,
When we both know the words are empty air.

– Calvin Harris, Sweet Nothing


We are spendthrifts with words,
We squander them,
Toss them like pennies in the air—
Arrogant words,
Angry words,
Cruel words,
Comradely words,
Shy words tiptoeing from mouth to ear.

But the slowly wrought words of love
And the thunderous words of heartbreak—
These we hoard.

– Words, by Pauli Murray

Plato’s The Apology of Socrates

On Saturday I saw Yannis Simonides’s moving performance of Plato’s The Apology of Socrates. Here’s a translation (not the one by Yannis). And an excerpt:

“Friends, who would have acquitted me, I would like also to talk with you about this thing which has happened, while the magistrates are busy, and before I go to the place at which I must die. Stay then awhile, for we may as well talk with one another while there is time. You are my friends, and I should like to show you the meaning of this event which has happened to me. […]

… we shall see that there is great reason to hope that death is a good, for one of two things: – either death is a state of nothingness and utter unconsciousness, or, as men say, there is a change and migration of the soul from this world to another. Now if you suppose that there is no consciousness, but a sleep like the sleep of him who is undisturbed even by the sight of dreams, death will be an unspeakable gain. For if a person were to select the night in which his sleep was undisturbed even by dreams, and were to compare with this the other days and nights of his life, and then were to tell us how many days and nights he had passed in the course of his life better and more pleasantly than this one, I think that any man, I will not say a private man, but even the great king, will not find many such days or nights, when compared with the others. Now if death is like this, I say that to die is gain; for eternity is then only a single night. But if death is the journey to another place, and there, as men say, all the dead are, what good, O my friends and judges, can be greater than this? […] What would not a man give if he might converse with Orpheus and Musaeus and Hesiod and Homer? Nay, if this be true, let me die again and again. I, too, shall have a wonderful interest in a place where I can converse with Palamedes, and Ajax the son of Telamon, and other heroes of old, who have suffered death through an unjust judgment; and there will be no small pleasure, as I think, in comparing my own sufferings with theirs. Above all, I shall be able to continue my search into true and false knowledge; as in this world, so also in that; I shall find out who is wise, and who pretends to be wise, and is not. […]”

Need motivation to vote?

This video of Koichi Toyama, part of a 2007 election campaign for the position of Tokyo governor, may help get you down to the polling station.

Choice quotes:

“To all the voters. This nation is horrible.”

“I have no interest whatsoever in political reform or any kind of reform.”

“This nation must be destroyed.”

“I do not have a single constructive proposal!”

“Annihilate everything that exists.”

“I despise each and every one of you.”

“If you think you can change something by voting, you are COMPLETELY wrong.”


“Please give me a phone call.”

(He wasn’t elected.)


Dov Gabbay, logician, fan of psychologism:

‘I got married in 1970. My wife is an artist, and I learned a lot from her; the fact that I can talk about things, for instance. I remember I was going out with her, before we were married, and we were walking from one part of the university to another part. My objective was to get from A to B, she wanted to stop and look at the moon, because it looked very nice. And I thought: “What the hell would I want to look at the moon for, when I want to go to B?” Now, of course, I will look at the moon at all times with her.’

(From an interview with “Ta!”) My reluctance to look at the lunar eclipse suggests I need to meet an artist—pronto 🙂