Your brain predicts your behaviour better than you can!

Falk et al (2010) have an interesting article showing how future behaviour, applying sunscreen, can be predicted by brain activity as people watch persuasive messages.

Activity in the (a priori selected) “region of interest” (ROI), a bit of the frontal cortex (MPFC), explained 24% of variance in the number of days of actual sunscreen use. Self-reported intention to apply sunscreen explained only around 3% of variance in days use. Nice result.

I think some caution is warranted, however, before people testing message persuasiveness, such as health organisations, invest all their money in brain scanning.

Behavioural methods include not only self report. For instance asking people how smart they are (e.g., indirectly, using Need for Cognition) is not as good as testing how smart they are (e.g., using Raven’s Advanced Progressive Matrices). Similarly there might well be behavioural ways to test the effect of persuasion, or to infer who is most likely to be persuaded, which can predict much better than the self-reported intention.

These results clearly demonstrate that the expensive and time consuming method of functional imaging is entirely unnecessary for testing message persuasiveness. Simply show people the message and then ask them later how they actually behaved. Very cheap and easy! Especially nice if you can try out a few different versions and select the one which works best.

There is always the danger that people cannot be trusted to report their actual sunscreen usage, a point acknowledged by the authors (as their data explain variance in a self-report measure): “direct observation of behavior would be preferred in future studies, our measurement of behavior through self-report is unlikely to have artificially enhanced our results.” So here the self-report of actual sunscreen usage is completely trusted.

The imaging might provide some theoretical clues for why people were persuaded. There is little of this theory in the paper. This is it, in fact: “… this region has been associated with selfreferential processing, but our ROI also overlaps with a more ventral portion of MPFC that has been associated with implicit valuation.” Such theoretical depth is sadly common in the function imaging literature.

I am very impressed that they found an ROI that predicts so well and I appreciate that a lot of work must have gone into this study. It will be interesting to see if it replicates and generalises. I also hope some theory is on the way explaining mechanisms of persuasion.

Reference

Falk, E. B., Berkman, E. T., Mann, T., Harrison, B., & Lieberman, M. D. (2010). Predicting persuasion-induced behavior change from the brain. Journal of Neuroscience, 30, 8421-8424.

Mill on patterns of human existence

“… it is important to give the freest scope possible to uncustomary things, in order that it may in time appear which of these are fit to be converted into customs. But independence of action, and disregard of custom are not solely deserving of encouragement for the chance they afford that better modes of action, and customs more worthy of general adoption, may be struck out; nor is it only persons of decided mental superiority who have a just claim to carry on their lives in their own way. There is no reason that all human existences should be constructed on some one, or some small number of patterns. If a person possesses any tolerable amount of common sense and experience, his own mode of laying out his existence is the best, not because it is the best in itself, but because it is his own mode. Human beings are not like sheep; and even sheep are not undistinguishably alike.”
—J. S. Mill, On Liberty

Some troubling and interesting things about investigating reasoning

Competence models are typically created and explored by a small number of experts.  Boole, Gentzen, Kolmogorov, Ramsey, De Finetti, …  The authority can often be shifted to the mathematics.   However, although non-experts can usually understand a statement of the theorem to proved, often they can’t understand the details of the proof.

There are problems with being an expert.  If you stare too long at the formalism, then you lose your intuition, and can’t see why someone would interpret a task “the wrong” way.  Often there are a priori non-obvious interpretations.

And who decides what constitutes a permissible interpretation?  Some obvious ideas for this are open to debate.  For instance, is it always reasonable for people to keep their interpretation constant across tasks?  Or is it rational to change your mind as you learn more about a problem?  Is it rational to be aware of when you change your mind?

To complicate things further, various measures loading on g predict interpretations.  Does that mean that those who have better cognitive ability can be thought of as having reasoned to the correct interpretation?

Embracing individual differences

This week I finished teaching/facilitating a course entitled Embracing individual differences in thinking and reasoning.  I was asked to give the gist of what this was about.

There’s a bunch of individual differences in how people solve reasoning problems. One way of thinking about this is that some people are very good at reasoning problems and others are not so good, with a continuum in between. But there’s evidence that people are interpreting the tasks in different (and reasonable) ways, and succeeding in reasoning from their interpretation. We examined these sorts of issues on the course.

A simple example is the meaning of “some”, discussed by J S Mill in his 1867, An examination of Sir William Hamilton’s philosophy: And of the principal philosophical questions discussed in his writings. “I saw some of your children today” implicates that I didn’t see all of them (if I had, then I’d have said so), even though the “all” conclusion is compatible with some.

There are many other examples: the degree to which people are affected by how information is presented; whether people can suspend their beliefs and reason from premises which are obviously false; whether people are sensitive to alternative causes of effects, or factors which can disable a relationship between a cause and effect.

The waters are muddied somewhat by complicated relationships with intelligence. So for instance people with higher intelligence (for several of the standard psychological ways to operationalize the concept as IQ) are more likely to go for the normative answer on some (but not all) tasks. But then one can wonder what exactly the IQ tests are measuring.

Things get particularly interesting when people with clinical conditions, such as autism, actually are more likely to give the normative answer for some tasks. There’s a nice example of where their ability to do so was used as an argument for why the normative answer was wrong. One researcher blogged:

Autistics were shown to perform with enhanced logical consistency, avoiding irrational and irrelevant biases that distorted decision-making in their nonautistic controls. However, autistics’ enhanced performance in this study was interpreted by the authors as a litany of autistic failures, imbalances, impairments, deficits, reduced capacities, weaknesses, and impoverishments (several invocations of some of these), none of which were actually found. […] In years to come, we can look forward to interventions designed to overcome this core autistic deficit and to ensure that autistics become as irrational as nonautistics.

There were plenty of issues to debate…

Working on your quirk

Another brief interlude from the stats and cog psych, but related to individual differences in reasoning I think! Interesting aside in a book by Robert Sutton on dealing with assholes (technical term here) in the workplace. He clarifies his defintion of the particular breed of asshole (technical term) in whom he is interested (pp. 16–17):

My focus is squarely on screening, reforming, and getting rid of people who demean and damage others, especially others with relatively little power. […] I am a firm believer in the virtues of conflict, even noisy arguments.

Here’s a special case he describes (pp. 18–19) of people who can occasionally appear to be assholes (technical term), but are not:

I also want to put in a good word for socially awkward people […]. I was struck by how many successful leaders of high-tech companies and creative organizations like advertising agencies, graphic design firms, and Hollywood production companies, had learned to ignore job candidates’ quirks and strange mannerisms, to downplay socially inappropriate remarks, and instead, to focus on what the people could actually do.

Examples he gives include autistic people and those with Tourette’s syndrome.

Reference

Sutton, R  (2007). The No Asshole Rule. NY: Business Plus.

Profiles of intelligence

Johnson, W. & Bouchard, Jr., T. J. Sex Differences in Mental Abilities: g Masks the Dimensions on Which They Lie. Intelligence, 2007, 35, 23-39:

“… we have presented evidence supporting the idealized notion of general intelligence as a general-purpose mechanism that accesses a toolbox made up of components that vary from individual to individual. Though everyone clearly has most if not all of the same tools, individuals appear to differ not only in the skill with which they use their tools, but also in the specific tools they habitually use. For some of the more specific tools, it would appear that using one tool means failing to use another. […] Performance on image rotation tasks is known to predict success in fields such as airplane piloting, engineering, physical sciences, and fine arts better than does general intelligence, and especially verbal ability. What has perhaps not been recognized is that inclusion of verbal ability in assessments used to recruit individuals to those fields may actually act to impair efforts to select those with the talents most relevant to the jobs in question.”

John Raven often quotes Spearman:

“Every normal man, woman and child is a genius at something … the problem is to identify at what … this must be a most difficult task because it occurs in only a minute proportion of circumstances … this cannot be done with any of the procedures in current use …”

(And Raven would still quite like to know how to fix psychometrics.)

"... we have presented evidence supporting the idealized notion of
general intelligence as a general-purpose mechanism that accesses a
toolbox made up of components that vary from individual to individual.
Though everyone clearly has most if not all of the same tools,
individuals appear to differ not only in the skill with which they use
their tools, but also in the specific tools they habitually use. For
some of the more specific tools, it would appear that using one tool
means failing to use another. [...] Performance on image rotation tasks
is known to predict success in fields such as airplane piloting,
engineering, physical sciences, and fine arts better than does general
intelligence, and especially verbal ability. What has perhaps not been
recognized is that inclusion of verbal ability in assessments used to
recruit individuals to those fields may actually act to impair efforts
to select those with the talents most relevant to the jobs in question."

Create your own economy (updated some more)

Recently I read Create your own economy by Tyler Cowen (thanks Michelle for the tip-off!). Interesting page-turner discussing autism, autistic(-like) traits in non-autistics, and implications for society.

Cowen points out (what is thankfully becoming more familiar) that although autism is often associated with tragedy, many autistics and not only savants have cognitive strengths, e.g., being infovores for their preferred areas of interest, better perceptual skills than non-autistics, less suspectiblity to false memories.  He argues that technological tools available today such as iTunes and Facebook allow non-autistics to have the same abilities.  Non-autistics are driven to do the same sort of organisation and searching for information as autistics are, and this is being made possible by technology. He argues that education is even designed to teach non-autistics some of the cognitive strengths of autism.

One side of autism mentioned in the book and not frequently discussed is that autistics are more likely to talk about feelings than make small talk (has this been studied? Is it true? I would like to know more). The emotional experience of autistics is rarely acknowledged.  Cowan gives examples of people who despite appearing outwardly aloof are deeply sensitive, caring, and who are shocked when they’re told otherwise.

There are plenty of examples in the book of people, real and fictional, who appear(ed) to have autistic traits. I found this a tad tiresome (there has been a lot of it about elsewhere), especially when suffixed with hedges saying that of course we don’t know whether they were autistic. The key point is that “what-we-call-what-it-is-that-I-am-talking-about” (to quote Cowan) probably ought not be derived from a name for a disorder. So viewed this way, most of the book is not about autism, but about a cognitive and emotional profile which many people in society have. This is not to say that autistics do not have cognitive strengths—and he discusses some examples in the book—but I do not see what is to be gained by conjecturing that people are/were autistic. What does this explain?  The details matter, not a one word label. (However this could be because I am deeply suspicious of labels in general!)

Lots of good stuff in the book. In general I think it does a great job of defending the eccentric, and argues successfully that many of the traits eccentrics possess are desirable. Good news for academics!

There are plenty of important points on respecting the individual. I like this of course, and am a big fan of positive individual differences research, e.g., discovering the strengths of people diagnosed with various developmental and psychiatric conditions. But I think my favourite sentence in the book is this (relatively unimportant) one:

“In June 2009, a group of Norwegian astronomers broadbast a Doritos ad to a distant star, forty-two light years away.”

This is genius, and I think it’s a good author indeed who can spot and report such facts. It’s these kinds of things that make society fun.

Individual differences (continued)

Continuing the theme.

“I am surprised that the author has used this data set. In my lab, when we collect data with such large individual differences, we refer to the data as ‘junk’. We then redesign our stimuli and/or experimental procedures, and run a new experiment. The junk data never appear in publications”
—An anonymous reviewer in 2005, commenting on research that sought to model individual differences in cognition.

From the intro to Navarro, D. J.; Griffiths, T. L.; Steyvers, M. & Lee, M. D. Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology, 2006, 50, 101-122

Some quick thoughts on individual differences

I view individual differences as trying to pull out some variance from the residual term. Contrast these approaches:

  1. If I do A to a load of people then on average B happens (but with a bit of noise).
  2. If I do A to a load of people, then those who have property P do B, those with property Q do C (and there’s a bit of noise, but less than in setup 1, hence the derogatory response to individual differences research: “Oh you’re just modelling the noise term”).

Much individual differences research is a bit dull, so I’m not surprised it is criticised: collect a load of questionnaire data or IQ scores and examine the correlations (sometimes disguised with structural equation modelling).

Here’s an example I happen to like: some elephants are mammals. Some people interpret some A are B as meaning some and possibly all; others interpret it as some and not all. Two things you can do: try to force people towards one or other interpretation or take an individual differences approach: find some other way to guess whether people are good at language pragmatics, and see if that predicts their interpretation of some A are B. I think both are important. The former is about trying to increase the probability that more people will understand what you mean. The individual differences approach is interesting as much of the information we deal with in life is ambiguous, and people seem to differ in how they deal with language and social context, how often they ask questions, etc. It’s interesting to leave things a bit ambiguous, one might even go as far as to say more ecologically valid, and characterise how people deal with it.

(Well actually there’s at least one other thing you can do: focus on areas of psychology where there are negligible individual differences.)

Ignoring individual differences can be misleading. Suppose you want to model processes based on empirical data. If you assume everyone is doing the same thing, and just average across responses, then your model is not going to be very good; from the abstract of Guasti et al. (2005):

“… some of the manipulations of the experimental context have an effect on all subjects, whereas others produce effects on just a subset of children. Individual differences of this kind may have been concealed in previous research because performance by individual subjects was not reported.”

See also Stenning and van Lambalgen (2005, p. 924):

“… although a considerable number of Byrne’s subjects (about 35%) withdraw the modus ponens inference after the second conditional premiss is presented, many more (about 65%) continue to draw the inference. What interpretation of the materials do these subjects have? If it is the same, then why do they not withdraw the inference too? And if it is different, then how can it be accommodated within the semantic framework that underpins the theory of reasoning? Does failure to suppress mean that these subjects have mental logics with inference rules (as Byrne would presumably would have interpreted the data if no subject had suppressed)? The psychological data is full of variation, but the psychological conclusions have been rather monolithic.”

I quite like this too, on the subject of longitudinal analyses but much more general (Bauer et al 2002, p. 202):

“The logic of deductive longitudinal analyses represents a clear advance over the assumption that all children follow a universal language trajectory. A priori hypotheses about group differences are evaluated with reference to individual developmental trajectories. However, individual variation that is not explained by group differences is relegated to a residual or error term. Given the crudeness of many of the hypotheses in the social sciences, it is often the case that much of the observed variation in individual trajectories is allocated to the residual term. The deductive approach thus seems to be at odds with an explicit focus on individual differences, in that departures from normative patterns are regarded as random error and typically are not investigated.

“The alternative, “inductive” approach seeks to maximize the information that can be gained from the individual trajectories themselves. In contrast to theory driven deductive methods, which evaluate differences between predefined groups, inductive methods are data driven and are used to examine the natural structure of individual differences. These methods are sometimes referred to as pattern oriented, person centered, or personalogic because they begin by examining similarities and differences in individual developmental patterns (see Cairns, Bergman, & Kagan, 1998, for a review). Using this bottom-up strategy, decision rules or clustering procedures are used to aggregate individuals into groups that display similar developmental patterns, irrespective of their status on theoretically relevant predictor variables.

References

Bauer, D. J.; Goldfield, B. A. & Reznick, J. S. Alternative approaches to analyzing individual differences in the rate of early vocabulary development. Applied Psycholinguistics, 2002, 23, 313-335

Guasti, M. T.; Chierchia, G.; Crain, S.; Foppolo, F.; Gualmini, A. & Meroni, L. Why children and adults sometimes (but not always) compute implicatures. Language and Cognitive Processes, 2005, 20, 667-696

Stenning, K. & van Lambalgen, M. Semantic Interpretation as Computation in Nonmonotonic Logic: The Real Meaning of the Suppression Task. Cognitive Science, 2005, 29, 919-96

Gender differences in psychology

My take on this:

  1. There are gender differences in ability, but not many, and the effect size is typically small* (Hyde 2005).
  2. Brain structure development is affected by a range of factors, environmental and genetic. For instance brain structure changes as a result of learning (e.g., Maguire et al, 2000). (And the phrase “hard-wired” is annoying.)
  3. A mean difference between groups, mean(group 1) > mean(group 2), on some measure does not imply that everyone in group 1 is better than everyone in group 2. So when selecting someone for a job, say, you could (a) grab a load of people with the gender which, on average, has (very slightly—see point 1) more of the ability you want and choose someone at random, or (b) you could choose someone who has more of the ability you want, and not focus on what gender they happen to be.
  4. The designers of IQ tests hack their tests to remove gender differences, for instance the designers of the British Ability Scales (version 2) “used three strategies to test for fairness and to remove items likely to increase bias” (Hill, 2005). Blinkhorn (2005) says: “Where there are sex differences to be found, detailed study of the internal workings of the test tends to show why. That’s not based on instinct, but on my professional experience in designing gender-fair tests.”

Blinkhorn, S. (2005). Intelligence: a gender bender. Nature, 438, 31-32.

Hill, V. (2005). Through the Past Darkly: A Review of the British Ability Scales. Second Edition. Child and Adolescent Mental Health, 10, 87-98.

Hyde, J. S. (2005). The gender similarities hypothesis. American Psychologist, 60, 581-592.

Maguire, E. A.; Gadian, D. G.; Johnsrude, I. S.; Good, C. D.; Ashburner, J.; Frackowiak, R. S. & Frith, C. D. (2000). Navigation-related structural change in the hippocampi of taxi drivers. Proceedings of the National Academy of Sciences of the United States of America, 97, 4398-4403

* Cis women are better at giving birth than cis men, with a large effect size.