Security through obscurity

Vaughan at Mind Hacks writes:

“Almost every psychological test relies on the fact that the person being assessed has no foreknowledge of the material.”

There is an assumption that psychologists are out to trick people into revealing their psychological traits.  Undergraduate students, for instance, often complain that personality questionnaires are rubbish because it’s obvious what they’re asking or what the answers should be.

I don’t think it’s a problem if what is being tested is transparent.  For instance take selection for a job or a university.  The point should be that a good selection process benefits both the candidate and the folk making the selection.  If you cheat your way into a job or onto a course you’re not capable of doing by obsessively practising a test, then it won’t be long until the pressures of performance will force you out.

Oh okay. This assumes that tests used for selection have predictive validity. And… well you can imagine how this argument would continue, how some jobs might require people who are good at pretending, how validity might depend on people being motivated enough to try some practice IQ tests—acquiring foreknowledge might (unknowingly to the tester) be part of the test, how, er, capitalism needs to be destroyed, and so on…

I find it frightening that tests used in clinical diagnosis should somehow trick patients into revealing their complaints. There is a diagnosis tool for autism spectrum conditions which works basically by tricking people into revealing how socially inept they are by various “social presses”, including during a period which appears to be a break between testing sections. The most frightening part of this was the obvious power trip the person who explained this test to me was on every time she used it.

Admittedly it’s a bit messy, but be suspicious of tests which are designed to trick people.