Data has recently been published on the percentage of civil servants who are LGB or other, broken down by profession and department, as of 31 March 2020. Here are pictures, created using R (code here).
This year as part of Covid-enforced “digital transformation” I ended up writing longer tutorial notes than usual so that students could work at their own pace. The module I teach assumes that students have already taken an intro stats course using software other than R, covering up to regression, but that they are likely to have forgotten how to do the latter.
The core texts I use are Fox and Weisberg (2019), An R Companion to Applied Regression (Third Edition) and Healy (2019) Data Visualization: A Practical Introduction – both excellent.
These notes add explanations where students were likely to be struggling and exercises with solutions.
I’ll be putting them all online over here.
I’ll be updating this, but first thoughts:
Fox, J., & Weisberg, S. (2019). An R companion to applied regression (3rd ed.). London: SAGE Publications Ltd.
See also online material, including free appendices and R code.
Healy, K. (2019). Data Visualization: A Practical Introduction. Princeton University Press. (Free online version.)
Wickham, H., & Grolemund, G. (2017). R for Data Science: Import, Tidy, Transform, Visualize, and Model Data. Sebastopol, CA: O’Reilly. (Free online version.)
Chang, W. (2020). R Graphics Cookbook (2nd ed.). Sebastopol, CA: O’Reilly. (Free online version.)
Lüdecke D (2018). ggeffects: Tidy Data Frames of Marginal Effects from Regression Models. Journal of Open Source Software, 3(26), 772. doi: 10.21105/joss.00772
This is very handy for getting predictions from models, focusing on the effect of predictors of interest whilst holding covariates at some fixed values like a mean or (for factors) mode.
See also the package website for illustrative examples.
Gelman, A. (2011). Tables as graphs: The Ramanujan principle. Significance, 8, 183.
Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition.. Chapman & Hall/CRC. Boca Raton, FL. (Free online version.)
See also the package website.
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31, 337–350.
“… correct use and interpretation of these statistics requires an attention to detail which seems to tax the patience of working scientists.”
Colquhoun, D. (2014). An investigation of the false discovery rate and the misinterpretation of p-values. Royal Society Open Science, 1, 140216. doi: 10.1098/rsos.140216
This generated lots of debate – I like how it attempts to use Bayes rule to turn p-values into something useful and the explanation in terms of diagnostic test properties. See also this on PPV and NPV.
Rafi, Z., & Greenland, S. (2020). Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise. BMC Medical Research Methodology, 20(1), 244. doi: 10.1186/s12874-020-01105-9
Interesting proposal to use s-values, calculated from p-values as −log₂(p). It’s a simple transformation: p is probability of getting all heads from −log₂(p) fair coin tosses. For example if p = 0.5 then s = 1; toss a coin once then the probability of head is 0.5. If p = 0.03125 then s = 5; toss a coin 5 times then the probability of all heads is 0.03125. But the s-value is supposedly easier to think about. I’m not sure if it really is, but I like the idea!
“… tensions between quantitative and qualitative methods can reflect more on academic politics than on epistemology. Qualitative approaches are generally associated with an interpretivist position, and quantitative approaches with a positivist one, but the methods are not uniquely tied to the epistemologies. An interpretivist need not eschew all numbers, and positivists can and do carry out qualitative studies (Lin, 1998). ‘Quantitative’ need not mean ‘objective’. Subjective approaches to statistics, for instance Bayesian approaches, assume that probabilities are mental constructions and do not exist independently of minds (De Finetti, 1989). Statistical models are seen as inhabiting a theoretical world which is separate to the ‘real’ world though related to it in some way (Kass, 2011). Physics, often seen as the shining beacon of quantitative science, has important examples of qualitative demonstrations in its history that were crucial to the development of theory (Kuhn, 1961).”
“Winwood Reade is good upon the subject,” said Holmes. “He remarks that, while the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.”
—The Sign of Four by Sir Arthur Conan Doyle (hat-tip MP)
(Updated 24 April 2015)
One of my day jobs is teaching psychology students how to do data analysis. Occasionally I quote famous statisticians, for instance to illustrate ways of thinking about analysis, the subjective nature of modeling data, and other fun things. I mention the likes of William Gosset (Guinness and t-tests), George Box (all models are wrong), and Bruno de Finetti (probabilities don’t exist).
Most—often all—of my students are women. Most of my current collection of quotations are from men. This is a problem. So, I’m currently looking for examples of famous statisticians who are women (broadly interpreted; including data scientists, economists, quantitative social scientists). Here’s my current list. Suggestions for others would be most welcome, especially if you have a quotation I can use (turns out that statisticians write in maths most of the the time so it can be hard to find nice quotes).
Quotations—work in progress
“The newly mathematized statistics became a fetish in fields that wanted to be sciences. During the 1920s, when sociology was a young science, quantification was a way of claiming status, as it became also in economics, fresh from putting aside its old name of political economy, and in psychology, fresh from a separation from philosophy. In the 1920s and 1930s even the social anthropologists counted coconuts.”
—Deirdre McCloskey, The Trouble with Mathematics and Statistics in Economics
“The Cabinet Ministers, the army of their subordinates… have for the most part received a university education, but no education in statistical method. We legislate without knowing what we are doing. The War Office has some of the finest statistics in the world. What comes of them? Little or nothing. Why? Because the Heads do not know how to make anything of them. […] What we want is not so much (or at least not at present) an accumulation of facts, as to teach men who are to govern the country the use of statistical facts.”
—Florence Nightingale in a letter to Benjamin Jowett; quoted by Kopf, E. W. (1916). Florence Nightingale as statistician. Publications of the American Statistical Association, 15, 388–404.
“To understand God’s thoughts we must study statistics, for these are the measure of His purpose.”
“The statistician who supposes that his main contribution to the planning of an experiment will involve statistical theory, finds repeatedly that he makes his most valuable contribution simply by persuading the investigator to explain why he wishes to do the experiment.”
—Gertrude M Cox
“It is no use, as statisticians, our being sniffy about the slapdash methods of many sociologists unless we are prepared to try to guide them into more scientifically acceptable thought. To do this, there must be interaction between them and us.”
—Stella V Cunliffe (1976, p. 9). Interaction. Journal of the Royal Statistical Society. Series A (General), 139, 1–19.
… to everyone who sent suggestions!
Randomised controlled trials (RCTs) are great, the gold standard of empirical research. The only thing better than RCTs are systematic reviews of lots and lots of RCTs. (So the story goes.) The reader may have noticed that RCTs evaluating CBT for psychosis have been vigorously debated for many months after a review was published in the British Journal of Psychiatry (Jauhar et al., 2014). Maybe not everyone agrees that RCTs are great (disclosure: I have analysed a couple), but I think it’s fair to say they are unavoidable whether you are trying to design or demolish them.
High-quality psychotherapy research by Patricia Arean and Helena Chmura Kraemer sets out to be a “practical, step-by-step guide” to designing and running RCTs. So why bother with an RCT? Observational trials, the authors explain, might involve studying participants who choose one of two or more interventions of interest by simply observing how they get on. This is problematic as differences in outcomes might be due to whatever factors led to them ending up receiving an intervention rather than the effect it had. RCTs use randomisation to overcome this problem so that people differ only in terms of the intervention received. That’s about it for the “why”: don’t expect debate on the epistemology.
The book’s strengths emerge as it develops: it catalogues issues that should worry study investigators and the authors draw on their own experience to offer hints. The Delphi consensus-building approach is introduced to solve the problem of developing an intervention manual and examples are given of how to word a letter asking for feedback on the proposed result. Randomisation techniques are introduced including horror stories of how they have gone wrong and invalidated RCTs. Ideas are provided for control groups, e.g., waiting list, usual care, and “gold standard” controls, and their strengths and drawbacks. The importance of not using pilot study results to determine sample size choices is explained. Guidance is provided on the people required; for example you need three or more therapists, at least two research assistants in case one takes ill, and a good statistician amongst other people. The Appendix includes a sample budget justification. All practical advice.
The text runs to under 200 pages so this could never be a comprehensive guide to all aspects of RCTs. What this book does do well is provide a systematic menu of options and ideas for things to consider. It might possibly give some ideas of what to demolish too, should you be so inclined, but this book is really only for those who are already sold on RCTs and want to get on with the seemingly painful task of designing and running one.
Areán, P. A., & Kraemer, H. C. (2013). High-quality psychotherapy research: from conception to piloting to national trials. Oxford University Press.
Jauhar, S., McKenna, P. J., Radua, J., Fung, E., Salvador, R., & Laws, K. R. (2014). Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. The British Journal of Psychiatry, 204, 20–29. doi:10.1192/bjp.bp.112.116285
Stephen Senn (2002) explains the need for statistical power analysis.
“An astronomer does not know the magnitude of new stars until he has found them, but the magnitude of star he is looking for determines how much he has to spend on a telescope.”
Senn, S. J. (2002). Power is indeed irrelevant in interpreting completed studies. BMJ, 325, 1304–1304
Computing Cohen’s (1988) U1, U2, and U3 from d (Table 2.2.1, p. 22)
U3 = function(x) pnorm(x) U2 = function(x) pnorm(x/2) U1 = function(x) (2*U2(x) - 1) /U2(x) cohenTab = data.frame(d = c(seq(0.0, 2.0, 0.1), seq(2.2, 4.0, 0.2))) cohenTab$U1 = U1(cohenTab$d) * 100 cohenTab$U2 = U2(cohenTab$d) * 100 cohenTab$U3 = U3(cohenTab$d) * 100 cohenTab
“Models of data have a deep inﬂuence on the kinds of theorising that researchers do. A structural equation model with latent variables named Shifting, Updating, and Inhibition (Miyake et al. 2000) might suggest a view of the mind as inter-connected Gaussian distributed variables. These statistical constructs are driven by correlations between variables, rather than by the underlying cognitive processes […]. Davelaar and Cooper (2010) argued, using a more cognitive-process-based mathematical model of the Stop Signal task and the Stroop task, that the inhibition part of the statistical model does not actually model inhibition, but rather models the strength of the pre-potent response channel. Returning to the older example introduced earlier of g (Spearman 1904), although the scores from a variety of tasks are positively correlated, this need not imply that the correlations are generated by a single cognitive (or social, or genetic, or whatever) process. The dynamical model proposed by van der Mass et al. (2006) shows that correlations can emerge due to mutually beneﬁcial interactions between quite distinct processes.”
Fugard, A. J. B & Stenning, K. (2013). Statistical models as cognitive models of individual differences in reasoning. Argument & Computation, 4, 89–102.