Big White Wall (edited: now in 2020 known as “togetherall”) has been marketing its service using this graphic:
Sounded too good to be true, so I had a look to see if I could find out more.
Their website states, “A copy of the independent review is available on request”. When I asked for a copy (May 2014), they replied that “there is some potentially commercially sensitive data in this review, so we’re not able to share it directly.” I continued asking and even had a meeting with one of their psychiatrists, explaining how important it is that they are transparent about their evidence, especially since they are marketing to CCGs and competing for NHS funding – still nothing.
I accidentally discovered today (Jan 2015!) that the report is online over here, linked to a post on the Mental Health Innovation Network forum. (I have also mirrored it here in case it disappears.)
Now the 95% claim makes sense. The report explains that this is the percentage of users who “reported one or more improvements in mental wellbeing”. There are problems with this kind of evaluation, the most obvious of which is the absence of any comparison group. This means we don’t know what percentage of non-users “feel better” too, so it is difficult to attribute any improvement to Big White Wall.
Additionally, the “one or more” obscures important detail provided in the report. As the tables below show, the improvement rates are often very much lower than 95%.
