Surveying religious belief needs social science not hard science [View all]

[cbayer]

http://www.guardian.co.uk/commentisfree/belief/2012/dec/05/nones-on-the-run-religion

People's responses to surveys about the emotive question of religion are notoriously difficult to pin down

Linda Woodhead
guardian.co.uk, Wednesday 5 December 2012 05.09 EST


The Centre for Longitudinal Studies suggests that 'a quarter of responses to any question on religion are unreliable'. Photograph: Dean Murray/Rex Features

Surveyitis is a disease that afflicts people who stay indoors too long poring over data. It can be alleviated by fresh air and meeting people. Symptoms include credulity about the accuracy of survey responses and morbid attachment to outdated questions. It is particularly dangerous in relation to religion.

A recent upsurge in interest in "nones" suggests a new outbreak of surveyitis. Nones are those who declare on surveys that they have no religion or belief. The fact that their number has been rising is cited by humanists and other there-is-no-God-botherers as proof of the demise of religion. A new report from the thinktank Theos points out that the "nones" encompass "nevers" who don't participate, "atheists" who don't believe, and the "non-religious" who don't belong to a particular religion, and that a significant percentage retain some religious beliefs and practices. The report concludes that we are dealing with shades of grey rather than black or white religion or secularity. Fair enough. But only surveyitis could have led to the idea that a bald religion question could tell us anything useful in the first place.

When it comes to surveys, the simpler, more concrete and less emotionally freighted a question the better. "Did you eat an egg for breakfast?" is good and "What party did you just vote for?" is OK, so long as it's anonymous. The problem with religion is that there aren't many questions like this. "Did you go to church last Sunday?" might seem fairly straightforward, but when American researchers counted cars in church parking lots they found that conservative Christians massively exaggerated their actual attendance. That's because it mattered to them – and the more an issue means to people, the more difficult it is to get simple and reliable answers.

It's even worse when questions are also vague and contested. Anything with the word "religion" in it falls into this category. No one has ever been able to agree a definition of religion, nor will they. So it's amazing to expect that survey respondents will miraculously understand the word, and mean the same thing. Even the smallest of changes to the wording of a religion question can make a significant difference to the responses. The Office for National Statistics demonstrated this beautifully when it experimented with small changes to the question on religion in preparation for the 2011 census. A new report from the Centre for Longitudinal Studies suggests with some exasperation that "a quarter of responses to any question on religion are unreliable".

more at link