Tuesday, June 13, 2006

Polling Morality

A couple of days ago, a Leger Marketing poll on "morality" was released. It's findings weren't terribly exciting - pedophilia registered as "immoral" by 81% of respondents, and divorce rattled out at the bottom of the list, considered "immoral" by less than 20% of respondents.

Inevitably, the alarmists started crying that nearly 20% of Canadians obviously must think that Pedophilia is morally acceptable. Clearly, Canada must be on the road to hell.

Skeptical of such inferences, I went digging and found the poll results on Leger's website. The document itself is here. Since Leger doesn't share their raw data, we are stuck with examining the results of their analysis and the raw questions used in the survey.

The 81% number, is naturally a very broad average number, applied to the entire population from a tiny sample (~1500 respondents). As Leger's report points out, there are a lot of subtle variations on this:

As shown in the following table, Canadians view pedophilia as the most immoral type of behaviour. It is condemned by women (84%) more than men (78%), by 45 to 54 year olds (88%) more than 18 to 24 year olds (74%), by Quebeckers (91%) more than respondents from all other provinces combined (80%), by high income earners (86%) more than low income earners (66%) and more by professionals (89%) more than students (70%).


Consider, 45 to 54 year olds (88%) more than 18 to 24 year olds (74%) - 45 year old people are far more likely to have children of their own, and perhaps have given some thought to protecting their children from predation, than someone in the 18-24 year old bracket. In other words, someone between 18 and 24 may simply have not even thought about the issue yet, much less come to any clear conclusions in their minds about its moral status.

The next question we must examine is how did Leger frame the questions? It turns out that they didn't do much at all to frame their questions.

The entire survey consisted of the following style of question:

PC2. Do you consider the following behaviour as IMMORAL?
a) ... Contraception?
Yes...................................................................................1
No....................................................................................2
DNK/Refusal......................................................................9


The first thing that becomes glaringly apparent is that Leger's results do not indicate if they distinguished between negative and "Don't Know or Refusal" responses in their analysis. A reader is left to presume that the latter two categories were lumped together. Here, we encounter the first problem with the alarmist cry that 19% of Canadians think that pedophilia is "morally acceptable". We simply cannot make such an inference - topics such as what are contained in Leger's poll are very sensitive subjects for some people, and the question itself may well cause a lot of people to refuse to respond simply because they are so polarized by the subject that even a question about its moral status throws them off guard.

The second point is the assumed commonality of definition in the terms used. While some questions are centered around concepts that are fairly clear - for example divorce or contraception, others are less so. The yes/no/don't know model of responses doesn't provide any room for "grey zones" or subtle responses. For example, while I don't necessarily have a problem with either contraception or abortion per se, I may well have a moral objection to someone using abortion instead of contraceptives repeatedly.

Of course, a ten question questionnaire does not leave any room for such subtleties. I think it's important to recognize that morality is quite different from a survey about shopping preferences. (Sadly - the surveys I've encountered have provided more room for nuanced responses to my preferences for breakfast cereal than Leger's Morality poll does) Morality is a complex subject, often coloured by individual experiences. For example, theft is immoral (in my mind), but is theft "immoral" if I am stealing from the thief that stole from me previously? The Catholic Church teaches that contraception is immoral, and yet is it immoral to protect myself and a partner from disease by using a condom? The world is filled with conundrums like this, shades of gray where black is not black, nor is white actually white.

Even the term "immoral" is dangerously loaded from an interpretative standpoint. Many people rely upon religious teachings to guide their morality, yet few religions agree on the moral status of any two topics, and when you throw into the mix those who are atheistic in their worldview, it seems unlikely that there is any clear codification of "morality" that we could agree upon. (A reality that has led Canada's Supreme Court towards a "measurement of harm" approach to many "moral" topics in their deliberations)

Further, the way that Leger has weighted and analyzed the data (and the lack of raw distribution data in their report), it is very difficult, if not impossible, to derive any serious conclusions from it. Being generous, I would suggest that Leger's poll might give a broad sense of how Canadians see various moral topics, but that's like expecting a 2" paint chip from the hardware store to give you a clear idea what the paint will look like through an entire house.

[Note]:

I am absolutely not defending any specific behaviour here - merely pointing out that the Leger poll is very weak in terms of its informational content, and that I am very cautious about inferring anything from the "negative/don't know" response set other than it is there - it's meaning is another thing entirely.

No comments:

The Cass Review and the WPATH SOC

The Cass Review draws some astonishing conclusions about the WPATH Standards of Care (SOC) . More or less, the basic upshot of the Cass Rev...