Questionnaires are often used in medical research to find out what people know or believe. Studies based on questionnaires can be very informative but need to be read critically.

https://oncologypro.esmo.org/Meeting-Resources/ESMO-2017-Congress/European-survey-of-907-people-with-cancer-about-the-importance-of-nutrition is a report presented at a meeting of the European Society for Medical Oncology. It concludes that most of the people with cancer who participated in the survey would like to receive more information about nutrition.

Does this mean that most people with cancer would like to receive more information about nutrition? 907 people completed the questionnaire, so you might think that this is a valid conclusion. But I think that we can’t be sure.

The problem is whether the 907 people are a representative sample of everyone who has cancer. The questionnaire was sent to all cancer patients and survivors, in ten countries, except for those with brain and breast cancer. So the people who received the questionnaire should be a representative sample.

But it is likely that not everyone who received the questionnaire replied. The authors of the report should have told us either (i) how many questionnaires were sent or (ii) what percentage of recipients replied (the response rate).

Why is this information important? Because if only a small proportion of the recipients replied, they may not be a representative sample of people with cancer. For example, it is possible that many of the people who didn’t reply had no complaints and so didn’t bother to complete the questionnaire.

In other words, unless everyone who received the questionnaire replied, the 907 people were partially self-selected and so may not be a random sample. Perhaps the authors of the report gained the impression that more nutritional advice was needed because many of the respondents wished to complain about some aspect of their treatment (not necessarily nutritional advice).

This problem arises in all research based on questionnaires. It can’t be avoided because, inevitably, some people won’t reply. If the authors had provided the response rate, or information for us to calculate it we would have been able to judge the reliability of their study.

Remember that science is not about simply believing the experts (post 17.9) – we need the information to be able to think for ourselves!

Related posts

19.4 Dietary information
17.33 Lord Kelvin and flying machines
17.9 Scientists believe that
17.1 It’s obvious
16.46 The placebo effect
16.36 Good and bad
16.32 Faith in science
16.28 Significant differences
16.1 Drug safety testing