Scientists know that all data is not created equal. The instruments and methods used can greatly affect the results. Then, there are things like normalization and error bars. The term 'comparing apples to oranges' is over used, but still applicable. Scientists know all of this and we work with the data to try and get a comprehensive picture of what ever it is we are measuring.
But, the public doesn't always understand this. To them, the results are an absolute and absolutes are not apples and oranges. They are apples and apples and can be compared. This sometimes leads to problems in communication and interpretation.
A recent report on polar bear population is just such a case in point. A new study came out that appears to say that the polar bear population is increasing, despite claims by climate scientists that their habitat is disappearing and putting the species at risk. Naturally, climate change deniers are pointing this out. What is not explained very well is that this study was done by aerial surveys and it is being compared to counts done by a different method, catch and recatch. This really is comparing apples and oranges.
The catch and recatch method consists of doing exactly what it says, they catch bears, tag them and then release them. These bears are tracked and later caught again. This data is then used to calculate how many bears there are in a given area. The latest report used aerial photographs, over a larger area, to visually count the number of bears in the target area. Both of these methods have their problems and give a count only withing a certain error range, a plus or minus amount.
So, the issue is the new report, the aerial counting one, is giving a higher number of bears than the latest catch and recatch report. If you take the two as being equal then the number of bears appears to be increasing. But, we don't know if the counts are equal. Can we take the numbers of one and compare them straight up to the numbers from the other? We don't know yet. There are plans to do comparison studies, but it is not yet known how the two compare. We may find that the catch and recatch method gave very different numbers than the aerial count. If that happens then the claim of increasing population means nothing. It is also possible that the two compare well. In that case, the population would have increased over the last few years. The point being, we don't know at this time and it is too early to drawing conclusions. This is just one datum point.
It is interesting to note that the new study does not conclude the population is increasing. It merely states the count. It is the media and public that is drawing the conclusion from that data.
But, if you are going to draw that conclusion, then you also have to note a disturbing note from the study: the number of young bears is very low. Only 3 percent of the population consists of yearlings, as compared to a normal of about 15 percent. If this holds up it would indicate a bad future for the bear population.
Why do we care? After all, the polar bear is one of the most vicious animals in the world. Will the world really care if the polar bear population goes down by a big percentage, or even goes to zero? The answer is that the polar bear population isn't really the story, its the change in their habitat that we are interested in. As Arctic sea ice melts and reduces the bear's habitat, the population will decrease. So, if we see a drop in bear population then that helps to confirm what we have been measuring by other means.
However, we cannot take a single data point in this debate and draw conclusions, one way or the other. We need years and decades of data. No matter what this latest survey says once it is calibrated to the other counting method, it will be only one datum point in many.