What can we do?

We all read or hear about “studies” that claim this or that. How good are they? How can we know? Certainly there can be differences in opinion, differences in interpretation, and more commonly perhaps, differences in the significance we attach to each study. This page presents information on how to answer each question. One important branch of mathematics is statistics. Descriptive statistics deals with what is and done right will be accompanied be a defined error bar. On the other hand, inferential statistics provides means for interpretation and decision making, again with defined error bars. Most basically, these two branches of math comprise the science of designing experiments, measuring events, selecting data, learning from a base of data, controlling outcomes, and communicating probabilities and uncertainties. As a science, statistics is hard to master, not because the math is arcane, but because it is not always self-evident to beginners which “test” to apply, nor does the average person accustomed to thinking in probabilistic terms. That wisdom comes with experience for most of us having some expertise. The sub-discipline of industrial statistics, for example, requires little more than the four functions of a hand calculator to perform elaborate calculations that can lead to astonishing results.

In practice, it is in the data acquisition and selection, experimental design, and selection of an appropriate inferential test (each with its own assumption of data distribution function) that bedevil otherwise good researchers or reporters. Errors somewhere along the line can creep into even the most careful of studies.

Not by any means is it all lost to the laypersons. A few simple rules can be used to help you decide which study to believe or rely upon. Look for them and you will be richer for the effort. You don’t even have to know how to accumulate sums; all you need to know is how to think in forming your judgment of the solidity underlying a finding or conclusion.

A valid plan of inquiry will answer the following questions:

• Were the samples or subjects selected randomly, where every sample or subject has the same likelihood of being selected as all others in the block or universe of possibilities?
• Were the samples representative? The perspective here involves the universe of data. One example, extreme enough to be remembered should fix the issue in your mind. “A truckload of gold ore is worth $45,000. It is one thing if every pebble and rock contains its fair share of gold. It is quite another if all the gold is concentrated in a single nugget! In that case, only as sample comprised of the entire truck load is representative, no matter how random the sampling plan.”
• In medicine, the sampling issue is equally acute. It is rarely possible to ignore genetic, gender, or other similar differences. To what group or sub-group do the results apply specifically?
• Were there enough samples taken to define an error of measurement, the standard deviation, SD, (square root of the average sum of squares of the quantity being measured) that when divided into the mean (average) and multiplied by the square root of the number of samples, yields a value of “t”, that defines the probability that the quantity being measured has a mean value that is not real? That was a long sentence, but use it a few times and it becomes second-nature.
• Was a baseline or control group employed with which to compare the data?
• What is the probability that the reported conclusions were right? What is their probability of being wrong—though the former is more generally useful?

Treating trends in data, regression (also known as least squares) is a common method.

• How well was it defined?
• What distribution function was assumed?
• Was data transformation applied?
• Was the residual error normal?

Differences in level of response between groups, “t” test for two groups, ANOVA for two or more groups are most common:

• Was the distribution function or any transformations of data reported?
• Dry and arcane it is—known to only a few.
• Making research more competitive, powerful and efficient by using balanced experimental designs that yield data bases that provide trends if effects and their corresponding significance levels from;
  • Two or more main variable.
  • Interaction among main variables.
  • Residual test errors against which all main effects and interactions can be tested against.
  
  Consider the following questions.
  
  
  • Young parents: is our child normal, gifted, retarded or harboring a genetic defect? And what then? Should our child be vaccinated? What are the individual and society tradeoffs?
  • Homeowner: How much insurance do I really need? What are the real risks? What about all the other opportunities to buy insurance? Are any cost effective to me? Where can I find out?
  • Commuter from the countryside: Which route to work is safest? How can I be sure?
  • Older parent: Should my child go to college and if so which one? And what then?
  • Sally citizen: Is the earth getting warmer? And so what if it is?
  • Young engineer designing a flying car: How can I avoid letting a drunk kill him/herself and others? How can I answer that?
  • Police Chief: Are my policies making any difference in view of social trends that naturally occur? If so in what direction and by how "sure" can I be? How sure can I be?
  • Voter deciding on candidate: Are those people across the water really as bad as the candidates make them out to be? If so, what can I do? If not, same question.
  • Concerned politician: How can I forecast what is to come and be ready with policy or response in time? What does history really have to say?
  • Budding scientist: Is this effect I see real? How sure do I want to be? How do I interpret the data? Is any of it false?
  • Child realizing death is inevitable: How long will I live?
  • Economic Policy: Does economic history support or deny claims made by politicians?
  • Data Mining: Theses studies are fraught with two kinds of error. Are both reported? How many null hypotheses were considered?
  
  The crucial issue is that no one knows or can ever know the outcomes exactly. But you can know the odds. Knowing the odds is how insurance companies and casino operators can be sure of their profits—to very, very, high degrees of probability.
  
  Probability reigns supreme in this universe, and the best each of us can do is understand it, hopefully on both the intuitive or mathematical levels akin to those above, which should be enough to get you started.
  
  For example visit Clinical Trials for what the government is making available, though not all concerned scientists are in fact reporting.

  Posted by RoadToPeace on Monday, April 30, 2012.

  Rate this article:  |  (current rating: 173)