Somehow I doubt the full humor of the situation will be apparent to most people, but this conversation occurs at my job on an occasional basis. A sample size of N=3 is a barest minimum for even a t-test, and on that basis alone probably isn't enough, but I'm willing to set that aside for the bigger issue, because it depends on the question.
It is a matter of the question asked. Not all experiments are the same, nor are all samples the same. In my conversation this morning, there was a basic misunderstanding of the sample unit. There was a sample size of N=3 in one group (treatment), and another group of 3 serving as a control. The problem (well one of the problems) was that the controls were not used as an independent group, but rather as a way to normalize each of the first 3. Instead of having two groups of 3 each, that we really had was a single group of 3 pairs of subjects (matched pairs). This lead to a few hours of trying to untangle What had been done versus what needed to be done. Frustrating, but then education is an important part of my job too.
I don't completely understand, but I think I get it. Basically they both need to know about each others fields to communicate. I get the sample size, intuitively bigger is better but only 3? thats tiny.
ReplyDeleteDid you just type it up and sent it to that movie making thingamabob?
There is definitely some difficulty in communications. The lab folks sometimes get so wrapped up in the science they forget there are many types of experimental designs. The biostatistician wants to know all about the experimental design and doesn't really understand the science.
ReplyDeleteA sample size of 3 is indeed tiny, but is not uncommon in some areas of lab science. In some cases the materials for each sample cost thousands of dollars, require weeks of work to produce, or there could be hundreds of combinations that need to be tried. They are looking for very strong effects that haven't yet been observed, rather than subtle difference that would need a big sample size.
This video was being circulated at my work, but I don't know who made it.