When is a difference really a difference?
loading...
I often see businesses compare their data and then they come up with two means that might be, for example, the return of investment of an adverts campaign versus the return of investment of a Facebook advertisement campaign. The means might describe a year, so that would be the result of a whole year. What you need to take in mind when making a decision and using means is that there is a random factor, at least if you agree with the underlying propositions of the general linear model. The truth is that the statistics we use today might not really be suited to do what we want do.
Increasing effectiveness of two means
If you have two means, there is a chance that those would differ just by random fluke. For example, if you’re at a big conference and you ask one person to rate the conference between 0 and 10, you will get one score. However, if you really want to know what people thought about this conference, it would be better to ask them all if that’s possible. There might be a thousand people and you might not be able to ask all of them, so you ask just some people.
That’s alright, but then what is some people? Obviously if you ask one person, the chance that you make an error is very high. The chance is less when you ask two people because then you can take a mean. Still, with two people, the chance that you have a biased result is quite high. What about ten people? What about hundred people? You might get quite good results that really represent everyone who is at the conference if you ask a hundred people. The more people you ask, the better.
Consider other factors
On the other hand, there might be costs involved for having participants. More participants are more costly, as is more months running a campaign. There’s a function about this cost, and the chance that you might be wrong because you didn’t include this function. As a rule of thumb, you can use this: if you compare two groups with each other you would like to have 30 data points in each group. That’s a very basic rule of thumb. That means if you have two conferences, and you want to know what is the overall evaluation of each conference, you should at least ask 30 people from each.
Data points are important
Getting back to the advertisement campaigns, it might not be a good idea to have your campaign run for 30 months before you judge what’s best. Instead, you can probably take 30 days, just make sure that in those 30 days, something actually happens. If you have days in which nothing happens, or let’s say on that given period, your campaign only had 20 visitors in total, that wouldn’t be enough obviously. However, if both of your campaigns have a minimum of 30 visitors, you can start saying something about them.
What it actually comes down to is this: don’t take the mean and compare it without looking at the number of data points this means consists of. It’s because you might make wrong decisions. If you are making business decisions based on statistics, make sure that you understand the statistics or you have someone in your team with knowledge about statistics. I can’t mention enough how important that is, but if you do otherwise, you might make a wrong decision that could cost you a lot of money.
Best you can do: hire me (-:
loading...
Related Posts:
Posted in General