Can We Stop Arguing Over What’s The “Best” Methodology?

By Ron Sellers

Blogs and message boards are a great way to discuss research questions.  Some discussions I’ve seen recently:

• Is it better to do one long survey or multiple shorter surveys?
• Is qualitative research or quantitative research more likely to result in usable insights?
• Should researchers be using CATI interviews or online panels?
• Are focus groups superior to IDIs, or do IDIs provide greater insights?
• Are traditional focus groups better than online qualitative, or is it the other way around?
• Are online panels better than social media sampling, or vice versa?

The fascinating thing is that in every case, the posed question is then followed by multiple answers and discussion of which option is right.  People passionately defend their selection and denigrate the other choice.  I’ve even seen some nasty comments and arguments arise from these debates.

The funny thing is that to have a blanket debate like this on these topics is largely pointless.  There simply is no overall answer to whether qualitative projects should be online or face-to-face.  There is no way to answer whether qualitative or quantitative work generates greater insight.  That’s like arguing whether every American would be better off driving a minivan, a family sedan, or a pickup truck.  If you want to haul lumber and tow your boat, you drive a pickup.  If you want to take your six kids to the soccer game, you drive a minivan.

To put it bluntly, saying a minivan is simply a better vehicle than a family sedan is, well, rather dumb.  So is saying IDIs are simply better than focus groups (or the other way ‘round).

As researchers, part of our responsibility is to know the best way of getting the insights we seek, taking into consideration the target audience, the geography, the budget, the timeline, the topic, and the information objectives.  There are times when telephone research is the best option.  There are other times when the project is much more appropriate for an online approach.  And guess what – there are even times when mail surveys or intercept interviews are the preferable way to go.

So can we stop arguing over what’s the “best” methodology?  There is no “best” methodology or approach, other than the fact the “best” methodology is the one that will be most ideal for the specific project at hand.

Unless you really think that everyone in the world should drive a minivan.