Market Research and Online Forums [Archive]

View Full Version : Market Research and Online Forums

Jason Dunn

11-11-2004, 06:30 PM

The following colunn was written by Dave Austin in response to an earlier conversation that we had in the forums regarding USB hosting. Dave was particularly passionate about the subject, so I encouraged him to write his thoughts down and I'd publish them as a column. I'd be interested to hear what people think. - Jason Dunn Online forums are the great democratic arenas of the new millenium. They’re the best source of answers in the world. They nurture a feeling of community, allow people to debate pros and cons of new and different things, and they also let us just “chew the fat” with others who share our hobbies and inclinations.<!> Forums also provide an ideal window through which PPC industry decision-makers can see into the brain of their ideal target-customer base. And that’s good, right? We say just what they need to hear, right? From my occupational observations as an engineer and as an SMB consultant for over a dozen years I’m not so sure they get the right messages from us. I’ve observed that most market research is done haphazardly and “off-the-cuff”, enough that I believe that handheld-market-researchers who might lurk in the corners of our forums probably don’t take the time to see beyond polling results and thread titles. Here are 5 reasons why I’m skeptical of how often they get the right message: Sometimes We Don’t Know All The Details When we vote in a poll about a proposed feature for new PDAs, each one of us has to make some assumptions. For example: will it increase the price much, will it diminish battery life much, will it make the PPC larger, or will it decrease reliability, or will existing features be compromised, etc. Too many assumptions like this can invalidate the statistical results of a poll. Forums Provide Solutions, But That Doesn’t Excuse The Problem Are unreliable alarms on the PDA not a big deal because so many people use their PDA for other things, or because you can buy a 3rd party workaround? I’ve heard such comments over the past 4 years whenever I’ve seen the “unreliable alarm” issue come up, and although they can be worthwhile comments do decision-makers ever use those comments to excuse the problem? Could that be why the alarm problem persisted off-and-on for 4 years? We Have Hopes And Fears Many of us voice our opinions out of fear or hope but not always from direct experience because we just don’t have enough cash nor time to try everything – but we do all have valuable opinions, even if they are heavily laden with subjectivity. Also, some of those who do have relevant experiences may sometimes be a little biased, understandably so because the technology in question was poorly executed the first time (especially true with “first adopters”, a huge part of our community). Do decision-makers take that into consideration? We Exaggerate And We Use Superlatives. That’s What You Do In A Forum. Ever hear anyone trash or praise a product or a feature simply because of what they do or don’t already have? Sure you have, but it’s all forum talk, and we’re generally always open to new things – but do decision-makers understand that? We Often Vocalize The Symptoms, But Not The Root Problem USB Hosting is the perfect example. In a recent poll over 70% of the users said they would not expect that feature in their next PDA. However, the issue with USB Hosting is not a lack of enthusiasm, but rather a lack of WinCE USB drivers which has limited USB Hosting usefulness. So the right message should have been “Make WinCE USB Drivers”, instead of the message: “Most of us don’t care too much about WinCE USB Hosting”. Having said that, I earnestly believe that forums are all about the free exchange of unfettered ideas and opinions. However I also believe due consideration can lessen these 5 pitfalls. Let’s do more research and provide more information when we post a poll or solicit opinions. Let’s insist that the solutions that we provide don’t excuse the problems. Let’s be open-minded and objective. Let’s have some solidarity and adopt a mindset of shaping the industry in something that serves the consumer’s best interests. Then we will have done all we can to help decision-makers easily and accurately check our online pulse (assuming that they even try). Hey, a little paranoia never hurt anybody.

johncruise

11-11-2004, 06:48 PM

Having said that, I earnestly believe that forums are all about the free exchange of unfettered ideas and opinions. However I also believe due consideration can lessen these 5 pitfalls. Let’s do more research and provide more information when we post a poll or solicit opinions. Let’s insist that the solutions that we provide don’t excuse the problems. Let’s be open-minded and objective. Let’s have some solidarity and adopt a mindset of shaping the industry in something that serves the consumer’s best interests.

Well said Dave :way to go: "open minded" that is what some people in the forums should consider when they are voicing out. Those people tend to "shove" the idea that "you don't need this" or, "that is not good", and "this is the best". It's ok for people to share their opinions but not to the point where they flame you because we have different point of view on what we like.

I'll be watching out for more post from Dave... Thanks Jason for posting this too!

surur

11-11-2004, 07:33 PM

If we want our opinions and polls to be useful to market researchers, we should be sure to use the mind set of a consumer.

This means demanding and prioritising features and fixes, and describing the way we want things to work.

We should not ever take on the mind set of the engineers and marketers, and say "I dont want that feature, because it will impact battery-life too much/its too difficult to implement/ will be too expensive" etc. Its our job to ask for features, and the engineers job to find a way that will implement them in a useful and usable way.

Ive seen this especially with regard to the hard drive question. When we ask "do you want 20GB on your device" people always say yes, but as soon as we say "do you want a hard drive" all we hear is about reliability issues, battery life issues and size issues.

These should not be our problem. Its for the engineers to sort out (and Im sure they can). As the article says, we should stop speaking from our preconceptions and assumptions, and rather judge the final product on its ability to deliver, produce advantages and mitigate the disadvantages.

So I agree with this article. Remember the customer is always right, and its the job of the companies to satisfy even our most exotic request (if they want our money). This also means not making excuses for them for example when they make a marketing decision (like the wm2003SE update) and blame it on so called engineering challenges.

More consumer activism!

Surur

Kati Compton

11-11-2004, 07:40 PM

I think the other problem is that it's difficult for marketers to sift through some forums that have a higher percentage of insults, etc, to find the useful information.

Especially when rants appear personal and involve a lot of swearing, I would think the marketer would be inclined to discount the ranter as a crazy anomaly.

Phoenix

11-11-2004, 08:04 PM

Up to this point, I agree with everything said in this thread.

DarkHelmet

11-11-2004, 09:28 PM

I don't.

I have been contacted more about my corporate purchasing activity (i.e. what am I going to buy?) and virtually never contacted about what features I want in a new gadget.

I think all you have to do is look at CTIA as an example of what the current trend in marketing is - me, too!

Truth be told, we need less market research and more design research.

USB hosting is a no-brainer for me - look at all of the USB devices out there - nearly all of them are unusable with PDAs.

Phillip Dyson

11-11-2004, 09:52 PM

Truth be told, we need less market research and more design research.

I agree with this comment.
Marketing research is as follows:
--What are people buying.
--What would is minimum effort to re-stimulate buying. (e.g. Phone sales are flat. Hmmm.... lets add cameras)

Design Research is more like:
--What do consumers want?
--What are consumres asking for?

johncruise

11-11-2004, 10:15 PM

DarkHelmet

11-11-2004, 10:28 PM

Phoenix

11-11-2004, 10:31 PM

I have been contacted more about my corporate purchasing activity (i.e. what am I going to buy?) and virtually never contacted about what features I want in a new gadget.

Truth be told, we need less market research and more design research.

And that's really where I'm coming from. Maybe I interpreted what was said here differently than you. But I agree with you 100%. Providing the proper feedback so that companies know how to design their products according to what we want, is what I'm all for.

Companies that focus more on what they think might generate more money (and often to their detriment) instead of focusing on what people really want and are asking for, never made any sense to me. If they'd pay attention to what people really want, then they wouldn't have to be as concerned with the bottom line.

johncruise

11-11-2004, 10:51 PM

look at all of the USB devices out there - nearly all of them are unusable with PDAs.

Not for my case. I needed a USB keyboard, mouse and harddisk to use in my PDA and I got them all connected and running. Sorry if you cannot take advantage of this feature. Peace!

My "point" was that most PDAs do not have USB hosting capability - apparently your Casio E-200, with PocketPC 2002 OS has the capability...

So - you helped me make my point - market research told all of the other vendors that USB hosting wasn't important - Casio knew better...

Ahh... point taken. Thanks for the correction.

Where are they now? <laments>
Unfortunately, all of us know that they won't be coming back any time soon. People like me are now looking for alternatives but unfortunately, units that offers that feature we wanted hasn't yet landed in the US territory (Mio 558, Asus A730, Toshiba 830, Fujitsu Loox 720). :( Oh well... first one to get here gets me as their next loyal customer.
:lol:

davea0511

11-12-2004, 06:40 AM

Truth be told, we need less market research and more design research.
And that's really where I'm coming from. Maybe I interpreted what was said here differently than you. But I agree with you 100%. Providing the proper feedback so that companies know how to design their products according to what we want, is what I'm all for.

Companies that focus more on what they think might generate more money (and often to their detriment) instead of focusing on what people really want and are asking for, never made any sense to me. If they'd pay attention to what people really want, then they wouldn't have to be as concerned with the bottom line.

Right, Phoenix - you got my gist. And you're totally right about how technology companies do research - they don't focus on what people really want, but that's largely because they don't know what people really want because, like you said, they don't "pay attention", but also because we don't send the right messages.

Regardless of the research ("Market Research", "Design Research", etc) conclusions are only as accurate as the data, and my point is that we don't always give them clear and consise data. So they take a "yes" for a "yes" or a "no" for a "no" when all that we really meant (but didn't say) was "maybe, if you fix this and that".

In short I think most of these misleading messages could be avoided if we all adopted a pro-consumer paradigm. Each member of the community should ask himself "Do I want to just satisfy my own immediate technological desires, or do I want to make the PPC world a better place for consumer?", and if we all adopt the pro-consumer paradigm then I believe decision makers would start spitting out the right stuff and responding much more readily to our needs.

USB Hosting is the perfect example of this. For most consumers it doesn't satisfy any immediate needs so for many it's easy to trash the idea, but with a little vision and pro-consumer attitude the PPC community could convince decision-makers to make it very powerful and useful for all of us. There are thousands of issues like that that get discussed here every day, and behind each issue is a manufacturer who wants to know what to do in the future.

I suggested it's paranoia, but it isn't really. Rather it's like a game a chess when you're thinking how the guy on the other side is going to respond to your response, and having that affect what you say and do. Some people don't live that way, and that's fine, but being cerebral can pay real dividends in the long run, especially when you have a whole community doing it.

Phoenix

11-12-2004, 11:03 AM

That's good insight, and I agree.

I think often times we'll have a synthesis of the two - beneficial suggestions for the masses that also gratify our own desires.

Along with many others, I too, have voiced my requests regarding technology, with the idea in mind that it would benefit all involved - topics including battery implementation, hardware interface and aesthetics, etc.

It pays to be mindful. But I do have to add, that as important as it is for all of us to share our ideas and desires, and to do so clearly and with vision, it would be far more effective and beneficial for the evolution of products and services and of course, consumers and companies alike, if we could see a true presence - a true voice - and consistent feedback in these forums from all of the companies that we support and are attempting to communicate with (and hope are listening). That would certainly foster much greater interest and contributive effort within our online communities from all of us, and it would help produce the results we all seek in regard to the development of the technologies that we have such a passion for and which draw us here in the first place. Not to mention, more revenue for the companies. And why couldn't things be modeled this way? Under those conditions, I don't think anyone here would hesitate to chime in about what they'd like to see or what they feel would be beneficial, so why not regular inquiries and feedback from the companies too? After all, we're all regular people who share a common interest to make great technology that benefits our lives.

So I'd like to see that kind of change. The kind of change that brings keyplayers from the companies we want to talk to, out of the confines of their boardrooms, where they're difficult or impossible to get in touch with, and into the communities here where we can openly exchange ideas. Personally, I see this type of exchange as necessary to achieve the results everyone is truly seeking. Because like anything else that requires communication, it's a two-way street. But it has to start somewhere. Why not here?

JvanEkris

11-12-2004, 01:57 PM

A very valuable comment indeed,

I do have some remarks,
Forums Provide Solutions, But That Doesn’t Excuse The Problem
Are unreliable alarms on the PDA not a big deal because so many people use their PDA for other things, or because you can buy a 3rd party workaround? I’ve heard such comments over the past 4 years whenever I’ve seen the “unreliable alarm” issue come up, and although they can be worthwhile comments do decision-makers ever use those comments to excuse the problem? Could that be why the alarm problem persisted off-and-on for 4 years?That is only partially true. Problem is, as with many market reseach data, people stop complaining at a certain point. It still is a problem, but i do not see a very long thread about this dreaded problem, although many people experience it and many are indeed anoyed by it. So the information on the board more or less reflects the current situation: people experience the problem, but have stopped complaining (and good marketing research will probably say the same).

The trick is, as a good developper, to listen to these complaints and adress them in some way. It is a legitemate complaint that should be treated as such. Problem is that few companies do listen to every complaint posted at a board. The few that do have a hell of a job doing so.....

We Have Hopes And Fears
Many of us voice our opinions out of fear or hope but not always from direct experience because we just don’t have enough cash nor time to try everything – but we do all have valuable opinions, even if they are heavily laden with subjectivity. Also, some of those who do have relevant experiences may sometimes be a little biased, understandably so because the technology in question was poorly executed the first time (especially true with “first adopters”, a huge part of our community). Do decision-makers take that into consideration?I would go even further than that. Many research has proven that when consumers talk about the future of a product they only try to extrapolate the current trend. That becomes their horizon and frame of reference for future developments. On these forums and even when conducting solid market research, this horizon is very prevalent in the answers. So not only our hope and fears that are clearly present, but also an extrapolation of the current products. This implies that consumers can not look beyond that point, and that when gathering information about future needs people will stick to what they see now. New needs are very seldomly identified by market research.

Biggest example is SMS. Reseasch never indicated it would be a good idea. They put it in there anyway because of the possibility of voicemail and some persistent designers. It is now the biggest generator of revenue for mobile operators. People simply could not inmagine the concept of text messages at that given point of time.

As said, a more design oriented approach could be better. It would break with this development within the current framework of the consumer, and could introduce really inovative products. However, also with the added risk of market catastrophes because a product is too far away from the customer base that it simply does not understand it's use. You are solving a problem the majority does not see. So solving our problems one step at a time is the safest way to go......

Jaap

davea0511

11-12-2004, 03:30 PM

Yes, although we can improve our communication, without a doubt the developers and manufacturers are to blame when they miss the mark with their products and solutions, regardless of what we say or do.

The sad reality is that most of them probably don't spend much time in forums like this to look inside our heads, simply relying instead on customer returns, sales figures, and the opinions of an executive or two who think they know what we want better than we ourselves do.

So "How can we get them to listen to us in the first place?" is the golden question. I think part of the answer is making it simple enough for them that they can easily observe our specific needs and desires by taking just a couple of minutes. Reviewers do this for them by listing a detailed pros and cons list, but even then the manufacturer doesn't know how significant each item is and it's only the opinion of a single reviewer.

I've bounced an idea around in my head for some time about a more interactive and dynamic poll, where each voter can vote on specific pros-and-cons that they personally find important, but even more importantly they can add their own pros-and-cons that other voters can vote on. The poll results evolve over time as a weighted profile, and each voter can go in and modify their votes if they want as the profile takes shape or as pros-and-cons appear or resolve. Ultimately each poll becomes something that the manufacturer can easily look at and say "Here's why it is or isn't popular, and here's what we need to fix or continue doing."

I think that would be very powerful, especially if they could review the evolution of each poll on a timeline - showing how voting trends change as they address each issue. Now I'm just dreaming.

JvanEkris

11-12-2004, 03:43 PM

Well,

I have been playing with the idea of making reviews in a WiKi. This way people can amend the review. This would make it a living thing and when new pro's con's are discovered people can simply add them. One of the most important things to add are things like abnormal wear and tear found during the use in the field for a couple of months (like the grips from the 2210), but also things the reviewer has missed. Perhaps that is a way to comunicate to designers.....

jaap

davea0511

11-12-2004, 04:14 PM

SassKwatch

11-13-2004, 02:12 AM

I couldn't agree more with the large majority of opinions expressed here.

Though I feel I get pretty good use out of my device, there's at least one area of MAJOR disappoint for me...#1 of which is your USB hosting mention. I'm a fairly avid amateur photographer and tend to shoot 99% of my digital pics in RAW format. Even with a 1gb microdrive, it doesn't take long before I'm running out of space and need to transfer the current crop of pics off the MD on to *something* else. I would LOVE to be able to connect a 20-30gb hard drive to my pda, throw the combo in my camera bag and take out in the field with me. But because I didn't see such capability likely arriving on the horizon any time soon, I wound up having to purchase a 7.5lb laptop pc.....which is too much to carry around to many locations. So, I'm stuck hoping a day's shoot won't fill up the MD until I can get back to the hotel/tent and do the transfer. And there have been several occasions when I've had to deliberately stop shooting for that very reason.

As far as I'm concerned, the PPC OS has been woefully underdeveloped since my initial purchase of an iPAQ 3870 in late '01. The progress made in that time only equates to a Win95 to Win98 state of progress, IMO.

And I think at least partly as a result of that lack of progress, the market for devices like OQO/Fliptstart/etc has been opened up. I may not be an 'early adopter' of such devices, but there's a high likelihood I'll be purchasing one of them in the future before I'll buy another pda. And I find that a pity, because there shouldn't be any *need* to do so.

upplepop

11-13-2004, 02:21 AM

I would LOVE to be able to connect a 20-30gb hard drive to my pda, throw the combo in my camera bag and take out in the field with me. But because I didn't see such capability likely arriving on the horizon any time soon, I wound up having to purchase a 7.5lb laptop pc.....which is too much to carry around to many locations.
I don't want to go too far off-topic, but perhaps you might be interested in either a Gmini 220 (http://www.archos.com/products/prw_500452.html) or Gmini 400 (http://www.archos.com/products/prw_500637.html) from Archos. Both of these are mp3 players with 20 GB hard drives AND CF card slots. Both are very compact; about the same volume as an iPod.

SassKwatch

11-13-2004, 04:11 PM

I don't want to go too far off-topic, but perhaps you might be interested in either a Gmini 220 (http://www.archos.com/products/prw_500452.html) or Gmini 400 (http://www.archos.com/products/prw_500637.html) from Archos.
Yep, those would certainly be alternatives. And there at least a few such items designed with the photographer in mind. But my point was that the PPC platform is SO close to having that capability already that a purchase of one of such items *should* be unnecessary......if only for a USB (or firewire, or whatever) option. "Close, but no cigar."