"Comparison of RIM Blackberry 4.0 and Microsoft Windows Mobile 5.0 with MSFP"

[Raphael Salgado]

<div class='os_post_top_link'><a href='http://www.smartphonethoughts.com/index.php?action=expand,11347' target='_blank'>http://www.smartphonethoughts.com/i...on=expand,11347</a><br /><br /></div><i>"Wipro Technologies, a leader in the IT services and consulting industry, conducted a comparative lab based study of mobile platforms by benchmarking the Microsoft Windows Mobile 5.0 Messaging and Security Feature Pack (MSFP) coupled with Exchange Server 2003 SP2 solution against the RIM Blackberry Handheld 4.0 and Blackberry Enterprise Server 4.0. Wipro conducted detailed testing of both environments in a simulated production environment and extrapolated the efforts and costs obtained from the lab environment to project the total cost of ownership (TCO) of managing each mobile platform in an enterprise production setting."</i><br /><br /><img src="http://www.pocketpcthoughts.com/images/web/2003/salgado-20060607-rimvsmsft.jpg" /><br /><br />For IT departments and enterprises that are evaluating solutions for a more mobile workforce, this comparative study may be a helpful read. After the RIM settlement and the Microsoft marketing blitz, it's good to see some hard numbers that could help decide which way your organization can go when it comes to deploying the right mobile device for your needs.

[choyboy]

It's a shame the comparison is sponsored by microsoft. Hardly unbiased.

On the upside, at least this is clearly stated at the start of the report.

I don't put my weight in any sponsored research. Anybody who has a working knowledge of statistics knows that numbers are quite easy to fudge one way or the other, or conveniently leave out variables.

I'll wait for a more even-handed report thanks. Maybe gartner or one of the *real* analyst firms will look into this.

[eagle63]

no kidding. sorta like all the windows vs. Linux comparisons that they 'sponsored'. Gee, I wonder why MS came out ahead....

[Jon Westfall]

Before we all jump and say "BIASED", lets think about it:

If Microsoft doesn't sponsor this, who will?

RIM certainly won't, as they have enjoyed the de facto lead in push email for some time now. What marketing exec at RIM would EVER think "Gee, we're already number 1, guess we should commission a study to see if this windows mobile thing is as good as we are". just acknowledging that Microsoft has a push solution is a dangerous thing for RIM.

I don't know of a single third-party group that would sponsor it either. It's not a matter of public health, it isn't something that businesses would buy a report on (that's what their own R&amp;D teams should be writing up), and it isn't something that the government would ever fund.

So if Microsoft doens't commission a third party team to compare these, no one ever will. It's the same argument people make with Tobacco companies funding smoking research - sure there may conflict of interest, but no one else would fund a study to see if brand x cigarette was better for you than brand y - they'd simply state both brands are bad for your health!

[Jon Westfall]

I don't put my weight in any sponsored research. Anybody who has a working knowledge of statistics knows that numbers are quite easy to fudge one way or the other, or conveniently leave out variables.

All research is sponsored. The independently wealthy scientist is a thing of the past. I take your post to mean that you don't put weight on corporate sponsored research, so conversely, you do value government funded research, which can still be biased by the government funding it. A few years ago USDA sponsored a write-up on bioterrorism related to agriculture in the US. That report, which merely collected information already readily available, was censored by the government. The agency publishing the report was asked to make it available only in print (not online, as they had with all the other reports they had done), and to remove chapter 3 unless someone had a specific "need" for chapter 3. Mind you none of this was new information, and the government admitted that the information did not meet the criteria for classification. So government funded research leads to censoring, private funded research leads to bias.

You do make a valid point - numbers can easily be manipulated. Get a large enough sample, and Anything will be statistically significant. Get a small enough sample and your correlations can be wonderful, yet you'd have no power. Play around with your alpha level in the name of "exploratory analysis" and there is no stopping you. Perform invalid tests on data or exclude data that you deem an "outlier" and you can certainly manipulate your results.

The only way we can trust reports is to have many of them, independently testing the same thing, to replicate or fail to replicate previous results. We should not, however, immediately dismiss something as "biased" simply because it is sponsored, or accept something as "real" simply because it is funded by a government we trust.

Wipro (http://www.wipro.com/) the company who wrote this report seems to be a "real" analysis firm. I find it dangerous to put trust into any group, such as gartner, just because it then can lull you into accepting their results later, when you should be skeptical!

[eagle63]

I disagree that all research is sponsored. There's plenty of independent research out there, in a variety of fields. Oftentimes when research IS sponsored, it's being paid for by customers/companies/people who want to use that research to help them make decisions. That's entirely different than research that is being paid for by a company that stands to make a financial gain based on the outcome.

Did you believe the tobacco companies back in the 60's when doctors paid by them decided that smoking wasn't harmful to your health? (despite what the rest of the medical community was saying)

Did you believe wall street analysts who put huge buy recommendations on lousy stocks because it meant that their firm would gain IPO underwriting business as a result? (fortunately wall street finally took this seriously enough to put an end to the conflicts of interest)

I realize most everyone here is rah-rah-Microsoft - and that's fine - but I'm stunned that you could take this type of thing very seriously. That isn't to say that there isn't some good factual information in it, but the glaring conflict of interest invalidates the whole thing.

[choyboy]

eagle63 and I are making the same point. Yes, alot of research is sponsored. And this need to be taken into account. Sponsored research in itself is not inherently bad, but I don't think the researchers would have much success if they didn't come up with favourable findings for Microsoft. If it didn't turn out the way Microsoft liked it, obviously they wouldn't want it published. But if it did show the Microsoft push was *worse* than RIM, and they tried to hide it, imagine what a field day RIM and the press would have with that!

Most of my experience is in medical research and I can site a recent example - a drug called Pravachol vs. Lipitor (the latter being the gold standard for hypercholesterolaemia). the makers of pravachol decided to do a head-to-head study comparing the two, certain it would come out in favour of their drug. It didn't and the other company wiped the floor with them. Unsurprisingly, very few drugs these days are pitted in head to head trials. They are generally double blinded placebo controlled.

There are other examples of research not done by companies with vested interests. On a consumer level, Choice here in Australia does research comparing products.

I have actually *read* this article by the way and there are quite a few pounts that are curiously left out of the article. Some of the licensing pricing is inaccurate, other areas are just plain wrong. There is no mention of the free server licensing which RIM offers in some instances.

I take this approach whatever the company - if it is company sponsored, you know the figures aren't going to look bad, and you know you have to take it with a grain of salt. You're right, you shouldn't necessarily ignore it, but I certainly wouldn't base *any* decisions on such an article and would look for third party review and research before making a decision.