Winners of the Best Software Awards 2007 Announced

[Darius Wey]

<div class='os_post_top_link'><a href='http://www.pocketpcmag.com/awards/main.asp' target='_blank'>http://www.pocketpcmag.com/awards/main.asp</a><br /><br /></div><i>"The Winners in Smartphone &amp; Pocket PC magazine's Seventh Annual Best Software Awards for Windows Mobile software have been announced at <a href="www.Smartphonemag.com/Awards">www.Smartphonemag.com/Awards</a>. One hundred members of the Smartphone &amp; Pocket PC magazine Board of Experts judged 889 Windows Mobile products in 188 Pocket PC and Smartphone categories. Winners are also listed in the Smartphone &amp; Pocket PC magazine 2008 Buyer's Guide first seen late November."</i><br /><br /><img src="http://www.pocketpcthoughts.com/images/web/2003/wey-20071119-bwa2007.gif" /><br /><br />The winners of <a href="http://www.pocketpcmag.com/awards/main.asp">Smartphone &amp; Pocket PC Magazine's Best Software Awards 2007</a> have just been announced. A 25-page report containing the list of winners and finalists, GPS software reviews, and comments from members of the Board of Experts panel is <a href="http://www.pocketpcmag.com/awards/main.asp">available now</a> and will also appear in the upcoming Buyer's Guide and February/March 2008 issue of the magazine.<br /><br />Congratulations to all the winners and finalists. Your contributions make the Windows Mobile platform stronger every year.

[Menneisyys]

Also, I would really like to know if anyone disagrees with the results - most importantly, for the why's.

[buzzard]

The only thing missing from the list of winners and losers is "why". I certainly can understand why some programs are better than others but as an example when it comes to Pocket Informant and Agenda Fusion, what made the team decide that PI was better?

[Menneisyys]

The only thing missing from the list of winners and losers is "why". I certainly can understand why some programs are better than others but as an example when it comes to Pocket Informant and Agenda Fusion, what made the team decide that PI was better?

I haven't judged in that category because I still haven't written a PIM Bible; that is, I don't know all the secrets of all these PIM apps so I refused voting in the category. That is, I can't answer the "why" myself with this category (as opposed to the cats I've judged in - and several others where I haven't, but I've published a Bible on them too).

Judges that did judge in the PIM category, however, will surely publish their thoughts. Hope even in this very thread.

[buzzard]

Just a note: I was only asking about the two PIM's as an example.

When I read which apps are winners and others are finalists I don't see a reason given as to why the winner is the winner. Maybe it's just personal preference or maybe the reviewer actually has a positive or negative reason that could be added so that the readers could understand why a program won or lost

[Menneisyys]

Just a note: I was only asking about the two PIM's as an example.

When I read which apps are winners and others are finalists I don't see a reason given as to why the winner is the winner. Maybe it's just personal preference or maybe the reviewer actually has a positive or negative reason that could be added so that the readers could understand why a program won or lost

Well, if you see a category I've published a Bible on (or at least separate reviews of all the nominated apps), you can find the answer there. While in not all the categories I've judged in / I know like the palm of my hand did the title that I consider the best win (for example, in the PPC racing game category, K-Rally wasn't even made a finalist, while I consider it without doubt the best 2D racing game for the PPC) , the vast majority of the results closely correspond to my opinions / judgement accessible to everyone thorugh my previously published reviews / roundups / Bibles.

Dunno if the other judges publish their opinions / chime into our discussion.

[burtcom]

I've found this annual list to be quite helpful, generally :)

One question though -- why was Microsoft's Pocket Streets even nominated? My understanding is that it is no longer available -- it certainly does not seem to be included with any of the recent Microsoft Streets products.

[Jason Dunn]

The only thing missing from the list of winners and losers is "why".

As a past judge of several years, I can say that the "why" is quite often hard to nail down - the finalists are all going to be great products to even get that far...all the dysfunctional software is weeded out early. So when you're looking at a few finalists, it quite often comes down to which software product delivers the most bang for the buck, and how well the developer delivers a working solution. That can be hard to explain...it's rarely every one feature that sets the product above the rest.

That's my 2 cents (1.9 cents USD) on being a judge at any rate. ;-)

[jeffx]

no category for Navigation software? What's up with that??

[tabenate]

I also find it puzzling why there is no category for GPS navigation software.

About the winners, the one that really surprised me was Contactbreeze over Inesoft Address Book. I use Address Book and have looked at Contactbreeze, there just is no comparison. I just don't see how an objective judge can make the decision in favour of Contactbreeze. I know price is not always a good guide, but this time I think the price difference of $25 vs $10 is a more reliable indicator of the difference in features between the two applications.

This is not a rant against Contactbreeze, which does what it's supposed to do, just that Address Book offers SO much more.

Maybe the organisers should tell us what the criteria are, and their weightings: eg,
features -10%
bugs -30%
interface -20%
price -20%
support -10%
ease of use -10%

Do the judges actually use a standard scorecard method like the one above or they just rank in order of 'preference' 1-2-3?

[Hal Goldstein]

I'll do my best to respond to questions, but I am actually in Uruguay (where my wife's family resides). Connection is usually OK, but sometimes can be flaky.)

First GPS - we should have made it clearer on the Awards page. About three years ago, we eliminated the category "GPS Voice Guidance" for two reasons. First of all, it was a nightmare to administer - tons of coordination between judges and vendors, lots of miscommunications, etc, etc. The other reason was that a simple vote could not do justice to these complex programs. So one of our judges, Myron Swartz (originally Shawna Kelley) took the lead. He and about a half a dozen other judges evaluated packages - 2 or 3 judges per package and Myron evaluated all of them. We publish the reviews in our Buyers Guide where we also announce other winners. Myron makes his recommendation for best, and we have an depth feature comparison chart. The GPS section is also part of the $5.95 special report that is on www.pocketpcmag.com/awards (and is given free to subscribers).

Secondly, with 188 categories it is difficult to either come up with one set of criteria that fits all, or 188 different sets of criteria. So, we did several things. First, one of our judges, Nate Adock, developed a spreadsheet which listed possible criteria (installation, ease of use, feature richness, system requirements, visual interface, WOW factor, functionality, reliability, support, documentation). The spreadsheet gives each judge the ability to weigh the criteria (these or others) depending on the category. Also, at our Judges' Home Page, there are a list of criteria various judges have used for various categories. As Jason implied, the judges are ultimately on their own. However, we give them lots of tools, and there is a private judges' forum where discussions can take place. Also, judges within each category can talk among themselves if concerns arise.

For the second year, we had judges make comments about each piece of software (pros and cons) that they evaluated. Currently, each vendor who had a nominated product can look at judges' comments about their software. The goal is to help vendors improve their software. We also publish some of the judges' comments in the magazine throughout the year, which means it eventually shows up on our web site in our archives. This year, we will very likely make all the judges' comments about the different software available to our subscribers at our Subscriber-only VIP site, www.PocketPCmag.com/vip. Note that judges were not required to comment, but the majority did, and there is a lot of interesting information.

I will hopefully send an E-mail about this thread to the judges tomorrow, so you can get others' perspective.

Finally, probably most reading this are aware of Werner's (Menneisyys') incredible contribution to the Windows Mobile community. I can't begin to acknowledge how helpful and fundamental his contribution was to this year's awards. He put in countless hours creating new categories, making sure products were properly categorized, creating the original nominations list, and serving as final arbiter as to whether a product was ultimately nominated. (Judges and the public made suggestions for nominees). Frankly, without him, given our loss of Diane Dumas, one of our key employees, who left us due to health reasons, we may not have had the awards at all.

Hal Goldstein
Publisher
Smartphone &amp; Pocket PC magazine
FOunder and Overall Administrator of Best Software Awards

[natestera]

Last year, I posted my finished judging scoring/comment sheets on-line (with Hal's permission of course), but haven't had the time this year. As an example you can go to our blog site (ppcmag.com/blogs) and simply enter "2006 Awards:" in the search box down the right-hand side of the site. By the way, I just wanted to make sure that Roberto Valerio (one of our judges) gets credit for adding the weighting of categories in our scoring sheet, which I had never even thought of. The sheet also includes a list of pointers and guidelines from past judging efforts.

[phmurphy]

Hi all,
I am not one of the judges, but I went through a long process of trying out the top database programs that are typically listed and SprintDB was so much better and easier that I don't understand why it never makes it even onto the list of competitors.

It makes me think that there is something other than a through test process going on.

It would be extremely helpfull if a more detailed assessment were made available, and included price and functionality and ease of use.

Thanks.

[natestera]

phmurphy...Sounds like a nomination issue, which is something that our judges, and the general public all have a say in prior to the awards. If the product was not nominated, or if the vendor of the product was not interested in registering the product, then that might explain why it is not included in the awards. Functionality, ease-of-use, etc. are all included in the judging determinations (our suggested criteria).

[Menneisyys]

Hi all,
I am not one of the judges, but I went through a long process of trying out the top database programs that are typically listed and SprintDB was so much better and easier that I don't understand why it never makes it even onto the list of competitors.

It makes me think that there is something other than a through test process going on.

It would be extremely helpfull if a more detailed assessment were made available, and included price and functionality and ease of use.

Thanks.

I've long been planning (and promising) a database engine comparison (another Bible). Let's hope I'll be able to devote some time to this "project"; now, I'm very busy with my forthcoming Multimedia Bible, and I'd also like to publish a Push Mail Bible and a Calculator Bible after that.

That is, I'm pretty sure I'll publish an extensive evaluation &amp; comparison of all these technologies - some time in the future.

[Prof Julie]

I was a Judge this year. There were many categories where the top two applications were so close, it was hard to choose. After getting it down to the top two, I chose the application I liked the most - not very objective, I know. In these cases I think the top two contenders are both winners. It really boils down to personal preference. What I prefer may not be what someone else prefers. If I was interested in installing a particular category of software to my Pocket PC, I'd probably try out the winner and the finalists before making a final decision. (I reviewed the PIM applications and my top choice did not wind up being the winning application).

I think Menneisyys did a great job pulling together a comprehensive list of applications to review and I nominated a few applications that were included. It was a huge list, but there's always a chance that a really great application could be omitted. If an application was missing in this year's competition, be sure to nominate it in next years' competition.

[Menneisyys]

If an application was missing in this year's competition, be sure to nominate it in next years' competition.

Yes, user and developer nomination requests are VERY important because all these requests are seriously considered for nomination. With over a thousand(!) nominees, it's VERY hard not to miss some 1-2 otherwise great products.

So far, I "only" know of SpaceTime, in the Calculators category, as a serious miss. We would surely have nominated it if it had been requested for nomination. Unfortunately, it wasn't by anyone. All in all, it's very important that both end users and developers check out the list of nominations before it is finalized so that we can quickly fix mistakes like missing titles.

[buzzard]

I'm almost afraid to ask but how do the programs get nominated?

I do believe that the judges do a fantastic job but that's exactly why I'm interested in their process. The comments from NatesterA at his blog were interesting but understandably brief what with the enormous number of programs to review. I'll be looking forward to reading the 2007 reviews in a future issue of the magazine. That will give us all winter to moan and groan about our favorites and why they didn't win. :lol:

[Prof Julie]

buzzard, nominations are accepted from the public around the end of July. See this link for more information:

http://www.pocketpcmag.com/awards/main.asp#How_do_products_become_nominated_in_2007

[Hal Goldstein]

Products get nominated as follows:

To keep track of things, we require that all products be in our on-line Encyclopedia of Software and Accessories (http://www.pocketpcmag.com/_enc/encyclopedia.asp) . This is a free listing for any Windows Mobile vendor. If a product is suggested for nomination and not in the Encyclopedia, then we contact vendor to enter his product. On occasion we will enter product if vendor is non-responsive, but we prefer not to do it

We start with Winners and Finalists from previous year. Note that the awards are for the best products within a category, not the best new products. That is why we start with best products from previous year.

Then Nominations Manager (this year, and hopefully for future years, Werner Ruotsalainen -- Menneisyys) created initial nominations list in May. In the process, Werner reworked our category structure, adding new categories, and moving products to proper location.

Throughout June, our experts made suggestions to add or remove nominations. I believe Werner accepted 80-90% of the suggestions.

In July we published the preliminary list of nominees for feedback from publci and vendors. There is a nominations form at http://www.pocketpcmag.com/survey/nominationsfeedback.htm available all year at our www.pocketpcmag.com/awards site. We asked the public to use the form. Werner accepted many of the suggestions. We froze the nominations list late July.

In August and early September judges evaluated and voted. Nominations, Finalists, and Winners are found in several issues of our publication and at our web site.

[daS]

As a judge in a few categories I thought I'd add my two cents here:

First, as a judge since the first annual awards, I must say that the judging just keeps getting better each year. While we now have some "guidelines" I'm actually pleased that each judge can use their own criteria in determining their picks in each category.

For myself, I weigh heavily the "usability" of an application for the general public. Thus, programs that might be very comprehensive in their feature set, but are difficult to install or understand score lower than you might think is justified.

As an example, I voted to place the Thunderhawk browser in second place above Opera Mini because the mini version of Opera required the installation of a Java VM that I didn't feel was easy enough for the average user. Other judges felt that the features of Opera Mini outwayed the complexity of the installation.

My second criteria has to do with the general application of the program. So, for example a sophisticated program that is for a small target user base would not do as well as a more simple program that has universal appeal. This is true even when I personally might be part of the target user base.

In the end, since there are more than a couple of judges per category, personal preferences typically don't factor into the final results too much, but it's still good to read some of the judges comments that Hal publishes in the magazine.