First, I think it would be very useful to have a way to find reviews of products more easily, be they written by the Review Team or regular members. Anything that makes finding something easier is a good thing, as long as people can still comment on the reviews. A reputation system is a nice, but not necessary, feature.
Second, I don't buy the concept that "professional" reviews and "lay" reviews are mutually exclusive. Right now, anybody can post a review in the (hopefully) appropriate forum, and other users can comment on it. The only difference with reviews written by the Review Team is the home page exposure and the template they have for reviews.
Third, let's not call reviews here "professional" unless they truly are. I know that some of the reviewers (like Chris Spera) are professional reviewers, but I bet many of the Review Team are not. "Professional" should mean either the writer gets paid for the review (which the Review Team does not, although they may get to keep some of the products) or that the writer writes reviews a significant part of his life.
The reviewers here are volunteers, and I bet most of them have jobs that don't involve writing reviews. The reviewers did have to go through a screening process involving writing one review and do have a template to follow (I know, because I tried out for the review team), but that does not make them professionals (again, I know, because I wouldn't consider myself a professional reviewer even if I had made the team).
So, to summarize, let's have a place where reviews, both from the Review Team and regular users, can be located. Make sure people can still comment on the reviews. If you can add a reputation system for reviewers (or for posters in general), that's fine, but not necessary.
By the way, would the review database only host complete reviews, or would it allow just linking to other reviews? If I put a review on my site, it would be redundant to put it here, too (and a waste of Jason's disk space), but people might still want to know about it and comment on it.
I agree... but with ONE CONDITION: the reviewer has to include the serial number of the device he is reviewing to ensure that he's really reviewing a device, and not just posting his biased opinion.
I mostly agree with Chris's comments on this subject. I think the current format works well because you get a single review that is structured, had time spent on it and offers a concise conclusion. The readers can then ask any questions that they feel are pertinent and add more body to the overall impression of the review that perhaps was missed or felt at the time of reviewing that it was not worth mentioning.
My impression of user reviews is that while many are well written, a good percentage can be a first impression or do not focus on the correct elements that will give a reader the information they require to help them forumulate a buying decision. I have also read many user reviews for a product and the overall 'score' of the product ends up in the 30 - 70% range with many differing views making it hard to come to a definite conclusion.
The 'professional' reviews you get here should be very focused with a clear final impression of the merits of the product. While many people may not agree with the review, at least they can follow and see the opinion of the reviewer. We are not paid for these reviews so professional is a loose term but do spend a lot of time and effort on these reviews and try to cover the main aspects that will assist a buying decision. We are human though and may miss things and we also are one opinion so I think the follow up feedback is important to get opinion from a different angle or even just for people to say they agree with the review. One other thing to be wary of regarding user reviews is a quick look at a product that may undermine a 'professional' review. I don't mean to say that the professional review is better, however the product is picked by a member of the review team because they have the relevant experience or knowledge of the subject matter to review that product and they set aside the time to conduct a thorough evaluation. It would be a shame if impressions of a product were tarnished by a few less thorough reviews as soon as a product is released. You only need to look in the forums to see user reviews of a product the day they have received it. While some products are easy to cover in a day, some are not but people still want to be the first to post regarding it.
Isn't that exactly what is happening with the Thought's review team? With each review that one of them posts it builds their reputation...with you and me.... <snip> You build their reputation in your mind. Reading reviews from people you are not familiar with, or are rated by people you are unfamiliar with doesn't build that confidence.....IMHO
While this is true, I was thinking also about someone who comes to the site for the very first time.. yes they can assume (as I've done) that the reviewers have been selected because they can actually do a good review, and even though I can think of a few ways a ratings system could be hijacked slightly to "ratings bomb" (like a google bomb) someone, it's nice to see a tangible indicator of how trustworthy someone's review is.
As for the difficulty behind integrating the review software into the user database for this site, according to censura's site they integrate easily with phpBB (which I'm assuming also means the user database).
As for the whole issue with censura not providing the functionality for user "reputations", that's valid. But as a web developer who loves PHP, that's more of a positive thing than a negative thing to me. :mrgreen:
__________________
"And then she understood the devilish cunning of the enemies' plan. By mixing a little truth with it they had made their lie far stronger."-C.S. Lewis
Professional or not every review is going to be biased, unless you just want to list the spec sheet of the product.
How often have we read "Professional Reviews" with statements like "The Pocket PC is just too complex compared to Palm" ... "Palm has many more applications available for it's platform." Not too mention a few overly biased Pocket PC reviews on this site.
That aside, it's just a Jason decision on what he sees this site becoming. At first I loved this site, cuz it was just a blog on this crazy Canadian who, like me, was trying to learn all the in's and out's of the Pocket PC's. Now it's mainly a news site with some great back end dicussions.
Attention: "My opinion coming next"
I think that if Jason sees this as the end all be all Pocket PC site, then he would have to open up the user community side of things, to attract an even larger audience. Eventually doing user reviews, chat rooms, downloads etc. Or live with the fact that sites like Aximsite will pop up and fill needs that aren't offered here, and continue to provide only very targeted information to a smaller hi-tech audience.
I think that if Jason sees this as the end all be all Pocket PC site, then he would have to open up the user community side of things, to attract an even larger audience. Eventually doing user reviews, chat rooms, downloads etc. Or live with the fact that sites like Aximsite will pop up and fill needs that aren't offered here, and continue to provide only very targeted information to a smaller hi-tech audience.
Your analysis of this site was very interesting to read, although your conclusions about Pocket PC Thoughts being over "targeted" information to a "smaller hi-tech audience" is pretty amusing. :lol: I never would have described this site in that way in a million years...
__________________ Want to contact me personally? Use this. Want to read my personal blog? Check it out. Want to follow me on Twitter? Here you go.
I think that if Jason sees this as the end all be all Pocket PC site, then he would have to open up the user community side of things, to attract an even larger audience. Eventually doing user reviews, chat rooms, downloads etc. Or live with the fact that sites like Aximsite will pop up and fill needs that aren't offered here, and continue to provide only very targeted information to a smaller hi-tech audience.
Your analysis of this site was very interesting to read, although your conclusions about Pocket PC Thoughts being over "targeted" information to a "smaller hi-tech audience" is pretty amusing. :lol: I never would have described this site in that way in a million years...
No kidding. When I first started visiting Pocket PC Thoughts I was totally ignorant about most tech stuff, yet I found the forums here to be very welcoming and helpful.
Many of the problems encountered by individuals that post negative reviews can be traced to, in my experience, user error. End users tend to just pop the top on an application without reading any of the installation instructions, Read Me's or other important documentation. They download and install in many casees, all in one step; and can in the blink of an eye screw up their computer, their Pocket PC, or both. They then blame the developer for their own ignorance. For example, a backup or Restore Point (if they run XP or Windows ME) doesn't exist or didn't get made before they installed the app in question...and that's the developer's fault (according to the end user).
Chris, I know you aren't a software developer, but developers often have that orientation. They design and build software that they understand how to use and often they write documentation that makes it clear how to use the software. (Actually, much of the documentation is not clear, but that's another topic.)
Unfortunately, that's not how most people learn and use software. They do it exactly as you describe. Great software is designed with that understanding. Usability is built in. That requires usability testing. Even a lone developer can test usability, initially with paper prototyping. (Yes, I'm a software designer, but anyone can do this and it's very inexpensive or even free.)
When I have designed software I've always assumed that if a user has problems in the first few minutes, he or she will set aside the software, intending to return later when there's more time. But unless the user is highly motivated, later never comes and we've lost that customer. One reason is that many users assume the software is logical but they are too dumb or illiterate to understand it, and people don't like feeling that way.
The wise user who is frustrated by new software blames the software. That may result in a negative user review. The "I'm too dumb" review is useful as well, though, because it warns off other beginners who may have problems. Yes, it may all be user error in the view of the developer. But it's really designer and developer error when a user can't figure out the software. The developer wants the user to read instructions but can't count on it. How many of us have tried software but just haven't been able to get our minds around it? But I'll bet the developer understood it and couldn't figure out why we were so stupid.
User reviews, then, are good for both the user and the developer. They let potential users know what others experienced and they tell developers (the smart ones, at least) what they need to do to improve their software. This feedback is invaluable. In pre-Internet days a good developer would crave such feedback. It may be embarrassing to have it out there for everyone to see, but that can be avoided through low-fidelity prototyping and low-cost usability testing. It's like quality control: you can spend the time to do it before you release the software or you can let real users identify your bugs and damage your reputation. Usability works the same way.
Quote:
Originally Posted by SassKwatch
A lot of this depends on one's definition of 'Professional'. Unfortunately, 'Professional' all too often equates only to 'Paid', and has little to do with 'Quality'. I've read far too many 'Professional' reviews that left me knowing as little about the reviewed product as I did before, and feeling as though the writer was simply compiling words to draw a paycheck.
Professional review can be very superficial. I have often received far better feedback from actual users than from professional reviews. One reason is that users are drawn to the software because they have a personal or professional need they hope it will meet. They use the software to solve a real problem and may have extensively exercised its various features. A professional may not even be a potential user of the software and can miss aspects that are important to real users. Or they may not get the software's intent and really get things wrong. Have you ever read a Pocket PC review written by someone who has always used a Palm?
When I've read professional reviews of software I've designed or managed, I've been surprised by the rare review that got it right. Seriously. My primary interest in such reviews was what kind of publicity they might give us, good or bad. User feedback, on the other hand, was a far better indicator of how well the software worked. User reviews may not be well-structured, well-written, comprehensive or fair, but they are from real users. This compensates for many shortcomings.
Let me quickly add that most "professional" reviews found on enthusiast sites are different from those found in magazines and such publications. On these sites the reviewers normally are not paid and are reviewing software they've selected the same way a user would: because they have an actual interest in the software. I put much more stake in these reviews because they better represent a user's experience.
I haven't looked theCensura reviewing tool Jason is considering, but it might provide tools that can help structure user reviews. For example, we would get more structured user evaluations if people were asked to rate a product's Ease of learning, Ease of Use, Performance, Value and Overall Quality. Yes, there is a potential to stack the deck. I don't know how to address that. A review from someone who has been a registered member for at least three months and posted 50 times might be more valuable than the opinion of someone who just happened to show up around the date a new product is released.
Not too sure about "over" targeted, just happens to be the niche it's ended up in. Remember when you asked how many of us were developers?
I'm sure many of the new users find a great wealth of information on this site and get many great answers from the community. But most that stay and continue to comment seem on the higher end even for the Pocket PC community.
The fact that this board talks about vertical market products (fire, medical etc), developer products, but has very limited coverage of games, themes, and the like draws at least a more mature crowd. Hence the reason I'm here I guess. I think I would trust the reviews of most of the people that come and visit PPCT on a regular basis.
Obviously you wanted this site to be more than a blog, but trying to attract a larger audience and keeping the user community feeling is a tough line walk. Either way I'm sure most of us are along for the journey.
User edited comprehensive software listings and comparisons
I think user input is a great idea and if done right is a great way to build up more valuable content. I would like to see an extensive software database (although www.pocketpcmag.com already does a good job) and extensive up-to-date comparisons of software in the same categories (e.g. dictionaries, PIMs, input methods, databases). I think the only way to keep this really comprehensive and up-to-date is a collaborative solution with users adding and editing the data.
Review Database
Linear Mode
