another free tool to benchmark devices

[The PocketTV Team]

We have added a new Benchmark feature in PocketTV Classic, that you can use to benchmark different devices against each other.

Benchmarks are useful to determine the device with the fastest combination of CPU and memory speed. Naturally this new benchmark will give you just one more rating, and other benchmarks may give different relative ratings for the same devices.

PocketTV Benchmark is somehow similar with the benchmark feature of Betaplayer, i.e. it will cause it to decode and display all frames as fast as possible.

There are a few differences in the PocketTV benchmark, compared to the Betaplayer benchmark:

- The audio is always disabled during benchmarking.

- You can benchmark using any part of a file (Betaplayer always start from the beginning)

- You can benchmark in Portrait or Landscape view (Betaplayer always run the benchmark in Landscape)

- The result is the average speed (the only important benchmark value), which is the percent ratio of the actual fps over the nominal fps.

Most of the other parameters displayed by the Betaplayer benchmark dialog (e.g. Bench time, framerate, sample rate, datarate etc) convey little more information as they are be derived from this "average speed" ratio and from the file Properties.

You can use any MPEG stream to do a benchmark. Just make sure to use the same file and the same settings if you want to compare two devices. There are many test MPEG streams here: http:://www.pockettv.com/mpg .

Naturally, depending on the settings of PocketTV, the result of the benchmark will be very different. For example, if you de-select "Fast Video", PocketTV will use GDI, so the benchmark will be heavily impacted by the performances of the GDI routines BitBlt or StretchBlt on the device (StretchBlt is used when the video is scaled, BitBlt when it is played at nominal size). If "Fast Video" is selected, PocketTV uses direct access to the frame buffer (GAPI), so the performance will reflect the speed of the video memory. etc, you get the picture :) .

To do a benchmark, select Tools > Options > Files > Benchmark, then play the file. Don't interact with the device while playing. Benchmark result will be displayed at the end (or when you stop).

Currently this benchmark feature is available in the (stable) beta version of PocketTV Classic 0.15.4, available from http://pockettv.com/bin/PocketTVSetup-0.15.4-BETA1.exe

Please let us know if you have any suggestion to improve this feature.

Enjoy!

[picard]

thanks for the feature.

- You can benchmark in Portrait or Landscape view (Betaplayer always run the benchmark in Landscape)
you can change the fullscreen orientaion in BetaPlayer and benchmark in Portrait mode if you want (but you can't emulate the user interface clipping)

yes, all info is derived from one benchmark value. but example the overal bitrate is kinda usefull. it gives a good approximate what overal bitrate your device can handle (if benchmarked with audio).

i will think about not seeking to the begining. btw do you empty the read buffer before starting benchmark? because BetaPlayer loads the buffer even it's not in play mode and if benchmark wouldn't drop the buffer, it would not give identical results (it would depend on previous state)

[The PocketTV Team]

> but example the overal bitrate is kinda usefull. it gives a good approximate what overal bitrate your device can handle (if benchmarked with audio).

yes, I suppose music lovers want to code their MP3 at 10 Mbit/sec :)

[picard]

yes, I suppose music lovers want to code their MP3 at 10 Mbit/sec :)
i mean with video and audio (and not with audio disabled)

ps: i modified my last post about buffering...

[The PocketTV Team]

> btw do you empty the read buffer before starting benchmark? because BetaPlayer load the buffer even it's not in play mode and if Benchmark wouldn't drop the buffer, it would give not identical results.

The read buffer is emptied whenever you seek, and it starts filling-up immediately (i checked), before you hit play. so most likely our pre-buffer is full when you will start playing (unless you start absolutely immediately after seeking), and it remains full all the times until the end. So the benchmark continuously run the read thread as soon as you start, and until you stop, and the buffer is always full (when reading from a file).

in other words, even on a short benchmark, there is no side effect that would be caused by spending more time filling-up the buffer initially. the buffer is in a stable, always full state, filling-up continuously while playing.

so i guess it works about the same way?

[picard]

The read buffer is emptied whenever you seek, and i think it start filling-up immediately, before you hit play. so most likely our pre-buffer is full when you will start playing (unless you start absolutely immediately after seeking)
i can understand this approach too. still currently it depends on the previous state, i would suggest to wait for a full buffer when you start play in benchmark mode.

how large is the buffer in PocketTV? now i have to think about modifing BetaPlayer behaviour, because it uses 2MB buffer and loading it from flash card could take about 2 seconds which can influence quite much the benchmark result for a short movie.

[The PocketTV Team]

> i would suggest to wait for a full buffer when you start play in benchmark mode.

it is almost certainely full, since the default buffer, when playing from file, in only 300 KB. This will fill-up immediately, in a few milliseconds.

> it uses 2MB buffer and loading it from flash card could take about 2 seconds which can influence quite much the benchmark result for a short movie.

IMHO it is totally un-necessary to use such a large pre-buffer when playing from files, unless you play from Microdrive or harddrive (and want to fill-up only by burst, to reduce power use), or unless your files are located on, say, a networked drive (an unlikely scenario).

[picard]

>IMHO it is totally un-necessary to use such a large pre-buffer when playing from files, unless you play from Microdrive or harddrive (and want to fill-up only by burst, to reduce power use), or unless your files are located on, say, a networked drive (an unlikely scenario).
unfortunattely not all files are interleaved that well. example i have many high bitrate m-jpeg avi files where even 2MB buffer is not enough to able to decode ahead 1/8 seconds audio. but you are right, i probably will decrease the default buffer size.

[The PocketTV Team]

unfortunattely not all files are interleaved that well.
In the case of mpeg1, they are, it's a requirement. check the section about VBV buffer in the spec. basically audio and video samples corresponding to simultanous times cannot be far way in the file.

i don't know about other compression formats like DivX, that may have different requirements for buffering.

[picard]

i don't know about other compression formats like DivX, that may have different requirements for buffering.
avi is kinda dummy format :)
(divx is only the video codec)

[The PocketTV Team]

i don't know about other compression formats like DivX, that may have different requirements for buffering.
avi is kinda dummy format :)
(divx is only the video codec)
Isn't ASF a better container format than AVI ? I think ASF corrects many of the shortcoming of AVI and is much better for streaming video formats.