How much CPU time do Today plug-ins/synchronized emails/Wi-Fi/BT kbd. drivers "eat"?

[Menneisyys]

It has always bugged me how many CPU cycles my resident apps / Today plug-ins "eat", mainly because the latter seemingly make the Today screen very slow and sluggish. This makes many people think they slow all their other applications too, even when the Today screen isn't visible. This is why I've made some benchmarks.

First, I've tried to test the programs using Spb Benchmark – running a CPU check before isntalling/starting aprogram and after that. However, it resulted in benchmark results that could not be taken seriously. For example, everybody knows installing any resident application/ Today plug-in decreases the performance. Spb suggested exactly the opposite. For example, when I benchmarked the "clean" Fujitsu-Siemens Pocket Loox 720, I've got the CPU index 941.87.
- when I started WZC (more on these abbreviations later), I got 1234.55 as if the PPC had become much faster.
- When I installed and activated the TO SA driver, I got 952.81 (as opposed to 941.87)
- when I installed and activated MultiIE, I got the result 1337.73.

The most significant "noise" was in the JPEG decompression results (even differences of 50-60%!), but other test results (MFLOPS etc.) have also differed in 4-5%.

All in all, because Spb Benchmark gave me wildly varying and hard-to-take-seriously results, I've chosen to use a much more reliable benchmarking I've been using for two years: searching in very large Mobipocket documents, which is a very CPU-intensive (and, to a certain degree, I/O-intensive) task.

I've put a very large, 101 Mbyte, self-created text-only Mobipocket Reader compressed PRC document on my 40* Ridata CF card. This size equals to 223 Mbytes of input HTML before compression (actually, the full textual database of AVI Forum, as of 03/2003.). I've searched for the word 'testinen' in the text (I've chosen a word that does not exist in the test so that Mobi does search for the end of the text).

I've used a source file this big in order to be able to measure very small differences in the searching time. This test, as opposed to that of the Spb Benchmark, resulted in what I've been waiting for: upon installing resident programs / starting background tasks / putting thousands of files in the main memory, the searching took slightly longer.

Mobipcoket 4.8 was able to search the entire PRC file in 6 minutes and 18 seconds on a totally clean, freshly hard-reset machine.

Incidentally, exactly the same test, under the same conditions, ran for 9:52 on an iPAQ 2210. Astonishing result if you take into account that the 2210 is a snappy, fast device and there's only 120 MHz difference between the two machines. The difference in the clock speed is only 33%, while in the raw CPU+I/O power, 56%! The difference is not that visible in real life, though, especially when browsing with File Explorer. Then, the PL720 (or, for that matter, any VGA machine) seems to be much slower than any decent WM2003 XScale PXA-255 QVGA device.

Now, for the numerical results.

Spb Pocket Plus 2.2 ( http://www.spbsoftwarehouse.com/products/pocketplus/?en ) : a highly popular and configurable Today plug-in, with (compared to the latest versions of MultiIE / PIEPlus / ftxpBrowser) an excellent PIE plug-in. Offers almost everything you could think of.

It caused almost no performance hit. Everything I could measure was probably a +1 second searching time, but I'm not sure about that.

Spb GPRS Monitor 2.2.0 ( http://www.spbsoftwarehouse.com/products/gprsmonitor/?en ), a very handy tool to monitor GPRS connections. It, because it actively polls the connection (even when the connection isn't active), does cause some slowdown in all programs. However, this is not very significant, some 1.3%. (+4 seconds in the test.)

OzBTWF 0.67 ( http://207.153.195.134/ozbt.html ): another very handy tool because it switches on/off Wi-Fi/BT automatically, depending on the actve window's caption. Older versions did have slowdown problems; this is why I was especially interested in what the new version delivers. Fortunately, the latest version only increased the search time by +2 sec. This means you don't have to fear of OzBTWF because it only slows down your PPC by some 0.7%.

WM2003(SE)'s built-in Wireless Zero Configuration (WZC) . It, because constantly polls the network to find available partners to connect to, can also cause some slowdown. I wanted to find out how much it is, and also wanted to know whether we can win any CPU cycles by disabling WZC entirely. The results were astonishing: WZC didn't introduce any additional slowdown. Enabling Wi-Fi itself increased the search time by 5 sec, though.

ThinkOutside StowAway BT keyboard (hx4700) 4.1 driver: An add-on keyboard. Didn't cause any additional slowdown.

Spb Imageer 1.2's memory card-insertion event listener. I wanted to find out whether it causes any kind of slowdown. Fortunately, the benchmark results were very good: registering to events like the card-insertion doesn't cause any kind of slowdown.

Slowdowns depending on the number of user files in the main memory

WinCE has always very sensitive to the number of files in the Object Store. The more files in the main memory, the more apparent slowdown. This is, unfortunately, is apparent with WM2003SE as well.

I synchronized 2865 mails (28.6Mbytes) with 372 (1774 kbytes) attachments onto my (otherwise clean) PPC. Because Pocket Inbox (Messaging in WM2003SE's parlance) stores synchronized e-mails in the main memory in a way that is definitely the worst (no compression at all and one file is used for each mail), this meant 3237 new files in the main memory. Please note that it's the number of additional files in the main memory that counts and not (only) the free memory. I still had some 80-90 Mbytes free when counducting the Messaging test – still, the PDA was much harder to use in everyday situations because of its sluggish response.

This caused a 40-second hit in the searching. (I've double-checked this.) The "normal" operation of the PDA (starting programs etc.), as has already been stated, became much more sluggish.

This means you should always try to put
1, all your files/programs in storage cards
2, use registry hacks to put PIE cache (=a lot of small files) there too
3, use e.g. WebIS Mail to store all your locally downloaded (that is, POP3/IMAP) messages on storage cards
4, try to avoid using ActiveSync's Inbox synchronization (because it can only synchronize with the main memory)

Opinions/comments are welcome!

[ChristopherTD]

Thanks for the valuable insights. In particular the fact that additional files in Main Memory causes slowdown. I will have to cut back my Inbox "Days To Sync" setting.

I do have WebIS Mail, but alas, I cannot send mail with it (a common unsolved problem it seems) so I must still use Inbox.

You have given several areas to explore! Thanks again.

[Menneisyys]

Slowdowns depending on the number of user files in the main memory

WinCE has always very sensitive to the number of files in the Object Store. The more files in the main memory, the more apparent slowdown. This is, unfortunately, is apparent with WM2003SE as well.

I synchronized 2865 mails (28.6Mbytes) with 372 (1774 kbytes) attachments onto my (otherwise clean) PPC. Because Pocket Inbox (Messaging in WM2003SE's parlance) stores synchronized e-mails in the main memory in a way that is definitely the worst (no compression at all and one file is used for each mail), this meant 3237 new files in the main memory. Please note that it's the number of additional files in the main memory that counts and not (only) the free memory. I still had some 80-90 Mbytes free when counducting the Messaging test – still, the PDA was much harder to use in everyday situations because of its sluggish response.

This caused a 40-second hit in the searching. (I've double-checked this.) The "normal" operation of the PDA (starting programs etc.), as has already been stated, became much more sluggish.

This means you should always try to put
1, all your files/programs in storage cards
2, use registry hacks to put PIE cache (=a lot of small files) there too
3, use e.g. WebIS Mail to store all your locally downloaded (that is, POP3/IMAP) messages on storage cards
4, try to avoid using ActiveSync's Inbox synchronization (because it can only synchronize with the main memory)

Opinions/comments are welcome!

In http://www.pocketpcthoughts.com/forums/viewtopic.php?t=36521, I've just outlined a working way to store even thousands of synch'ed e-mails on the PDA without performance hits:

"A solution for freeing up main memory is ZIP'ing all the AS synched mails by hand and storing the resulting ZIP elsewhere. As the mail headers are stored in WinCE databases, totally independently of the mail bodies present in \Windows\Messaging, (temporarily) removed mail files won't result in re-synching them (it's the WinCE database tables that AS checks and not the physical mail files).

This way, you can even store / access thousands of e-mails in your PDA (which would mean disastrous slowdowns and intability if stored in the main memory uncompressed) - it's just that you'll need some manual work (finding & decompressing) before accessing a given e-mail.

Finding a given e-mail is pretty easy if you continuously synchronize your mails - just look for the file creation date. If, however, you synch'ed all your desktop e-mails one step, you'll need to peek into them to find out when they have been posted."

[Menneisyys]

In http://www.pocketpcthoughts.com/forums/viewtopic.php?t=36521, I've just outlined a working way to store even thousands of synch'ed e-mails on the PDA without performance hits:

"A solution for freeing up main memory is ZIP'ing all the AS synched mails by hand and storing the resulting ZIP elsewhere. As the mail headers are stored in WinCE databases, totally independently of the mail bodies present in \Windows\Messaging, (temporarily) removed mail files won't result in re-synching them (it's the WinCE database tables that AS checks and not the physical mail files).

This way, you can even store / access thousands of e-mails in your PDA (which would mean disastrous slowdowns and intability if stored in the main memory uncompressed) - it's just that you'll need some manual work (finding & decompressing) before accessing a given e-mail.

Finding a given e-mail is pretty easy if you continuously synchronize your mails - just look for the file creation date. If, however, you synch'ed all your desktop e-mails one step, you'll need to peek into them to find out when they have been posted."

BTW, while answering a question ( http://www.firstloox.org/forums/showthread.php?t=2918 ) over there at FirstLoox I've played around with how Windows CE stores the messages. The following information is pretty good at cases outlined above: having all your old messages on your PDA in a ZIP file, and only decrunching a file when it's really needed (because you want to show it to someone else). By directly handling the WinCE database data, you can find the corresponding message body in a second, even on a restricted/crippled PDA.

As has already been stated, the files in \Windows\Messaging only have message bodies – no headers at all. To access the headers (e.g., the sender / date / subject data) as well, you'll need to export the Windows CE database named 'fldr1001904' too. I recommend using Pocket dbExplorer ( http://www.phatware.com/hpcdbex.html ) for this. (Tap-and-hold the database and choose Export and then, transfer the resulting file to the desktop.)

A record in the above-mentioned fldr1001904 database looks like this:


50351703 1 50351717 "Topic Reply Notification - Summary of Media Players" 1/10/05 2:46:05 PM 3 4921 16783617 """Pocket PC Thoughts Team"" <[email protected]>" "Pocket PC Thoughts Team" 1354 268435456 0 16783616 1 1/15/05 2:40:47 PM

You can easily spot the subject, the date etc. part of it. The first number, 50351703 , however, is the most important of all. It identifies the corresponding file in \Windows\Messaging. The latter directory contains the body of each files of the following form:

03004e571000001f.mpb

where the bold section is the first number in each database record, converted into hexadecimal (on the PC, you can do this easily by the built-in Windows program "calc") and the italian section doesn't change. For example, the hexa 03004e57 equals to decimal 50351703 . This way, you can easily find the corresponding body to all your messages, especially from a program, if you write one to associate message headers with message bodies.

[Menneisyys]

Another trick to reduce the number of files in your main memory to speed up WinCE:

You can move all your Pocket Internet Explorer cache (which can be as high as 20-40 Mbytes in size!) to some not-that-slow flsh-based storage medium by directly editing the directory name at

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\Cache

in the registry.

It's preferable to benchmark your flash ROM/memory cards before choosing which one to use. For example, both the Belkin eFilm Pro cards (see e.g. http://www.firstloox.org/forums/attachment.php?attachmentid=224 ) and most built-in Flash ROM areas are pretty slow to write (for example, in the Pocket Loox 720, while SD cards can be written at 800-1200 kbytes/s, the built-in ROM only at some 150 kbytes/s).

[Menneisyys]

Another trick to reduce the number of files in your main memory to speed up WinCE:

You can move all your Pocket Internet Explorer cache (which can be as high as 20-40 Mbytes in size!) to some not-that-slow flsh-based storage medium by directly editing the directory name at

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\Cache

in the registry.

It's preferable to benchmark your flash ROM/memory cards before choosing which one to use. For example, both the Belkin eFilm Pro cards (see e.g. http://www.firstloox.org/forums/attachment.php?attachmentid=224 ) and most built-in Flash ROM areas are pretty slow to write (for example, in the Pocket Loox 720, while SD cards can be written at 800-1200 kbytes/s, the built-in ROM only at some 150 kbytes/s).

On BrightHand at http://discussion.brighthand.com/showthread.php?s=&postid=753572 , I've been told the following in response to my recommending moving the PIE cache to storage media:

"If you move the cache out of RAM, expect your performance in IE to suck, big time. Didks is many, maybe dozens of times slower than RAM, and IE uses it a lot."

Because my answer can be pretty interesting for all PPC freaks, I post it here too (it's better to keep all advanced benchmarks together in one thread):

It all depends on if you're displaying pictures or not, and, if yes, how many of them there are on the page. If, for example, you switch off images because you mostly use GPRS-based connections from your PDA's, there won't be severe slowdowns if you use storage cards. (Actually, the opposite may even be true: the more files in the main memory, the slower the PPC in general. And, PIE caches are renowned for containing a lot of files.)

If you entirely switch off displaying pictures in PIE, the difference between the SD- and the main memory-based solution can be as low as 30%. If, on the other hand, the page you download has, say, 150 different (meaning: all of them must be stored/retrieved from cache) images, the difference can be as high as 100%.

Using traditionally slow storage media (e.g., built-in flash storage – iPAQStore, LOOXstore etc.) causes real slowdowns of even 300-400%, even when rendering plain HTML pages, however. So, NEVER set your internal cache to flash ROM areas.

I've done two times three benchmarks with PIE to demo the slowdown. I've used a local web server (Apache), right on the desktop PC my PDA is connecting to via USB ActiveSync. I've always checked its server log to be absolutely sure that the HTTP server indeed returns a 304 Not Modified, forcing PIE to read the file back from the cache, when reading a page again.

In the first series, I've only read the 131874 bytes deprecated-list.html file from Java 2 Platform SE v1.4.1 and benchmarked all the three (main memory, plain SanDisk SD and LOOXstore) cache configurations for both writing and retrieving. The test device was a Pocket Loox 720. All tests have been conducted under the same conditions.

The results:

RAM SD LOOXstore
saving to cache & rendering ~8s ~11s ~23s
retrieving from cache & rendering ~8s ~11s ~23s



So, using the SD, the difference in speed is about 30%.

The second series consisted of retrieving a small HTML file linking 150 different images. Here, the results were drastically different:



RAM SD LOOXstore
saving to cache & rendering 1:41 3:15 7:50
retrieving from cache & rendering 1:41 2:53 6:45


This test is more of a drastic, sadistic test of PIE and the storage medium because most pages will only contain 10-15 different images (e.g., smileys) at most.

It should be pointed out, in addition, that NetFront 3.1 is a LOT faster at rendering pages like this.

Verdict: if you switch off displaying images, the speed difference is barely visible. With common pages not containing more than 5-10 different images, the difference is still under 40%, and it is even smaller under slow (e.g., GPRS) connections (don't forget that I've done all this over fast (150-200 kbytes/s), HTTP/1.1 connections. GPRS is orders of magnitude slower and some GSM providers don't even support HTTP/1.1, further slowing down the connection.).

BTW, I've used the following self-written Java program (initially written for my forthcoming Java & TCP/IP book) to generate a HTML and 150 small pics:



import java.io.*;
class BrowserTestGenerator
{
public static void main(String[] s) throws Exception
{
String imageNamePrefix = ""+Math.random();
int[] b = new int[]{71, 73, 70, 56, 57, 97,
15, 0, 15, 0, 179, 14, 0, 255,
234, 0, 69, 69, 69, 0, 0, 0, 255, 206, 0,
255, 201, 0, 255, 180, 0, 254, 157, 0, 255, 254, 147, 255,
253, 19, 255, 255, 255, 255, 255,
199, 51, 51, 51, 255, 255, 235,
255, 229, 0, 0, 0, 0, 0, 0, 0, 33,
249, 4, 1, 0, 0, 14, 0, 44, 0, 0, 0, 0,
15, 0, 15, 0, 0, 4, 91, 208, 73, 25, 106, 157, 216, 85,
166, 206, 25, 23, 22, 112, 8, 2, 156,
131, 17, 76, 65, 39, 8, 39, 240, 166, 171, 118, 152,
113, 14, 106, 10, 14, 159, 63, 130,
42, 112, 3, 198, 130, 133, 202, 33, 199, 4, 16, 146,
1, 211, 162, 9, 88, 60, 43, 3, 217,
107, 187, 21, 174, 2, 89, 65, 98, 60, 22, 16, 118,
154, 193, 160, 193, 53, 211,
88, 106, 130, 92, 254, 22, 5, 10, 248, 194,
48, 195, 178, 212, 38, 17, 0, 59};
PrintStream HTMLps = new PrintStream
(new FileOutputStream ("i.html"));
for (int i=0; i<150; i++)
{
HTMLps.println("<p><img src="+
imageNamePrefix+i+".gif>");
FileOutputStream fos =
new FileOutputStream (imageNamePrefix+i+".gif");
for (int j=0; j< b.length; j++)
fos.write(b[j]);
fos.close();
} // for
HTMLps.close();
} // main
}

[Menneisyys]

Addition to the cache test:

I've also benchmarked my Ridata (a comparatively fast; was quite famous for this some 3-4 years ago) 256M CF card. The results are worth considering: it supports very quick file creation. This is why switching to using it in the many-image-files test, the initial page reading time, compared to keeping the cache in main memory, isn't considerably more.


RAM SD CF LOOXstore
to cache 1:41 3:15 1:53 7:50
from cache 1:41 2:53 2:07 6:45


Of course, in the on-file-test, because of the speed bottleneck of accessing a memory card, the results aren't as spectacular. Still, it's still faster than keeping the cache on the SD card.


RAM SD CF LOOXstore
to cache ~8s ~11s ~11s ~23s
from cache ~8s ~11s ~9.5s ~23s


What does this mean?

1. Plain transfer/reading/writing rate of a card won't clearly show how much time it will take to create a file on it. So, you should choose storage medium based on not just the plain file transfer speed (which is best measured by Total Commander, Resco File Explorer or Spb Benchmark), but also file creation speed.

2. using really good cards, keeping the PIE cache on a card won't really increase loading times. (Please note that I've emphasized it's PIE. NetFront 3.1 is far faster at rendering pages containing tons of images, so keeping its cache on a memory card may be not as similar to keeping in main memory than with PIE.)

[Menneisyys]

Because I was really interested in how NetFront (NF) 3.1 fares against the latest, WM2003SE PIE, I've also tested it, installed in both the main RAM and CF. All tests were conducted on a Pocket Loox 720, under the same circumstances.

The test with 150 images was conducted in 12 secs (an order of magnitude faster than with PIE!); the Java HTML in 8.8 secs. A simple reload of the 150-image-page, which didn't send any request to the HTTP server (it entirely depended on the local cache!) took about 2.5 secs; forced reloading took 12 secs – it resulted in an unconditional HTTP request being sent out to the HTTP server, which, therefore, responded with sending over all the images again.

Operating on a CF, the image test's unconditional HTTP download was conducted in 17 secs in QVGA and 8 secs in native VGA; subsequent cache-based loading about, as with the main memory, 2.5 secs. It should be pointed out that it's only when leaving the page that the files are actually written to the cache. It took NF3.1 some 3-4 secs to write all the 150+1 files to the cache when on the CF (when also deleting the previous versions, 15 secs) – this time, the GUI doesn't respond; with the main memory, it happened almost instanteously.

Some people are complaining about the slow loading times of NF (e.g. at http://discussion.brighthand.com/showthread.php?s=&threadid=115345 ), so I also benchmarked it. Passing to starting to read the pre-set homepage took 6 secs when both installed into main memory and on the CF card. It is not at all slow IMHO.

Verdict:

- older NetFront versions didn't really work from storage cards – they weren't able to store their configuration. The new version is OK in this respect. It's only in the lack of the forced VGA mode that a card-based setup is clearly worse than a main-memory based one. However, as the forced VGA mode, even with the hack described at http://discussion.brighthand.com/showthread.php?s=&threadid=105664 , is pretty useless because of the lack of horizontal scrollbar and a still too big URL field, it's not a big problem IMHO. In native VGA mode, everything is OK with NF. (Also note that the location of its cache is not modifiable and is always in a subdirectory in its home.)
- NetFront 3.1 is an order of magnitude faster at rendering pages containing tons of images than WM2003SE's PIE. There's no difference, however, at rendering large pages without images.
- used with fast cards, its cache reading/writing overhead is almost negligible, even with 150 files to create + write / read.

So, a big thumbs-up for Netfront – it behaved much better than I've thought and it was good to see how well it performed even on a (fast) storage card. If only it worked well in forced VGA...

[Menneisyys]

I've just read the CF/SD write speed section of the Canon EOS-1D Mark II review over at DPReview ( http://www.dpreview.com/reviews/canoneos1dmkii/page12.asp ). The camera writes to hi-speed SD cards much faster than to CF cards. The given link is really worth checking out, it's pretty instructive!

[ctitanic]

Menneisyys, I have noticed that the PIE in WM5 renders faster than previous version but I have not hard evidences of this. Would be good once you update your machine to WM5 if you run these benchmarks again with the new PIE on WM5 ;)

[Menneisyys]

Menneisyys, I have noticed that the PIE in WM5 renders faster than previous version but I have not hard evidences of this. Would be good once you update your machine to WM5 if you run these benchmarks again with the new PIE on WM5 ;)

Sure I'll retest this stuff as soon as I get a WM5 device.

[pocketpcadmirer]

Hi all

I used to have lot of today plugings but now I have changed myself. Now i have only 3 plugins namely pocket weather, owner info and spb diary.

I know that spb diary causes some system slowdown but at the same time it really saves a lot of screen space on my JAM

[Menneisyys]

I know that spb diary causes some system slowdown

Thanks for pointing this out; will make some quantitive comparative tests with these Today plug-ins.