Log in

View Full Version : Who's Using My RAM? Mike Calligaro Answers


Jason Dunn
11-30-2005, 09:00 PM
<div class='os_post_top_link'><a href='http://blogs.msdn.com/windowsmobile/archive/2005/11/29/498154.aspx' target='_blank'>http://blogs.msdn.com/windowsmobile.../29/498154.aspx</a><br /><br /></div><i>"In my recent "Where's My RAM" entry, I explained why you tend to see a difference between what the WM5 device reports as "Total" RAM and what you know the device actually has. In other words, "Why does my 64M RAM device say it only has 50M?" In the comments for that entry people asked about the reported "In Use" numbers. "In Use" shows how much of the total RAM the system is using right then. The general consensus among the commenters was that WM5 took three times as much RAM to boot as WM2003 SE. This was shocking to me, because I know the steps we took to keep that from happening. WM5 is a massive upgrade, with a new OS, new apps, and a ton of new features. We were very worried about bloat, and did a number of things to combat it. We funded a group whose sole job was to analyze performance (including RAM usage) and keep it under control. Those folks did a bunch of work to reduce RAM usage. We also purposely limited the amount of RAM available on the devices our developers used. Almost all of our devices only had 32M of RAM, and the one device that had more, we artificially limited to only use 32M. We didn't want anyone to get lazy."</i><br /><br />Some further details about why Windows Mobile 5 seem to have so little RAM available after a soft reset. Everything that Mike is saying is logical, but that still doesn't dispel the feeling that Windows Mobile 5 devices are a step back in some ways when it comes to memory usage and speed. Using a VGA device (the Jasjar) with Flash ROM is a double-whammy on performance, but I was hoping for more speed that what I'm seeing. The comment from Alex Kac of WebIS was especially interesting, and one that I agree with - now that Microsoft has hit the 50% overall market share number, I think they can afford to lean on the OEM/ODMs a bit more to ensure a better device experience for end users (and that includes OS upgrades).

JonMisurda
11-30-2005, 09:18 PM
This is the only WIMR? blog post about WM5 that hasn't angered me over the attitude that MS seems to have over the users of the devices. I'm not sure I can appreciate the "we know better than you" attitude that they have about not reporting actual memory numbers. It reminds me of the smart minimize button.

That said, I'm guessing the comparison they did here involves the Axim x50v (reading between the lines, I could be wrong.) After I did the WM5 upgrade I found that I had 28MB of RAM free. Before I did the WM5 update I had... 28MB of RAM free.

I understand the issues with XIP and the horrors of Flash ROM, but I agree that this is starting to feel like a step backwards in the usefulness of devices. If flash RAM is so slow that you need a 4MB filesystem cache to hide the performace hit, maybe this isn't the proper design for the OS.

The solutions of adding more, larger caches to keep write backs to ROM feeling speedy is also creating a dangerous situation that not many peope have discussed: If your device locks up before these (large!) buffers are written back, you can lose a ton of data. I've already seen the issue of Registry edits not taking hold because I did a hardware reset instead of a software one (which forces a flush before it resets.)

I'm afraid to use the hardware reset button because I'm not sure what will still be on the device and uncorrupted when it reboots.

Maybe WM2003SE was the pinnacle of handheld OSes. I thought I was happy with WM5, but after writing this, I'm wondering if PowerPoint Mobile is really enough to keep me from downgrading.

/rant off

Jon

mpaque
11-30-2005, 09:29 PM
This whole WM5 memory thing sounds troubling to me. I have an X50v that I planned on upgrading to WM5. After reading this I think I'm going to stick with WM2003SE.

Hugh Nano
12-01-2005, 12:25 AM
Hmm... Now I don't know whether to blame Microsoft or Dell. As an "early adopter", I was among the first to obtain the both the Axim x50v and the WM5 upgrade for it. I struggled for about a week or so to get used to the speed and performance hit after the upgrade, even uninstalling some of my favourite (WM5 compatible) programs to try to revive my x50v's speed. To no avail. I "downgraded" my x50v last week, and am now more than happy with my Pocket PC's speed and stability.

So, where does the problem lie? Obviously some of the fault lies with Dell's bug-ridden implementation of WM5 (which brought back a number of the same problems I had initially had with versions A00 and A01 of their 2003SE x50v ROM, on top of the slowdown issues), but I wonder where the fault for the main flaw - the slowdown issue - lies. Even at full speed, with almost nothing installed, I found the x50v agonizingly slow running WM5. Is the flash ROM on the x50v sub-standard in terms of speed? Or is WM5 just over-estimating flash ROM speeds and thus prematurely implemented?

jpf
12-01-2005, 12:33 AM
heh! wm5 is going to be the windows Me of the ppc.

onepieceman
12-01-2005, 01:27 AM
Yes, definitely feels like a step backwards for me. But give it time. The service pack isn't out yet, and I tend to find it's only after that that it's possible to judge the OS. The joys of being an early adopter...

JonMisurda
12-01-2005, 01:31 AM
I have the same problem: who's to blame? I'm guessing that the hardest part of the update is writing drivers. Now some drivers should be part of the Windows CE base that WM is built upon. I'd bet anything that the filesystem driver is mostly written by Microsoft.

Flash memory is quite slow compared to RAM, but really only for writes. Reading from memory (i.e. launching a program, or loading a file) should be snappy. We've all seen the slowness of Flash memory because we've all used storage cards.

One thing that sets off my BS meter about the explanation of XIP and the difficulties of reading and writing to the same flash medium is the fact that in WM2003, we could run a program from the OS (in flash) and save into Built-in Storage (same flash.) Are they saying that WM2003 didn't have XIP either, because I was under the impression that it did do it? If it did, why couldn't WM2005 do it too?

A question I also have is was the filesystem compression dropped? In the RAM based filesystem of Windows CE, a file was somewhat compressed when it was store. If you go search on PocketMatrix.com you can read about it a little. If there was any place that compressing a filesystem might help, it'd be with a slower medium like flash RAM so your writes could be smaller.

Another thing that doesn't sit well with me is just how slow writes are. I've been spoiled by RAM writes for many years, but I've also used a laptop computer with a 4200 RPM harddrive for many years. Harddrives are still slower than Flash RAM, to my knowledge, and I've not seen the performance hit in desktop applications.

The final thing that really really makes me wonder is the fact that Microsoft had to test this on real devices. I was under the impression that they were likely using Axims since the Axim x50 was built with WM in mind (large Flash, medium RAM, graphics accelerator...) Surely their internal test version didn't run this poorly, or it'd have never shipped (I hope... hmm that's awfully naive of me.) Where are those drivers? Why didn't Dell use them.

This whole situation doesn't sit well with me as a Computer Scientist, a consumer, and PocketPC user for many years now. Things don't add up.

Jon

Janak Parekh
12-01-2005, 02:31 AM
Are they saying that WM2003 didn't have XIP either, because I was under the impression that it did do it? If it did, why couldn't WM2005 do it too?
Actually, some 2002 and 2003 devices didn't support XIP, and others did. I think the iPAQ 1910 was the first that used the cheaper NAND ROM and didn't support XIP -- and on startup, of the 64MB of RAM only 49MB was useable because the rest had the OS loaded in.

Another thing that doesn't sit well with me is just how slow writes are. I've been spoiled by RAM writes for many years, but I've also used a laptop computer with a 4200 RPM harddrive for many years. Harddrives are still slower than Flash RAM, to my knowledge, and I've not seen the performance hit in desktop applications.
Actually, if you look at digital photography sites that review Microdrives against flash memory, hard drives are much much much faster at writes than flash ROM. RAM is indeed faster, but we're talking about ROM, right?

The final thing that really really makes me wonder is the fact that Microsoft had to test this on real devices.
That's my exact question too. You'd think that the OEM documentation would include "best practices" for WM5 memory management.

--janak

BrandonMiniman
12-01-2005, 02:54 AM
The final thing that really really makes me wonder is the fact that Microsoft had to test this on real devices. I was under the impression that they were likely using Axims since the Axim x50 was built with WM in mind (large Flash, medium RAM, graphics accelerator...) Surely their internal test version didn't run this poorly, or it'd have never shipped (I hope... hmm that's awfully naive of me.) Where are those drivers? Why didn't Dell use them.

I have the same question, Jon. It seems only natural that MS would ensure that the upgrade to 5.0 would please users in terms of performance, but perhaps they were focused too much on numbers (RAM usage and whatnot) and really didn't spend the focus to test real world performance (multitasking, etc).

Actually, if you look at digital photography sites that review Microdrives against flash memory, hard drives are much much much faster at writes than flash ROM. RAM is indeed faster, but we're talking about ROM, right?

Yes, I think it's known that Microdrives are faster than Flash memory which is faster than ROM. What would be interesting is if somehow there was an option/hack to allow for external storage to *take the place of* ROM entirely. Yes, you can save most programs to external storage, but it would allow for three things if every bit of data (including core OS files) residing in ROM could operate on external storage - 1. faster reads/write means better performance in the OS environment and programs 2. WM devices could be touted as having almost "infinite" storage capability (up to the confines of external storage card capacities) 3. WM devices suddenly act more like PCs, in that RAM is RAM, and external storage is analogous to a hard drive: everything exists on there, it can be removed, formatted, etc. This could also help non WM users understand how the hardware works a bit more, if it’s comparable to something they already know.


Or am I just off my rocker? :D

JonMisurda
12-01-2005, 03:17 AM
Actually, if you look at digital photography sites that review Microdrives against flash memory, hard drives are much much much faster at writes than flash ROM. RAM is indeed faster, but we're talking about ROM, right?


I wasn't aware this was the case. Could you point me to any such reviews, I'm just a simple CF card user for my digital camera, no type II slot for me, so I've never looked into microdrives.

It seems that the write latency to NOR flash ROM is a highly guarded secret, or I've lost my internet searching touch. I'm curious to see some benchmarks (i've been in grad school way too long, i'm asking for benchmarks... :roll: )

Is it possible that the small platter size of a microdrive allows better performance than a traditional harddrive?

The final thing I dislike (for now) about flash memory is the limited write cycles. 10,000 low end writes for a cell in NOR RAM is only 30 writes a day for a year. I hope the driver balances the writes evenly, but why do i doubt that....

Jon

Janak Parekh
12-01-2005, 04:05 AM
I wasn't aware this was the case. Could you point me to any such reviews, I'm just a simple CF card user for my digital camera, no type II slot for me, so I've never looked into microdrives.
This article (http://www.dpreview.com/articles/mediacompare/) was the one I was thinking of... but you know, it's ancient. I did a quick search for more recent data, and this study (http://www.robgalbraith.com/bins/multi_page.asp?cid=6007) implies differently with newer flash media. That said, I'm not sure if a) Pocket PCs use the same type of flash as the newer CF cards; b) what the latency is, as opposed to throughput. (And you're right, latency seems to be far more ignored by the general media.)

The final thing I dislike (for now) about flash memory is the limited write cycles. 10,000 low end writes for a cell in NOR RAM is only 30 writes a day for a year. I hope the driver balances the writes evenly, but why do i doubt that....
Most NOR implementations do, I believe, but I don't have documentation for it at the moment. Practical experience suggests longer; I have a Smartphone with persistent storage for more than 12 months already.

--janak

JonMisurda
12-01-2005, 04:17 AM
This article (http://www.dpreview.com/articles/mediacompare/) was the one I was thinking of... but you know, it's ancient. I did a quick search for more recent data, and this study (http://www.robgalbraith.com/bins/multi_page.asp?cid=6007) implies differently with newer flash media. That said, I'm not sure if a) Pocket PCs use the same type of flash as the newer CF cards; b) what the latency is, as opposed to throughput. (And you're right, latency seems to be far more ignored by the general media.)


Ah, thanks for those. Throughput is almost meaningless for the types of things that we're using pocket pc's to do from Flash ROM. You can get the best of both worlds with a microdrive for throughput if you have a large on-device RAM buffer, and I'd be surprised if they didn't. Most harddrives have some sort of write buffer.

On the pocket pc, we're seeing frequent, short writes (like ActiveSync writing a log, the cab installer writes a log, etc.) and the thing that I hope the driver is doing is coalescing small writes into a single larger write.

I think part of the problem on the Axims at least, is that we're seeing processor speed scaling issues that also hurt the performance and every little thing is just adding up to a bad user experience (sometimes, I still have WM5 installed, and probably will keep it installed with the hope of a ROM update, without one I may go back to WM2003SE by the end of the year though.)


Jon

JonMisurda
12-01-2005, 04:22 AM
Over on Aximsite, it's been noticed that the filesystem driver can consume 90-95% of the processor.

http://www.aximsite.com/boards/showthread.php?t=106009

It seems that the bottleneck is in msflash.dll which sounds to me like the microsoft flash filesystem driver.

If it has ms on it, it would seem to make it MS's fault, but it could always be that Dell is using it wrong?

Anyone else confirm this spike?

Jon

alex_kac
12-01-2005, 05:44 AM
The one thing that makes me think its almost all Dells fault (or HTC rather ) is the great results seen on the ETEN M600.

ctmagnus
12-01-2005, 07:43 AM
Who's Using My RAM? Mike Calligaro Answers

That makes it sound almost like Mike Calligaro's owning up to using your RAM. ;)

Maybe it's a ploy by MS to get everyone to buy WM2007 upgrades (provided by MS, rather than the OEMs) for their shiny, new WM2005 devices. 8O

jlp
12-03-2005, 03:27 AM
[i]...
WM5 is a massive upgrade, ...

WiMp5 is a massive BUG :twisted:

Too many problems, not enough testing, worst many steps back in lost and even lacking functionalities, blatent lies ("RAM does not need to be powered when device is off" or worst), etc.

Jonathon Watkins
12-03-2005, 05:40 PM
This whole WM5 memory thing sounds troubling to me. I have an X50v that I planned on upgrading to WM5. After reading this I think I'm going to stick with WM2003SE.

I'm with you on that one. I've not gotten aound to getting the upgrade and now I'm wondering if I should just now.

Ed Hansberry
12-03-2005, 11:19 PM
blatent lies ("RAM does not need to be powered when device is off" or worst), etc.
Who said that? The only thing I can think of is "Ram does not need to be powered to preserve your data" which is true. Can you provide a link to Microsoft's site to your statement?

jlp
12-04-2005, 07:14 PM
blatent lies ("RAM does not need to be powered when device is off" or worst), etc.
Who said that? The only thing I can think of is "Ram does not need to be powered to preserve your data" which is true. Can you provide a link to Microsoft's site to your statement?

NO Ed, your above affirmation is WRONG: RAM DOES need power to preserve data, ROM does NOT!!

Ed Hansberry
12-04-2005, 08:14 PM
blatent lies ("RAM does not need to be powered when device is off" or worst), etc.
Who said that? The only thing I can think of is "Ram does not need to be powered to preserve your data" which is true. Can you provide a link to Microsoft's site to your statement?

NO Ed, your above affirmation is WRONG: RAM DOES need power to preserve data, ROM does NOT!!

True or False:
If a device running Windows Mobile 5 loses power and RAM is erased, the user's data is preserved.

IOW, "Ram does not need to be powered to preserve your data."

Posting some words in all caps JLP doesn't make your statement valid. :roll:

And you still haven't answered my questio. Who said "RAM does not need to be powered when device is off?"

Nurhisham Hussein
12-05-2005, 01:28 AM
I really think all this confusion is really about how you define "off". FWIW, I wouldn't consider MS to have lied on this issue - if the device is OFF, then RAM doesn't need to be powered. However, if the device is SUSPENDED i.e. how everybody commonly uses it, then RAM still needs to be powered. Question is, how often do people fully turn off their PDAs as opposed to suspending them?

jlp
12-10-2005, 12:02 AM
read here: http://www.firstloox.org/forums/showthread.php?t=4524&amp;highlight=ram some posts link back to PPCT.

I don't have time to search thru many posts in many threads about WiMp5.

On the above linked thread Duncan is quite clear about his explanation that WiMp5 devices are supposed to be completely off.

Let's ask him if he remembers where he got this information.