Log in

View Full Version : WM2003 and SE: file handles become un-usable after power OFF/ON


The PocketTV Team
05-04-2004, 06:58 AM
It appears that MSFT managed to break something in WM2003 and WM2003-SE, and this can cause serious head-haches for developers, as there is no easy workaround.

The problem affects applications that read or write files located on removable storages (SD Cards, CF Cards).

In Pocket PC 2000 and Pocket PC 2002, when the device was turned OFF (i.e. "suspended"), then turned ON again later ("resumed"), all the files handles of files located on removable storages were re-mounted correctly by the filesystem drivers, i.e. basically the suspend/resume operation was totally transparent to all applications, including those that had open file handles to files located on removable storages.

The only caveat was described in this KB article: http://support.microsoft.com/default.aspx?scid=kb;en-us;811693 . Basically ReadFile could return an error when trying to read the file too soon after the power-resume. The workaround was simple: sleep a bit and retry. If it keeps failing after say 4 seconds, then abandon.

Now, in WM2003 and WM2003-SE, it appears that those file handles are not always remounted by the driver upon the power-resume, i.e. all file handles can become un-usable after a power suspend/resume for files located on removable storage (even though the storage card was left in the device while the device was suspended).

You may think: no big deal, the application can just re-open the file.

But think a bit more, and you will realize that by re-opening the file based on its pathname (e.g. "\SD Card\foo.xxx"), you have no guarantee that you are in fact opening the same file! While the device was suspended, the user may have replaced the Storage Card by another one that have a different file with the same name. Only the driver can verify that the Storage Card is the same, based on the filesystem signature, the Card ID etc. That's why the filesystem should be re-mounted by the driver, and all the file handles should remain valid, unless the storage card was changed.

And even if you re-open the file, it is not as simple: You also need to keep track of the current offset in the file (seek position), in order to get a new file handle that points to the same offset.

And your application must now expect each File System call (e.g. GetFileSize, SetFilePointer, ReadFile, WriteFile, GetFileAttributeByHandle, FlushFileBuffer, etc ) to fail since it has become a "normal" thing that will happen each time the user press the POWER OFF button (if your file is on a storage card).

I complained to MSFT that this was a serious bug that will cause problems to a number of apps and reported the issue to several MSFT Engineers at the Mobility Developer Conference. They did not take it seriously enough to file a bug report in the Knowledge Database.

When I insisted, two months later they assigned a contractor to look into the issue, and the feedback I got from the guy was: "it's by design, the product group said it's normal that file handles to files located on removable storages become un-usable after the device is suspended/resumed". This guy told me that "there was many improvements in WinCE 4.2, so some things have changed". That's definitely what I would call a "negative improvement".

I just cannot believe that this new bug is "by design", since it was working (almost) perfectly in Pocket PC 2000 and 2002. And if this was a change "by design", I would expect it to be documented, which is not the case.

Furthermore, http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcemain4/html/cmconstoragemanagerarchitecture.asp clearly suggests that the storage Card drivers should remount correctly the filesystems on suspend/resume:

[...]This notification forces unmounts of the partition driver and any file systems. During this period, the mount points move into a suspended state and requested I/O access to the mount points fail with ERROR_DEVICE_NOT_AVAILABLE. The primary use for this is during suspend/resume cycles where a block driver might unload and reload on resume. When the Storage Manager receives a load notification, it examines the block device to see if it is identical to the previously unmounted device. If it is identical, the block device remounts under the previous mount points.

ChristopherTD
05-04-2004, 07:19 AM
This is bizarre because the latest ROM (1.10.10) for my iPaq 5550 fixed this problem!

Before the ROM update any application that was suspended with files open on a Storage Card would behave unpredictably. These apps included Microsoft Reader and Laridian Pocket Bible.

After the ROM update they behave as expected, allowing you to resume reading without incident.

I hope that WM2003SE (final release) doesn't break this because it would be a showstopper for me. Fortunately I am in no rush to upgrade!

The PocketTV Team
05-04-2004, 08:10 AM
> Before the ROM update any application that was suspended with files open on a Storage Card would behave unpredictably.

That's what we observe with all the WM2003-SE devices that we have tested.

We also observe this on the Toshiba e800 (WM2003).

socrates63
05-04-2004, 08:24 AM
So this seems to be the answer to the problems I experienced a couple of months ago. Things improved significantly when I installed the critical apps (SIP, Pocket Informant, Wisbar, etc.) to main memory upon advice from a PPCT member. Prior to that, my 2215 would freeze when I turned it back on after it had turned itself off. I never had problems of this sort on my PPC 2002 device. I almost returned the 2215.

The PocketTV Team
05-04-2004, 10:06 AM
So this seems to be the answer to the problems I experienced a couple of months ago. Things improved significantly when I installed the critical apps (SIP, Pocket Informant, Wisbar, etc.) to main memory upon advice from a PPCT member. Prior to that, my 2215 would freeze when I turned it back on after it had turned itself off. I never had problems of this sort on my PPC 2002 device. I almost returned the 2215.
Yes, I could be related if those applications open a data file located in their installation folder.

Fishie
05-04-2004, 10:36 AM
> Before the ROM update any application that was suspended with files open on a Storage Card would behave unpredictably.

That's what we observe with all the WM2003-SE devices that we have tested.

We also observe this on the Toshiba e800 (WM2003).

So thats what it was.

I noticed that after removing my CF card and taking pictures with my digicam upon reinsertion it hanged and I could no longer search the file system, sometimes even a reset didnt change that.

Its a major bug and made half my machine unfunctional.

Resco file explorer came to my rescue and after using that to open some files on the card it decided to work again last time that it happened.

The PocketTV Team
05-04-2004, 10:48 AM
Fishie,

When you take out the Storage Card, change its contents (e.g. take photos), and put it back, the Pocket PC should detect that the filesystem has changed, so it should remount a new one (and invalidate any previously opened handles). Normally you should be able to search the newly mounted filesystem, but there may be another bug preventing it.

Even though you describe a problem that may have to do with mounted filesystems, it seems unrelated to the problem we are talking about here.

The problem we are discussing here involves only suspending/resuming the device, without taking out any storage card.

Fishie
05-04-2004, 03:40 PM
Ah, ok.

Thanks for the clarification.

KenClunk
05-04-2004, 11:14 PM
This kind of thing is going to make me... wait to upgrade. I think I am going to set a limit of 1 month after release before I upgrade. I want to let everyone else test it out first. Of corse I have no will power so I will probably do it anyways. Oh well.

The PocketTV Team
05-05-2004, 08:13 AM
This kind of thing is going to make me... wait to upgrade. I think I am going to set a limit of 1 month after release before I upgrade. I want to let everyone else test it out first. Of corse I have no will power so I will probably do it anyways. Oh well.
This problem is already present in some WM2003 and WM2003-SE devices (e.g. on the e800).

I think the real source of the issue, here, is that the test suite provided by MSFT to the Manufacturers (to test if all the features of the OS are implemented correctly) does not contain any tests to check that things work as they should upon power suspend/resume.

So the OEM just presses the Power button and check that the device turns back ON. But that's not enough! Many invisible things can happen. Like the one this thread is about (i.e. all the handles of files located on storage cards are not re-mounted correctly and become permanently invalid).

In the process of testing this, we also found that very often, devices have problems with their audio driver upon power suspend/resume, i.e. audio starts making funning craking sounds after the device was powered OFF while playing audio, or audio just stops working all together (require a soft reset). That's the case with the e800, too.

pepegg
05-05-2004, 04:10 PM
My 2215 has had this problem since ever, and the new 1.10 ROM didn't make any difference. The new SD driver didn't either.
It is most noticeable with applications installed in Storage Cards than with documents.
The worst app for this "feature" is by far Acrobat Reader, a binary released for PPC2000 and not updated since then (shame on you Adobe :evil:)
-when installed in my _slow_ SD card it hangs easily when I turn the PPC on after suspension;
-if installed in the CF card things improve a lot, but still sometimes it hangs.

How would you explain this sort of improvement ?

Cheers PTV team, good work

The PocketTV Team
05-07-2004, 03:02 AM
MSFT just confirmed that:

On WM2003, applications should not expect storage card file handles to be remounted correctly after power suspend/resume (even if the storage card was not removed from the device while suspended).

Developers, you need to be aware of that.

As a side effect, it means that applications installed on storage cards can quit unexpectedly upon power-on, even if they don't access any file located on a storage card.

Something to keep in mind, too.

Dave Beauvais
05-10-2004, 03:55 AM
I've noticed this when running Mapopolis (http://www.mapopolis.com/) since upgrading my h5455 to WM2003. If I turn off the iPAQ with Mapopolis open, it behaves poorly when turned back on. As an example, I can see the map, but can no longer pan around it, jump to the next waypoint, etc. The only way to get it functioning again is to exit and restart the software. This is exceedingly annoying.

ChristopherTD
05-10-2004, 07:06 AM
MSFT just confirmed that:

On WM2003, applications should not expect storage card file handles to be remounted correctly after power suspend/resume (even if the storage card was not removed from the device while suspended).

Something to keep in mind, too.

Then it is definitely worth chasing anyone who has contacts at HP because they fixed this problem in the latest ROM release for the 5550. Even if it was an "accidental" fix, they may be able to shed light on how it was fixed?

The PocketTV Team
08-10-2004, 03:16 AM
The problem was indeed confirmed by MSFT.

The good news is that there appears to be a workaround (although not an official recommendation, it was suggested by someone at MSFT)

Try setting the following registry value as follows:
HKEY_LOCAL_MACHINE\System\StorageManager\PNPUnloadDelay : 4096

Essentially, on WM2003, the default value of PNPUnloadDelay was decreased. PNPUnloadDelay is a timeout that can impact how quickly a file handle is torn down with events around SD card suspend/resume, insertion/removal, etc.

Looks like this number is much lower on newer devices to make the OS aware at a faster interval. When PNPUnloadDelay is increased to 4096 (basically the default on 2002) the problem appears to get solved on most devices.

Stephen Beesley
08-10-2004, 08:57 AM
Ahhhhh......

I wondered what was going on with MS reader and books stored on a CF card on my WM2003 Toshiba e755. I was beginning to think that I had a faulty card or device, but no - just a plain ol' MS stuffup!

Dave Beauvais
08-10-2004, 04:33 PM
Looks like this number is much lower on newer devices to make the OS aware at a faster interval. When PNPUnloadDelay is increased to 4096 (basically the default on 2002) the problem appears to get solved on most devices.
Thank you! This seems to have solved a problem I had with Mapopolis (http://www.mapopolis.com) on my h5455 w/WM2003. If the PPC turned off, I'd lose the ability to pan around the map or zoom in when turning it back on. It was as if the map data were no longer loaded. This never happened under PPC 2002 on the same device.

ctitanic
08-25-2005, 03:10 PM
The default value for that key in the e800 is 800 and if fails to open a file after resuming. In my hx4700 the default is 1500 and it does not fail to open a file after resuming.

May be 1500 is enough for all PPCs.

optimusmaximus
09-09-2005, 12:16 PM
Yeah! The postings on this forum helped me use my Laridian Bibles once again with no problem. I never like messing around with registry changes, but this one did the job wonderfully well. Once you change the registry, remember to restart your computer before the change takes place. I didn't do this the first time and didn't notice any change in my stystem operation. After resetting my Toshiba E405 (this might be a Toshiba problem it seems), my Pocket PC suddenly showed my Laridian Bibles whether I turned the machine on or off.

I did try to look at my PDF Adobe Pocket Reader 2.0 and this still crashes my system if I turn the machine off while a PDF document is open, but I feel this is Adobe's fault for building pocket software that is buggy and could be improved drastically (Adobe, support your Pocket Reader software please).

Remember, you must change the registry as was mentioned to the following (the default value for my machine was 800):
HKEY_LOCAL_MACHINE\System\StorageManager\PNPUnloadDelay : 4096


I used my Resco Registry Editor that is part of the Pocket Browser to do this. Restart your Pocket PC and your Laridian Bibles should now always show up, whether you power your Pocket PC off or on.

ctitanic
09-09-2005, 12:58 PM
Well, the good news for those who are afraid of editing the registry manually is that I have included this registry hack in the latest version of Tweaks2K2. :devilboy:

yankeejeep
09-09-2005, 02:03 PM
And like all the hacks in Tweaks2k2, it works very nicely. Thanks.

ctitanic
09-09-2005, 03:57 PM
And like all the hacks in Tweaks2k2, it works very nicely. Thanks.

Thank you! 8)

Ed Hansberry
11-08-2005, 06:46 PM
I am seeing the same issue on a WM5 JasJar but the problem is, the registry key is already set to 4,096. :?

Robb Bates
11-08-2005, 07:09 PM
Same thing here with X50v WM5. Before, I could pause RealPlayer while listening to an audiobook, power off, power back on and then resume. Now when I resume, the file stops playing after a few second. I assume the few seconds are because of buffered audio.

Robb

alese
12-10-2005, 03:17 PM
Maybe this is possible sollution for the unmounting problem.

Just leave the PNPUnloadDelay as it is and change the other key:

HKEY_LOCAL_MACHINE\System\StorageManager\PNPWaitIODelay

from default (on Universal it's 4096) to 1.

I'm not 100% sure yet but for now it's working for me.

kenny1
02-06-2008, 07:11 AM
Thanks for the info. :)

Craig Horlacher
09-16-2009, 06:33 PM
I am seeing the same issue on a WM5 JasJar but the problem is, the registry key is already set to 4,096. :?

I know I'm a few years late on this post but I figured I'd try. This is the issue where applications crash if they are running off a memory card or if the running application has a file open from a memory card. I can't beleive it's possible for this to still be an issue in WinMo 6.1. I was sure Microsoft would have fixed it by now.

I'm having the same problem with my Imate Ultimate 6150. I was able to fix it with the registry change on my Toshiba E830 but the registry change made no difference on this device. Then I figured Imate would fix it with the WinMo 6.1 update. Well, they finally released the WinMo 6.1 update a while back and that still hasn't fixed it. I've tried the reg-hack on 6.1 and it still doesn't fix it. I thought you may have some insight since your JasJar was an Imate device. I've asked imate about this many times and they sort of ignore it and/or don't understand what I'm talking about.

Thank you!

Craig