Unfortunately, the h4355 cannot be upgraded to WM2003SE. None of HP's previous generation of devices will be able to be upgraded to this new version. It's probably the news you didn't want to hear, but trust me, we've all had our rants on this!
As with folding keyboards, you can easily get IR or Bluetooth enabled keyboards that work with the device. I would recommend some but there are simply too many to choose from. Have a search through PPCT for some suggestions.
And lastly, is it a "dumb idea" to be purchasing the h4355? Well that really depends on what you want in a Pocket PC. If WM2003SE is not a big deal, then you can generally find a vast range of devices out there with WM2003 that have been marked down in price. Are you solely limited to HP when choosing a Pocket PC? If not, there are other manufacturers out there such as Dell, Toshiba, and Asus...to name a few. If so, have you considered other devices in HP's previous range? Although you have grown attached to the idea of a thumboard, if you feel as though you can make do without it, then there are many other devices out there that you can opt for with similar specifications. The h4150 and h2210 may appeal to you in this case.
These are just a few suggestions. If you have any other questions, feel free to ask.