Log in

View Full Version : TUTORIAL: Windows Mobile Smartphone Internationalization Issues


Menneisyys
07-25-2007, 12:25 PM
In this tutorial, I explain how you can add new word completion (XT9) dictionaries to your Windows Mobile Smartphone and how you can redefine their built-in keyboard layout (http://en.wikipedia.org/wiki/Keyboard_layout) to match the official keyboards used in different languages (QWERTZ in German, AZERTY in French etc.). Based on my tutorial, you will be able to define any keyboard layout – not just the official ones.

One of the drivers that made me write this was the complete lack of any kind of a tutorial or even usable forum posts on these questions (make a Google search for any of the Registry subkey names!). The available ones (see for example THIS (http://www.modaco.com/WM6-Rom-keyboard-layout-t256039.html)) are pretty much useless. That is, this article, as with most of my other articles, is the first REAL article / tutorial / description of all these issues never ever published by anyone before.

Note that this article only refers to touchscreen-less MS Smartphone (Windows Mobile Standard) devices, NOT Pocket PC’s. Should you want to know how you can redefine the default word completion directory used by Pocket PC’s, check out for example THIS (http://www.pocketpcthoughts.com/forums/viewtopic.php?p=365548) article of mine. In addition, should you want to know how to redefine the built-in thumbboard layout, just read the “Change slide-out keyboard layout” section of the related XDA-Dev Wiki page (http://wiki.xda-developers.com/index.php?pagename=Changing_Device_Language).

1. Adding new XT9 dictionaries

My biggest problem with my TC Vox (s710) was the lack of a Finnish dictionary, as I do most of my chatting in either English or Finnish. The English dictionary works just great and really greatly increases the text entering speed; this is why I’ve tried very hard to find a Finnish dictionary. This forced me to discover how the XT9 dictionaries work and how additional dictionaries can be registered.

First, some good news. Unlike with the Pocket PC platform, you can easily switch between the XT9 dictionaries by just long-pressing the * dial button (or, on the keyboard, the Sym + Space keys) and just selecting the new dictionary. With the Pocket PC, you couldn’t do this very easily – you would need to quickly change the two Registry values HKCU\ControlPanel\ Sip\DictFile and HKCU\ControlPanel\ Sip\ProbFile. (Of course, using a scripting language / environment like the excellent nScriptm either executable from Start / Programs, you can quickly create a program that switches between the two (or more) dictionaries quickly. You can even assign this file to a hardware button. See my past nScriptm-related articles on how this can be done.)

First, an elaboration on how this all works. Note that you will NOT need this in most cases, unless you want to add your DLL’s not existing in the two CAB’s provided in the next section; in this case, you can safely skip this section and move to the next, 1.2.

1.1 In the Registry

First, some advanced stuff. Let me clarify how the Registry keeps track of the different dictionaries.

1.1.1 eT9\AvailableLanguages

First, let’s have a look at HKEY_LOCAL_MACHINE\ Software\Tegic\eT9\ AvailableLanguages. This lists the supported languages. That is, if a given language is enabled in here, then, it will be selectable in the dictionary selector menu (regardless of whether its dictionary DLL really exists or not).

The language codes here come right from the official ISO country codes (listed for example HERE (http://www.microsoft.com/globaldev/reference/winxp/xp-lcid.mspx), in the LCIDHex column), with the exception of 04 being changed to 01.

That is, while the official language code of English is 0409, you must use 0109 to refer to English in here. The same stands for all other languages: for example, French is 040C; therefore, it must be referred to as 010C in here.

Note that the order of the numbers of the different languages is also important. For example, if you want to make sure French is the first, just move 010C to the first place of the list in this Registry value; then, the latter will be something like this:

010C,0109,0107,0110,010A,0113,0105,0108,0118,0115,0116,011B,011F,010E,011D,0114,010B,0106

(This registry import script is also available HERE (http://www.winmobiletech.com/072007SmartphoneI18n/FrenchFirst.reg), should you want to play with it.)

Doing this will result in French being offered as the first in the list:

http://www.winmobiletech.com/072007SmartphoneI18n/FrenchAssFirst.png

You will need this information particularly useful when you want to move a language otherwise not in the first section (for example, not accessible with a phonepad shortcut key). For example, this THIS (http://www.winmobiletech.com/072007SmartphoneI18n/FinnishAtTheEnd.png) screenshot, you can see Finnish is at the end of the list - you both need a lot of scrolling (even if you press Up at first) to be able to access it and you can’t use a keypad shortcut to activate it either. To fix the problem, you only need to find the language code for Finnish (040B ), change the first 04 to 01 and just move the resulting 010B to the beginning of the list as can, for example, be seen in the following screenshot:

http://www.winmobiletech.com/072007SmartphoneI18n/FinnishAsFirst.png

(registry import file available HERE (http://www.winmobiletech.com/072007SmartphoneI18n/FinnishFirst.reg); in it, the list is as follows: "010B,010C,0109,0107,0110,010A,0113,0105,0108,0118,0115,0116,011B,011F,010E,011D,0114,0106")

See how easy is to bring the languages you REALLY need to the beginning of the language list?

Note that, as with all the other Registry hacks in this article, contrary what some people state (for example HERE (http://www.modaco.com/WM6-Rom-keyboard-layout-t256039.html)), the changes will be visible as soon as you modify the Registry. You do NOT need to restart your handheld.

Now, let’s move on to providing the real dictionaries for the system.

1.1.2 eT9\IME\KdbDlls

As one can easily guess, dictionaries take up a lot of memory (in general, between 100 and 400 kbytes); therefore, more scare languages don’t come with all Smartphones to conserve ROM memory. (For example, a Smartphone sold in the U.K. doesn’t necessarily need the Swedish or the Finnish dictionary built-in; therefore, they’re left out.)

This means, in addition to listing the language in eT9\AvailableLanguages (see section 1.1), you must also provide the given DLL for the system.

To do this, the given DLL, firs, must be got from somewhere (for example, extracted form a ROM of a localized (non-general) Smartphone having it) and, after being copied to the \Windows directory of your handheld, must be listed under the HKEY_LOCAL_MACHINE\ Software\Tegic\eT9\ IME\KdbDlls subkey.

The values in here are pretty straightforward: their names end up at the lower byte of the country code and their values contain the name of the DLL. For example, with English (language code in hex: 0409; therefore, lower byte: 09) will be represented like this:

"00010609"="eT9.Dis.9Keys.English.kbd.dll"

The upper three bytes will always be 000106 with eT9.Dis.9Keys DLL’s and 000107 with the much rarer (for example, Greek) eT9.HQD.Libra DLL’s.

1.2 Ready-made dictionaries

Should you want to avoid direct Registry editing, all you need to do is importing the CAB file containing the language dictionary you need. For example, if you need one or more of the English, French, German, Italian, Spanish, Dutch, Czech, Greek, Romanian , Polish, Portuguese, Slovak, Turkish, Hungarian set, you can just download and install the CAB file provided HERE (http://www.modaco.com/Paul-s-Vox-eT9-Language-P-t254428.html) (for the HTC Vox / s710) or HERE (http://www.modaco.com/Excalibur-eT9-Language-Pa-t255603.html) (HTC Excalibur / s620).

Should you, on the other hand, need Scandinavian languages (yes, including Finnish, Mr. Rutkowski ;) ), you’ll need THIS (http://www.winmobiletech.com/072007SmartphoneI18n/Scandinavian_XT9.cab) file, originally compiled by XDA-Developers forum member mriz (http://forum.xda-developers.com/showthread.php?t=318744). (I provide a link to the version hosted by me because mriz hosts his version on Rapidshare, which is known to take down its downloads after a while.)

A screenshot of the language list is HERE (http://www.winmobiletech.com/072007SmartphoneI18n/Alt9lLanguages.png); an in-action screenshot (http://www.winmobiletech.com/072007SmartphoneI18n/Finnisht9.png) with the Finnish dictionary.

1.2.1 A caveat: very important!

When you install ANY language pack, HKEY_LOCAL_MACHINE\ Software\Tegic\eT9\ AvailableLanguages (see section 1.1) will be overridden and the old dictionaries NOT present in the just-installed one will just disappear! Therefore, you MUST for example save the original contents of this entry and, after installing ANY language pack, just concatenate these numbers to the new ones.

For example, if you install the Scandinavian pack on a handheld that previously had the MoDaCo multilingual pack, eT9\AvailableLanguages will contain only the following after the import (that is, four languages; not even English):

011D,0114,010B,0106

instead of the combination of both packs, listing all the languages in both of them; that is,

0109,010C,0107,0110,010A,0113,0105,0108,0118,0115,0116,011B,011F,010E,011D,0114,010B,0106

(I’ve used italic to denote the languages installed by the previously and, by bold, the Scandinavian pack.)

Therefore, you’ll need to re-add the, now, invisible languages to eT9\AvailableLanguages. Should you be aware of this (by manually editing the Registry), just use for example THIS (http://www.winmobiletech.com/072007SmartphoneI18n/FrenchFirst.reg) registry import script.

1.2.2 Compatibility

These files worked just great on my HTC Vox. They, however, didn’t work at all on my WM5 AKU3 HTC Oxygen (s310).

2. Redefining the layout of the built-in hardware keyboard (thumbboard)

Unlike on the Pocket PC, where you could quickly swap the keyboard layout by just editing (as is described in the “Change slide-out keyboard layout” section of the related XDA-Dev Wiki page (http://wiki.xda-developers.com/index.php?pagename=Changing_Device_Language))

With this platform, the situation is a bit more complicated: you must supply the full keyboard mapping file and, in addition, you must also register it in the Registry.

2.1 Keyboard layout files

The recommended (but not compulsory) naming convention for a keyboard mapping file is the following:

eT9.Vox.XXXX.kmap.txt

where XXXX is the official language code (see the already-linked MSDN page (http://www.microsoft.com/globaldev/reference/winxp/xp-lcid.mspx) for them) and you can change “Vox” to “Excalibur” on the HTC Excalibur (or anythign similar on other, keyboard-enabled devices). Note that, the Smartphone platform (as opposed to Pocket PC) lacking related file name conventions (you must explicitly refer to the keyboard mapping file in the Registry), it can be named anything. I only use this name to be compliant with the naming conventions used by the Vox version of the MoDaCo dictionary installer.

The contents of these files may seem a bit complicated at first but you will soon be able to navigate around. For example, let’s have a look at the following (Greek) configuration file:

http://www.winmobiletech.com/072007SmartphoneI18n/GreekFile.png

As one can see, there are five fields (referred to as {VK VK' CH1 CH2 KeyIndex} by the code samples) in each record defining a button. The first of them just lists what is physically printed on the key and doesn’t need to be touched – it’s only for human consumption, to give you a helping hand when editing the keyboard map file. The second is more important: it’s what you will want to modify when defining a different letter to be input when you press a button. The third and fourth fields define the symbols produced by Fn + key and the symbols entered in 123 mode (the latter two is the same). Finally, you don’t need to touch the last field either.

Note that you can entirely leave the last field (KeyIndex) off (as is in the German layout I provide below). Then, only the first field in the record will be used to determine which key you’re talking about. An example of this (with the German layout):

{Q Q @ @}
{W W ! !}
{E E 0 0}
{R R 1 1}
{T T 2 2}
{Y Z 3 3}
{U U + +}
{I I / /}
{O O ( (}
{P P ) )}
{A A ? ?}
{S S * *}
{D D 4 4}
{F F 5 5}
{G G 6 6}
{H H - -}
{J J _ _}
{K K : :}
{L L ; ;}
{Z Y 0x09 Z}
{X X # #}
{C C 7 7}
{V V 8 8}
{B B 9 9}
{N N 0xFA N} // 0xFA = Launch symbol window
{M M 0xFB M} // 0xFB = Launch SMS
{0xBE . , ,} // 0xBE = VK_PERIOD
{0x26 0x26 0x21 0x26} // 0xF1 = up arrow, 0x21 = page up
{0x28 0x28 0x22 0x28} // 0xF2 = down arrow, 0x22 = page down
{0x25 0x25 0xFD 0x25} // 0xF3 = left arrow, 0xFD = launch pIE
{0x27 0x27 0xDE 0x27} // 0xF4 = right arrow, 0xDE = '

Let’s take a closer look at the Z and the Y records above: that is, the two records you need to modify to get the German layout out of the English one (on limited keyboards not having a number row and non-alphabetic keys):

{Z Y 0x09 Z}

{Y Z 3 3}

As you can see, if you need to instruct the handheld to enter a ‘Y’ character when you press the “Z” button, you need to modify the “Z” record.

(Note that this map file is HERE (http://www.winmobiletech.com/072007SmartphoneI18n/eT9.Vox.0407.kmap.txt) and the accompanying Registry import file – see the next, 2.2 section – HERE (http://www.winmobiletech.com/072007SmartphoneI18n/GermanKeyboardEnable.reg))

2.2 Registering keyboard layout files in the Registry

Only providing keyboard layout files (see section 2.1) is insufficient. You must also register these files in the Registry so that the system knows you use a, from English, different keyboard layout.

To do this, you must register they layout file under both HKEY_LOCAL_MACHINE\Software\ Tegic\eT9\IME\ KeyboardLayouDlls and HKEY_LOCAL_MACHINE\Software\ Tegic\eT9\IME\ LanguageKeyboardIDMapping.

With the former, you must create a new value named 000007XX, where XX is the lower byte of the language code (for example, 07 for (the 0407 hexa) German), with the value containing the map file (for example, for a German map, "eT9.Vox.0407.kmap.txt").

With the latter, you must map the language and the keyboard layout together. To do this, you must create a value named after the language ID (with the changes outlined in Section 1.1.1; that is, changing 4 to 1 in the upper byte). Its value must be the lower (16-bit) word of the KeyboardLayouDlls registration – as a String (incidentally, as with, for example, the Pocket PC keyboard language string under HKCU\ControlPanel\ Keybd\Locale).

That is, with the German example, the two new values you must create are the following:

[HKEY_LOCAL_MACHINE\Software\Tegic\eT9\IME\KeyboardLayouDlls]
"00000707"="eT9.Vox.0407.kmap.txt"

[HKEY_LOCAL_MACHINE\Software\Tegic\eT9\IME\LanguageKeyboardIDMapping]
"0107"="0707"

Again, the value of the second value refers back to the first.

Note that, unfortunately, defining a keyboard mapping file to fix the well-known, annoying HTC Vox bug affecting many games / emulators (for example, ScummVM (http://www.pocketpcmag.com/blogs/index.php?blog=3&p=2060&more=1&c=1&tb=1&pb=1)) won’t work. Yes, I’ve tested this (HERE (http://www.winmobiletech.com/072007SmartphoneI18n/NoNumbers.txt) is the file completely switching off numbers in Fn mode) and it didn’t work – pressing the E, R, T, Y, D, F, G, C, V and B keys on the keyboard still resulted in the numbers 0, 1,… 9 being input.

2.3 What about AEKMap (http://ae.inc.ru/aekmap.php)?

The well-known AEKMap has gone a LONG way since its first versions (then, only usable with external USB keyboard). This, unfortunately, also means it has gone commercial – at least as far as accessing its advanced features are concerned. The current version is pretty powerful and also supports Smartphones. Unfortunately, very few ready-made Smartphone-related config / layout files are available now, which means you will end up having to write one yourself if you want to use a non-English mapping.

http://www.winmobiletech.com/072007SmartphoneI18n/Aekmapmain.png

As I find the non-AEKMap way is more system-friendly, I’d, myself, prefer defining your keyboard layout using it, the traditional way. However, you will still want to give AEKMap a try to see whether you can use its advanced capabilities.

3. Tips for ThinkOutstide StowAway keyboard support

Unfortunately, currently, the StowAway keyboards by Mobility Electronics, Inc. (ex- ThinkOutstide) are, officially, not supported by the manufacturer on several Smartphone models. However, this is in no way a problem: if you have a HTC Vox, just download the driver meant for Excalibur (accessible HERE (http://www.thinkoutside.com/support/Drivers.asp?ManufacturerID=14&ModelID=501)), it’ll work just great. The same may stand for other, officially, not supported models - just give a try to drivers meant for other smartphones (preferably running the same operating system version).

This also means you can safely redefine the keyboard layout by both ThinkOutstide’s official tool (which must be requested by mail as can be read HERE (http://www.thinkoutside.com/forums/toast.asp?sub=show&action=posts&fid=5&tid=416)) and, most probably, AEKMap.

Menneisyys
09-27-2007, 04:15 PM
UPDATE (09/27/2007): there are other language packs HERE (http://melgurth.ovh.org/index.php?page=downloads).
(end of update)

Menneisyys
09-27-2007, 04:24 PM
The just-posted link ( http://melgurth.ovh.org/index.php?page=downloads ) also has a downloadable T9 word editor.