Extract contacts names and numbers from ActiveSync and HTC Xbackup backup files

[Menneisyys]

Over at a non-English PDA board, I was asked to help with a pretty common problem (http://www.pdamania.hu/forum/forum1.php3?szuro=4&topic=12223): extracting WindowsCE databases from ActiveSync STG files. Unfortunately, the only tool to be able to extract stuff from STG files, STGViewer (http://www.pocketpcfreewares.com/en/index.php?soft=354), is unable to do this – it can only extract files, but not databases.

Unfortunately, ActiveSync is pretty useless when it comes to restoring backup files to other device types (even after a ROM upgrade), and, unlike with FSC Backup, it can't be hacked either (see this thread (http://www.firstloox.org//forums/showthread.php?t=3585) on making FSC Backup cross-restore backup files) to do so. (Not to say Sprite Backup, the by far the best Pocket PC backup apps, also supports cross-restoration).

This is pretty annoying with the Windows CE PIM databases, which are 100% compatible and, therefore, could be freely restorable on Pocket PC's running the same Windows Mobile version.

This is why I've set out to scrutinize the more closely the STG format and succeeded in finding the contacts database and writing a tool that does this automatically.

The utility, finally, has turned out to be not only compatible with STG, but also with the HTC Xbackup Contacts format. On the other hand, it won't restore FSC Backup (it stores the name/number pairs in the opposite direction) and iPAQ/Sprite (the number/name pairs are are separated by several bytes) contacts, however.

Getting it:

It's, as usual with my apps, free. Download the Java class from here (http://winmobiletech.com/sekalaiset/stg/STGContactNumberAndNameExtract.class). As usual, I also make the source code freely available, get it from here (http://winmobiletech.com/sekalaiset/stg/STGContactNumberAndNameExtract.java).

Usage:

If you already have a Java Virtual Machine on your desktop PC (if you don't have one, get it from here (http://java.sun.com/j2se/1.5.0/index.jsp). Note that, if you want to reduce bandwidth/disk usage – older JVM's are smaller – the app will work even with older JDK versions – even with JDK 1.1), just issue the

java STGContactNumberAndNameExtract inputfile outputfile.csv

command (using any inputfile and outputfile name; the latter should have the csv extension).

After doing this, go to Outlook and in the Contacts view, go to File/Import and Export; choose Import from another program or file, click Next, choose Comma Separated Values (Windows) and click Next again. After this, it'll be pretty easy to go on.

It's highly unlikely I'll rewrite the tool into C# today: I'm really busy and I'm already late from swimming. Tomorrow I'll have tons of meetings + swimming + sauna so won't really have time either.

Other stuff:

You may also want to check out this thread (http://pocketpcmag.com/forum/topic.asp?TOPIC_ID=16922) on my Xbackup file extractor tool – you may find it useful if you have a HTC Phone Edition device.

Also, here're my posts on Yahoo -> Outlook/PDA export (http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=17393). Note that it's a bit outdated: when I wrote it, Yahoo had no explicit Outlook-specific export capabilities. The current Outlook export file is 100% compatible with Outlook on the desktop.

[carrausse]

Hello Menneisyys,

I've tried to use your tool, but I have the following message :

J:\Utils\PocketPC\STG Tools>dir
Volume in drive J is MAXTOR
Volume Serial Number is 6818-DA2F

Directory of J:\Utils\PocketPC\STG Tools

06/01/2006 06:33 <DIR> .
06/01/2006 06:33 <DIR> ..
05/11/2005 02:28 17,185,792 Backup_20051105.stg
06/01/2006 06:25 2,086 STGContactNumberAndNameExtract.class
2 File(s) 17,187,878 bytes
2 Dir(s) 63,947,157,504 bytes free

J:\Utils\PocketPC\STG Tools>java STGContactNumberAndNameExtract.class Backup_200
51105.stg Backup_20051105.csv
Exception in thread "main" java.lang.NoClassDefFoundError: STGContactNumberAndNa
meExtract/class

Do you have any idea if this error could come from the application or from a misuse or a pb on my pc?

(I've just downladed the latest Java package from Java.com)

Best greetings...

Alex
London

[carrausse]

Hi again,

Just a quick thought :

The word "lang" "in java.lang.NoClassDefFoundError" make me thinking it might be a problem with language or regional settings...

My PC runs XP SP2 in US/English version and regional settings are "United Kingdom".

My PDA (a HP iPaq 2210) is a UK version but at the time the backup file was created it was under "French settings". I had trouble restoring it after a hard reset and I was able to restore it after changing regional settings to "French" on the PDA. Unfortunately even though the restore is successfull, the PDA is not starting (freeze when boot bar reaches 100%).

You can see this previous post about my problem:
http://www.ipaqhq.com/forums/showthread.php?t=25433

I found your tool by browsing Google and I was expecting it could save my life by allowing to export my business contacts (I don't have any other source than this bloody .stg file)....

Hope you will read this...

Cheers,

Alex

[Menneisyys]

J:\Utils\PocketPC\STG Tools>java STGContactNumberAndNameExtract.class Backup_200
51105.stg Backup_20051105.csv
Exception in thread "main" java.lang.NoClassDefFoundError: STGContactNumberAndNa
meExtract/class

Do you have any idea if this error could come from the application or from a misuse or a pb on my pc?

Just leave .class from the command so that the command looks like as follows:

java STGContactNumberAndNameExtract Backup_200 51105.stg Backup_20051105.csv

[carrausse]

Thanks for your reply.

I've tried without the .class but I have the same error :

J:\Utils\PocketPC\STG Tools>java STGContactNumberAndNameExtract Backup_20051105.
stg Backup_20051105.csv
Exception in thread "main" java.lang.NoClassDefFoundError: STGContactNumberAndNa
meExtract

Any idea?

Alex

[Menneisyys]

Thanks for your reply.

I've tried without the .class but I have the same error :

J:\Utils\PocketPC\STG Tools>java STGContactNumberAndNameExtract Backup_20051105.
stg Backup_20051105.csv
Exception in thread "main" java.lang.NoClassDefFoundError: STGContactNumberAndNa
meExtract

Any idea?

Alex

Then, if it's in the current directory, it seems it isn't in CLASSPATH. Then, use the following command:

java -classpath <current directory path> STGContactNumberAndNameExtract Backup_20051105.stg Backup_20051105.csv

[carrausse]

Thanks Menneisyys!

This time it worked but the results where not those I expected, there is only two columns : "Name","Mobile Phone", I don't have all my contacts (only a hundred instead of more than 2000), and the contacts seems corrupted (lots of name are missing and most of the numbers are not telephone number)...

Here the file :

"Name","Mobile Phone"
"","5001736"
"","3002560"
"","1001979"
"","1001978"
"","1001818"
"","1002094"
"","1001977"
"","2001763"
"","1001976"
"","2001762"
"","1001975"
"","2001761"
"","9001846"
"","1001974"
"","1001879"
"","1001814"
"","1001973"
"","1001972"
"","1001877"
"","1001971"
"","5003733"
"c0685836707","0130770878"
"","2002017"
"","1001970"
"","1001810"
"","2002016"
"","1001873"
"??????????? ","?????????"
"","2001959"
"","4001739"
",Delphine","0616302197"
"","1002189"
"","1001969"
"","1001968"
"","7002015"
"","1001967"
"","1001966"
"","1001806"
"","2001751"
"","1001965"
"Tomlins, Simon","+441442884560"
"","2001750"
"Crossman, Andrew","07812733939"
"","1001802"
"","1001867"
"b /Security\Software\Microsoft\AirSync\Connection","0010409"
"","1001865"
"g016-211-6520","3225107291"
"","1001861"
"??????????","????????"
"","9001891"
"???????????????????????????????????","????????????"
"","2001749"
"","2001748"
"","2001747"
"","2001745"
"Nigeria, Ambassade de France","095235510"
"","2001744"
"eAussibal, Stephane","0492114414"
"","2001743"
"","2001742"
"","2001741"
"","2001740"
"","8002156"
"","1001856"
"","5001577"
"","1001853"
"?????????????","?????????"
"??????????????? ","???????????"
"","6003014"
"","9002032"
"CYou, Yeunsu","0182920667"
"","2001795"
"","2001794"
"","2001793"
"","2001792"
"","2001791"
"","2001790"
"","1001844"
"","1001840"
"%d","0000000"
"85293039829","85225205225"
"","3002035"
"r","+2348033118315"
"","1003255"
" 0607727672","0142661205"
"Leroy, Philippe","09015385332"
"","2001789"
"","2001788"
"","2001786"
"","9000866"
"","2001785"
"","1001998"
"","1001838"
"","1001997"
"","1001996"
"","8001684"
"","1001995"
"","2001781"
"","1001994"
"","1001834"
"","1001993"
"","1001992"
"","1001897"
"","4002072"
"","1001990"
"","1001830"
"","2002130"
"","9001860"
"?????????????????r","08003160245"
"","1001893"
"","7001889"
"a+6598249625","+33615316253"
"","1002147"
"","1003894"
"","6002051"
"????????? ","???????"
"85228267878","85228267336"
"","7002133"
"","1001989"
"","1001988"
"??????????","??????????"
"","1001987"
"","1001986"
"","6001639"
"","1001985"
"eCarteret, Olivier","0404101220"
"","1001984"
"","1001983"
"","1001982"
"","1001822"
"","1001887"
"","1001981"
"","8002183"
"","1001980"
"","1001883"
"","5002546"
"Holt (Podium T2), Bryan","+442087456855"
"","6003791"
" Gueneau, Catherine","0143662294"

Any idea?

Alex

[Menneisyys]

Thanks Menneisyys!

This time it worked but the results where not those I expected, there is only two columns : "Name","Mobile Phone", I don't have all my contacts (only a hundred instead of more than 2000), and the contacts seems corrupted (lots of name are missing and most of the numbers are not telephone number)...


I'll try to fix this in the near (1-2-3 days) future (also with extending the functionality to extract everything); I'll let you know when I'm ready.

[Menneisyys]

I have some very good news for you: much as STG Viewer can't restore WinCE databases, a very old title, HPC Vault, from the Handheld PC times, can.

It must be used just like STG Viewer: you open the STG backup file. Then, just click the WinCE Databases folder in the left pane and, in the right pane, highlight the database(s) you want to restore (in your case, Contacts Database) and choose Restore to HPC:

click for screenshot (http://www.winmobiletech.com/sekalaiset/HPCVaultRestoreDatabase.png)

Before doing so, make sure your PDA doesn't currently run Contacts and it's connected via ActiveSync. This is because HPC Vault is pretty weak at error reporting: it won't report any error message when there's no currently connected PDA and/or it can't restore the given database onto it because an application is locking the database on the PDA.

Please note that you should not right-click the three in the left pane to, for example, restore all databases as easily as possible (without highlighting them in the right pane): it will result in a crash.

You can download the trial version from Tucows (http://www.tucows.com/get/32503_45679). To be on the safe side (the file is no longer hosted by the original developer and is, therefore, long forgotten), I've also uploaded it on my site (http://www.winmobiletech.com/sekalaiset/hpcvault.zip) so that later access can be guaranteed.

[Menneisyys]

I've also posted an extended version of this tutorial to my blog: http://www.pocketpcmag.com/blogs/index.php?blog=3&p=442&more=1

[WiedseL]

Can you explain to me how to make sure my contacts aren't running on my PPC. I tried a safe mode and checked my running programs....
HPC Vault doesn't work either, probably because of the same reason recovering doesn't work.

[lumbanraja]

Hi Menneisyys,

just like many others (as I've seen afterwards in the net) I've busted my contacts database with an AS's backup. And OF COURSE I did NOT sync it before with Outlook. So I've tried HPC Vault, but if I select on the contacts database "Restore to HPC" the status window lits up for a 10th sec. and nothin happens. But then I found your java app, and I had to insert the classpath-thing just like "carrusse" had to.
But now I get the VM "Access denied".

Here's my cmd and the VM:

D:\Robin\Backups>java -classpath <D:\Robin\Backups> STGContactNumberAndNameExtract Backup.stg robin.csv
Zugriff verweigert (that means:Acces denied)

I'd really apreciate if you, or anyone else, could help e ou of this...

Thanxx

[lumbanraja]

Well, I finally made it...
The problem was that I had the files on my data drive...
I've given it one more try and copied both fies, the java applet and the backup.stg direct on my C:-drive ==> it worked!!
Since it might be interesting for somebody out here, I just would like to tell what happened thereafter.
I couldn't import the created csv-file correctly since my outlook didn't
realize the komma-seperated values (Name, "Mobile Phone") as two different fields and so only imported the names into my contacts without the numbers.
So I drag'n dropped the whole data from the csv-file (in Excel) into a simple txt-file (Editor) and saved it. Thereafter I imported the txt-file as semicolon seperated value (Windows) into my outlook and most of my contacts were back!!!
Well, some of them still are missing ore displayed somehow incorrect, e.g. numbers within the names or some numbers were added to the phone numbers, but it took me just like 20 minutes to get that fixed manually.
Finally I just wont to say a BIG THANK YOU to Menneisyys, for giving me the possibility of getting my contacts back. :lol:
At least one person out there, who fixes up Gill Bate's f***ing crab....

Greetz

lumbanraja

[adamfinn]

Hi
Thanks for the clear and helpful advice in the thread and blog.
This response is a little tangential but you may be able to help...
PocketPC (Imate) smartfone crashed and reset (as it often does).
Restore from .stg file brought back settings but no database info (calendar, contacts, tasks) (as it doesn't)
Sync with desktop Outlook - instead of bringing back all the contacts to the fone, did the opposite and deleted them all from Outlook.
:-( (& why, oh why was it so stupid as to imagine I wanted it to do this??)
I wonder what Activesync setting I need to change to prevent this happening next time I'm in this situation..
Outlook, it seems, was not backed up...
:-(
Using HPC Vault, following your advice, confirmed my worst fears - as soon as Activesync had finished synching/deleting all the contacts, it immediately did a new backup and saved the new, empty, contacts file in the new .stg over the old one.
:-(
Seemed like the whole thing had collapsed house-of-cards-like.
Then... when I opened Outlook on my laptop - taking care NOT to go on line via Outlook connector (we are an Oracle Calendar institution..) and have the same thing happen again - there all the contacts were, looking at me! Goodness knows how they got there!
Trouble is, no one seems to be able to figure out where Outlook, off line, actually keeps this information nor, indeed, where the laptop could have got it from!!
I know it's not really a Pocket PC problem by now, but, I guess, if nothing else, it's a cautionary tale on how easy it is to have this happen. If you know how I can get these data off the laptop/Outlook and into the smartfone, or you know someone who might know, please let me know!
Otherwise it's going to be fun typing them all back in by hand...
Best wishes
Adam Finn

[chuckf]

Hi Menneisyys,

I have the same failure as carrausse.
How did you figure out the format of the .stg file?
Did you just look at it with a hex editor?
Is there Microsoft documentation?
I'm not a java programmer but it looks like your looking for a number of length 7 through 15 digits followed by a string that would be the name.
I'm not familiar with UTF-16LE format. I'll have to read up on that.
Any info at all would be appreciated.
Thanks,
chuckf

[ericlowe]

Hi Menneisyys,

I have downloaded STGViewer Vo02 ( Although it says it is 0.1b in help).

I have had trouble with restoring the backup.stg file which was created just 1 week ago. On completion of the restore I have 15 file errors made up mostly of dll's but importantly my contacts database. I would therefore like to restore these using SGTViewer. My Calendar and Notes are OK but one of my two programs that I have added is corrupt,

I can get as far as seeing the contacts database and the files therein. But, when I highlight the files and then right click and select restore files the drop down box shows all my folders etc in 'My computer' but does not show my mobile device (XDA2i) so I cannot select the place to restore to. Also copy / past will not work, I can highlight / copy but when I want to paste the option does not show.

THE CAUSE
The sequence of events that have caused me to have a catastrophic failure are:

1. Problem 1 – Backed up Outlook pst file.

Outlook. Because I have a slow computer I try to do good PC housekeeping. Part of this was to use the Microsoft defragmentation tool which works very well. However, this showed that I had 21% fragmentation and gave me a list of 7 files that were causing this. I was able to delete 5 of them but the other 2 were a copy of my Outlook.pst file and the working Outlook.pst file. I then cut and pasted (moved) both copies to a separate hard drive. Carried out a C:/ drive clean up and a defragmentation which came down to 7%. On replacing the working pst file there were major problems and after using the Inbox repair tool I ended up with Notes ok but most e-mails lost, corrupted contacts and no Calendar. I am still trying to recover everything form the original pst back up file. I have tried a free recovery tool called pst-repair-t.exe, but this was only marginally better than the Inbox repair tool, with no contacts restored which is vital to me. I am not sure where contacts, calendar and notes these are stored by Microsoft.

2. Problem 2 – Activesync synchronisation PC with mobile (XDA2i)

Activesync, to synchronise mobile information back into Outlook. I then tried to use activesync to get my Contacts and Calendar (Notes was OK) back into Outlook. I chose the normal option to merge. This lead to the wiping out of all my contacts on my mobile, all 2,000+!

3. Problem 3 – Activesync restore from backup to mobile (XDA2i)

Activesync, to restore data to my mobile. I had a 1 week old backup of my mobile data and I tried to restore this data. This gave me 15 errors where many were dll’s but most importantly my Contacts database. This is where I found the STGViewer.exe viewer on your website but cannot get it to work to reinstate my 15 files.

So all of these above problems are caused by the multiple failures of Microsoft products, especially as I had backups. Can you help with any or all of my problems please as I use my e-mails and my phone for business and this is serious to me?

I have spent hours on the internet with GOOGLE and including Microsoft and O2, in trying to resolve these problems.

SUMMARY

I need to be able to restore from my backed up

A. outlook.pst file to my PC. To reinstate my e-mails, contacts, calendar and notes

B. backup.stg to my mobile file. To reinstate contacts, calendar and notes and provide a full restore to all 15 failed files.

C. Fully synchronise my PC/Mobile once again.

I am low tech so simple explanations would help please?

This is also my first ever forum posting so not sure of protocols.

Many thanks.

Eric
:confused:


Over at a non-English PDA board, I was asked to help with a pretty common problem (http://www.pdamania.hu/forum/forum1.php3?szuro=4&topic=12223): extracting WindowsCE databases from ActiveSync STG files. Unfortunately, the only tool to be able to extract stuff from STG files, STGViewer (http://www.pocketpcfreewares.com/en/index.php?soft=354), is unable to do this – it can only extract files, but not databases.

Unfortunately, ActiveSync is pretty useless when it comes to restoring backup files to other device types (even after a ROM upgrade), and, unlike with FSC Backup, it can't be hacked either (see this thread (http://www.firstloox.org//forums/showthread.php?t=3585) on making FSC Backup cross-restore backup files) to do so. (Not to say Sprite Backup, the by far the best Pocket PC backup apps, also supports cross-restoration).

This is pretty annoying with the Windows CE PIM databases, which are 100% compatible and, therefore, could be freely restorable on Pocket PC's running the same Windows Mobile version.

This is why I've set out to scrutinize the more closely the STG format and succeeded in finding the contacts database and writing a tool that does this automatically.

The utility, finally, has turned out to be not only compatible with STG, but also with the HTC Xbackup Contacts format. On the other hand, it won't restore FSC Backup (it stores the name/number pairs in the opposite direction) and iPAQ/Sprite (the number/name pairs are are separated by several bytes) contacts, however.

Getting it:

It's, as usual with my apps, free. Download the Java class from here (http://winmobiletech.com/sekalaiset/stg/STGContactNumberAndNameExtract.class). As usual, I also make the source code freely available, get it from here (http://winmobiletech.com/sekalaiset/stg/STGContactNumberAndNameExtract.java).

Usage:

If you already have a Java Virtual Machine on your desktop PC (if you don't have one, get it from here (http://java.sun.com/j2se/1.5.0/index.jsp). Note that, if you want to reduce bandwidth/disk usage – older JVM's are smaller – the app will work even with older JDK versions – even with JDK 1.1), just issue the

java STGContactNumberAndNameExtract inputfile outputfile.csv

command (using any inputfile and outputfile name; the latter should have the csv extension).

After doing this, go to Outlook and in the Contacts view, go to File/Import and Export; choose Import from another program or file, click Next, choose Comma Separated Values (Windows) and click Next again. After this, it'll be pretty easy to go on.

It's highly unlikely I'll rewrite the tool into C# today: I'm really busy and I'm already late from swimming. Tomorrow I'll have tons of meetings + swimming + sauna so won't really have time either.

Other stuff:

You may also want to check out this thread (http://pocketpcmag.com/forum/topic.asp?TOPIC_ID=16922) on my Xbackup file extractor tool – you may find it useful if you have a HTC Phone Edition device.

Also, here're my posts on Yahoo -> Outlook/PDA export (http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=17393). Note that it's a bit outdated: when I wrote it, Yahoo had no explicit Outlook-specific export capabilities. The current Outlook export file is 100% compatible with Outlook on the desktop.