Menneisyys
05-24-2005, 12:04 PM
Saving precious communication bandwidth using compression
Especially mobile phone-based communication may be very expensive unless you use a flat rate service. This is why data compression comes into picture.
Let's have a look at the most known TCP/IP protocols commonly used on any of today's computers.
The most important of them is HTTP, the protocol that requests and delivers Web pages and resources.
HTTP per se supports compression, but very few (if any!) of today's Web servers return compressed pages (due to the computational overhead that on-the-fly compression causes, unless the server also stores the given page in a compressed version, not only uncompressed). A solution for this is
1, using a ZIP'per proxy like RabbIT ( http://rabbit-proxy.sourceforge.net/ ) run on your (or your friends'), say, workplace desktop PC
2, using an online service like Skweezer, which has just introduced content compression. Please see http://www.pocketpcthoughts.com/forums/viewtopic.php?p=345365 on this question and my remarks on Skweezer and its alternatives.
Both of the above solutions have drawbacks.
First, it's highly unlikely you will be allowed to run a ZIPper proxy on any of your 24/7 desktop machines. Your work computer is most probably behind a corporate firewall which doesn't allow for running proxies. DSL or cable connections (if you wanted to run the ZIPper proxy server at home) may suffer from another problem: most of them use dynamic IP's that change, in general, in every 24 hours. Sure, you can use a dynamic IP lookup service, but you would end up changing the proxy address in your PDA after the IP of your home PC changes, which can be quite tedious.
Second, as for Skweezer (or any online service that does ZIP HTTP compression), it has other problems, which I've outlined in the above-linked PPCT thread. Most important is the need for manual cookie editing if you want to save cookies between sessions.
Third, if you also want to use SMTP, IMAP and/or POP3, the protocols in charge of sending/receiving e-mail, you run into another problem: these protocols don't support content compression (unlike HTTP), so, you can't use any kind of remote 'proxy server' (without a local, compression-enabled client) for guaranteeing that you only send/receive highly-compressed contents to reduce bandwidth usage.
With e-mailing protocols, therefore, the only solution is using a client run on the local Pocket PC (or just client device, in a broader meaning, let it be a Symbian, a Palm OS or a WinCE-based one) that use some proprietary protocol to communicate to a proprietary server run somewhere on the Internet. This server will then communicate with your POP3 mailbox and/or SMTP server.
While on the desktop Windows there're several solutions that make even mail crunching possible (for example, the Bytemobile Macara Optimization Client), on the Pocket PC platform, there haven't been any, up until now.
In my article, I explain how the toonel.net client, one of the first solutions to the problems outlined above, works and how it can be used on a Pocket PC device.
The clients available at http://www.toonel.net/ are written in Java. This is why, even without explicit Pocket PC support, they run on them too. Unfortunately, this also means that it requires a JVM to be run, which, taking the the meagre CPU power/memory of PDA's into account, isn't the best compared to a native implementation, but the performance/memory impact on today's hi-end Pocket PC's (for example, the 520 MHz and 128 Mbyte RAM-equipped Fujitsu-Siemens Pocket Loox 720 or the 624 MHz iPAQ hx4700/Dell Axim x50v) is (close to) negligable.
First, get the latest version from the Toonel Generic Swing Package section. At present, it's version 0.0.50.20 and is available at http://www.toonel.net/generic/005020/toonel.jar .
Please note that, technically, you don't need the Swing-compatible version at all because current Pocket PC JVM's don't support Swing without additional, manual hacking. It's just that the non-Swing-enabled (and, therefore, slightly smaller) distribution package (see the toonel.net PersonalJava section - the direct link to the downloadable file is http://www.toonel.net/pjava/005020/toonel.net.tar.gz ) has bugs; the most important of them is the lack of the Java package jzlib (its old name was syncjzlib and the non-GUI-based version isn't properly tested; this is why the package with the old name is still there.), which renders them useless.
Second, install a JVM on your PPC. In this tutorial, I only discuss the Jeode (Insignia) JVM delivered with some Pocket PC's and been purchased by many Pocket PC users. (Old versions of this JVM will run even on the latest devices with very minor problems because of the Palm-size PC (PsPC)-compliant GUI.)
Third, copy toonel.jar to \Windows\lib on your Pocket PC and create a link file with the following content in \Windows\Start Menu\Programs[\<any subdirectory name>[\<any subdirectory name>]]:
99#\windows\evm.exe -cp \windows\lib\toonel.jar client.WebUI -v
Run it. The program will create a sample initialization parameter file named toonel.ini in the root directory of your PDA. Transfer it to your desktop PC and edit the mail-related server entries so that it looks like the following:
# This is a sample toonel.ini file used by toonel.net client application.
# This file contains the mappings of IP ports.
# Each entry should be kept on an individual line and has the following format:
# [local port number];[host];[remote port number]
# Additionally, comments (such as these) may be inserted on individual lines
8080;127.0.0.1;7999
25;smtp.netti.fi;25
110;pop3.netti.fi;110
# 143;IMAP Server [Incoming mail sample];143
Pay special attention to the two e-mail related rows in bold; the HTTP-related record ("8080;127.0.0.1;7999") doesn't need to be modified.
Copy it back to the root of your PDA.
Fourth, set up the HTTP proxy setting of the active connection group used by the "The Internet" drop-down list. (To know more where to find this, please read my article on proxy-related article at, say, http://www.firstloox.org//forums/showthread.php?p=28878 and http://discussion.brighthand.com/showthread.php?s=&threadid=118113 ). As the HTTP proxy is listening on your local device at port 8080, fill in the HTTP input textfield in the Advanced section of the proxy setup:
http://menneisyys.freeweb.hu/toonel/HTTPProxySettings2.gif
Then, the basic page of this will be as follows:
http://menneisyys.freeweb.hu/toonel/HTTPProxySettings1.gif
Please note that you can change the HTTP proxy server any time. To make this as easy as possible, you may want to read the method I've invented and described in my article in the above-mentioned article.
Fifth, overwrite the incoming/outgoing mail server addresses in your mailer client to contain the word 'localhost' or, as with the above example, the IP address '127.0.0.1', which equals to 'localhost':
http://menneisyys.freeweb.hu/toonel/ToonelMailerSetup.gif
Please note that, unlike the built-in mailer clients of Symbian devices, which support arbitrary port numbers, the built-in Microsoft-shipped Pocket Inbox/Messaging client doesn't support them:
http://www.toonel.net/res/emailp800_8.gif
(Unfortunately, this can't even be registry-hacked.)
Therefore, if you plan to use several POP3 clients with different server addresses (!) with Toonel, you will want to switch to some other, more advanced mailer (POP3/SMTP) client with port defining capabilities for each account you set up. An example of these advanced mailer clients is WebIS Mail ( http://www.pocketinformant.com/products_info.php?p_id=mail&dir=wm& ).
Note that if you close/kill the JVM by clicking the X button (it not only minimizes it!), your PIE browser and mailer clients won't be able to connect to the net because they'll try connecting to the (non-existing) local proxy. Then, just re-run the proxy starter link.
Please note that Toonel will crash on the PPC if you try to use its setter HTTP page at localhost:7999. It's best to leave it alone - fortunately, \toonel.ini has all the settable parameters, so, you don't really need the on-line page.
Bottom line
Pros:
- free
- fast
- reliable
- no custom HTTP proxy server is needed
- compresses mail too
Cons:
- (comparatively) noticeable impact on the PDA performance because of the heavy-weight JVM
- doesn't compress images (this is not a major problem though because many GPRS providers - for example, Vodafone - also offers on-the-fly image crunching if you use their special APN)
To be continued:
1, using toonel with other JVM's (Créme, IBM etc), also with comparative benchmarks on JVM speed etc.
2, comparison of toonel to the native ARM solution developed by Globility( http://www.globility.co.nz ). See http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=15684 on the latter.
Please post your remarks here. Do you need additional clarification? Was there anything in the article that you haven't really understood?
The author is an expert in TCP/IP protocols and wireless communications. He has authored the IBM JavaBeans FTP Java bean and library back in 2000-2001 and has co-authored the POP3 and SMTP libraries in the same section. Has also authored several HTTP proxy and filtering servers and libraries; for example, the Mobipocket Web Companion Support Pack ( http://menneisyys.freeweb.hu/mwc ).
Especially mobile phone-based communication may be very expensive unless you use a flat rate service. This is why data compression comes into picture.
Let's have a look at the most known TCP/IP protocols commonly used on any of today's computers.
The most important of them is HTTP, the protocol that requests and delivers Web pages and resources.
HTTP per se supports compression, but very few (if any!) of today's Web servers return compressed pages (due to the computational overhead that on-the-fly compression causes, unless the server also stores the given page in a compressed version, not only uncompressed). A solution for this is
1, using a ZIP'per proxy like RabbIT ( http://rabbit-proxy.sourceforge.net/ ) run on your (or your friends'), say, workplace desktop PC
2, using an online service like Skweezer, which has just introduced content compression. Please see http://www.pocketpcthoughts.com/forums/viewtopic.php?p=345365 on this question and my remarks on Skweezer and its alternatives.
Both of the above solutions have drawbacks.
First, it's highly unlikely you will be allowed to run a ZIPper proxy on any of your 24/7 desktop machines. Your work computer is most probably behind a corporate firewall which doesn't allow for running proxies. DSL or cable connections (if you wanted to run the ZIPper proxy server at home) may suffer from another problem: most of them use dynamic IP's that change, in general, in every 24 hours. Sure, you can use a dynamic IP lookup service, but you would end up changing the proxy address in your PDA after the IP of your home PC changes, which can be quite tedious.
Second, as for Skweezer (or any online service that does ZIP HTTP compression), it has other problems, which I've outlined in the above-linked PPCT thread. Most important is the need for manual cookie editing if you want to save cookies between sessions.
Third, if you also want to use SMTP, IMAP and/or POP3, the protocols in charge of sending/receiving e-mail, you run into another problem: these protocols don't support content compression (unlike HTTP), so, you can't use any kind of remote 'proxy server' (without a local, compression-enabled client) for guaranteeing that you only send/receive highly-compressed contents to reduce bandwidth usage.
With e-mailing protocols, therefore, the only solution is using a client run on the local Pocket PC (or just client device, in a broader meaning, let it be a Symbian, a Palm OS or a WinCE-based one) that use some proprietary protocol to communicate to a proprietary server run somewhere on the Internet. This server will then communicate with your POP3 mailbox and/or SMTP server.
While on the desktop Windows there're several solutions that make even mail crunching possible (for example, the Bytemobile Macara Optimization Client), on the Pocket PC platform, there haven't been any, up until now.
In my article, I explain how the toonel.net client, one of the first solutions to the problems outlined above, works and how it can be used on a Pocket PC device.
The clients available at http://www.toonel.net/ are written in Java. This is why, even without explicit Pocket PC support, they run on them too. Unfortunately, this also means that it requires a JVM to be run, which, taking the the meagre CPU power/memory of PDA's into account, isn't the best compared to a native implementation, but the performance/memory impact on today's hi-end Pocket PC's (for example, the 520 MHz and 128 Mbyte RAM-equipped Fujitsu-Siemens Pocket Loox 720 or the 624 MHz iPAQ hx4700/Dell Axim x50v) is (close to) negligable.
First, get the latest version from the Toonel Generic Swing Package section. At present, it's version 0.0.50.20 and is available at http://www.toonel.net/generic/005020/toonel.jar .
Please note that, technically, you don't need the Swing-compatible version at all because current Pocket PC JVM's don't support Swing without additional, manual hacking. It's just that the non-Swing-enabled (and, therefore, slightly smaller) distribution package (see the toonel.net PersonalJava section - the direct link to the downloadable file is http://www.toonel.net/pjava/005020/toonel.net.tar.gz ) has bugs; the most important of them is the lack of the Java package jzlib (its old name was syncjzlib and the non-GUI-based version isn't properly tested; this is why the package with the old name is still there.), which renders them useless.
Second, install a JVM on your PPC. In this tutorial, I only discuss the Jeode (Insignia) JVM delivered with some Pocket PC's and been purchased by many Pocket PC users. (Old versions of this JVM will run even on the latest devices with very minor problems because of the Palm-size PC (PsPC)-compliant GUI.)
Third, copy toonel.jar to \Windows\lib on your Pocket PC and create a link file with the following content in \Windows\Start Menu\Programs[\<any subdirectory name>[\<any subdirectory name>]]:
99#\windows\evm.exe -cp \windows\lib\toonel.jar client.WebUI -v
Run it. The program will create a sample initialization parameter file named toonel.ini in the root directory of your PDA. Transfer it to your desktop PC and edit the mail-related server entries so that it looks like the following:
# This is a sample toonel.ini file used by toonel.net client application.
# This file contains the mappings of IP ports.
# Each entry should be kept on an individual line and has the following format:
# [local port number];[host];[remote port number]
# Additionally, comments (such as these) may be inserted on individual lines
8080;127.0.0.1;7999
25;smtp.netti.fi;25
110;pop3.netti.fi;110
# 143;IMAP Server [Incoming mail sample];143
Pay special attention to the two e-mail related rows in bold; the HTTP-related record ("8080;127.0.0.1;7999") doesn't need to be modified.
Copy it back to the root of your PDA.
Fourth, set up the HTTP proxy setting of the active connection group used by the "The Internet" drop-down list. (To know more where to find this, please read my article on proxy-related article at, say, http://www.firstloox.org//forums/showthread.php?p=28878 and http://discussion.brighthand.com/showthread.php?s=&threadid=118113 ). As the HTTP proxy is listening on your local device at port 8080, fill in the HTTP input textfield in the Advanced section of the proxy setup:
http://menneisyys.freeweb.hu/toonel/HTTPProxySettings2.gif
Then, the basic page of this will be as follows:
http://menneisyys.freeweb.hu/toonel/HTTPProxySettings1.gif
Please note that you can change the HTTP proxy server any time. To make this as easy as possible, you may want to read the method I've invented and described in my article in the above-mentioned article.
Fifth, overwrite the incoming/outgoing mail server addresses in your mailer client to contain the word 'localhost' or, as with the above example, the IP address '127.0.0.1', which equals to 'localhost':
http://menneisyys.freeweb.hu/toonel/ToonelMailerSetup.gif
Please note that, unlike the built-in mailer clients of Symbian devices, which support arbitrary port numbers, the built-in Microsoft-shipped Pocket Inbox/Messaging client doesn't support them:
http://www.toonel.net/res/emailp800_8.gif
(Unfortunately, this can't even be registry-hacked.)
Therefore, if you plan to use several POP3 clients with different server addresses (!) with Toonel, you will want to switch to some other, more advanced mailer (POP3/SMTP) client with port defining capabilities for each account you set up. An example of these advanced mailer clients is WebIS Mail ( http://www.pocketinformant.com/products_info.php?p_id=mail&dir=wm& ).
Note that if you close/kill the JVM by clicking the X button (it not only minimizes it!), your PIE browser and mailer clients won't be able to connect to the net because they'll try connecting to the (non-existing) local proxy. Then, just re-run the proxy starter link.
Please note that Toonel will crash on the PPC if you try to use its setter HTTP page at localhost:7999. It's best to leave it alone - fortunately, \toonel.ini has all the settable parameters, so, you don't really need the on-line page.
Bottom line
Pros:
- free
- fast
- reliable
- no custom HTTP proxy server is needed
- compresses mail too
Cons:
- (comparatively) noticeable impact on the PDA performance because of the heavy-weight JVM
- doesn't compress images (this is not a major problem though because many GPRS providers - for example, Vodafone - also offers on-the-fly image crunching if you use their special APN)
To be continued:
1, using toonel with other JVM's (Créme, IBM etc), also with comparative benchmarks on JVM speed etc.
2, comparison of toonel to the native ARM solution developed by Globility( http://www.globility.co.nz ). See http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=15684 on the latter.
Please post your remarks here. Do you need additional clarification? Was there anything in the article that you haven't really understood?
The author is an expert in TCP/IP protocols and wireless communications. He has authored the IBM JavaBeans FTP Java bean and library back in 2000-2001 and has co-authored the POP3 and SMTP libraries in the same section. Has also authored several HTTP proxy and filtering servers and libraries; for example, the Mobipocket Web Companion Support Pack ( http://menneisyys.freeweb.hu/mwc ).