Menneisyys
05-31-2005, 02:33 PM
On-line bandwidth saver services for Web browser users
Author: Werner 'Menneisyys' Ruotsalainen, member of the Pocket PC magazine Board of Experts 2005, tech writer, PPCMag forum moderator, frequent contributor to, say, PPCMag/FirstLoox/PPCT/Brighthand/PDAMania.hu etc. forums
There're two problems with most web sites: they return dozens or sometimes even hundreds of kilobytes long pages that are full of unnecessary stuff (whitespaces, scripts and formatting that are useless for PDA users). Furthermore, the pages they return are pretty hard for a PDA browser to display, especially with pre-WM2003SE Pocket Internet Explorer (PIE) browsers without any additional PIE plug-ins that do something similar to the "One Column" view mode in the WM2003SE version of PIE.
Therefore, if you still use a pre-WM2003SE device without any of these PIE plug-ins, using an on-web content ripper and/or cruncher service may be advantageous.
There're quite a few services with radically different capabilities.
All-in-one solutions
Some (namely, Thunderhawk, http://www.bitstream.com/wireless/ ) not only heavily compress the Web content and make it much more PDA-friendly, but also use their own client instead of PIE (or an alternative browser like Netfront or Minimo). This has clear advantages:
• Thunderhawk has much better JS support than PIE. For example, the JavaScript in the Pocket PC Magazine or the Pocket PC Thoughts webpage don't work in PIE.
• Consumes way less bandwidth - also because it gets everything in a page in a single TCP/IP connection, without sending out additional GET headers to fetch the resources of the given page. Sure, all PIE's from Windows CE 2.11 have also used persistent, HTTP/1.1 connections to avoid the need for building up separate TCP/IP connections for fetching resources; but that alone doesn't reduce the overhead caused by HTTP headers in both directions, which can amount to 500-800 bytes for each resource.
Thunderhawk is a very good solution, with very few problems. These are as follows:
- as there is no VGA-optimized version, on VGA devices, the visual experience Thunderhawk delivers is not as good as it could be
- it only allows for displaying one page; there're no tabs unlike with all PIE plug-ins or Netfront. This makes it very hard to, say, copy information between different web pages.
Local PDA-based cruncher proxies
While the HTTP protocol allows for GZIP encoding of the HTML or textual bodies, almost no Web servers support this. This can be helped with compressor proxies like toonel.net. Please see http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=15684 on a more exhaustive description of the problem.
Now, only toonel.net (from now on, Toonel) has a Pocket PC-compliant client and the accompanying online service to connect to. The other one, the native ARM solution developed by Globility ( http://www.globility.co.nz ), has not been released yet. As soon as it becomes at least a beta, I'll test it against Toonel and update my review.
As you can read in my article on configuring and using Toonel on a Pocket PC, http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=16017 , the client and the service are working flawlessly.
This service is highly recommended for anyone that doesn't want to pay for the Thunderhawk service and/or still prefers 'traditional' Web browsers on his or her PDA for example, because of their tabbing capabilities or native VGA support.
Please note that Toonel not only compresses HTML bodies but also SMTP and POP3 communication. Please also note that it doesn't downscale images, unlike Thunderhawk or, if everything goes OK, the service offered by Globility.
Clientless and PDA-based compressor proxy-less, online-only compression/content ripper services
The thirs category doesn't require anything on the client device. This means its client can be virtually anything, even an old Commodore Amiga.
They both rip unnecessary and/or not renderable contents from Web pages and, sometimes, also compress it with standard, PIE and Netfront-compliant gzip. Some of them also downsample the referenced image resources, which is a big plus.
The services are as follows:
Skweezer
http://www.skweezer.net/
The most important, very good service that also has a free version.
Pros: not only supports GZIP HTML body encoding, but also POP3 access with (slight) compression. The Pro version, fortunately, only differs from the free service in that it's ad-free. Unlike some of its direct competitors, it always uses the right URL added to the leading 'skweezer.net' URL. Therefore, URL's can even be put in the local favorites.
Cons:
- it doesn't downsample images, unlike some of its competitors
- it doesn't pass server cookies to the client, unlike some of its competitors. This means all server-issued cookies are stored and handled on the Skweezer server. Between the client and the server, only the non-presistent ASP.NET_SessionId cookie is sent. By forced re-sending of these cookies, the original cookies can be forced to be re-sent to the server and user log-in will, consequently, work, but then, Skweezer won't answer with GZIP-encoded bodies any more. Furthermore, the web session is only kept for some 20-30 minutes; after that, all the cookies are lost. That is, its pretty useless to use any kind of cookie hacking with Skweezer.
The latter means you will always need to re-login to your account after you close PIE. This could be fixed vey easily by the people at Skweezer because it requires comparatively little work.
clickfisch.net
http://clickfisch.net/proxy/proxy.php
Seems to be removed as of 31/05/05. See http://www.pocketpcthoughts.com/forums/viewtopic.php?p=345365 for more info and a direct comparison.
WebXCope from Jenner Research
http://216.103.91.135/rfxDM/ppcframe.php
Doesn't compress but cuts contents pretty good.
Cookies are real server cookies (unlike with Skweezer) but they aren't persistent. Furthermore, if you want to navigate to different sections in a lot of pages (for example, Pocket PC Thoughts forum), you'll need to click [+] a lot of times, which really goes into nerves. Furthermore, if you have to use this link, all the subsequent pages will uses static HTML (like /rfx/ss1868008.htm ) URLs and not real URL's, unlike, say, Skweezer. This means you won't be able to bookmark a lot of pages.
MobileLeap
http://mobileleap.net/app/demo/translator
This is a simple service with really reduced capabilities as it doesn't support HTML forms at all (I've scrutinized the stripped HTML source it sends back of a page that originally contained FORM tags. The stripper engine just strips all <form>...</form> tags; this is why there is no way of logging in). This means you won't be able to, say, log in to most forums or run form-based searches, which is possible with almost all the other compression/content stripping techniques. In some cases, Web pages that also accept GET requests for log in, by passing the POST request body inside the GET request you can 'hack' the engine to let you in, but it won't work in most cases like with phpBB.
Pros:
- image compression (in the MobileLeap mode; in the two other, Lynx and Text Extractor modes, it completely strips images too)
Cons:
- no GZIP compression
- no form support at all
- no page-specific URL's (it's always http://mobileleap.net/app/demo/translator because, internally, it only uses POST to communicate parameters)
The WML Proxy conversion service of Google
http://www.google.com/xhtml
Much as this is, officially, only a Google front-end, you can enter the full page URL you want to access into it without the leading http://. Then, you'll be able to use this service to access the page without any URL hacking or manual editing (which would be rather compicated with WML Proxy). It doesn't have images and are, at times, because of its being a widely used service, very slow or doesn't even work, though.
Because it didn't work when I finished this article, I couldn't test its capabilities. This is why it's also missing from the comparison chart.
Service comparison chart
Skweezer clickfisch WebXCope MobileLeap
Price? free; Pro version: free free free
15 $/year
POST? + + + -(not even FORM support!)
ZIPed HTML? + - - -
Downscaled
images? -,not even in Pro + no images +
at all
URL's + - generally -
yes; after clicking [+], no
In-session
cookies? + ? + ?
Inter-session
cookies? - ? - ?
Bottom line
It's not very complicated to say which services are the best as of now (31/05/2005):
- Thunderhawk, if you can put up with the relatively high price of the service and the problems (QVGA, no tabs) of the Thunderhawk client
- Toonel, if you don't mind running a personal proxy server on your PDA
- Skweezer, if you don't mind the need for loggin in each time you revisit a page between browser restarts (and/or the Web session timeout on the Skweezer server).
Please note that this article can also be found at http://menneisyys.freeweb.hu/CompressionServices/
Author: Werner 'Menneisyys' Ruotsalainen, member of the Pocket PC magazine Board of Experts 2005, tech writer, PPCMag forum moderator, frequent contributor to, say, PPCMag/FirstLoox/PPCT/Brighthand/PDAMania.hu etc. forums
There're two problems with most web sites: they return dozens or sometimes even hundreds of kilobytes long pages that are full of unnecessary stuff (whitespaces, scripts and formatting that are useless for PDA users). Furthermore, the pages they return are pretty hard for a PDA browser to display, especially with pre-WM2003SE Pocket Internet Explorer (PIE) browsers without any additional PIE plug-ins that do something similar to the "One Column" view mode in the WM2003SE version of PIE.
Therefore, if you still use a pre-WM2003SE device without any of these PIE plug-ins, using an on-web content ripper and/or cruncher service may be advantageous.
There're quite a few services with radically different capabilities.
All-in-one solutions
Some (namely, Thunderhawk, http://www.bitstream.com/wireless/ ) not only heavily compress the Web content and make it much more PDA-friendly, but also use their own client instead of PIE (or an alternative browser like Netfront or Minimo). This has clear advantages:
• Thunderhawk has much better JS support than PIE. For example, the JavaScript in the Pocket PC Magazine or the Pocket PC Thoughts webpage don't work in PIE.
• Consumes way less bandwidth - also because it gets everything in a page in a single TCP/IP connection, without sending out additional GET headers to fetch the resources of the given page. Sure, all PIE's from Windows CE 2.11 have also used persistent, HTTP/1.1 connections to avoid the need for building up separate TCP/IP connections for fetching resources; but that alone doesn't reduce the overhead caused by HTTP headers in both directions, which can amount to 500-800 bytes for each resource.
Thunderhawk is a very good solution, with very few problems. These are as follows:
- as there is no VGA-optimized version, on VGA devices, the visual experience Thunderhawk delivers is not as good as it could be
- it only allows for displaying one page; there're no tabs unlike with all PIE plug-ins or Netfront. This makes it very hard to, say, copy information between different web pages.
Local PDA-based cruncher proxies
While the HTTP protocol allows for GZIP encoding of the HTML or textual bodies, almost no Web servers support this. This can be helped with compressor proxies like toonel.net. Please see http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=15684 on a more exhaustive description of the problem.
Now, only toonel.net (from now on, Toonel) has a Pocket PC-compliant client and the accompanying online service to connect to. The other one, the native ARM solution developed by Globility ( http://www.globility.co.nz ), has not been released yet. As soon as it becomes at least a beta, I'll test it against Toonel and update my review.
As you can read in my article on configuring and using Toonel on a Pocket PC, http://www.pocketpcmag.com/forum/topic.asp?TOPIC_ID=16017 , the client and the service are working flawlessly.
This service is highly recommended for anyone that doesn't want to pay for the Thunderhawk service and/or still prefers 'traditional' Web browsers on his or her PDA for example, because of their tabbing capabilities or native VGA support.
Please note that Toonel not only compresses HTML bodies but also SMTP and POP3 communication. Please also note that it doesn't downscale images, unlike Thunderhawk or, if everything goes OK, the service offered by Globility.
Clientless and PDA-based compressor proxy-less, online-only compression/content ripper services
The thirs category doesn't require anything on the client device. This means its client can be virtually anything, even an old Commodore Amiga.
They both rip unnecessary and/or not renderable contents from Web pages and, sometimes, also compress it with standard, PIE and Netfront-compliant gzip. Some of them also downsample the referenced image resources, which is a big plus.
The services are as follows:
Skweezer
http://www.skweezer.net/
The most important, very good service that also has a free version.
Pros: not only supports GZIP HTML body encoding, but also POP3 access with (slight) compression. The Pro version, fortunately, only differs from the free service in that it's ad-free. Unlike some of its direct competitors, it always uses the right URL added to the leading 'skweezer.net' URL. Therefore, URL's can even be put in the local favorites.
Cons:
- it doesn't downsample images, unlike some of its competitors
- it doesn't pass server cookies to the client, unlike some of its competitors. This means all server-issued cookies are stored and handled on the Skweezer server. Between the client and the server, only the non-presistent ASP.NET_SessionId cookie is sent. By forced re-sending of these cookies, the original cookies can be forced to be re-sent to the server and user log-in will, consequently, work, but then, Skweezer won't answer with GZIP-encoded bodies any more. Furthermore, the web session is only kept for some 20-30 minutes; after that, all the cookies are lost. That is, its pretty useless to use any kind of cookie hacking with Skweezer.
The latter means you will always need to re-login to your account after you close PIE. This could be fixed vey easily by the people at Skweezer because it requires comparatively little work.
clickfisch.net
http://clickfisch.net/proxy/proxy.php
Seems to be removed as of 31/05/05. See http://www.pocketpcthoughts.com/forums/viewtopic.php?p=345365 for more info and a direct comparison.
WebXCope from Jenner Research
http://216.103.91.135/rfxDM/ppcframe.php
Doesn't compress but cuts contents pretty good.
Cookies are real server cookies (unlike with Skweezer) but they aren't persistent. Furthermore, if you want to navigate to different sections in a lot of pages (for example, Pocket PC Thoughts forum), you'll need to click [+] a lot of times, which really goes into nerves. Furthermore, if you have to use this link, all the subsequent pages will uses static HTML (like /rfx/ss1868008.htm ) URLs and not real URL's, unlike, say, Skweezer. This means you won't be able to bookmark a lot of pages.
MobileLeap
http://mobileleap.net/app/demo/translator
This is a simple service with really reduced capabilities as it doesn't support HTML forms at all (I've scrutinized the stripped HTML source it sends back of a page that originally contained FORM tags. The stripper engine just strips all <form>...</form> tags; this is why there is no way of logging in). This means you won't be able to, say, log in to most forums or run form-based searches, which is possible with almost all the other compression/content stripping techniques. In some cases, Web pages that also accept GET requests for log in, by passing the POST request body inside the GET request you can 'hack' the engine to let you in, but it won't work in most cases like with phpBB.
Pros:
- image compression (in the MobileLeap mode; in the two other, Lynx and Text Extractor modes, it completely strips images too)
Cons:
- no GZIP compression
- no form support at all
- no page-specific URL's (it's always http://mobileleap.net/app/demo/translator because, internally, it only uses POST to communicate parameters)
The WML Proxy conversion service of Google
http://www.google.com/xhtml
Much as this is, officially, only a Google front-end, you can enter the full page URL you want to access into it without the leading http://. Then, you'll be able to use this service to access the page without any URL hacking or manual editing (which would be rather compicated with WML Proxy). It doesn't have images and are, at times, because of its being a widely used service, very slow or doesn't even work, though.
Because it didn't work when I finished this article, I couldn't test its capabilities. This is why it's also missing from the comparison chart.
Service comparison chart
Skweezer clickfisch WebXCope MobileLeap
Price? free; Pro version: free free free
15 $/year
POST? + + + -(not even FORM support!)
ZIPed HTML? + - - -
Downscaled
images? -,not even in Pro + no images +
at all
URL's + - generally -
yes; after clicking [+], no
In-session
cookies? + ? + ?
Inter-session
cookies? - ? - ?
Bottom line
It's not very complicated to say which services are the best as of now (31/05/2005):
- Thunderhawk, if you can put up with the relatively high price of the service and the problems (QVGA, no tabs) of the Thunderhawk client
- Toonel, if you don't mind running a personal proxy server on your PDA
- Skweezer, if you don't mind the need for loggin in each time you revisit a page between browser restarts (and/or the Web session timeout on the Skweezer server).
Please note that this article can also be found at http://menneisyys.freeweb.hu/CompressionServices/