ROUNDUP: Read / browse WikiPedia on your Pocket PC!

Menneisyys · 08-20-06, 12:06 PM

WikiPedia has become by far the best source of really up-to-date information – in many respects, even better than the online Oxford English Dictionary (see “The Definitive Roundup of All Pocket PC Dictionaries Part II – non-WordNet-based English Dictionaries” for more info if interested on the well-known, alternate sources of available information.)

In this roundup, I elaborate on how you can access information in WikiPedia on your Pocket PC other than directly browsing its pages in your Pocket PC Web browser.

Note that this article contains a LOT of never-before-published tricks and tips. Did you know, for example, that you can save a lot of money / greatly speed up your mobile phone-based online Wiki access by using data compression? Did you know that, for example, one of the offline Wiki databases, Lexipedia, offers excellent fuzzy searching capabilities, which make it possible to find keywords whose spelling you’re not really sure of? The list continues – trust me, even seasoned Wiki users will learn a lot of tricks and tips from this article.

Compressed / optimized online access to drive down communication costs/loading times

Unfortunately, Wiki (still?) doesn’t have a WAP or a PDA-optimized “official” interface that would return the content at least GZIP-compressed (please do read this article and the referenced articles to understand what this means) to heavily drive down the communication costs. Therefore, you need to rely on online proxies or repositories.

This article is probably the best to elaborate on these. It’s a bit outdated (for example, the Wiki proxy at 3g.co.nz doesn’t work any more) and lacks a lot of very important information.

The related comparison chart is here (sorry, couldn’t put it in here). Note that Pocket Internet Explorer / Internet Explorer Mobile in WM2003+ supports WAP (also see this article for more info) if you prefer accessing Wiki through WAP.

To summarize the chart's contents, you may want to give a try to pda.Language Code.wapedia.org but it’s not much better than a generic Web compression service (elaborated on here).

Offline access

There are three WikiPedia databases / applications on the Pocket PC that make it possible to have a local Wiki copy and browse it locally, without any kind of Internet access.

MDict

The database itself, MDict, free and, because of being an excellent dictionary / database application on the Pocket PC (please see this roundup of Pocket PC English dictionaries for more info on MDict itself), highly recommended. As is also pointed out in the article, there are some free databases for MDict, which also add to its usability – in this screenshot , it has for example the WordNet and the OPTED dictionary databases, among others.

However, the MDict Wiki database itself is very outdated (late 2003 – see, for example, the original PPC article here to see how outdated it is) and must be rebuilt in order to be usable. That is, while MDict is a free and decent engine, as its Wiki database is three years old, it’s not recommended.

TomeRaider 3 with Erik Zachte’s databases with TomeRaider3

This is by far the best engine and database – it supports HTML and has a considerably newer database than the other two alternates.

Its main disadvantage, compared to the most important alternative, Lexipedia, is the complete lack of fuzzy search and in-word (substring) searching, as far as searching its index is concerned. Also, it has no way of invoking (or even getting the contents of) external links, unlike the two alternates.

However, in all other respects it’s far superior to Lexipedia. For example, it has a freely scrollable index, which is a definite advantage. It is also able to make in-text searching. While it may be a bit slow, it’s unique: the alternate apps don’t offer this capability.

Lexipedia 2.4 by Revolutionary Software Front

It has some real advantages over TomeRaider: it has a quickly searchable index, allows for fuzzy (error correction) searches (just like Lextionary 2.4 by the same developer) and has clickable external links.

It, however, has major drawbacks. It (as with Lextionary) doesn’t allow for in-description search (unlike TomeRaider) and it renders the descriptions as plain text, without any HTML markup. This not only means the complete lack of even basic things like bold and italic markup (which would be essential to emphasize for example newly introduced stuff), but also the complete lack of any info boxes (which are widely available in a lot of Wiki articles – even in the “old”, one-year-old versions) and their contents.

Unfortunately, the complete lack of HTML support seems to be common with all Revolutionary Software Front products. Given that it’s not trivial to implement (unless the developer switches entirely to using the Pocket Internet Explorer plug-in to render the HTML content) I don’t think it’ll be implemented in the near future.

Also, the database reflects the Wiki state as of early August 2005 and can not be manually updated (the database format is proprietary and closed, as opposed to that of TomeRaider, which has freely available tools for database creation). In this respect, TomeRaider, reflecting the late December 2005 database state, is definitely better. And, again, don’t forget that you can create a newer, updated version of the TomeRaider database any time, unlike with Lexipedia, where you are at the developer’s mercy.

Please also see this full review on the engine (TomeRaider3) itself.

Comparison chart – offline solutions

The comparison chart is here. Note that it’s entirely different from the first comparison chart showing the online, Wiki-related compression / WAP/PDA-conversion services.

Verdict

Apart from the inability to (quickly) search for substrings in the titles, to make fuzzy searches (which are unique features of Lexipedia) and to click external links, go for the TomeRaider version. The openness of the format, the full HTML support, including info boxes, the ability to search for anything in-text, the fact that you get not only Wiki for the price of TomeRaider but also the ability to read any other TomeRaider documents are all big advantages of the TomeRaider version. (If you can and will pay more than twice the price of Lexipedia, that is.)

Lexipedia, while it has definite advantages over TomeRaider (fuzzy search, clickable links), due to the complete lack of HTML markup (including infoboxes) is bound to hide relevant information and is based on a slightly older database, which will be even more acute a problem as time goes by and the TomeRaider version is updated (I also plan recompiling and updating the TomeRaider version), if (and only if) the Lexipedia database remains not updated.

Therefore, I recommend checking out the TomeRaider version first and Lexipedia only as second (if at all).

Unfortunately, I can not recommend the MDict Wiki version at all. It’s really outdated. Its database MUST be updated to become usable.

Links

In addition to the links to my other articles, I also recommend this FirstLoox thread on the technical limitations of other e-book formats; for example, Mobipocket Reader (note that there are still Mobi conversion; for example, the pretty new (14.4.2006) German one here (thanks for Felix for the tip!) and for example this AximSite thread on the copyright issues of the TR database.

Note that since the article “The whole world in your pocket (almost)" by Shaun McGill (which you may also want to read) has been written, the TR databases have been relocated from here to here.

08-20-06, 01:00 PM

WOW as usual. Thanks loads for all you do!

Tim L · 08-21-06, 07:57 PM

Nice roundup. I hope Erik Zachte gets another database together soon though. I've had the December 2005 one since around January and I'm itching for a more recent version. I do wonder just how big it is now though - Wikipedia seems to have reached a popular critical mass in the past year.

08-22-06, 03:50 PM

I purchased TomeRaider3 and it is a VERY good app. I use not only the last Wiki upgrade put out for it but have several of the other databases put out. I've found them all to be very informative and nice to have around.

I hope they put out some more databases/documents in TomeRaider3 format and hopefully the Wiki database will be upgraded soon. With storage getting so cheap it would be nice to carry around the entire Wiki database.

My 6GB CF Microdrive still has plenty of free space on it. :)

szhrmp · 08-25-06, 07:15 AM

I just tried out the new tomeraider 3.20.0 and I think the new interface looks much worse than the old one, is there a way to go back to the old one (proper menu bar and icons on the bottom).

Also I would be nice if Tomeraider PPC would support external html links (ie.. fire up Opera if you click on a html link).

Peter

Menneisyys · 08-28-06, 12:02 PM

Article slightly updated with the latest TomeRaider3 and other information.

Menneisyys · 02-04-07, 07:01 AM

UPDATE (08/29/2006): PPCT frontpage

UPDATE (02/03/2007): In October 2006, a brand new dump has been posted. The text-only version, however, requires at least a 2GByte card, as opposed to the previous dump. Also, the LexiPedia database has been updated; now, the new database is dated 12/29/2006.

Menneisyys · 02-04-07, 02:40 PM

UPDATE (02/04/2007): I've thoroughly examined the database date of both new versions to find out which one is newer. Lexipedia is dated back to 09/25/2006 (while the release date is, as has already been pointed out, late December 2006) and is, therefore, more than a year more up-to-date than the previous version - and still fits in a 1Gbyte card.

The, technically, superior TomeRaider version, on the other hand, is definitely a disappointment this time: it’s more than a year old (don’t pay attention to the filedates, which “only” date back to October 2006): the database itself reflects the state of 12/12/2005. In this respect, it is in no way better / newer than the old version. The TomeRaider folks should, at last, release a more up-to-date database to remain competitive. All in all, you’ll want to prefer the Lexipedia version if you want the latest Wiki state.

(In the meantime, there haven’t been new Wiki database versions for MDict. That is, it should be entirely forgotten as an alternative.)

02-04-07, 05:29 PM

So, what about Eric's database, with the pictures; is that out of date? Is it worth the $25.00? Why would I want to install old data just to get the graphs and pictures? Has anyone found that much more utility with Eric's graphics version than the free version?
After the latest updates mentioned above, would that make Lexipedia a better choice? I am somewhat confused as to my best option.

Menneisyys · 02-05-07, 03:50 AM

Originally Posted by drb123

So, what about Eric's database, with the pictures; is that out of date? Is it worth the $25.00? Why would I want to install old data just to get the graphs and pictures? Has anyone found that much more utility with Eric's graphics version than the free version?
After the latest updates mentioned above, would that make Lexipedia a better choice? I am somewhat confused as to my best option.

Dunno about Eric's non-free version (I, generally, don't request non-trial versions so that I can avoid situations like biting the feeding hand.)

However, the TR folks have just announced in my blog that they will release a more up-to-date version of the pirctureless version.

Up until then, LexiPedia is a better choice because of the newer database. I only wish its rndering engine were better.

STi Sev · 02-05-07, 05:21 AM

You are a living god.

Thank you for this!

**Rishad** · 02-05-07, 05:58 AM

Tomeraider used to have a compact smaller database. Wonder what happened to that one.

Menneisyys · 02-05-07, 06:01 AM

Originally Posted by akheron

Tomeraider used to have a compact smaller database. Wonder what happened to that one.

The TR folks have just promised they would release a brand new & updated version of it.

10-09-07, 01:13 PM

What's the status on the best db to use now? Menn, I read an old posting in your blog that said you might fix up that Perl script used to convert the EN wiki database...if that can work again, I'd definitely try that - newest db if I do the conversion, and full image support...*drool*

Menneisyys · 10-09-07, 03:26 PM

Originally Posted by Lat

What's the status on the best db to use now? Menn, I read an old posting in your blog that said you might fix up that Perl script used to convert the EN wiki database...if that can work again, I'd definitely try that - newest db if I do the conversion, and full image support...*drool*

Sorry, didn't have the time... I don't think I will have now either...