Parse-MediaWikiDump Parse::MediaWikiDump is a collection of classes for processing various MediaWiki dump files such as those at http://download.wikimedia.org/wikipedia/en/; the package requires XML::Parser. Using this software it is nearly trivial to get access to the information in supported dump files. INSTALLATION To install this module, run the following commands: perl Makefile.PL make make test make install LIMITATIONS Parse::MediaWikiDump currently can not safely handle the full page dumps (a dump where each page have more than one revision). In this instance Parse::MediaWikiDump will abort processing of the archive. Parse::MediaWikiDump is not as fast as it could be but it is faster than using most other XML parsing frameworks. The parser could stand to be rewritten to be faster and handle the full page dumps. EXAMPLE Extract the text for a given article from the given dump file: #!/usr/bin/perl use strict; use warnings; use Parse::MediaWikiDump; my $file = shift(@ARGV) or die "must specify a MediaWiki dump of the current pages"; my $title = shift(@ARGV) or die "must specify an article title"; my $dump = Parse::MediaWikiDump::Pages->new($file); binmode(STDOUT, ':utf8'); binmode(STDERR, ':utf8'); #this is the only currently known value but there could be more in the future if ($dump->case ne 'first-letter') { die "unable to handle any case setting besides 'first-letter'"; } #enforce the MediaWiki case rules $title = case_fixer($title); #iterate over the entire dump file, article by article while(my $page = $dump->page) { if ($page->title eq $title) { print STDERR "Located text for $title\n"; my $text = $page->text; print $$text; exit 0; } } print STDERR "Unable to find article text for $title\n"; exit 1; #removes any case sensativity from the very first letter of the title #but not from the optional namespace name sub case_fixer { my $title = shift; #check for namespace if ($title =~ /^(.+?):(.+)/) { $title = $1 . ':' . ucfirst($2); } else { $title = ucfirst($title); } return $title; } COPYRIGHT & LICENSE Copyright 2005 Tyler Riddle, all rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.