WWW/Find version 0.01 ===================== SYNOPSIS use LWP::UserAgent; use HTTP::Request; use WWW::Find; $agent = LWP::UserAgent->new; $request = HTTP::Request->new(GET => 'http://www.bookmarks.example'); $find = WWW::Find->new(AGENT_FOLLOW => $agent, REQUEST => $request, # optional MAX_DEPTH => 2, # optional MATCH_SUB => \&$match_sub, # optional FOLLOW_SUB => \&$follow_sub ); ## example $match_sub finds *pl/*pm files and prints the complete URI $match_sub = sub { my $self = shift; if($self->{REQUEST}->uri =~ /(pl|pm)$/io) { print $self->{REQUEST}->uri . "\n"; } ## example $match_sub finds *pl/*pm files and prints the complete URI $match_sub = sub { my $self = shift; if($self->{REQUEST}->uri =~ /(pl|pm)$/io) { print $self->{REQUEST}->uri . "\n"; } return; }; ## example $follow_sub follows links with header content_type eq 'text/*' $follow_sub = sub { my($self, $response) = @_; $response->content_type =~ /text/io ? return 1 : return 0; }; DESCRIPTION Think of WWW::Find as a web version the Unix 'find' command. One can imagine various uses for it. For example, it might be used to recur- sively mirror multi-page web sites on your local hard disk. Or perhaps you might want to scour the web for resources matching certain HTTP header criteria; whatever you like. I've opted for maximum flexibility by allowing the user to pass in custom URL and header matching subrou- tines. Flexibility is both good and bad; care is required. Given bad parameters, you could easily begin the infinite task of downloading everything on the net! SEE ALSO http://www.gnusto.net is the offical home page of WWW::Find AUTHOR Nathaniel Graham, COPYRIGHT AND LICENSE Copyright 2003 by Nathaniel Graham This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. INSTALLATION To install this module type the following: perl Makefile.PL make make test make install