正體中文 | 简体中文 | English

:::

chklinks

What’s New?

2007-06-06
chklinks version 3.03 is released. Fixed the raw UTF-8 warning with HTML::Parser. Download chklinks version 3.03.
2006-06-02
chklinks version 3.02 is released. The digital signature is corrected. Download chklinks version 3.02.
2006-05-31
chklinks version 3.01 is released. This fixed a missing prerequisites in Makefile.PL. HTML documents are returned. Download chklinks version 3.01.
2006-05-30
chklinks version 3.00 is released. It uses HTML::LinkExtor to parse the page and LWP::RobotUA to make requests. More URL schemes other than http are supported. Cookies are supported. robots.txt rules are respected. Download chklinks version 3.00.
2005-10-02
Website style adjustment. WCAG Accessibility designs are added. <hr />s are replaced by CSS borders for accessibility reason.
2005-04-27
The text 繁體中文 on the website is corrected to 正體中文.
2003-05-26
chklinks version 2.02 is released. The default changed from one-level to recursive. This is more desirable in most cases. Download chklinks version 2.02.
2003-05-25
chklinks version 2.01 is released. An installer, accessary files and a POD document are added. Download chklinks version 2.01.
2003-05-25
chklinks version 2.0.0 is released. This is the first public release uploaded onto SourceForge. Only the main script file is available. Download chklinks version 2.0.0.
2003-05-19
SourceForge project space approved.
2003-05-18
Requesting a SourceForge project space.

Description

chklinks is a Perl link checker. It helps finding broken links on your website.

chklinks differs from linkchecker in that chklinks is non-threaded. It does not raises many simultaneously connections for its job. It won’t run out of the resources and crash your system in a moment. This is certainly more desirable for most webmasters and users.

chklinks follows robots.txt rules. If you disallow robots from your website and experience problems, you need to allow chklinks. Add the following lines to your robots.txt file to allow chklinks:

User-agent: chklinks
Disallow:

chklinks uses LWP::RobotUA and support the following schemes: http, https, ftp, gopher and file. You can also specify a local file. (To use https, you need to install Crypt::SSLeay. This is the requirement of LWP::RobotUA.)

chklinks supports cookies.

System Requirement

  1. Perl, version 5.6 or above. I have not successfully run this on earlier versions. Please tell me if you can. You can run perl -v to see your current Perl version. Perl is distributed with most Unix now. If not, or if you have an older version of Perl, you can download and install/upgrade it from Perl website. If you are using MS-Win32, you can download and install ActiveState ActivePerl.
  2. Required Perl modules: URI, HTML::LinkExtor, LWP::RobotUA. HTML::LinkExtor is part of the HTML-Parser package, and LWP::RobotUA is part of the libwww-perl package. You can download and install them from the CPAN archieve. If you are installing with the CPAN or CPANPLUS shell, they shall be automatically installed.
  3. Optional Perl modules:
    Crypt::SSLeay
    This is needed for LWP::RobotUA to support HTTPS.
    You can always search, download and install the missing Perl modules from the the CPAN archieve.

Download

chklinks’s official websites is at…

You can always download the newest version of chklinks from…

imacat’s PGP public key is at…

Install

chklinks uses standard Perl installation with ExtUtils::MakeMaker. Follow these steps:

% perl Makefile.PL
% make
% make test
% make install

When running make install, make sure you have the priviledge to write to the instalation location. This usually requires the root priviledge.

If you are using ActivePerl under MSWin32, you should use nmake instead of make. nmake can be obtained from the Microsoft FTP site.

If you want to install into another location, you can set the PREFIX. For example, to install into your home when you are not root:

% perl Makefile.PL PREFIX=/home/jessica

Refer to the docuemntation of ExtUtils::MakeMaker for more installation options (by running perldoc ExtUtils::MakeMaker).

Options

./chklinks [options] URL1 [URL2 [URL3 …]]
./chklinks [-h|-v]
-1,--onelevel
Check the links on this page and stops.
-r,--recursive
Recursively check through this website. This is the default.
-b,--below
Only check the links below this directory. This is the default.
-p,--parent
Trace back to the parent directories.
-l,--local
Only check the links on this same host.
-s,--span
Check the links to other hosts (without recursion). This is the default.
-e,--exclude path
Exclude this path. Check for their existence but not check the links on them, just like they are on a foreign site. Multiple --exclude are OK.
-i,--include path
Include this path. An opposite of --exclude that cancels its effect. The latter specified has a higher priority.
-d,--debug
Display debug messages. Multiple --debug to debug more.
-q,--quiet
Disable debug messages. An opposite that cancels the effect of --debug.
-h, --help
Display the help message and exit.
-v, --version
Output version information and exit.
URL1, URL2, URL3
The URLs of the websites to check against.

Notes

chklinks does not obey Crawl-delay: in robots.txt yet. This is a problem in WWW::RobotRules, but not chklinks itself.

If you encounter warnings like this:

Parsing of undecoded UTF-8 will give garbage when decoding entities at /usr/share/perl5/LWP/Protocol.pm line 114.

This is a LWP::Protocol issue when working with HTML::Parser version ≥ 3.40. See CPAN RT Bug#20274 for a LWP::Protocol patch on this.

Bugs

chklinks does not support authentication yet. W3C-LinkChecker have support on this. As a workaround, You can use the syntax http://user:pass@some.where.com/some/path for Basic Authentication, but this does not work on Digest Authentication. This practice is not encouraged. Your password would be visable to anyone on this system using ps, including hidden intruders. Also what you type in your shell will be saved to your shell history file.

mailto: URLs should be supported by checking the validity of its DNS/MX record. Bastian Kleineidam’s linkchecker have support on this.

Local file checking has only been tested on Unix and MSWin32. More platforms should be tested, especially VMS and Mac.

Support

chklinks is hosted on SourceForge, CPAN and Tavern IMACAT’s. For the latest infomation, see

There is a chklinks mailing list hosted at SourceForge. Please submit your questions, suggestions or bug reports there. It is a Mailman mailing list. For more information, see the chklinks mailing list page. Alternatively, you can send a mail to the e-mail command mailbox with the subject help for a list of available e-mail commands.

See Also

LWP::UserAgent, LWP::RobotUA, WWW::RobotRules, URI, HTML::LinkExtor, Bastian Kleineidam’s linkchecker and W3C-LinkChecker checklink.

Author

imacat.

Copyright

Copyright © 2003-2007 imacat. All rights reserved.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.


By imacat, first version 2003-05-25, last updated 2007-06-06
Copyright © 2003-2007 imacat. Please read the copyright before copying.