PHC

Phc 1.2 (author:skyroam)
20060712

NAME

Phc - customizable proxy hunter - version 1.2

SYNOPSIS

phc [option]...

DESCRIPTION

This program validates "free proxies" from a file named proxylist. Free proxies found are printed on screen and are also available in a file named proxyout at the same time so that you can stop the runing program at any time.

HTTP PROXY GET, HTTP PROXY CONNECT, SOCKS5 PROXY CONNECT, SOCKS4 PORXY CONNECT request will be sent to the testproxies accordingly (directly connection or via myproxy, a http or socks5 proxies with or without authentication) to start validation. Tt is useful to validate proxies through a tcp tunnel or not. It is suitable to verify a lot of proxies in the same time. I add commandline options in phc 1.2 to let it be invoked from scripts.

The route from phc to the site used to check testproxy is:
phc -------- (myproxy) ------ testproxy ----- checksite

FILES

phc.conf
Configuration file for phc. The name of this file can be customized by `-f' option, see below.
doc/phc.conf.example
An example of phc.conf and should be copied as phc.conf.
doc/proxylist
A file including proxies to be verified. One testproxy every line. The format is "proxyipv4ip:port@protocol.*",such as "127.0.0.1:8080@HTTPabc". String "protocol" can be "HTTP","http","SOCKS5","socks5","SOCKS4" or "socks4" now. All chars after protocol are ignored. You can appoint the other file as proxylist with option `-i'.
proxyout
A file including validated free proxies. It can be changed by `-o' option.
scripts/createlist
Script used to filter and create proxylist file from raw file
scripts/sortlist
Script used to sort proxyout

Both phc.conf and proxylist are needed and searched in current dir.

In phc.conf, the line after "#!"(not "#! if...") must be customed before run this program. See CONFIGURATION SYNTAX below.

In proxylist, only the line with string "@HTTP" ,"@http","@SOCKS5","@socks5","@SOCKS4" or "@socks4" will not be ignored and the "host:port" before "@" will be read as proxy.

In proxyout, you can get free proxies with delay data. See this table for detail:

method output format
HTTP_GET host:port@HTTP estab pget end
HTTP_CONNECT host:port@HTTP estab pcon get end
SOCKS5_CONNECT host:port@SOCKS5 estab pcon get end
SOCKS4_CONNECT host:port@SOCKS4 estab pcon get end
estab
delay between sending connect (SOCKS5 connect command,SOCKS4 connect command or HTTP CONNECT request or connect to testproxydirectly) and receiving established connection to remote testproxy (interact with myproxy if not in DIRECT mode).
pget
delay between sending HTTP "PROXY" GET request to testproxy and receving first byte of target.
get
delay between sending HTTP GET request to target and receving first byte reply of target.
end
delay between receving first byte and finding the key.
pcon
delay between sending connect command(SOCKS5 connect command,SOCKS4 connect command or HTTP CONNECT request) to remote proxies and receving connection established from remote proxies.

I have written a document to describe the process in detail. It will be available on the project homepage in the future. Please "info phc" for a short but enough to use description on it.

OPTIONS

Option Syntax

All of the options need argument except `-V'.

-V

display the version of Phc and exit.

-f

The argument of `-f' is configuration file. If phc is invoked without `-f', it will search `./phc.conf',then `~/.phc.conf', at last `/etc/phc.conf' until the first one is found.

The other options are for configuration.

Configuration Options

Here we give the table mapping command options to configuration options with brief description. See CONFIGURATION SYNTAX for detail.

-t

threads

-g

user_agent

-m

mytype, the type of myproxy.

-p

myport, the port of myproxy

-h

myhost, the ip or domainname of myproxy.

-u

myuser, the username of myproxy if need.

-s

mypass, the password of myproxy if need.

-c

checksite, the target site used to check testproxies.

-k

key, the keyword used to check whether testproxies give correct response.

-r

rcvtimeo, socket recv timeout in seconds.

-n

sndtimeo, socket send timeout in seconds.

-S

myconreplysize

-T

testconreplysize

-G

testtagreplysize

-H

httptestmethod

-5

socks5testmethod

-4

socks4testmethod

-l

logfilename

-i

phcin,the proxylist file used as input of phc.

-o

phcout,the file which phc output result to.

CONFIGURATION SYNTAX

Line begin with "#" is read as comment. Configuration line begin with "var=value". There must be no whitespace between "var" and "=" or "value" and "=". Whitespace inside "value" or "var" is permitted and all chars including quote(if have) in var or value string is read without expanding. The line with misspelled "var" is read as comment too. Here is a list of "var" acceptable:

threads
The number of threads opened at the same time. This should be set according to ulimit. The number can't be more than 65535. Though nocache header is used, it is expected that less threads give more precise delay information to evaluate the proxies better.
mytype
The type of myproxy. 1 for HTTP_CONNECT proxy without authentication. 2 for SOCKS5_CONNECT proxy without authentication. 3 for HTTP_CONNECT_AUTH proxy, that is to say, a proxy with basic authentication. 4 for SOCKS5_CONNECT_AUTH proxy, that is to say, a proxy with username/password authentication. -1 for direct connection to testproxy without myproxy(Testing).

You must set myuser and mypass if let mytype=2.
myhost
The hostname or ip of myproxy(http or socks5 proxy) used as tunnel to connect to remote test proxies.
myport
Of the same proxy as above, but port opened.
myuser
The username of myproxy if basic authentication(HTTP) or username/password authentication(socks5) is needed.
mypass
The password of myproxy if basic authentication(HTTP) or username/password authentication(socks5) is needed.
checksite
The target will be used as request sent to proxies. The format must be http://host:port/path.
key
Tt will be searched in the string returned to validate the proxies.
httptestmethod
Methods used to validate free http testproxy.1 to find http CONNECT proxies. 0 to find the common http proxies.
socks5testmethod
Method used to validate free socks5 testproxy. Only 2 (SOCKS5_CONNECT) is supported now. Don't change it.
socks4testmethod
Method used to validate socks4 testproxy. Only this method, that is to say, 5 (SOCKS4_CONNECT) is supported now. Please don't change it.
user_agent
Identified as Browsers or others to "cheat" some proxies.
myconreplysize
The size of buffer used to receive reply after sending CONNECT request to myproxy
testconreplysize
The size of buffer used to receive reply after sending CONNECT request to testproxy
testtagreplysize
The size of buffer used to stored data from the target. Key was searched in this buffer.
rcvtimeo
Receiving timeouts in second
sndtimeo
Sending timeouts in second
logfilename
The filename stderr is redirected to. Use "." to leave stderr as it is.
phcin
The input file of phc. It is a proxylist. Please don't use `~' as pathname here. If it is not set, phc will search file named as `proxylist' in current directory.
phcout
The output file of phc. Verified proxies with speed information are stored in it.If it is not set, phc will search file named as `proxylist' in current directory.

NOTES

OS
This program is only tested on linux(gentoo , debian, fedora), OSF1 and SunOS. If it doesn't work on other linux or unix platform, please mail me.
WWW
http://www.nongnu.org/phc You can found more useful information here.
COPYRIGHT
Phc and all the document, scripts and other files coming with the distribution are subject to the GNU General Public License (GPL). Please see the COPYING file in the top directory of the package for detail.
AUTHOR
Phc is written from scratch by Jia Wang <skyroam@gmail.com>.
BUGS
You are welcome to this Bug Tracking System at project page: http://savannah.nongnu.org/bugs/?group=phc