StatProxy
StatProxy conducts tests on a list of proxies. The list may be from the
command line, from a file, or from a URL.
All tests run in parallel, so large lists of proxies can be tested very
quickly (even on MS Windows systems!).
The tests run include basic proxy functioning, CONNECT capability,
non-censoring, Socks capability, and vulnerability to various URL
obfuscations.
The results are printed in a format suitable for use with mergeHosts.
perl -w statProxy.pl [-u] [-m] [-p] [-x n] [-q] [-a
<auth>] [-s n] [-C <CONNECT proxy>] [-t
<tests>] (-l <file>
|<IPaddress:port>|transparent|http://your.url.com/path)
where
-u asks for safe mode to be off. In safe mode, statProxy will not test
various unsafe looking proxy addresses and will not repeat a test on
the
same host wih a different port.
-m 'modified' tests. The tests are done in a minimum download bandwidth
mode. Test 0 becomes a HEAD test, instead of a GET test. This is for
dialup users testing a large list.
-p 'pretty print' mode. This doesn't display proxies which fail, and
adds the fqdn for each proxy. Ignored if test 0 not done.
-x n sets the debug level to n
-q asks for the F or P result to be changed to X or Y if the proxy
under test is not the one which gets the web pages for you (you're
probably looking at an array of proxies). The proxy getting the web
pages is mentioned in the data at the end of the test results
(connBack).
-a <auth> Send Basic authentication to the proxy. <realm>
is realm,proxy:port,username,password. One of realm, or proxy:port
*must* be specified. All commas must still be present.
-s n Insert a delay of n seconds between socket reads and writes.
StatProxy uses sockets as fast as it can normally, and can slow down
networking even on a shell. Useful to slow the whole thing down.
-C <CONNECT capable proxy> causes all tests to be done through a
CONNECT tunnel. This is handy if you want anonymity (choose an
anonymous
CONNECT proxy), or if you want to test proxies your firewall doesn't
normally allow you to get to. This is slower than the normal method.
-t <tests> specifies the tests to be done, where <tests>
looks like range0:range1:..., and range may look like n, or 0..15, or
0to15, or all, or -n (to disable a test from a previously specified
range). A typical use is -t all:-18 which runs all tests except the
Socks
proxy test (test 18).
-l file specifies a file with a list of proxies to check,
<IPaddress:port> may be numeric, or fqdn (which may resolve to
multiple IP addresses to test).
'transparent' checks your transparent proxy (if you have one).
http://your.url.com/path asks for this url to be treated as a list of
proxies to test. StatProxy can handle many formats, but is not as good
as findProxy at this yet.
The result is a list of one-line test results for each proxy tested
suitable for piping to mergeHosts for incorporation into the hosts or
user config data structures.
The results are saved to a uniqely named file (the file name is
announced with the results on the screen).
If you are testing a long list of proxies, a single control-c will
print the results so far, and
then continue testing.
If you want to abort, a double control-c will wait for the current
proxy/test to finish, print the results so far, and exit.
In benchmark tests on the worst lists of old proxies (lots of timeouts
and slow fails), statProxy was averaging something like two proxy tests
per second.
perldoc statProxy.pl
will show all of the documentation
embedded in the Perl code, including the meaning of each of the tests.
For those who just want to get started, test 0 is printed first on the
line. Test results are R (connection refused), T (connection timed
out),
P (pass), F (fail).
Test 0 is a normal proxy GET test.
Test 1 is a CONNECT panix.com:22 test.
Test 13 is the anonymity test.
Test 14 is the non-censoring test.
Test 18 is the Socks4 capability test.
Quick notes about statProxy:
People were sending me test results from their own ISP's proxies, but
that's now dropped to a trickle for some reason.
This was a good thing because
1) many of these proxies are firewalled off from the rest of the
Internet, so I can't test them. The only people who can test
them are inside that ISP. In practice this
means you!.
2) with this data, I can keep a reasonably good database for the
newbies who don't yet know how to run statProxy
themselves, but still want to run localProxy. It also allows me to
(using extractHosts) post lists of working proxies for each country.
3) I haven't yet decided how best to handle the procedures allowing
localProxy to reliably update the speeds and capabilities of
these proxies itself, so it's not done dynamically. There's an
'autoconfigure' button though.
To encourage this, and to make life easier for those doing these tests,
I'll mention typical uses here:
1) statProxy will allow you to specify the fixed ISP proxy fqdn address
to check. For example:
perl statProxy.pl -t all proxy.nesma.net.sa:80
will test all of the Nesma proxies currently up (212.71.32.95,
212.71.32.97, 212.71.32.98, 212.71.32.94).
2) The results are most useful to yourself, of course, so you can save
them into your own configuration (that way, you don't need to worry
about losing your results next time you download my latest hosts.zip).
Please send to me too, though!
Do it like this:
perl statProxy.pl -t all proxy.nesma.net.sa:80 > results.out
(or forget the results.out and use the unique filename SP
automatically saves
it's results to, then ...)
perl mergeHosts.pl results.out config-wayne.xml
(substitute your own proxy address to be tested and your own
config, of course). The ISP config file in the distribution
(config-KSA-nesma.xml) can be used, but you lose your results next time
you get a new one from CVS, or a package. I suggest copying it to one
with your own name first (e.g. config-john.xml), and then doing the
mergeHosts stuff above.
3) Keep your own information (and again, mine, please!) up to date on
the other proxies that localProxy will select for your use in each new
build. This can be done easily too. When you have the localProxy GUI on
the screen with a new build displayed, copy all the layer 0 proxies you
can see (click the service (probably 10080 - http), commStrat, layer 0,
then right-click 'clear clipboard', right-click again and 'copy/append
all proxies to clipboard') into a
file (say, current.in), and do this:
perl statProxy.pl -t all -l current.in > current.out
perl mergeHosts.pl current.out config-john.xml
(substitute your own config again)
When you restart services with configuration 'john', LP will be using
the new test results to select proxies for your build (and they
override anything from the hosts.xml default database).
4) statProxy also has other modes of operation:
perl statProxy.pl -t all -l proxyListInAFile.txt
perl statProxy.pl -t all http://proxy.list.com/specialList.html
perl statProxy.pl -t all 194.170.1.66:8080,194.170.1.99:8080
perl statProxy.pl 194.170.1.66:8080 (deprecated)
etc. etc. ...
Try:
perl statProxy.pl -h
or:
perldoc statProxy.pl
for more command line option details.