Final fixes in Naoko 4.5 (release version)
* Fixed issue where under some conditions 304 replies could cause
persistent connections to get out of sync (this could cause headers
to appear in the web page body for example).
* proxcert.pem updated
* Added check to prevent infinite list recursion
* Fixed bug in filter merge when a texture is not found.
* Fixed bad request following redirecting a POST URL to a file
(the posted data wasn't being "consumed" even though it's not used).
New Stuff in Naoko 4.5 (beta 4/18/2003)
* Added HTML escaping to block files and URL list under .pinfo
to prevent any included tags from causing trouble.
* Positional variables are now stored and reset if a match fails.
With luck it should make them behave more as they did in previous
versions while still allowing them to be read while matching.
* Web filters no longer import in reverse order.
* Ok I know I said no new features, but it's true all those local
file requests flood the log window. There's now an option to show
them which is disabled by default. :-)
* PriorityBoost = TRUE/FALSE flag has been added to the config. This is
a bit experimental, but when true will boost the priority of filtering
threads while allowing the main program to run at a normal level.
It might help especially when browsers up their own priority which may
starve Proxomitron for CPU time. Because of this I'd not recommend also
running Proxomitron itself at high priority at first or it make make the
thread priority too high; if you do, you may want to disable the boost.
* Changed 302 redirect handling a bit so Mozilla can process them
faster (the latest Mozilla seems to pause for some time if some HTML isn't
included with the redirect).
* Internal header filter sorting on config load has been fixed. This could
cause header filters to be called in an even more unpredictable order than
normal. :-) Still, it's best not to rely on header filter order as the
internal order they're called is "officially" undefined. Someday headers
will be more like web filters and have a order you can arrange rather than
just being sorted, but for now it's best to assume they may be called
in any order.
* TST matches must now be an exact length. For example given "foobar",
$TST(var=foo???) will match while $TST(var=foo??) will fail. This makes
it a bit easier to test for exact values.
* "w" added to $DTM command for current weekday (thanks to a great observation
by ScoJo that this can be used to create filters that are active only at
certain times or on certain days :-)
$SET(today=$DTM(w))
$TST(today=(Sat|Sun))
New Stuff in Naoko 4.5 (beta 4/12/2003)
* Try http://local.ptron/.pinfo/ - lots of stuff there ^_^
(views can be customized via the Info.css in the html folder)
* Added ability to import/export filter to and from the windows
clipboard. Right-click over either the web or header filter
lists - the import/export options are available in the context
menu.
Export will write any selected filters to the clipboard while
import will insert filters from the clipboard's text. The format
is the same as is used in Proxomitron's config files.
There's also an option in the merge dialog to accept input from
the clipboard rather than a file. It can be used to import any
config file settings - just be sure to include the filter section
headers with the text.
* Log window clipboard should be better behaved under NT/W2k/XP
* Proxomitron now has support for pre-request variables. These
can be set anywhere (even in the headers) and checked later
(say in the actual HTML). Each request has it's own set, and
is isolated from all other pages.
- To set a variable use...
$SET(variable=something)
- To clear a variable use...
$SET(variable=)
- To get a variable's value use...
$GET(variable)
- To test a variable's value use...
$TST(variable=match expression)
or also just...
$TST(variable)
can be used in a match to see if the variable's contents match
the current text. For example..
src="http://$TST(myhost)/"
note that this must be a literal match (except for case) - the variable's
value isn't treated as a matching expression with wildcards and such.
Unlike positional "\1 \2 \# ..." style variables, $SET expands a named
variable's value as soon as it's called (positional are expanded
in the replace). That means named variables can be set to themselves
and expanded - for instance...
$SET(foo=$GET(foo) more stuff) - adds to the end of "foo"
$SET(foo=more stuff $GET(foo)) - adds to the start of "foo"
$SET(foo=$GET(foo) \1) - adds value of "\1" to the end of "foo"
$SET(foo=$GET(foo)$GET(bar)) - adds value of "bar" to the end of "foo"
$TST can also be used on good-old positional type variables \0 ... \9
which allows you to do neat things like...
<\1\s * $TST(\1)>
which matches...
stuff
but not...
stuff
Note that to get this to work I had to make some big changes to the way
Proxomitron normally sets these variables. In this beta please watch out
for unexpected side-effects. Especially in cases where you use the same
variable more than once in a match. Proxomitron used to wait till the very
end to set variables which guaranteed that once matched it wouldn't change
till matched again. Now it may be possible for some failed match later on
to erase the previous value of a variable. Still I can't really find many
cases where this would actually be a problem and the trade off is probably
worth it.
Speed notes:
Like calling a list, testing a variable is a bit costly
It involves a lookup of it's name each time it's called so it's better
to not have it as the very first thing in a match...
SLOW: $TST(name=ayumu|osaka)
BETTER: $TST(name=ayumu|osaka)
Also more variables makes the lookup take longer, so if you know your
done with one, it's best to clear it: $SET(foo=)
* size specifiers on runs ( like "[a]+{0,500}" ) can now range up to
65535 rather than being limited to 255.
* Error messages now call "Errors.css" in the html folder
so you can customize the colors and fonts used
* Config name now shows in system tray tool tip (as per request ;-)
* Can now filter the CONNECT method used to establish
SSL pass-thru connections. That allows some minimal
filtering (basic site blocking and bypass list host
matching) even without using OpenSSL. CONNECT headers
are now also visible in the log window (though keep
in mind they're only used by the proxy and not passed
to the remote site).
* Bug fixed with $WESC not escaping closing parenthesis
* Added new DOM banner/container killer filter
* multi-part (push) content types not filtered by default
* Updated SSLeay/OpenSSL libraries used. Also added some basic
remote site certificate validation. See readme.txt for
new details
* Fixed bug that could sometimes cause characters not to be
filtered in the tester window when the match was exactly
1 character long.
* $LOG() command can now open the log window if it's closed.
just include a "!" as the first character before the color
indicator...
$LOG(!GLog Something green)
* Fixed "read overflow" bug (for real this time ;-)
* Fixed missing "&" in "Content-Type: Fix MIME types based on URL"
filter
* Altered Javascript filters to insert after the
tag on webpages (works better for IE6)
* Pop-up filters are improved a bit and now use the external
"WindowOpen.js" file in the html folder.
* $URL, $RESP, $IHDR, $OHDR, $TST all now work in the replacement
section too (where they can be useful to capture values).
* Latest DOM banner blaster and container killer included
this uses "DomConKiller.js" in the html folder
* Long command causing crash in NT/Win2k/XP fixed.
* Bug with long entries in log window fixed.
* Bug with handling certain malformed HTTP headers fixed.
* SSL socket reuse bug fixed (hopefully)
* \h will now match hostnames that don't have a trailing slash like
"http://foo.com"
* Proxy advanced options now allow a username with no password.
* Reloading a config no longer clears the debug flag.
New stuff in Naoko 4.4
* Added KEYCHK command. This can be used to test for keypress
combinations inside a filter. It works like any matching test
and can be used in a filter's match or a URL match. When the
match is done KEYCHK will return as "matched" if the keys
being tested for are currently pressed. Most keys can be tested
by using their *unshifted* values. However, there's several
special key tests which begin with a carat "^" symbol....
^C = CONTROL
^A = ALT
^S = SHIFT
^T = TAB
^F1 = Function key #1
^F2 = Function key #2
^32 = Virtual keycode 32 (spacebar)
^65 = Virtual keycode 65 ("A")
These are not case sensitive so either "^c" or "^C" may be used.
Multiple keys can be included to test for combinations. For
example...
$KEYCHK(^A^S) = ALT+SHIFT
$KEYCHK(^C^F6) = CONTROL+F6
$KEYCHK(^AG) = ALT+G
$KEYCHK(^C^A^S) = CONTROL+ALT+SHIFT
This will work for any combination the keyboard allows. Other
special keys can also be tested if you know the Windows virtual
keycode - this is a number between 0-255. For instance ^2 tests
for the right mouse button.
* Added LOGFILE option to blockfiles. Adding this keyword to the top
of any blockfile tells Proxomitron not to treat the contents as
matching tests or to load them into memory. This makes it safe to
use the file as a log file when using $ADDLST(). It also increases
speed - normally $ADDLST opens the file, appends the new entry, and
then closes the file again. However in LOGFILE mode the file will
remain open between writes to make inserting large amounts of data
much more efficient.
It's important to keep this in mind if you load a logfile into another
program while it's in use since recent entries may still be in the
file buffer and not written yet. However, reloading the config or
editing a log via the "Edit blockfile" option from a Proxomitron
menu will flush any unwritten data to the log. You can give your
logfiles a filename extension other than ".txt" then associate that
extension with a specialized logfile viewer if you wish. This allows
the "edit blockfile" option to be used to launch your log viewer.
* Added $FILE(filename) command. This will insert the contents of any file
into the replace section of any filter. It's most useful in web filters,
but will work in header filters too (keep in mind headers should only be
one line and no more than around 4k in size).
* Added $RESP(match) matching command. This command can be used to match
or capture the response code returned by a web server. Normally this
will look something like...
"200 OK" or "404 Not Found" etc.
The match starts directly with the return code so does not include the
"HTTP/x.x" portion of the reply line. However any message text after
the code is included in the match. For example, to match redirects
you could use "$RESP(302*)". This can also be useful to capture this
information for a log file.
* Entries added to a non-existent blocklist will cause a new "temporary"
list to be created with that name. This list will be memory only, and
can be used like a hash array to store information globally. Note that
the list won't appear on any menus and can't be edited (since it's not
a file), but can be called in a match and added to just like any other
list.
Also keep in mind reloading the config or restarting Proxomitron will
clear all temporary list. Use a normal file based list if you wish to
store the information longer term.
* Multi-match replace buffers can now insert any amount of data
as long as there's enough memory (previously they were limited to 32k).
Keep in mind when designing filters this does allocate extra memory,
so should be used carefully (especially if using the new $FILE command).
* Fixed error with stale file handles being reused - may help the
"half-loaded" page problem some people were reporting.
* Fixed reused connections still using the proxy after "Use remote
proxy" has been deselected.
* Fixed "Read overflow" warning from needlessly popping up.
* Fixed bug that could cause some hostnames not to resolve properly
when contacting a webserver on ports other than 80.
* Created a work-around for servers that send "deflate" content
encoding with/without the normal header bytes. Seems not all
servers format it the same.
* Updated proxcert.pem for another year. This is a self-signed
certificate Proxomitron uses for it's experimental SSL filtering
(not enabled by default). It's not an actual valid server
certificate.
Naoko 4.3
* Added $ASK(AllowList, DenyList, Prompt msg, item to add [,alternate match])
$ASK() automates the process of asking the user if a particular
item should be filtered at a given site. Although you can do the
same thing with combinations of some of the other new commands,
ASK wraps it up neatly in one small package.
* Added a NOAPPEND option to blockfiles. Blockfiles with this flag set
will not have new items added to the file itself when ASK or ADDLST
is called. The items will still be blocked, but only for the current
session or until the list is reloaded (whichever comes first).
It might be useful in cases where you only want to block something
temporarily, and I admit it's a bit of a hack. It may not stay
around too long if I get a better idea. ;-)
* Added $DTM(format) command. This can be use to insert date, time
and connection information into any replacement text. It uses a
simple format string to control the display and can be useful for
adding dates to a page, or including information for use with $LOG
or other similar commands.
* Added persistent connection support to local connections as well
as remote. For best results make sure your browser is set to use
HTTP/1.1 through the proxy and has pipelining enabled (if it's an
option). Also I recommend about 6-8 simultaneous connections. With
this, Proxomitron now supports nearly all of HTTP/1.1's optional
features.
For this to work Proxomitron must make heavy use of "chunked"
encoding since we can't know the size of a filtered item ahead of
time. This shouldn't be a problem, but it's possible some browsers
may not expect so many items to be "chunked" - especially images.
In testing Mozilla seems to have some random problems with chunked
encoding if pipelining is enabled, but the flaw doesn't seem to be
in Proxomitron's handling of things (Pipelining support in Mozilla
is new and may still be a bit buggy). Opera, which also heavily
pipelines, seem ok.
HTTP/1.0 persistence is also supported. However, since Proxomitron
frequently makes changes to an item's content-length and HTTP/1.0
does not allow for chunked encoding, fewer requests may be able
to persist if your filtering.
Local persistence can be turned off if necessary and it's keep-alive
time adjusted under the HTTP tab of the Config dialog. By default
local connections are killed after 10 seconds of inactivity. That
should be fine in most cases since new local connections are fairly
fast to create, but you may wish to increase this - especially if
you're using Proxomitron remotely over a LAN.
* Added new character hashing to matching routines. This can greatly
speed up OR separated lists where the initial item in each OR section
is not a wildcard. For example...
(one|two|(three|four|five)|six) is hashable while...
(one|*two|(three|four|five)|six) isn't, and neither is...
([to]ne|two|(three|four|five)|six)
Actually though each set of parens (...) has its own hash, so
in each case above the "(three|four|five)" is still hashed each
time. Also mixing in AND or ANDAND is usually ok too...
(one|two|(three|four|five)|six)*(end)&&*whatever*
(one|two|((three|four|five)*&*whatever*)|six)
This also means a complex match like (from the banner blaster)...
<(a\s*|i(nput*>|layer*|frame*))
is pretty much just as fast written like...
|||