[olug] Python based replacement for Procmail

[William E. Kempf]

Hi all,

I have a love/hate relationship with procmail.  Being the "defacto
standard" for filtering e-mail on Linux, I've been using it for several
years now.  It works, but there are definately things I dislike about it. 
Here's a list:

* The syntax is ugly.  Colon, no colon, trailing slash for maildir vs.
mbox, etc.  I know the syntax pretty well, but I still make typos that
bite me big because of these warts.

* Flags are not always intuitive (more complaints like the first).

* The $MATCH variable is limiting.

* Complex recipes required.

To explain the last two... I receive a ton of e-mail every day.  I
subscribe to and unsubscribe from mailing lists very frequently (I'm a
developer and find myself subscribing to many lists on programming topics
permenantly and even temporarily as I await a response to a query).  In
order to keep this chaos of e-mail under control, I've had to develop very
complex recipes that handle around 70% of mailing lists with out any need
for me to modify my .procmailrc.  Unfortunately, those complex recipes are
fragile, and often require multiple nested matches to split the $MATCH
variable up into multiple values (such as user and host portions of an
e-mail address).  And since the syntax is so ugly (IMO), I wind up
breaking my scripts more often than not when I try and tweak them to
handle some mailing list that manages to pass through uncaught by the
current scheme.

So, I decided I wanted a new solution.  Procmail is nice, but it's just
not for me.  I thought a filter in a full scripting language would be much
better, and since I like Python, I thought that would be the best solution
for me.

So, lots of searching revealed only one solution... Pycmail.  And I hated
it's design.  Despite the claim to be a Python version of Procmail, it
just wasn't much like Procmail, or what I felt a filter script should be
like.  So, I decided I needed to write my own.  I scoured the net once
again for other mail filtering scripts in any language, looking for
inspiration for a design for my own filter script.

Goals:

* Keep the syntax simple.  Non-python programmers should be able to write
simple filters with out much effort.

* Avoid global variables as much as possible, as they usually lead to
errors.  (Current library uses only one global variable).

* Behave as much like procmail as possible, but with out it's syntax
problems.  (For instance, by default storing a message ends the script,
though you can specify you want the script to continue.)

* Be as failsafe as possible.  I've done a reasonable job at this so far,
but my knowledge of this area is lacking enough that I may have missed
some important things.

The result is a Python module and a script I currently call "pyfilt"
(suggestions for a better name much appreciated).  I'm currently running
this script myself, with no issues, but I consider the code to be "alpha"
quality.  There's probably bugs lurking in there that I have not found
yet, and the interface still needs some adjusting in areas.  Despite that,
though, there may be others out there that are interested in such a
script.  If you're interested, let me know in private e-mail, and I'll
send you a copy.  I'd certainly appreciate the feedback and bug reports of
others.

In addition to feedback and bug reports, if there's anyone interested in
helping with the development of such a library, I'd be interested in doing
the collaboration.  In addition to the core library, I'm interested in
add-on functionality (such as interfacing to other filtering tools, such
as spamassassin), or with help in creating an add-on that will
"auto-magically" handle most mailing list traffic.  Using the tricks I
learned from Procmail, I can currently achieve about 70% on this, but want
to work on getting this up to 95% or better.

Sorry if this rambling was boring for anyone, but if you read this far and
are interested in the script or in helping in any way, PLEASE send me a
private e-mail.

Thanks,

-- 
William E. Kempf