Email has been the killer-app of networked computers for 40 years. It is a method of communication that everyone from the multinational corporation down to the lonely long haul trucker is using. These days, when anybody can have as many free gigabyte mailboxes as they want, email is as close as a netizen can get to an inalienable digital right. But grant someone a right, and it won't be long before they discover ways to exploit it for their own benefit and infringe upon the rights of others. And just like that, spam was born. Everyone gets it, so I guess it is just another inalienable digital right (or is it a wrong? Sigh.) The noise from all these unwanted emails has grown from an annoying buzz to a deafening scream relentlessly jangling the wires of the net. This noise is why people like Lefteris Chatzibarbas have created software like Imapfilter, to bring order to your email account.
At work I deal with two types of spam, internal and external. Internal spam is generated by all the servers on our network. They send out mail any time they have a hiccup, burp, or aneurysm. They cry out at night for their loving system administrators to come tuck them in, to listen to their nightmares, warnings, premonitions and misgivings. The next day I come to work and check my email only to find all these paranoid system notifications, shouts of triumph from successful cron jobs, and the occasional diarrhea of errors. External spam is the kind that everyone gets. It is the unsolicited din of all those netizens who exploit their right to email in order to make a buck. So along with the million or so obscure ramblings of the petulant servers on our network I am blasted by people trying to swell my penis to the size of a large marine mammal, trying to tell me about a man somewhere in Africa who wants to give me millions of dollars, about cheating housewives, discount pharmaceuticals, and a large vibrating egg that can be mine, free, with my paid subscription to "Studly Donkey" magazine. This and a million other types of garbage gluts my inbox. 99% of it is stuff that I don't understand or care about, but then there is that other 1 percent.
What is a netizen to do? How can we claim our right to email? How can I possibly read through the 1% of good clean mail without getting my hands dirty on all that other trash?
TaDaaaa! The answer is Imapfilter. Imapfilter is the ultimate tool for separating the the wheat from the chaff. Control freaks need not look further for a way to sort and organize mountains of mail across multiple accounts. Let me warn you now, if you don't feel comfortable writing short scripts then Imapfilter isn't for you, otherwise I encourage you to read on. Getting Imapfilter is easy. Any Linux distribution will probably have a package for it (Debian and Fedora do). Install it through you package manager. Then you need to create a directory in your home directory called .imapfilter. underneath that directory you put the imapfilter configuration file. It is called config.lua. This file tells imapfilter which servers and accounts to connect to, and lets you sort, delete, copy, and flag your emails according to your whims. (Read the man pages on imapfilter or imapfilter_config for details on configuring and running it.) This config file very cleverly uses an embedded scripting language called lua, which means you can do neat tricks with the mail you process. At work I run imapfilter every 10 minutes out of cron. It sorts my mail using a set of rules I have written: spam goes to the spam folder (thanks spamassasin), system noise (messages to root, cron output) goes to a system folder, mail from people on my white list (you know who you are) goes into my mailbox. My email is completely under control (I'm only reading that 1 percent).
Recently I decided to take my Imapfilter experience to the next level. I am using more of the lua scripting capabilities now. Certain emails (like errors) are handled like events. For example, I am responsible for dealing with bugs filed against my software in a timely manner, so whenever I get a notice from the bugzilla daemon Imapfilter sends my cell phone an sms notification containing the subject line and comments associated with the bug. I also configured Imapfilter to play a sound like a cricket chirping whenever one of my web applications sends out an error email. This is nice because I don't get notifications for every single email I receive, just the ones I think are important enough for me to take time out for. I also have Imapfilter log the subject lines of my application's error emails, that way I can tail the log when I hear the cricket chirp and see what kinds of errors are coming across without actually opening up my email client.
What follows is an excerpt from my code. I hope the comments are enough to make it make sense. I will also provide a few links to good lua programming resources and that should be all you need to really get going with custom email handling. In an attempt to make things shorter and more readable I have removed lots of the variables that I setup and use.... so this code wouldn't run by itself. If anyone is really interested in getting help with this or working code for this I'd be happy to be of help just drop a line in the comments.
[code]
---------------
-- Functions --
---------------
-- ------------------
-- last_index(a, b)--
-- ------------------
-- same as the first_index function below but for the last index
function last_index(a, b)
return b
end
-- ------------------
-- first_index(a,b)--
-- ------------------
-- a function you can call with the find command to get the first index of somethign
-- example first_index(find(string, regxp)) will return the starting index of the
-- the first match of regxp in the string.
function first_index(a,b)
return a
end
-- ----------------------
-- parse_bugzilla(body)--
-- ----------------------
-- parse a table of bugzilla messages for the "comment"
-- returns an array of parsed messages.
function parse_bugzilla(body)
result_array = {}
count = 0
if (body ~= nil) then
for id, text in pairs(body) do
--pull the index of the last string before the comment of the bug begins
index = last_index(string.find(text, ' %-%-%-%-%-%-%-', last_index(string.find(text, '%-%-%-%-%-%-%- '))))
--get 120 characters of the comment (our sms can only be 160 characters)
if (index ~= nil) then
end_index = first_index(string.find(text, '%-%-% ', index))
if (end_index ~= nil) then
if ((end_index - index) <= 120) then
text = string.sub(text, index, end_index)
else
text = string.sub(text, index, index + 120)
end
elseif (string.len(text) >= index + 120) then
text = string.sub(text, index, index + 120)
else
text = string.sub(text, index, string.len(text))
end
else
text = nil
end
-- store the parsed text into the array
result_array[count] = text
count = count + 1
end
end
-- return the array of parsed messages
return result_array
end
-- ----------------------------------------
-- set_table(subject_table, message_list)--
-- ----------------------------------------
-- take a table of message subjects that match up with a list of message
-- bodies (for example the result of the parse_bugzilla function) and
-- create a new table that pairs up the message subject and bodies
function set_table(subject_table, message_list)
new_table = {}
count = 0
if (subject_table ~= nil) then
for from, subject in pairs(subject_table) do
-- strip unnecessary string Subject: from the beginning of each subject
subject = string.gsub(subject, "Subject:", "")
new_table[subject] = message_list[count]
count = count + 1
end
end
return new_table
end
-- ----------------------------------------
-- send_sms(message_table, carrier, phone--
-- ----------------------------------------
-- take a table of messages of the form (subject, message) and send them
-- via sms to a mobile phone. messages with nil values are skipped.
-- carrier is a email server for a particular phone company tmobile = tmomail.net
-- phone is your 10 digit phone number.
function send_sms(message_table, carrier, phone)
if (message_table ~= nil) then
for subject, message in pairs(message_table) do
--send an sms text message of the text summary truncates at 160 characters.
if (message ~= nil) then
os.execute ("echo '".. message .. "' | /bin/mail -s '" .. subject .. "' " .. phone .. "@" .. carrier)
end
end
end
end
-- ---------------------------------
-- log_mail(results, user, folder)--
-- ---------------------------------
-- logs the subject of a mail to a file called maillog.
function log_mail(results, user, folder)
if(table.getn(results) ~= 0) then
subjects = fetchheaders(user, folder, {'subject'}, results)
FILE = io.open ("/home/josh/.imapfilter/maillog", "a")
for message, subject in pairs(subjects) do
-- log the subject field of the header.
subject = string.gsub(subject,"%s", "")
io.output(FILE):write(os.date() .. ": " .. subject .. "\n")
end
FILE:close()
end
end
-- ------------
-- doorbell()--
-- ------------
-- play a doorbell sound is a table has data
function doorbell(results)
if(table.getn(results) ~= 0) then
os.execute("/usr/bin/play /home/josh/.imapfilter/Bell_1.wav")
end
end
---------------
-- Filters --
---------------
-- this variable shows how you can create rules for individual emails
-- I use this "apps" rule for mail I am supposed to be watching. I can then
-- use imapfilter to flag these emails, move them to a particular folder, or whatever
-- else i can come up with. The invert=true thing changes the usual imapfilter behavior,
-- which is to OR things together to AND things together.
apps = {
invert = true,
{ 'from "Application@www.somwhere.org"', 'subject "A Notification"', },
{ 'to "foo-sysadmin@bar.org"', 'subject "User Rejected"', },
{ 'from "root@foo.baz.org"', 'subject "PERSEUS TPS Log Cleanup Report"', },
{ 'from "root@foo.baz.org"', 'subject "TPS PIPES Cleanup Report"', },
{ 'from "root@foo.baz.org"', 'subject "PERSEUS Log Backup Report"', },
{ 'from "randy@goofball.org"', 'subject "mcsb_extract"', },
{ 'from "ftpuser@foo.baz.org"', 'subject "board:cron:"'},
{ 'from "bob@foo.baz.org"', 'subject "LDAP PASSWORD CHECK"'},
{ 'from "www@balloey.org"', 'subject "Processing Error"'},
{ 'subject "DEMO:"'},
{ 'subject "DEVEL:"'},
}
----------------
-- Commands --
----------------
-- What follows is the actual imapfilter code that moves, flags, sorts my mail.
-- First archive all the old crap from my mailbox
-- put it in backup folders by year/month
results = match(josh, 'mail', SixMonthOld)
dest = "archive/" .. Year .. "/" .. Month
move(josh, 'mail', josh, dest, results)
-- Move all messages from inbox to a temporary folder 'ztmp' for processing
results = match(josh, 'INBOX', all)
move(josh, 'INBOX', josh, 'ztmp', results)
-- Move SPAM to SPAM folder
results = match(josh, 'ztmp', SPAM)
move(josh, 'ztmp', josh, 'SPAM', results)
-- Flag messages from my apps
results = match(josh, 'ztmp', apps)
flag(josh, 'ztmp', 'add', { 'flagged' }, results)
-- play a notification sound that important mail has arrived and log the subject lines
doorbell(results)
log_mail(results, josh, 'ztmp')
-- Move bugzilla messages to bugzilla folder and send sms alert
results = match(josh, 'ztmp', bugzilla)
parsed_body_table = parse_bugzilla(fetchbody(josh, 'ztmp', results))
subject_table = fetchheaders(josh, 'ztmp', {'subject'}, results)
ready_to_send_table = set_table(subject_table, parsed_body_table)
send_sms(ready_to_send_table, Carrier, Phone)
move(josh, 'ztmp', josh, 'bugzilla', results)
[/code]
References
1 comment:
Thanks for the useful example. I use imapfilter too, but I needed to know how to use the flag commando to mark messages as read.
Now I know that thanks to your example!
Thanks!
Post a Comment