iMS and SpamAssassin (populating OUTPUT_HEADERS unsuccessful)

Using iMS 5.2sp04 on Sol 8

Conversion channels work just fine (using it right now for Anti-Virus with success).

Trying to drop external mail through conversions to spamassassin for processing, but to no avail.

What I am able to do:

I can rewrite OUTPUT_FILE and pass on information

I can rewrite OUTPUT_HEADERS, but the changes don't take

This is what the conversion channel looks like at present:

!

in-channel=*; out-channel=*;

in-type=*; in-subtype=*;

message-header-file=2; original-header-file=1;

override-header-file=1; override-option-file=1;

command="/usr/iplanet/server5/msg-adhcpdns-1/imta/programs/tst.csh"

!

This is what tst.csh looks like:

#!/usr/bin/csh

mv $INPUT_HEADERS $OUTPUT_HEADERS

echo "X-Test: This is a test" >> $OUTPUT_HEADERS

mv $INPUT_FILE $OUTPUT_FILE

echo "This is a test" > $OUTPUT_FILE

What I'm running into:

I am able to manipulate the OUTPUT_FILE just fine. Anything I put in there ends up at the users' mail. However, though I am able to write to the OUTPUT_HEADERS, INPUT_HEADERS, MESSAGE_HEADERS, or other such files (env variables...), I still end up with the original headers by the time the message lands in the user's mail box.

Explanation of above settings:

in-channel=* --> for testing, we'll test ALL channels

out-channel=* --> ibid

message-header-file=2 --> have tried 0, 1, 2 to no avail

original-=header-file=1 --> have tried 0, 1 to no avail

override-header-file=1 --> have tried 0, 1 to no avail

override-optioni-file=1 --> have tried 0, 1 to no avail

I have replaced INPUT_HEADERS with:

MESSAGE_HEADERS

OUTPUT_HEADERS

ORIGINAL_HEADERS

in the script, to no avail.

Now, you wonder, so where is spamc in all of this? Well, I had to dumb it down to the csh above, since I wasn't getting the manipulation I was hoping for.

Using spamc, it would look something like this:

#!/bin/csh

set SERVER_ROOT =/usr/iplanet/server5

set INSTANCE=instance-name

set SPAMDIR="${SERVER_ROOT}/msg-${INSTANCE}/imta/queue/SPAMDIR" # Make sure this exists!!!

# Set a reference ID for the working file

set REFERENCE_ID=`/bin/cksum ${MESSAGE_HEADERS} | /bin/awk '{print $1}'`

set RECORD_FILE=${SPAMDIR}/SpamAssasin-${REFERENCE_ID}.spam # This is the file we'll use for processing

# put a blank in the INPUT_HEADERS file before feeding it to spamc

# This is because the HEADERS and FILE are separated in SMTP with a "\n", but they are

# not separated this way on the system

/bin/echo "" >> $INPUT_HEADERS

/bin/cat $INPUT_HEADERS $INPUT_FILE | /usr/local/bin/spamc -u spamd -f >& ${RECORD_FILE}

# Now, where does that first "\n" occur? In C-Shell, you cannot grep for "^$", which is more

# convenient. Furthermore, when you use the -n option in grep, you end up with

# a ':' separator and the name of the files. We have to stript this off.

# Finally, we subtract "1" so that we start at the proper location BEFORE

# the first "\n"

@ Split = `/bin/grep -nv "." ${RECORD_FILE} | /bin/head -1 | /bin/cut -f1 -d: ` - 1

# Now let's write out the HEADERS from the RECORD_FILE

/usr/bin/head -${Split} ${RECORD_FILE} >& $OUTPUT_HEADERS

#Shouldn't need to rewrite the body if you don't want the reports

#@ Split = $Split + 2

#/usr/bin/tail +${Split} ${RECORD_FILE} >& $OUTPUT_FILE

/bin/mv $INPUT_FILE $OUTPUT_FILE

# This is to demonstrate that spamc did it's processing, and everything worked....

cat $OUTPUT_HEADERS >> $OUTPUT_FILE

exit 0;

If you run it through this script, you will see that the RECORD_FILE (in SPAMDIR) gets the header rewrites just fine, and the OUTPUT_FILE gets populated with the header rewrites at the end. However, the OUTPUT_HEADERS don't follow the message through the messaging system, for some reason.....

[4254 byte] By [706874] at [2007-11-25 8:08:43]
# 1

Keep in mind the following:

OVERRIDE-HEADER-FILE 0 or 1. If set, then MIME headers are read from the OUTPUT_HEADERS symbol, overriding the original headers in the enclosing MIME part.

My point being that you're probably able to modify the headers of the MIME part of the message on which your script is running, but that is all. I don't recall what happens if you send in a message with only a text/plain part for example.

It has been a while since I've had to tweak with things. I know of others that are having difficultly with SpamAssassin with PMDF as well. I've heard rumors of a shared image being developed or written but not sure. I'll dig around my archives and if I find anything I can post I will do so.

707213 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 2

Problem is:

The message I'm using to test is text/plain, so it passes from tcp_local right to the conversion channel. I write to the OUTPUT_HEADERS file, and even cat the contents of this to a debug file to make sure the changes took, but when the message passes on, the contents remain unchanged (regardless of how burried or shallow the message part has been).

So, basically, I still cannot modify the headers (any of them) for the message in question.

There must be another flag somewhere to set?

706874 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 3
I just received word that iMS 5.2p1, which will release shortly (don't have a date yet), will include a methodology to write through libspamass.so to spamd. There should be some documentation with the update/release indicating how to set this up.So, for now, just wait and see.
706874 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 4
Is there an update on this? Patch 1 is out and it doesn't contain this functionality. Do you know if it's coming soon?Ryan
709309 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 5

It works with libspamass.so here.

Here is a copy 'n paste from: Alton Yu

Just compile spamassassin per the instructions that come w/ it.

I have only set it up MTA wide, not for individual users. It's kind of set up like for sieve filters. Couple of things you will need is to be running iMS 5.2 hotfix 0.9 or greater and direct ldap.

After installation, here are the steps:

add to the tcp_local channel keywords - sourcebrightmail

add to the option.dat file:

BRIGHTMAIL_CONFIG_FILE=<serverroot>/<msg-hostname>/imta/config/Spam Assassin

BRIGHTMAIL_LIBRARY=<serverroot>/bin/msg/imta/lib/libspamass.so

BRIGHTMAIL_NULL_ACTION=data:, redirect "test3";

(in this case, I'm redirecting all spam to test3 -- you can do whatever you want provided you follow RFC 3028)

and create the <serverroot>/<msg-hostname>/imta/config/SpamAssassin with contents:

HOST=127.0.0.1

PORT=783

DEBUG=1

MODE=0

Then do imsimta refresh and you should be set.

Before starting spamd however, I would set the /etc/mail/spamassassin level to something like 7-10 -- I think 5 is too little. I also have a huge whitelist.

Let me know if you have any problems.

Alton

bdelbono at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 6

Has anyone been able to get the sourcebrightmail keyword to work with SpamAssassin 2.50 and iMS 5.2? I followed the example attributed to Alton Yu. I can see in the debug logs that the incoming messages are sent to spamd by libspamass.so and spamd identifies them as spam, but the messages are not redirected. In face, in the slave_debug log I see this:

14:46:36.45: Creating socket to connect to SpamAssassin

14:46:36.45: Binding SpamAssassin socket

14:46:36.45: Connecting to SpamAssassin

14:46:36.47: Sending SpamAssassin announcement

14:46:36.47: Sending SpamAssassin the message

14:46:36.48: Performing SpamAssassin half close

14:46:36.48: Reading SpamAssassin status

14:46:36.95: Status line: SPAMD/1.1 0 EX_OK

14:46:36.95: Reading SpamAssassin result

14:46:36.95: Result line: Content-length: 200

14:46:36.95: Reading SpamAssassin symbols

14:46:36.95: Symbols line:

14:46:36.96: Verdict line: X-Spam-Flag: t-length: 200

14:46:36.96: Closing connection to SpamAssassin

14:46:36.96: Brightmail verdict #1, status = 0

14:46:36.96:Brightmail verdict: Deliver normally.

So I am missing something here. I created a support ticket with Sun/iPlanet and I was told that SpamAssassin was not supported. That was all the help I got.

704944 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 7

There is an easy fix to this. SpamAssassin 2.50 sends content length along with the result. The SpamAssassin interface in ims is expecting the result, where spamd is sending other info. You need to modify the following line in spamd (line#587). -

From -

printf $client "Content-length: %d\r\n\r\n%s",

length($msg_resp), $msg_resp;

To -

printf $client "$spamhdr \r\n%s\r\n";

Also, you do not need to enable directldap unless you are wanting to take advantage of more advanced PMDF features like fileinto.

Brad

704647 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 8
Hi:Can somebody give me the complete step-by-step procedure for configuring SpamAssassin-2.52 on iMS 5.2 on a solaris machine.Thanks Sunny
704823 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 9

Have you figured out how to write to $MESSAGE_HEADERS and the changes take effect ?

I am able to write to the file $MESSAGE_HEADERS but can't see what I added when receiving the message. why is that ?

I am doing this because I want to redirect a message depending on its attachment. I am trying to change its header ( the "To" field ) in order to redirect it.

please it's urgent, I would be very thankful if you help.

704854 at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 10
Anybody succeeded configuring SA with SQL server and iMS to allow individual user to change their preference? Could you please share your experience? Thanks.
Yang Chen at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 11
If I do anti-virus scanning using conversion channel and anti-spam scanning through libspamass.so at the same time, which one will happen first? I hope the anti-virus scanning will happen first.
Yang Chen at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 12
If I do anti-virus scanning using conversion channel and anti-spam scanning through libspamass.so at the same time, which one will happen first? I hope the anti-virus scanning will happen first.
Yang Chen at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 13

Hello. I run iMS for a small company. We are running 5.2p1. I've been trying to get my hands on the libspamass.so file to integrate with SpamAssassin but having no luck. Apparently it only comes with a hotfix, not the patch, and we are too small to have a service contract. Does anyone have any suggestions on how to get this? All I need is the hotfix that includes the SpamAssassin support.

Thanks,

-Matt

Matt Mosley at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...
# 14
You can log a one-time service call, and get the hotfix. Unfortunately, we're not allowed to send it out otherwise.
jay_plesset at 2007-7-1 14:00:49 > top of Java-index,E-Mail, Calendar, & Collaboration,Sun Java System Messaging Server...