Update 24 May 2010: I have added some more features to Dropbox and I have released my new “Dropoff” over at www.dropoff.me. Get all the latest and greatest there in future!

I have found a great solution to the common problem of sending and receiving files from other sites and research partners, and generally sending large files around the web where email won’t do the trick.

It’s called “
Dropbox” and was originally written by the University of Delaware.

I am launching it as a service at work called “
Dropoff” in order that people don’t think you are talking about the service provided by www.dropbox.com which is a totally different thing.

The idea is that you don’t even need to login to send a file to a user within your site/company/University/institution, so external people can use it to send files to people in your institution. It can handle arbitrarily large files, there are no fixed limits. If you log in to the Dropbox website, you can send files to people outside your institution. People who cannot log in can only send files to people within your institution. This stops the rest of the world using it to send people to other people who aren’t members of your institution.

I have added various extra features to it:
  • Active Directory AD authentication (to multiple AD sites at once if needed)
  • Virus scanning of uploaded files, using ClamAV

I have customised it quite a bit just for our site, so if you want a copy of my patched version, along with a guide as to what changes I have made, then please
contact me.

Anti-Phishing and Spear-Phishing Version 2

Update 11 October 2009: This has now been moved to www.ScamNailer.com. Please check there for all future information and updates to this package.

Update 20 September 2009:
The Google-hosted data file has been moved to SourceForge, so I have updated the URL it downloads it from. You need to update your script to the new version 2.05.

Update 16 June 2009:
I have changed the rule structures to make them considerably faster than the old ones. Download the updated script from the link below.

I have acquired a new reliable feed of email addresses used in phishing attacks. These addresses have all been checked by real people, and they come from a very reliable and well-known source.

The new data file is provided by means of DNS and an Anycast network, which makes it pretty resilient to attack. The previous spear-phishing data is gathered from the project hosted by Google in the traditional way, that hasn’t changed.

I have updated my script so that it fetches both sets of data. It makes use of a temporary directory under /var/cache, which is configurable at the start of the script, and which needs to be writable by the user the scripts runs as (normally just ‘root’ so this doesn’t present any problem at all to most people).

You can
download version 2.05 of the script.

If you are not using MailScanner with this script, you will need to comment out or delete the line that mentions “service MailScanner reload” about 1/3 of the way down the script (search and ye shall find!).

For more explanation of this whole problem and the way this script works, please refer back to
my earlier article.

Anti Spear Phishing

Update 2009-October-11: This package is now hosted at www.scamnailer.com. Please check there for all future information and updates.

Update 2009-June-15:
There is now a brand new additional data feed of known phishing email addresses, which I have added to my script.

Spear phishing is a technique used by spammers and scammers to try to get your email username and password. They send you an email claiming to be from your email provider, in which they say that your account will be deleted unless you supply them with your username and password “for authentication” or some other similar ruse.

If they get your username and password, they then use your email account and email provider to send out millions of spam messages. Because the spam comes from a genuine email system (yours!) it will be accepted by most sites and will automatically pass many spam checks.

I have written a script which takes a file of addresses commonly used in these attacks. It also allows an additional list of addressed you can add to. From these, it generates a set of SpamAssassin rules that detect the presence of these addresses, which can be used in MailScanner to stop the spear-phishing attacks completely.

Download the script
here. Note that the script is gzipped to ensure your browser doesn’t do anything silly when fetching it, so you’ll need to “gunzip” it before doing anything with it. To start with, just copy it into your “/etc/cron.hourly” directory, and run the command “chmod a+rx /etc/cron.hourly/Spear.Phishing.Rules” to make it run every hour.

It is pretty much a finished script, and is directly usable by you guys without you having to do much to it except read the settings at the top and tweak the filenames if you want to change where it puts things.

I have taken a lot of care to ensure that this won't match any false alarms, I don't just dumbly look for the strings in any surrounding text, which certain commercial AV vendors have been caught doing in the past!

I make a suggestion in the comments at the top of the script about how I use the rule within MailScanner, you probably want to do something similar, and not just delete anything that matches, just in case you do get any false alarms.

It also looks for numbers at the end of the username bit of the address, and assumes that these are numbers which the scammers may change; so if it finds them, it replaces them with a pattern that will match any number instead. There's starting to be a lot of this about, as it's the easiest way for the scammers to try to defeat simple address lists targeted against them, while still being able to remember what addresses they have to check for replies from your dumb users. Happy I thought I would make it a tiny bit harder for them...

You can also add addresses of your own (which can include "*" as a wildcard character to mean "any series of valid characters" in the email address), one address per line, in an optional extra file. Again, read the top of the script and you'll see it mentioned there. That file is optional, it doesn't matter if it doesn't exist. As a starter, you might want to put
m i c h a e l l o u c a s * @ g m a i l . c o m
(without the extra spaces) in that file, as it will nicely catch a lot of "Job opportunity" spams.

It looks for any of these addresses appearing **anywhere** in the message, not just in the headers. So if you start talking to people about these addresses, don't be surprised when the messages get caught by the trap.

It does a "wget", so make sure you have that binary installed, or else change the script to fetch the file by some other means.

The very end of the script does a "service MailScanner restart", so if you need some other command to restart MailScanner or your SpamAssassin setup, then edit it for your system. It needs to be a "restart" and not a "reload" as I have to force it to re-build the database of SpamAssassin rules. If you don’t use MailScanner, but do use “spamd” in some setup or other, then a simple “service spamd restart” would do at the end of the script.

My aim was that, on a RedHat system running MailScanner, you could just copy the script into /etc/cron.hourly and make it executable, and it will just get on with the job for you. I do advise you read the bit in the script about "SpamAssassin Rule Actions" though.

Please do let me know how you would like me to improve it, and tell me what you think of it in general. (be polite, now!)

Update 13th January 2009:
A colleague on the MailScanner mailing list has made this simpler to use. You don’t have the flexibility of adding your own addresses to the list, but you can get the latest list along with all your regular SpamAssassin updates with the “sa-update” command.
Here are his instructions:
wget http://www.bastionmail.co.uk/spear.txt
sa-update --import spear.txt
Add “spear.bastionmail.com” to the list of channels that you update from (either add “--channel spear.bastionmail.com” to your sa-update command, or add “spear.bastionmail.com” to the file pointed to by the sa-update “--channelfile” command-line option).
Add the key “06EF70A3” to the trusted keys (either add “--gpgkey 06EF70A3” to your sa-update command, or add “06EF70A3” to the file pointed to by the sa-update “--gpgkeyfile” command-line option).
Then these SpamAssassin rules will be automatically updated every time your system runs the “sa-update” command, which is daily on a standard MailScanner system.

Talking from sendmail to Exchange over SMTP auth

There are various things you can do in Exchange, such as control who can address distribution lists, that can be restricted to authenticated senders only. So how do you make your sendmail box an authenticated sender?

Start at
http://www.sendmail.org/~ca/email/auth.html -- about half way down it starts talking about “Using sendmail as a client with AUTH”. That tells you how to setup your sendmail box (which is the client) so that it talks SMTP auth to Exchange (which is the server).

Building an MX the Easy Way

Boot off RHEL5 disk 1.
linux rescue
Activate network interface eth0 (first interface).
Give IP and so on.
Use fdisk to create
/dev/sda1 /boot Linux 100Mb
/dev/sda2 Linux swap 2048Mb
/dev/sda3 / Linux all the rest
mkfs.ext3 /dev/sda1
mkfs.ext3 /dev/sda3
mkswap /dev/sda2
mkdir /mnt2
mount /dev/sda1 /mnt2
ssh crow ‘dump 0f - /dev/sda3’ | ( cd /mnt2 && restore -rf - )
umount /mnt2
mount /dev/sda3 /mnt2
ssh crow.ecs.soton.ac.uk ‘dump 0f - /dev/mapper/VolGroup00-LogVol00’ | ( cd /mnt2 && restore -rf - )
or else
ssh crow.ecs.soton.ac.uk ‘cd / && tar clf - .’ | ( cd /mnt2 && tar xvBpf - )
Fix /mnt2/etc/fstab so it points to all the right partitions.
If you really want to use partition labels, use the “e2label” command to set the label of each partition so that your shiny new /etc/fstab can find them. Syntax is obvious: /sbin/e2label device [ new-label ]
umount /mnt2

Installing Grub
Cloning an RHEL4 system using the RHEL5 rescue disc? You must use Grub from RHEL4 to setup a boot record for an RHEL4 system. So mount the (newly copied) root filesystem in /mnt2 and copy /mnt2/sbin/grub to /sbin/grub. Then follow the instructions below.
mkdir /boot
mount /dev/sda1 /boot
rm -rf /boot/boot
grub> root (hd0,0)
grub> setup (hd0)
Edit /boot/grub/grub.conf and change the root command to (hd0,0) and the kernel root-filesystem argument to /dev/sda3.
Repeat that edit for all the other kernels available.

Unplug network interface
Reboot and it should boot from hard disk
cd /var/spool/mqueue.in
rm -f *
cd /var/spool/mqueue
rm -f *
cd /var/spool/MailScanner/quarantine
rm -rf *
cd ../incoming
rm -rf *
cd ../archive
rm -rf *
cd /var/log
Remove all old logs
service syslog restart
Fix ethernet and IP address in /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network
Fix /etc/hosts
Repair ownership and permissions of /home/* and /usr/local/share/clamav (and subdirectories).
Fix extra ClamAV databases so that “MailScanner --lint” runs correctly.
Reboot with network interface connected.

Re-register with RedHat network for yum updates, get the info from the Systems KB.
yum update