Troubleshooting Email Sent From Scripts

Your script is supposed to send an email to you. Yet, you never receive it.

What's the problem?

Finding the answer to that can be quick or a long, involved process. Here are the steps I generally take when email troubleshooting:

Reverse Script-Breaking Changes

If the script previously sent email, restore the previous version and settings. If the previous version is not available, one can try to undo any changes that might have been done since emailing was successful or continue with the rest of these steps until the breaking change is found.

Especially check for a -T flag at the end of the first line of the script. That can seriously interfere with sending information to sendmail. (If the -T flag must be on the first line for security, there are workarounds, but outside the scope of this article.)

Check for Typographical Errors

It's easy to mistype something and not notice it, especially if it's something done so often it has become rote.

1. Check the location of sendmail is correctly specified. Common typos are absence of the leading / character and spelling "usr" as "user".

   Also, verify sendmail has the -t flag correctly specified. Some scripts automatically add this flag, others don't.

2. Verify the email address is valid. To do this, copy the address you're using and paste it into the destination field of your email program and send the email. See whether or not the email arrives.

   Do a copy 'n paste of the address when doing that test. If retyped instead of pasted, a previous typo may be missed.

Compare With Scripts Successfully Sending Email

If other scripts installed on the same domain successfully send email to the same email address, find the part of the email or emailing code that is different than the one with issues.

This requires enough programming skill to read the code and spot where the emailing occurs. The differences might be subtle, like different sendmail flags or "From" email header line.

When comparing differences, consider all sendmail flags and email header lines.

Verify the Script Is Properly Formatting the Email

Maybe the script isn't formatting the email like it is supposed to. Or maybe it's not piping anything to sendmail at all.

One way to check this is to make the script write the email to a file instead of piping it to sendmail. In many scripts, this is quite easy to do.

If you have a line something like this:

open MAIL,"|/usr/bin/sendmail -t";

change it to:

#open MAIL,"|/usr/bin/sendmail -t";
open MAIL,">emaildump.txt";

Notice the original line is commented out and the added line, using the same file handle name, opens a file for writing. This caused whatever would have been sent to sendmail to be written to the file, instead.

Now run the script and review the content of emaildump.txt

When done with this testing, remove the added line and uncomment the original.

Identify Restrictions

Restrictions I've found, some contradictory, include:

1. The sendmail program won't send email to a domain on the same server it is installed on.

2. The sendmail program won't send email to any domain except domains on the same server it is installed on.

3. The Return-Path header line must be specified with a valid email address. (With sendmail, the Return-Path header line is specified with the sendmail -f flag, like: -fname@example.com (no space between "-f" and the email address.)

   The email address in the -f flag might be restricted to an address of the same domain where sendmail is running on.

4. The Return-Path header line must be omitted (-f flag must be omitted).

5. sendmail has outgoing email restrictions like:

   1. Limit on number of emails in a certain time period.

   2. Allow only authorized accounts to send email.

   3. Filter outgoing email for spam-like characteristics.

6. Any email sent from the server must be sent by a special script the hosting company provides.

7. Email filtering at the receiver's ISP and/or at the destination mailbox itself prevent the email from being delivered or route it somewhere unexpected.

Checking these steps used to take be a lot of work and take a lot of time. Each variation needs to be tested.

Recently, I got the brilliant idea to create a script to automatically test many of the above. Leave it to Will to automate the arduous.

The script is here:

#!/usr/bin/perl
use strict;


# Between the parenthesis, list the location or locations of sendmail to be 
#    tested, without -t or other flags, one per line (blank lines are okay):
my $SendmailLocations = qq(
/usr/sbin/sendmail
/usr/lib/sendmail
);


# Between the parenthesis, list any gratuitous email header lines to be 
#    included in the outgoing email, one per line (blank lines are okay):
my $ExtraHeaderLines = qq(

User-Agent: Thunderbird 1.5.0.5 (Windows/20060719)

Content-Transfer-Encoding: 7bit

Content-Type: text/plain; charset=ISO-8859-1; format=flowed
MIME-Version: 1.0

);



#####################
#####################
#####################

sub SendTheMail
{
   my ($mailer,$destination) = @_;
   my $email = <<EMAIL;
To: $destination
From: $destination
Subject: [emailtest] ($mailer)
$ExtraHeaderLines
Mailer: $mailer
EMAIL
   print "<p> <br>Now sending:<br><textarea cols=77 rows=10 wrap=off>$email</textarea></p>";
   if(open MAIL,"|$mailer")
   {
      print MAIL $email;
      close MAIL;
   }
   else { print "<p><b>$!</b></p>"; }
} # sub SendTheMail


sub ConformCustomEntry
{
   my $s = shift;
   $s =~ s/^\s*(.*?)\s*$/$1/s;
   my @ta = split /[\r\n]+/,$s;
   for (@ta) { s/^\s*(.*?)\s*$/$1/s; }
   $s = join "\n",@ta;
   return $s;
} # sub ConformCustomEntry


print "Content-type: text/html\n\n<head><body><form>";

unless($ENV{QUERY_STRING})
{
   print '<p>Append a question mark and the address to send test email to the URL. Example: <b>http://example.com/cgi-bin/emailtest.cgi?name@example.com</b></p>';
   goto BOTTOM;
}

my $Destination = $ENV{QUERY_STRING};
$Destination =~ s/[\n\r]|(?:x|\%)0[ad]|\\0?1[25]//sig;

my $FromAddy = $ENV{SERVER_NAME};
$FromAddy =~ s/^www\.//;
$FromAddy = "webmaster\@$FromAddy";

$SendmailLocations = ConformCustomEntry $SendmailLocations;
$ExtraHeaderLines = ConformCustomEntry $ExtraHeaderLines;
$ExtraHeaderLines .= "\n" if $ExtraHeaderLines;

for my $Mailer (split /\n/,$SendmailLocations)
{
   SendTheMail("$Mailer -t",$Destination);
   SendTheMail("$Mailer -t -f$FromAddy",$Destination);
   SendTheMail("$Mailer -t -f$Destination",$Destination);
}

print '</form><p>D O N E</p> </body></html>';
BOTTOM:
# end of script

The script lets you specify more than one sendmail location (some servers do have more than one sendmail installation).

And it lets you specify additional email header lines to use in the outgoing email. This can be good for testing if destination filters are sensitive to certain email header lines.

It allows you to test both plain text and HTML emails, for example. You can also simulate email being sent from other email programs, like Eudora or Thunderbird. This header line simulates the latter:

User-Agent: Thunderbird 1.5.0.5 (Windows/20060719)

Some email software identifies itself with the X-Mailer header line instead of User-Agent. If you wish to simulate certain software, have it send an email to yourself then view full headers. You should find an identifying line or lines. Copy and paste that line or lines into the script.

Similarly, other header lines from received email can be used in this testing script.

Install the script and run it with your email address as the parameter. Example:

http://example.com/cgi-bin/emailtest.cgi?name@example.com

The script will send three emails to that address, one without the -f flag, one with the destination address specified at the -f flag, and one with a home domain email address specified at the -f flag, for each sendmail location you've specified.

Run the script with various addresses. One of these addresses should be to a mailbox on the same domain sendmail is running on. Others can be to your mailbox at your ISP. Others to a web mailbox like Hotmail or Yahoo!

The various destinations are to bypass filters that might be in place at or for one destination but not another.

Every time you run the script, it sends out another set of three for each sendmail location you've specified. The email is marked with the sendmail location and flags that were used for its sending.

When any of the emails arrive at a mailbox, you know what does work. By eliminating what does work, what may not work is revealed — although further testing might be needed to determine with certainty what does not work.

If you must test the points in this section, use the script. It makes life so much easier.

Will Bontrager