Debugging SMTP and TLS errors in Outlook
Written by Mike Rede on October 5, 2009Sending secure email often involves the process of also having to troubleshoot error messages related to TLS and SMTP in Outlook.
Transport Layer Security (TLS) is a cryptographic protocol used to encrypt traffic over networks such as the Internet. Use TLS encryption for servers that require basic authentication. With so much critical information such as usernames and passwords passing through your network, why take the risk that someone snooping could eavesdrop and pull out important corporate information? Implementing encryption and other security measures can help to protect your corporate jewels. The enforcement of security will require users to use the same encryption level that you set when they try to negotiate access to your network and servers. Without the same level of security, messages will be returned and non-delivery reports (NDR) will be generated.
Simple Mail Transfer Protocol (SMTP) is used for sending outgoing mail for both POP and IMAP clients and is well known for its vulnerabilities such as spoofing of emails.
To minimize your security exposure and to ensure that your corporate SMTP communications are protected you can start by implementing the Transport Layer Security protocol in the Exchange Server.
If you are like most email or system administrators you will be supporting both end users who work at corporate offices and also end users working from remote locations such as out of state offices or home offices.
Occasionally you will get a complaint from end users who are unable to send email but they can still receive email. After changing the SMTP addresses in their outgoing email servers you find that they are still not able to send email and that they are getting an error message such as the following:
“Verify the email address in your account properties. Server responded: 530
5.7.0. Issue a STARTTLS command first.”
The STARTTLS command takes plain text communications and provides a secure connection without having to use a separate port for encrypted communication. It is an extension to plain text communication protocols and makes a plain text connection become an encrypted connection such as a TLS or SSL connection.
The benefits to using STARTTLS include the ability to verify the identity of the client and/or server in an e-mail transmission. It can also be used to encrypt mail transmissions with or without the identity verified between two mail servers. And it provides the capability to authenticate a user for relaying through a mail server.
Now as previously mentioned, if for some reason a client is not able to send email and they are receiving an error message about having to issue a STARTTLS command first then a solution is to enable an encrypted Secure Socket Layer (SSL) connection for the SMTP server. You make this change in the account setup. Look for the advanced tab of the More Settings dialog box.
Another troubleshooting mechanism you can use is to turn on transport logging in Microsoft Outlook. This will allow you to log all communications between Outlook and many of the email servers that are out there. By reviewing the logs you will be able to identify any communication problems that might occur when using Outlook and email servers.
- You can turn on logging in Outlook by going to the Tools menu and clicking Options.
- Next, select the Other tab and then click on the Advanced Options button.
- You will get a check box labeled “Enable mail logging”.
- Put a check mark in the box by clicking on it.
- Then save your changes by clicking OK.
- Click OK again after you have returned to the main options.
- Finally exit Outlook and then restart it.
If your problem is that you are having trouble establishing a secure connection to Microsoft Exchange server it might be because you are trying to use a different port other than the default (SMTP) port 25. This may have happened if another administrator or an end user changed the default port by selecting the check box labeled, “This server requires a secure connection”. This will also affect other email servers that require a STARTTLS negotiation.
When Outlook 2000 is used to create a secure connection for SMTP it will issue a STARTTLS command which then starts the TLS handshake process for a connection using the default port 25. But this is not the same process for ports other than port 25.
A solution to this kind of problem is to upgrade to Microsoft Outlook 2002. After the upgrade the email client will be able to issue the STARTTLS command and initiate the negotiation process for a secure socket on a different port number that is not port 25.


