• g-mail-relayer
• Features
  • Current status
  • Not supported features
• REST API Usage
  • Extended documentation
  • Examples
  • /api/mail/send method (POST)
    • EmailMessage
    • Attachment
    • Header
    • SendMailResult
  • Error handling
    • 400 error code - validations
    • 500 error code - something went wrong
• SMTP Relaying
  • Sending method selection headers
• Configuration
  • application.properties
  • json config
  • Server matching algorithm
  • I'm confused
  • Still confused...
  • Configuration file rules

Table of contents generated with markdown-toc

This project develops a Java Spring Boot SMTP relayer and an API to send emails. The need comes due to some limitations found in my projects and I felt like building it myself. I need to talk to old NTLM servers and not all applications can. Also, sending a lot of emails synchronously in my applications is slow, so an async sending method was desirable. Multiple relay servers are also supported and selected depending on the incoming message.

The basics are:

• A Spring Boot REST API application able to relay emails coming from another application, asynchronously, so the sender can forget about waiting to say EHLO! to a external server.
• It includes a vanilla SMTP server to cover legacy code that cannot change to call to an API or third-party applications (as an example of my needs, a replacement for MultiSMTP OTRS plugin).
• The relaying server is selected depending on the parameters of the incoming message and the configuration of the relay servers.

Thanks to @avthart and his spring-boot-james-smtp-server project as it enlightened me in the using of Apache James.

This is the list of intended features. Not all of them are implemented, so check below.

1. Email sending/relaying:
   • HTML or text emails
   • Attachments (attached or inline)
   • Specific headers
   • Support for several SMTP destination severs
2. Defer email sending to a specific time
3. Store information about the email
   • Reception time
   • Application sending the email, so it can support different applications
   • Message type (application dependant so the application can search about specific type of mails)
   • Status
   • Last sending time (last trying to send the message)
   • Number of retries
   • to/cc/bc
   • ¿Body of the message? Still thinking about it, maybe gzipped to avoid space usage
4. Support for NTLM authentication
5. Support for several relaying servers
6. Able to send email through a REST API (no auth)
7. Able to relay email through a standard 25-smtp port
8. Able to retry email sending for a specified number of times and specified amount of time.
9. Support for callbacks in message sent or message NOK events
10. Search/statistics methods.
    • Able to search
      • Received in a period of time
      • By message status / type
    • Able to give information about
      • Number of received emails
      • Number of OK/NOK/pending/retrying messages
11. Separate logs for each application
12. Plugins for beforeSend and afterSend events
13. Hooks for success / error messages
14. API method to fetch status of sent mails
15. Client GUI for the statistics module

I'm trying to keep this up to date, but no release is still out, so this refers to the master branch:

#	Description	Status
1	Send emails
2	Defer email sending
3	Store information about the email
4	Support for NTLM authentication
5	Support for several relaying servers
6	REST API
7	Standard 25-smtp port
8	Retry email sending
9	Support for callbacks
10	Search/statistics methods
11	Separate logs for each application
12	Plugins for beforeSend and afterSend events
13	Hooks for success / error messages
14	API method to fetch status of sent mails
15	Client GUI for the statistics module

This list includes the features that have been thought of and have been rejected:

• HTTPS: As this is intented for internal use an not to be exposed, don't think this feature is needed. Anyway, it's very simple to configure an Apache/Nginx/whatever-the-server-you-like in front of the application.
• API authentication: Same reason. This is intended for internal use. Anyway, a proxy with basic auth could be set in front of the api or more professional applications like WSO2 API Manager.

Further documentation can be found in:

• Swagger.io
• api.json file
• Locally generated HTML doc

Examples of api calls can be found in samples directory.

The only existing method is POST /api/mail/send. It receives a EmailMessage json object and sends an email with the matching configuration for that email.

This is the json object representing an email message that is going to be sent:

This is the json object representing an attachment included in the message.

This is the json object representing a header included in the message.

This one wasn't tough, was it?

When a sending operation is successful, the following information is returned. Keep in mind that this doesn't mean that the email is already sent:

All exception are handled by a subclass of ResponseEntityExceptionHandler. Only validation errors are handled explicitly in the application.

Validation errors are handled with a 400 response code and the following response body:

{
  "timestamp": "2020-04-29T22:24:15.404Z",
  "status": 400,
  "errors": [
    "textFormat must match \"(HTML|TEXT)\"",
    "deliveryType must match \"(PRIORITY_SYNC|PRIORITY_ASYNC|QUEUE)\""
  ]
}

An example of the output produced by an error 500 return code.

{
  "timestamp": "2020-04-30T11:23:03.146Z",
  "status": 500,
  "error": "Internal Server Error",
  "message": "Mail server connection failed; nested exception is com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 26; timeout -1;\n  nested exception is:\n\tjava.net.ConnectException: Connection refused: connect. Failed messages: com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 26; timeout -1;\n  nested exception is:\n\tjava.net.ConnectException: Connection refused: connect",
  "trace": "org.springframework.mail.MailSendException: Mail server connection failed; nested exception is com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 26; timeout -1;\n  nested exception is:\n\tjava.net.ConnectException: Connection refused: connect. Failed messages: com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 26; timeout -1;\n  nested exception is:\n\t... 95 more\r\n",
  "path": "/api/mail/send"
}

The application binds a SMTP server to the specified local port. Reads incoming messages and relays them to the specified remote SMTP Server. This means we can send emails from cron or similar small utilities that do not support smtp server sending. The server asks for no authentication, so it should be placed in a secured environment.

Message is relayed 'as it is', with the only addition of a "Received" header and a "Message-ID" header:

Received: from 0:0:0:0:0:0:0:1 (EHLO [IPv6:::1]) ([0:0:0:0:0:0:0:1])
          by LOCALHOSTNAME (Spring Boot g-mail-relayer SMTP Server) with ESMTP ID -88888888
          for <[email protected]>;
          Tue, 31 Mar 2020 13:09:24 +0200 (CEST)
Message-ID: <[email protected]>

This means that the original from address of the email sent must be allowed in the relaying server to be the sender of the message with the authentication provided in the configuration. This is, provided username and password must be authorized to send emails on behalf the original from address.

This is true if the configuration selected for that email does not override the from address (overrideFrom property). In that case, the authorized sender will be the one indicated in the overrideFromAddress property.

As with the API, the email can be classified to match one of the defined relaying methods. An explanation of the matching algorithm is ahead, but the data needed for this classification in the SMTP Relaying sending method is:

• Original 'from' address
• X-GMR-APPLICATION-ID and X-GMR-MESSAGE-TYPE headers

The following properties are supported in the application.properties file.

The application needs a json config file (in UTF-8 format) to set the configuration for the relaying servers and to get the information to select which one of the listed servers is selected. The json object has four main properties:

The objects api-defaultand smtp-defaultshare the following properties:

apiand smtpare both an array of objects. This objects have the same properties as api-defaultand smtp-default plus these ones, that specify the relaying server selection rule. The selection method and further rules for configuration specification are explained below:

There are configuration examples here.

First of all, from address, application id and message type are retrieved:

• For api, from the post body arguments
• For smtp, the from address and the X-GMR-APPLICATION-ID and X-GMR-MESSAGE-TYPE headers.

With this data the following server matching algorithm is performed:

1. Application ID takes precedence over from address so if this id is not null, the message type is retrieved from the message.
2. If both message application id AND message type match any of the server configuration, this one is selected.
3. If only application id matches a given configuration, this could happen because:
   • Email message type is null: then a configuration with an application id but no message type is searched. If found, it is selected
   • Email message type is not null but there is not a matching configuration. Then, a configuration with the same application ID AND a null message type is searched. This is: it will never be matched if application id is the same AND not the message type AND the configuration message type is not null.
4. If application id and message type did not give a match, the message from address is searched in the configuration. For this matching, the personal information is removed. This means, John Doe <[email protected]>is the same address as [email protected].
5. If none of the steps below did give a match, then the default configuration is used for the applicable sending type (api or smtp).

Yeah, i know. I prepared this list of examples. I'm not sure if I had understood the previous chapter without having programmed it myself.

Consider the following set of configurations. Not considering if api or smtp because the selection works the same:

In the next table, it is stated the chosen configuration when different emails are received:

Sorry, don't know what else I can do. You can check the algorithm in the ConfigFileHelper.java file.

This matching algorithm means that for each group of configurations (api or smtp) the following rules must be met:

1. The configuration can be selected by application or from address, not both, so an error will be raised if forFrom and forApplicationId are both not null.
2. A message type is a subclass of application so an error will be raised if forMessageType is set and forApplicationId is null.
3. Each configuration must be unique so:
   • Can't set two identical forFromaddresses
   • Can't set two identical forApplicationId and forMessageType pair values. Keep in mind that null is a valid value for the message type. So you can have an application id with no message type and the same application with a specific message type.

... PENDING ...