Java Email Parsing and Email Signature Extraction

If you use Java and need to parse emails then this guide is for you.

We’ll cover:

• Why parsing emails is hard
• Java Code Example for parsing emails with SigParser

Why parsing emails is hard

Splitting Email Chains

If you want to split emails on the headers with Java, that is tough. No two email clients seem to produce the same header format and over time the email clients change the way they format headers.

Signature and Contact Detail Detection

Parsing email signatures with Java is also difficult. Many think they can just use a couple regex expressions and they'll be done. But the more you start working the problem the harder it becomes. Here are some of the major things you'd need to handle.

• No signature is formatted the same
• Phone numbers can have many different formats
• Need to attribute the type of phone number (Fax vs Mobile vs Work phones).
• The phone type indicator has lots of variations. For example, Mobile vs M: vs Cell vs C: and many others
• Titles can be incredibly difficult to capture without getting too much wrong information.
• Locations are tough. Very few people put full addresses. Often they'll only put the city and state but no country. Even street addresses are massively different by country.

Then there is identifying where the email signature is in the email which is really hard. We use a machine learning algorithm with lots of labeled validation emails. We've been labeling our test set for years across many organizations.

This is why we expose a simple email parsing API for use with Java

All of the above is why we suggest using our email parsing service to parse emails from Java. As we improve you'll automatically get any improvements without the need to redeploy any of your code. If an email client like Gmail starts using a new reply header format, we'll have a fix deployed with days and you won't have to do anything.

The SigParser Email Parsing API

The SigParser Email Parsing API is a serverless, stateless email parsing API which is easy to call from Java. It can extract contacts and split emails into sections. It can find phone numbers, titles, addresses and attribute them to the correct contact. It even takes care of deduping contacts for you if the same email address appears in the email.

Stateless means we store none of the email contents. It is a processing only service. We only store some high level statistics like length of the email or how long it took to process.

Example: Parse an Email with Java

Here is how to call the SigParser API with Java and convert the data to nice model. The model definition is below.

import com.fasterxml.jackson.databind.ObjectMapper;
import java.net.http.*;
///////////////////////////////////////////
ObjectMapper mapper = new ObjectMapper();
Map<String, String> requestBody = new HashMap<>();
requestBody.put("plainbody", "This is an email.");
requestBody.put("from_address", "jsmith@example.com");
requestBody.put("from_name", "John Smith");
HttpClient httpClient = HttpClient.newBuilder()
		.version(HttpClient.Version.HTTP_2)
		.build();
HttpRequest request = HttpRequest.newBuilder()
		.uri(URI.create("https://ipaas.sigparser.com/api/Parse/Email/Message/JSON")) // or https://ipaas.sigparser.com/api/Parse/Email/Contact/JSON
		.header("cache-control", "no-cache")
		.header("x-api-key", "YOUR-API-KEY")
		.header("content-type", "application/json")
		.POST(
				HttpRequest.BodyPublishers
						.ofString(mapper.writerWithDefaultPrettyPrinter().writeValueAsString(requestBody))
		)
		.build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
EmailResponse emailResponse = mapper.readValue(response.body(), EmailResponse.class);

The Java class which you can bind the response to looks like

import java.util.Date;
import java.util.List;

public class EmailResponse {
    public String error;
    public List<Contact> contacts;
    public boolean isSpammyLookingEmailMessage;
    public boolean isSpammyLookingSender;
    public boolean isSpam;
    public String from_LastName;
    public String from_FirstName;
    public String from_Fax;
    public String from_Phone;
    public String from_Address;
    public String from_Title;
    public String from_MobilePhone;
    public String from_OfficePhone;
    public String from_LinkedInUrl;
    public String from_TwitterUrl;
    public String from_TwitterHandle;
    public String from_EmailAddress;
    public List<Email> emails;
    public String from_LinkedInHandle;
    public double duration;
    public String cleanedemailbody;
    public String cleanedemailbody_ishtml;
    public String cleanedemailbody_plain;
}

class Contact {
    public String firstName;
    public String lastName;
    public String emailAddress;
    public String phoneNumber;
    public String mobilePhone;
    public String voipPhone;
    public String officePhone;
    public String fax;
    public String address;
    public String title;
    public String twitterUrl;
    public String twitterHandle;
    public String linkedInUrl;
    public String linkedInHandle;
}

class Email {
    public String from_EmailAddress;
    public String from_Name;
    public String textBody;
    public List<String> htmlLines;
    public Date date;
    public boolean didParseCorrectly;
    public List<To> to;
    public List<To> cc;
    public String htmlBody;
    public boolean spammyLookingEmail;
    public String subject;
}

class To {
    public String name;
    public String emailAddress;
}

The JSON looks like this.

{
    "error": "string",
    "contacts": [
      {
        "firstName": "string",
        "lastName": "string",
        "emailAddress": "string",
        "emailAddressDomain": "string",
        "emailAddressDomainWithoutTLD": "string",
        "phoneNumber": "string",
        "mobilePhone": "string",
        "voipPhone": "string",
        "officePhone": "string",
        "fax": "string",
        "address": "string",
        "title": "string",
        "twitterUrl": "string",
        "twitterHandle": "string",
        "linkedInUrl": "string",
        "linkedInHandle": "string",
        "companyName": "string",
        "website": "string"
      }
    ],
    "isSpammyLookingEmailMessage": true,
    "isSpammyLookingSender": true,
    "isSpam": true,
    "from_LastName": "string",
    "from_FirstName": "string",
    "from_Fax": "string",
    "from_Phone": "string",
    "from_Address": "string",
    "from_Title": "string",
    "from_MobilePhone": "string",
    "from_OfficePhone": "string",
    "from_LinkedInUrl": "string",
    "from_TwitterUrl": "string",
    "from_TwitterHandle": "string",
    "from_EmailAddress": "string",
    "emails": [
      {
        "from_EmailAddress": "string",
        "from_Name": "string",
        "textBody": "string",
        "htmlLines": [
          "string"
        ],
        "date": "2019-05-05T22:27:56.124Z",
        "didParseCorrectly": true,
        "to": [
          {
            "name": "string",
            "emailAddress": "string"
          }
        ],
        "cc": [
          {
            "name": "string",
            "emailAddress": "string"
          }
        ],
        "htmlBody": "string",
        "spammyLookingEmail": true,
        "subject": "string",
        "cleanedBodyHtml": "string",
        "cleanedBodyPlain": "string"
      }
    ],
    "from_LinkedInHandle": "string",
    "duration": 0,
    "cleanedemailbody": "string",
    "cleanedemailbody_ishtml": true,
    "cleanedemailbody_plain": "string",
    "from_CompanyName": "string",
    "from_Website": "string",
    "from_EmailAddressDomain": "string",
    "from_EmailAddressDomainWithoutTLD": "string"
  }