TA-dmarc add-on for Splunk supports ingesting DMARC XML aggregate reports from an IMAP/POP3 mailbox or local directory with mitigations against:

• ZIP bombs
• gzip bombs
• various XML attack vectors like billion laughs, quadratic blowup, external entity expansion and so on
• malformed reports
• false reports (alpha)

Additional requirements:

• Splunk heavy forwarder instance: Because of Python dependencies Splunk Universal Forwarder is not supported
• KVstore: used to keep track of which IMAP messages or local files have already been processed. KVstore is enabled by default on Splunk instances.

The event output format changed to JSON. This is the new default because JSON is a better match for the structured aggregate reports than a flat key=value (KV) format. See for example issue #4 on Github. If you want your inputs to to keep using the KV format:

• Disable your inputs before upgrading
• Upgrade the add-on to version 3.x
• Edit your inputs and change the output format to KV
• Enable your inputs

Note that KV format output is likely to be deprecated in future versions, and enhancements like multiple XSD validation aren't available in KV output format.

Because of changes to KVstore logic, the add-on will re-index every report on IMAP, leading to duplicate events.

1. In Splunk, click on "Manage Apps"
2. Click "Browse more apps", search for "TA-dmarc" and install the add-on

The following table lists support for distributed deployment roles in a Splunk deployment:

Steps:

1. Add an account
2. Add an input

If you're using this add-on from a disconnected network, you can skip account creation and create a new directory based input. This assumes you have an offline method for getting DMARC attachments to this directory.

In the Configuration tab, create a new account:

• Account Name: descriptive account name, e.g. google_dmarc_mailbox
• Username: the account to identify with
• Password: the password to authenticate with

This add-on can ingest DMARC aggregate reports from:

• IMAP mailboxes
• POP mailboxes
• Local directories (for offline environments)

Go to the add-on's input tab and configure a new modular input by clicking on the "Inputs" menu:

• Click "Create new input"
• Select "DMARC imap" or "DMARC pop3"
• Configure:
  • Name: e.g. dmarc-google
  • Interval: how often to poll the mailserver for aggregate reports.
  • Index: what Splunk index to send the aggregate reports to
  • Global Account: select the account to authenticate with
  • IMAP server: the imap server to poll
  • Resolve IP: Whether or not to resolve the row source_ip in the DMARC XML aggregate reports
  • Validate XML: Whether or not to validate the DMARC XML against the DMARC XSD
  • Validate DKIM: Whether or not to validate the DKIM signature(s) in the mail
  • IMAP mailbox: Select the specific IMAP mailbox folder to poll. Default: INBOX
  • Output format: Send events to Splunk in JSON format (default) or key=value (left for compatibility reasons)

TA-dmarc can fetch DMARC aggregate reports from an IMAP or POP3 server. It will look for:

1. messages with "Report domain:" in the subject.
2. attachments with .xml, .zip or xml.gz file extentions
3. attachments with mime-types:
   • application/zip
   • application/gzip
   • application/x-gzip
   • application/xml
   • text/xml
   • application-x-gzip (Non-standard mimetype used for Comcast DMARC reports)
   • application/x-zip-compressed (Non-standard mimetype used for Yahoo DMARC reports)

The add-on doesn't modify, move or delete messages on the IMAP server but insteads keeps a record in the Splunk KV-store of which mails have already been read.

DMARC XML aggregate reports contain a source_ip that can be reverse resolved at index time. This is the default but can cause considerable delay in processing aggregate reports due to unreachable nameservers. Second, because the reverse DNS record cannot really be trusted, another forward lookup is performed. Only if the reverse and forward lookup match, the result is included in the output.

DMARC XML aggregate reports can be validated against multiple DMARC RUA XML schema definition versions (XSD):

• The XSD from draft-dmarc-base-00-02 (march 2012)
• The XSD from RFC7489 (march 2015)
• A custom relaxed XSD that should succesfully verify RFC7489-based reporters and pre-RFC7489 reporters

The result of the validations is added as new event fields in Splunk: vendor_rua_xsd_validations

DKIM signatures from email messages can be verified. Currently the results of this validation are only available in debug log. Future versions will add a new event field in Splunk.

TA-dmarc can watch a folder for new DMARC aggregate reports. This can be useful for loading DMARC reports in non-internet connected environments. It will look for files with extentions:

1. .xml
2. .zip
3. .xml.gz

TA-dmarc doesn't modify, move or delete files in the directory: it uses internal checkpointing to keep track of which files have been previously read. Any invalid .xml, .zip or .xml.gz files are ignored and logged.

1. Go to the add-on's configuration UI and configure a new modular input by clicking on the "Inputs" menu.
2. Click "Create new input"
3. Select "DMARC directory"
4. Configure:
   • Name: e.g. "production_dmarc_indir"
   • Interval: how often to poll the directory where DMARC XML aggregate reports are read from
   • Index: what Splunk index to send the aggregate reports to
   • Directory: Location where DMARC aggregate reports should be read from
   • Quiet time: Ignore files that have a modification time of less than n seconds ago. You can use this to prevent ingesting large files that are dropped on a network share but take some time to transfer
   • Resolve IP: Whether or not to resolve the raw source_ip in the DMARC XML aggregate reports
   • Validate XML: Whether or not to validate the DMARC XML against the DMARC XSD
5. Click add

Relevant fields within an aggregate report are mapped from the CIM Authentication datamodel, because DMARC is short for "Domain-based Message Authentication ... etc".

Mapping from the Authentication datamodel has the following advantages:

• DMARC aggregate report results are automatically incorporated in the relevante Splunk Enterprise Security dashboards.
• Accelerated |tstats searches can be performed against the normalized fields by selecting where All_Authentication.app=dmarc

From the XML sample below, the following values and fields are mapped:

<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
  <report_metadata>
    <org_name>google.com</org_name>
    <email>noreply-dmarc-support@google.com</email>
    <extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info>
    <report_id>13190401177475355109</report_id>
    <date_range>
      <begin>1506988800</begin>
      <end>1507075199</end>
    </date_range>
  </report_metadata>
  <policy_published>
    <domain>example.com</domain>
    <adkim>r</adkim>
    <aspf>r</aspf>
    <p>none</p>
    <sp>none</sp>
    <pct>100</pct>
  </policy_published>
  <record>
    <row>
      <source_ip>192.0.2.78</source_ip>
      <count>1</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>fail</dkim>
        <spf>fail</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>example.com</header_from>
    </identifiers>
    <auth_results>
      <spf>
        <domain>example.com</domain>
        <result>fail</result>
      </spf>
    </auth_results>
  </record>
</feedback>

From the DMARC XML sample above, the following fields are created:

The DMARC-imap input saves checkpointing data in KVstore. To see its contents: |inputlookup ta_dmarc_checkpointer_lookup

If you want to reindex an entire mailbox, you can do so by deleting the TA-dmarc KVstore checkpointing data through this Splunk command:

|inputlookup ta_dmarc_checkpointer_lookup
|search state!="*input=dmarc_imap, server=imap.gmail.com*"
|outputlookup ta_dmarc_checkpointer_lookup`

If you want to reindex a single DMARC report, you can do so by deleting its corresponding record from KVstore:

| inputlookup ta_dmarc_checkpointer_lookup 
| search state!="*Report-ID: 3596274322387252907*" 
| outputlookup ta_dmarc_checkpointer_lookup

Reindexing a DMARC report from a directory input is left as an excercise for the reader.

We use the following guidelines for developing this add-on:

This add-on is maintained by Jorrit Folmer. These people and organisations have contributed pull requests, issues, ideas or otherwise spent time improving this add-on:

• Christopher G Andrews (ChristopherGAndrews)
• John (john-9c54a80b)
• Mike Kolk
• Steve Myers (stmyers)
• Steven Hilton (malvidin)
• SMTware

The following software components are used in this add-on:

1. defusedxml version 0.5.0 by Christian Heimes
2. xmljson version 0.1.9 by S. Anand
3. IMAPClient version 1.0.2 by Menno Finlay-Smits
4. dkimpy version 0.6.2 by Scott Kitterman
5. dnspython version 1.15.0 by Bob Halley
6. Splunk Add-on Builder version 2.2.0 by Splunk and the third-party software it uses

See CHANGELOG.md

This is an open source project without warranty of any kind. No support is provided. However, a public repository and issue tracker are available at https://github.com/jorritfolmer/TA-dmarc