Read message body of an email using Apache Nifi

Viewed 2833

Is it possible to retrieve the body content of email, email header details and email attachments in Single step using Apache Nifi.

If so Please help me how to achieve this.

2 Answers

It is not possible in a single step unless you write your own processor or script (using ExecuteScript or InvokeScriptedProcessor). However it is possible in a single flow with something like the following:

ConsumePOP3 -> ExtractEmailHeaders -> ExtractEmailAttachments -> ...

At the end of the flow above, you will have one flow file per attachment, each flow file containing the email headers as attributes and the attachment as the content.

You can use the processor "ExecuteScript", not developing custom processor.

import email
import mimetypes
from email.parser import Parser
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from java.io import BufferedReader, InputStreamReader
from org.apache.nifi.processors.script import ExecuteScript
from org.apache.nifi.processor.io import InputStreamCallback
from org.apache.nifi.processor.io import StreamCallback

class PyInputStreamCallback(InputStreamCallback):
    _text = None

    def __init__(self):
        pass

    def getText(self) : 
        return self._text

    def process(self, inputStream):
        self._text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

flowFile = session.get()
if flowFile is not None :
    reader = PyInputStreamCallback()
    session.read(flowFile, reader)

    msg = email.message_from_string(reader.getText())
    body = ""

    if msg.is_multipart():
        for part in msg.walk():
            ctype = part.get_content_type()
            cdispo = str(part.get('Content-Disposition'))

            if ctype == 'text/plain' and 'attachment' not in cdispo:
                body = part.get_payload(decode=True)  # decode
                break
    else:
        body = msg.get_payload(decode=True)

    flowFile = session.putAttribute(flowFile, 'msgbody', body.decode('utf-8', 'ignore'))

    session.transfer(flowFile, ExecuteScript.REL_SUCCESS)

Screenshot

enter image description here

Related