SOA and Email in action – Starting a BPEL process instance by sending an email with XML content- on decoupled interaction between web-site and back office

In what is to become a series of articles illustrating a full blown business case for a Conference Registration Processing process implemented using Oracle BPEL PM, this is the third installment, discussing how the BPEL Process Instance can be fired off upon reception of an email. The idea is that the front-office application – a very simplistic website – collects data from the candidate Conference attendee and submits that information by means of an email to a designated mail account. This mail account is to be monitored by the back office – whenever a new email is received, a registration handling process should be started – either by one of our back office staff or by our back office BPEL engine. The latter is obviously the objective of these articles.

In the first installments, Generic Service – HTML Form Post to Email Servlet and Implementing a stand alone Email Server – Getting Started with Apache James we have created a very simple Web Site (a static HTML Document) that allows registration of our conference visitors, implemented a generic Servlet that handles an HTML Post and turns it into an email, configured a dedicated Email Server using Apache James and learned how to access that email server using Thunderbird and plain Java code. This article will demonstrate – using the excellent weblog by Matt Wright as well as Chapter 15 in the Oracle BPEL PM Developer’s Guide for release 10.1.2 (see resources for hyperlinks) – how we can implement as BPEL process that is instantiated whenever an email is received into the designated account. The process instance will read the email and act upon its contents. To clearly show that some is happening, the BPEL process instance will conclude by sending an email to the front office email account, informing on the Conference Registration just taken care of.

Summary of steps

The steps required to get our BPEL process going are roughly the following:

1. create a website with an HTML Registration Form that can be posted to a specific URL
2. implement a servlet – sitting behind that URL – that receives the posted HTML Form data and turns it into an email and sends it to the email address specified in the request
3. install and configure an Email server that can be used for sending and receiving emails (we used Apache James)
4. access the Email Server – through a Mail Client such as Thunderbird and programmatically from Java using the Java Mail API
5. install Oracle BPEL Process Manager (I used release 10.1.2.0.2)
6. create a new BPEL process – following the example in the blog by Matt Wright (see resources) – based on an Email Activation Agent
7. configure the Email Activation Agent through a webaccount file in the BPEL_HOME\integration\orabpel\domains\default\metadata\MailService directory
8. enhance the HtmlPost2Email servlet, to create XML content in the Email Body
9. specify a message type with a part based on a complex type in the WSDL for our BPEL process, matching the ConferenceRegistration XML fragment sent in the email
10. parse the Email received by the Activation Agent and turn the content into an XML variable based on the ConferenceRegistration part in the message type specified before
11. handle the Conference Registration as stored in the process variable (currently this step does nothing in reality, although this is where the hard work will take place eventually)
12. send an email to the front-office, informing on the ConferenceRegistration that was received via email and has now been handled; this email is sent by the standard Oracle BPEL Notification Task; configuration of the Email Account and Server to send the email from is done in the file BPEL_HOME\integration\orabpel\system\services\config\ns_emails.xml

The first four steps have been implemented and discussed in the first two installments in this series (see resources). Step five is generic and described elsewhere (besides, installation is dead easy). Starting with step 6, things become more interesting. Let’s take a detailed look, shall we?

 

Create a new BPEL process based on the Email Activation Agent

– following the example in the blog by Matt Wright (see resources) – based on an Email Activation Agent

Open the BPEL Process Designer. Create a new Application Workspace (COA-EMAIL-ACTIVATON) and a new BPEL Project (COA-EmailActivation) based on an a-synchronous activity.

In the default BPEL process created for us, remove the callBackClient Invoke Activity – as we do not need it.

becomes:

Since we are going to deal with emails in this process, we need to set up the structure of emails through an XSD. Oracle BPEL ships with a Schema Definition for emails, as this structure is standardized. The XSD is called Mail.xsd and can be found at: http://<host>:<port>/orabpel/xmllib/Mail.xsd where <host> is the server on which your BPEL PM Server is installed and <port> is the port that it’s listening on (on my laptop therefore it is at http://localhost:9700/orabpel/xmllib/Mail.xsd).

We have to import this XSD into our BPEL Project. Go to the structure window (lower left hand corner) for our BPEL process. Right click on the Project Schemas Node:

Select Import Schema. The following popup is shown:

Enter the url for the Mail.xsd document and click OK.

The XSD is added to the structure of the BPEL process:

Update Project WSDL – set payload message type

We need to specify the message type of the payload of the request message. To do this go into project structure and expand the Message Types -> Process WSDL structure. Within this expand the Request Message Node (in this example COA-EmailActivationRequestMessage) and right click the payload element and select Edit Message Part

We will select an element from the Mail.xsd document, so click on the browse elements icon for the Element Radio Button:

The Type Chooser opens. Locate the Mail.xsd schema and select the mailMessage element.

Now we have specified that the payload part of the Request Message is of type mailMessage. We can use this information later on to extract specific pieces of information from the request message variable.

Include and Configure the Email Activation Agent

The BPEL process as such is ready! It may not do much – in fact, it does nothing except receiving a RequestMessage – but it is a process! However, we do not intend to invoke this process ourselves – we want the BPEL Container to iniate process instances whenever an email arrives at a designated email account. For that to happen, we need to add the Email Activation Agent to the BPEL process definition. And we do that in the bpel.xml file. Open that file, and add the following fragment, between the closing partnerLinkBindings tag and the BPELProcess closing tag:

<activationAgents>
  <activationAgent className="com.collaxa.cube.activation.mail.MailActivationAgent" heartBeatInterval="35">
    <property name="accountName">coa-backoffice</property>
  </activationAgent>
</activationAgents>

Here we have specified – using the MailActivationAgent, a standard component of Oracle BPEL PM – that every 35 seconds (heartBeatInterval), the BPEL Container should check the coa-backoffice mail account for new emails. Any new email found should trigger the start of a new process instance. The email message itself is used as payload for the RequestMessage with which the process is started.

One very good question at this point would be: how does Oracle BPEL PM know where to look for these emails? Which Mail Server should it use? That is indeed something you need to tell it, somewhere and somehow. The answer is that we configure the mail account with a file in the BPEL_HOME\integration\orabpel\domains\default\metadata\MailService directory. This file should be called coa-backoffice.xml, after the value of the accountName property in the bpel.xml file. This file looks like this:

<mailAccount xmlns="http://services.oracle.com/bpel/mail/account">
	<userInfo>
		<displayName>Lucas Jellema</displayName>
		<organization>AMIS</organization>
		<replyTo>coa-backoffice@localhost</replyTo>
	</userInfo>
	<incomingServer>
		<protocol>pop3</protocol>
		<host>localhost</host>
		<port>110</port>
		<email>coa-backoffice</email>
		<password>coa-backoffice</password>
	</incomingServer>
</mailAccount>

As was described in a previous installment, I have installed and configured James, the Apache Email server. I have created a few email users or account, including coa-backoffice and coa-web; the passwords are the same as the usernames. James supports the pop3 protocol can be reached at port 110 on my localhost. That is where applications such as Thunderbird and also Oracle BPEL PM can read emails for specific useraccounts.

With this configuration file, Oracle BPEL PM knows how to link the Email Activation Agent that was specified in the BPEL process with accountName coa-backoffice to the email account coa-backoffice – their names do not need to be the same by the way, the link is in the name of the file – on port 110 of the localhost.

Proof Eating this half baked pudding

Well, the proof is in the eating so let’s try to deploy this very basic process. Note that very shortly we will flesh out some more interesting aspects of the process.

Ensure that the Oracle BPEL PM is up and running. Now we can deploy our BPEL Process to the Oracle BPEL Process Manager.

Right click the project, click deploy and select the default domain on the LocalBPELServer. The Oracle BPEL Process Designer starts compiling the project and creates a BPEL suitcase that subsequently is deployed.

Now let’s start the BPEL Console, when we look at the dashboard, we can see our newly deployed process:

In order to start a process instance, all we need to do is send an email to the coa-backoffice@localhost account, and it should be picked up by the BPEL Process Manager.

From Thunderbird, I send a very simple email:

After no more than 35 seconds, this email should trigger a BPEL Process instance.

If you click on the receiveInput icon, we can see the data that was retrieved from the email message:

If it does not work for you – as it did not for me – the gotcha from Matt’s blog is very very extremely useful (thanks Matt!): Gotcha!! – If you modify the BPEL process in JDeveloper, the bpel.xml file may lose its changes (i.e. the activationAgent definition), and as a result the process will never get initiated – so always check the bpel.xml file is correctly defined just before deploying the process.

Upgrade  HtmlPost2Email servlet – create XML content in the Email Body

In the Generic Service – HTML Form Post to Email Servlet I introduced a servlet that receives an HTML Form post and turns it into an Email, with all posted Form parameters included in the email body as key,value pairs. Now is a good time to take the next step and have this servlet create XML content in the email. This obviously allows for much easier processing in the BPEL Process.

With just a little change in the code, we create an XML Fragment that contains XML Elements for all Parameters:

package nl.amis.util;

import java.io.IOException;
import java.io.PrintWriter;

import java.util.Enumeration;

import javax.servlet.*;
import javax.servlet.http.*;

public class HtmlPost2XMLEmailServlet extends HttpServlet {
    private static final String CONTENT_TYPE = 
        "text/html; charset=windows-1252";

    public void init(ServletConfig config) throws ServletException {
        super.init(config);
    }

    public void doPost(HttpServletRequest request, 
                       HttpServletResponse response) throws ServletException, 
                                                            IOException {
        String emailAddressee = "";
        String emailSubject = "";
        String emailSender = "";
        String rootElementName = "";
        emailAddressee = request.getParameter("email_addressee");
        emailSubject = request.getParameter("email_subject");
        emailSender = request.getParameter("email_sender");
        rootElementName = request.getParameter("root_element_name");
        String content = "<"+rootElementName+ ">";

        Enumeration paramNames = request.getParameterNames();
        while (paramNames.hasMoreElements()) {
            String paramName = (String)paramNames.nextElement();
            String paramValue = "";
            String[] paramValues = request.getParameterValues(paramName);
            if (paramValues.length == 1) {
                paramValue = paramValues[0];
            } else {
                for (int i = 0; i < paramValues.length; i++) {
                    paramValue = paramValue + (i > 0 ? "," : "") + paramValues[i];
                } // for
            }
                content = content + "<"+paramName + ">" + paramValue + "</"+paramName + ">" + "\n";
        } //while
        content = content + "</"+ rootElementName+">";
        response.setContentType(CONTENT_TYPE);
        PrintWriter out = response.getWriter();
        out.println("<html>");
        out.println("<head><title>HtmlPost2EmailServlet</title></head>");
        try {
            EmailSender.sendEmail(emailAddressee, emailSubject, emailSender, 
                                  content);
            out.println("<body onload='alert(\"Your registration was received successfully. Thank you! \");'>");
            out.println("The email has been sent. <a href='javascript:history.go(-1);'>Return to previous page.</a> ");
        } catch (Exception e) {
            out.println("<body onload='alert(\"Unfortunately, an error occurred while processing your request! \");'>");
            out.println("Could you please try again. <a href='javascript:history.go(-1);'>Return to previous page.</a> ");

        }
        out.println("</body></html>");
        out.close();
    }
}

We need two small changes in the HTML Conference Registration document that we first saw in the article referred to above: we need to reference the new servlet (HtmlPost2XMLEmail) and we need to include a hidden form parameter that specifies the name of the root element in the email body (ConferenceRegistration in this case):

          <form action="http://10.0.0.156:8988/COA_WEB-ViewController-context-root/htmlpost2xmlemailservlet" method="post"
                enctype="application/x-www-form-urlencoded">
            <input type="hidden" name="root_element_name"
                   value="ConferenceRegistration"/>

When we now enter our conference registration details on the html form

and press the Invoke icon to submit, an email is generated with XML content; we can see this when we look at the BPEL process instance that is initiated as a result of our registration:

Here we see that the process instance created upon sending the email with XML content takes the XML fragment into the BPEL process. Now we need to take the email content – an unstructured string – and turn it into a structured BPEL Process Variable, XML type.

Create Message Type in BPEL Process for Email XML Content

We will first specify a message type with a part based on a complex type in the WSDL for our BPEL process, matching the ConferenceRegistration XML fragment sent in the email. Open the COA-EmailActivation.wsdl file. Add a new message, called RegistrationDetails with a part called COAWebConferenceRegistration. Inside the <types> element node, add a definition for the complexType ConferenceRegistration, mapping to the XML fragment in the Email body:

...
        <xsd:schema targetNamespace="http://nl.amis.coa" elementFormDefault="unqualified" >
          <xsd:complexType name="ConferenceRegistration">
            <xsd:sequence>
              <xsd:element name="Name" type="xsd:string" />
              <xsd:element name="Address" type="xsd:string" />
              <xsd:element name="City" type="xsd:string" />
              <xsd:element name="Telephone" type="xsd:string" />
              ... <!-- here we add all other elements from the HTML Registration Form -->
            </xsd:sequence>
         </xsd:complexType>
       </xsd:schema>
    </types>
    <message name="RegistrationDetails">
      <part name="COAWebConferenceRegistration" type="coa:ConferenceRegistration" />
    </message>
...

Also add two namespaces to the root of the WSDL file:

     xmlns:coa="http://nl.amis.coa"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema"

Save the WSDL file. Now we can create a variable in our BPEL process that is based on this type definition and subsequently parse the XML fragment from the Email body into this variable, for further processing.

Parse the XML content from the Email into a BPEL variable

First, let’s create a new variable, called COARegistration. It is based on the RegistrationDetails message(type), and specifically its COAWebConferenceRegistration part. Go to the structure window for the COA-EmailActivation.bpel file. Locate the node Variables. Right click and select Create.

Select the radio button Message Type and click on the Browse Types icon. The Type Chooser opens. Select the Registration Details message that we have just created in the project’s WSDL file.

The Create Variable window is shown again. Click OK.

The new variable is added to the BPEL document and is displayed in the Structure Window:

Now we make use of this variable: it will provide the structured ConferenceRegistration, parsed from the unstructured “XML” String in the email body. We will create an Assign activity that will copy the parsed as XML content from the email to the COAWebConferenceRegistration part of the COARegistration variable. Key for this operation is the function ora:parseEscapedXML that I found out about more or less accidentally. This function takes a piece of unstructured text (a string) and parses it into a DOM element, that can futher be manipulated.

For our purpose, let’s create the Assign Activity, by dragging it from the Component Palet onto the BPEL diagram, just below the receiveInput activity.

Now double click on the Assign activity to specify the copy operation.

Note how we have retrieved the payload part from the inputVariable, extracted the content element and parsed it into a DOM element, using the ora:parseEscapedXML function. The result of this extract and parse operation is stored in the variable COARegistration’s COAWebConferenceRegistration part. After this assignment, all steps in our process can easily access the Conference Registration details that were initially fed into the process from an unstructured email.

Handle the Conference Registration

This is where the bulk of the process(ing) takes place. Various services are invoked to perform financial checks, calculate the entrance fee, validate address details, add the registree to marketing files etc. However: this part of the process is not yet implemented – as it is not relevant to the story of this article.

Send an Email Notification to the Front Office

The last step in our process will be to send an email to the front office (coa-web@localhost), informing on the ConferenceRegistration that was received via email and has now been handled. Sending emails is a standard operation in Oracle BPEL PM. We can use the Notification Task. Drag the Notification activity from the component palet and drop it just after the ParseXMLfromEmailIntoVariable activity. The Notification Service Wizard is started. We can choose several communication mechanisms for sending out our notification, including the classic fax and SMS text messages. We select EMail.

The second step in the wizard has us specifying the destination and origin of the email, the subject and the content of the message to be sent.

Note how we can mix plain text content (or HTML for that matter) and data derived from variables. We have cherry picked a few interesting details from the COAWebConferenceRegistration part of the COARegistration variable, for example the Name and City element.

The BPEL Process now looks like this:

Sending emails has to be configured: the Oracle BPEL Process Manager reads on of the deep-down-hidden-inside configuration files to determine how to send an email. We have specified to use the default account for sending the emails to coa-web@localhost. This default account is configured in the file BPEL_HOME\integration\orabpel\system\services\config\ns_emails.xml on the server that hosts the Oracle BPEL PM. For me, this file looks like this:

<EmailAccounts xmlns="http://xmlns.oracle.com/ias/pcbpel/NotificationService">
   <EmailAccount>
      <Name>Default</Name>
      <GeneralSettings>
         <FromName>COA-BACKOFFICE</FromName>
         <FromAddress>coa-backoffice@localhost</FromAddress>
      </GeneralSettings>
      <OutgoingServerSettings>
         <SMTPHost>localhost</SMTPHost>
         <SMTPPort>25</SMTPPort>
      </OutgoingServerSettings>
      <IncomingServerSettings>
         <Server>localhost</Server>
         <Port>110</Port>
         <Protocol>pop3</Protocol>
         <UserName>coa-backoffice</UserName>
         <Password ns0:encrypted="false" xmlns:ns0="http://xmlns.oracle.com/ias/pcbpel/NotificationService">coa-backoffice</Password>
         <UseSSL>false</UseSSL>
         <Folder>Inbox</Folder>
         <PollingFrequency>1</PollingFrequency>
         <PostReadOperation>
            <MarkAsRead/>
         </PostReadOperation>
      </IncomingServerSettings>
   </EmailAccount>
</EmailAccounts>

For our current purpose – sending emails – is the IncomingServer of no relevance.

Eat the Pudding now it is completely done

Time to re-deploy our BPEL process. When deployment is complete and successful, we can do another Registration for the Conference on the website. This should cause an email to be sent, a BPEL Process instance to be initiated and eventually another email to be sent to the coa-web account in the front-office.

First, the BPEL Console tells us the process was run after we posted the Registration Form on the COA Website. That is very good, although not new.

Zooming in on the invoke Step in the Notification Flow, we can see how the Email is constructed:

Looking in the Thunderbird Mail Client for the coa-web@localhost account, we find the email that our helpful backoffice has sent us – fully automatically!

Resources

The source code for this article (an Oracle BPEL PM Designer project): COA-EmailActivation.zip

Prior installments in this series:  Generic Service – HTML Form Post to Email Servlet and Implementing a stand alone Email Server – Getting Started with Apache James

Oracle BPEL Process Manager – Packt Publishing (October 2005) on CodeGuru: http://www.codeguru.com/cpp/sample_chapter/article.php/c10789__7/ with good descriptions of some important data manipulation functions.

Matt Wright’s Blog – Using Email to initiate a BPEL Process (30 march 2006)

Oracle BPEL PM Developer’s Guide (10.1.2)