How to send an email with a text that has new lines (\n) in ESB 5.0.0

Assume that we want to send an email with the following text.

Hello world!!!

My Name is Evanthika Amarasiri.

I work for the support team.

I need to send this email in the above format with new lines between each sentence.

How can we make this possible with WSO2 ESB?

So to support this, what you first need is to configure the WSO2 ESB to support email sending. This can be done by following the configuration mentioned in our product documentation.

Once done, start up the ESB server and create a Proxy service with the following configuration.

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="EmailSender"
       transports="https http"
       startOnLoad="true">
   <description/>
   <target>
      <endpoint>
         <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
      </endpoint>
      <inSequence>
         <log/>
         <property name="messageType" value="text/xml" scope="axis2" type="STRING"/>
         <property name="ContentType" value="text/xml" scope="axis2"/>
         <property name="Subject" value="Testing ESB" scope="transport"/>
         <property name="OUT_ONLY" value="true"/>
         <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
         <payloadFactory media-type="xml">
            <format>
               <ns:text xmlns:ns="http://ws.apache.org/commons/ns/payload">$1</ns:text>
            </format>
            <args>
               <arg evaluator="xml"
                    expression="concat('Hello world!!!','&#10;','&#10;', 'My Name is Evanthika Amarasiri.','&#10;','&#10;', 'I work for the support team.')"/>
            </args>
         </payloadFactory>
         <log level="full"/>
         <send>
            <endpoint>
               <address uri="mailto:evanthika@wso2.com"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
</proxy>


Note the below line that is inside the PayloadFactory mediator.

<arg evaluator="xml" expression="concat('Hello world!!!', '&#10;','&#10;', 'My Name is Evanthika Amarasiri.','&#10;','&#10;',  'I work for the support team.')"/>

To support new lines what you need to add is ' ' in between the text you want the new line to be.

Once the above proxy service is deployed, send a request to the proxy service and you should get an email attachment with the below content.

 NOTE: In WSO2 ESB 4.8.1 this cannot be done from the UI and has a known issue. Therefore, as a solution, you need to add the configuration to the physical proxy configuration file which resides under wso2esb-4.8.1/repository/deployment/server/synapse-configs/default/proxy-services
 

Validating JSON arrays when the payload is sent as a query parameter with WSO2 ESB

In my previous post, I explained how JSON payloads can be validated when it's sent as a query parameter. Using the same Synapse configuration without doing any changes, we will see how JSON arrays can be validated by tweaking the JSON schema.

Assume that my requirement is to send a JSON array as a query parameter as shown below.
http://localhost:8280/jsonAPI/jsonapi?jsonPayload=[{"getQuote": {"request": {"symbol": "WSO2"}}},{"getQuote": {"request": {"symbol": "MSFT"}}}]

Create an API using the same configuration which we have used in the previous post.

   <api context="/jsonAPI" name="jsonAPI">
        <resource methods="GET" protocol="http" uri-template="/jsonapi">
            <inSequence>
                <property expression="$url:jsonPayload"
                    name="jsonKeyValue" scope="default" type="STRING"/>
                <payloadFactory media-type="json">
                    <format>$1</format>
                    <args>
                        <arg evaluator="xml" expression="get-property('jsonKeyValue')"/>
                    </args>
                </payloadFactory>
                <validate>
                    <schema key="conf:/schema/StockQuoteSchema.json"/>
                    <on-fail>
                        <payloadFactory media-type="json">
                            <format>{"Error":"$1"}</format>
                            <args>
                                <arg evaluator="xml" expression="$ctx:ERROR_MESSAGE"/>
                            </args>
                        </payloadFactory>
                        <respond/>
                    </on-fail>
                </validate>
                <respond/>
            </inSequence>
        </resource>
    </api>

The StockQuoteSchema.json which you have created under the path conf:/schema/StockQuoteSchema.json should be written in the following format.

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "array",
    "items": [{
        "getQuote": {
            "type": "object",
            "properties": {
                "request": {
                    "type": "object",
                    "properties": {
                        "symbol": {
                            "type": "string"
                        }
                    },
                    "required": [
                        "symbol"
                    ]
                }
            },
            "required": [
                "request"
            ]
        }
    }],
    "required": [
        "getQuote"
    ]
}

Note the text marked in blue. In the previous example, when a simple JSON payload was sent, the value of the type was set to object whereas in this scenario, since it's handling JSON arrays, it should be set to array.

On the other hand, since your JSON payload is sending an array, the schema should list elements to be checked inside a block called items with the JSON body wrapped inside square brackets i.e.; [] as highlighted above. 

So once the above configuration is done, and the GET request is sent to the API, you should see the following output if everything goes well.

[
  {
    "getQuote": {
      "request": {
        "symbol": "WSO2"
      }
    }
  },
  {
    "getQuote": {
      "request": {
        "symbol": "MSFT"
      }
    }
  }
]

Validating JSON payloads when the payload is sent as a query parameter in WSO2 ESB

In cases where we want to validate a JSON payload against a particular schema that is being sent by the client before sending it to the backend, we can use the Validate mediator of WSO2 ESB. This support has been added from WSO2 ESB v5.0.0 onward. The samples given in the WSO2 ESB documentation are for scenarios where the JSON payload is sent as a message body.

However, if the JSON payload is being sent as a query parameter, the configuration given in the samples will not work and we will have to tweak the configuration to support this. Given below  is an example scenario which explains this in detail.

I have an API deployed in WSO2 ESB which does a GET call by passing the JSON message payload as a query parameter.

 http://localhost:8280/jsonAPI/jsonapi?jsonPayload={"getQuote": {"request": {"symbol": "WSO2"}}}

The schema (StockQuoteSchema.json) used to validate the incoming payload is as below. Note that this schema is saved in the registry under the path /_system/config/schema
 
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "getQuote": {
      "type": "object",
      "properties": {
        "request": {
          "type": "object",
          "properties": {
            "symbol": {
              "type": "string"
            }
          },
          "required": [
            "symbol"
          ]
        }
      },
      "required": [
        "request"
      ]
    }
  },
  "required": [
    "getQuote"
  ]
}

To validate the JSON object passed as a query parameter in the URL from the parameter jsonPayload the following API configuration should be used.

    <api context="/jsonAPI" name="jsonAPI">
        <resource methods="GET" protocol="http" uri-template="/jsonapi">
            <inSequence>
                <property expression="$url:jsonPayload"
                    name="jsonKeyValue" scope="default" type="STRING"/>
                <payloadFactory media-type="json">
                    <format>$1</format>
                    <args>
                        <arg evaluator="xml" expression="get-property('jsonKeyValue')"/>
                    </args>
                </payloadFactory>
                <validate>
                    <schema key="conf:/schema/StockQuoteSchema.json"/>
                    <on-fail>
                        <payloadFactory media-type="json">
                            <format>{"Error":"$1"}</format>
                            <args>
                                <arg evaluator="xml" expression="$ctx:ERROR_MESSAGE"/>
                            </args>
                        </payloadFactory>
                        <respond/>
                    </on-fail>
                </validate>
                <respond/>
            </inSequence>
        </resource>
    </api>


With this Synapse configuration in place, the validation should happen flawlessly.

Mediation flow of WSO2 ESB

WSO2 ESB is one of the most important products in the WSO2 product stacks which enables users to do all sorts of transformations. Instead of having to make each of your applications communicate directly with each other in all their various formats, each application simply communicates with the WSO2 ESB, which handles transforming and routing the messages to their appropriate destinations. While working with this product, we believe it is important to understand how the messages flow through ESB. So when a message comes inside ESB, it goes through the transport layer and then the Axis2 engine which converts the incoming message into a SOAP envelope.

Once converted, it is then handed over to the Mediation engine, which is considered as the backbone of the product. This is where all the mediation work happens. In this post, I will be explaining in detail what happens to a message which comes inside ESB and the path it takes until a response is delivered back to the client.

To explain the scenario, I will use the below Proxy Service which talks to a simple back-end and logs a message in the inSequence as well as the outSequence.

<proxy name="DebugProxy" startOnLoad="true" transports="https http">
        <description/>
        <target>
            <endpoint>
                <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
            </endpoint>
            <inSequence>
                <log level="full">
                    <property name="IN_SEQ" value="Executing In Sequence"/>
                </log>
            </inSequence>
            <outSequence>
                <log level="full">
                    <property name="OUT_SEQ" value="Inside the out sequence"/>
                </log>
                <send/>
            </outSequence>
            <faultSequence>
                <log level="full">
                    <property name="FAULT_SEQ" value="Inside Fault Sequence"/>
                </log>
            </faultSequence>
        </target>
    </proxy>   
           
The entry point to the mediation engine for a message which comes in to a Proxy Service, is the receive method of the ProxyServiceMessageReceiver while for the message out-flow, the entry point is SynapseCallbackReceiver. Below I've listed down each method that is being called in each class inside the Mediation engine.

In-flow

Out-flow 

Enabling SSL Tunneling through a Squid Proxy Server with authentication

This post will describe how we can proxy our outgoing requests through ESB using a Squid Proxy Server. For more information on the scenario, you can refer the WSO2 ESB documentation.

Step 1 - Setting up Squid Proxy Server

To setup a Squid Proxy Server locally, you can follow the instructions available here.

Step 2 - Configuring Squid Proxy Server - updating the squid.conf file

Add the following line under the acl section

acl squid.proxy.server src appserver.wso2.com


Note: If you will not be able to start the squid server with the above config, you might have to set an /etc/hosts entry for the host appserver.wso2.com.

The following should be added before the http_access TAG

http_access allow squid.proxy.server


Note: We will be referring to this proxy server instance by the name squid.proxy.server. Hence, you need to add this entry to the /etc/hosts file which resides in your local instance as well as to the instance where the Squid server is running.

Add the following port information before the https_port TAG section

http_port 8888


Once the above is added to the squid.conf file, restart the Squid server

sudo service squid3 restart

Step 3 - Enabling the proxy configuration in WSO2 ESB

To do this, add the below configuation to the axis2.xml under the PassThroughHttpSender, PassThroughHttpSSLSender configuration

<parameter name="http.proxyHost" locked="false">squid.proxy.server</parameter> <parameter name="http.proxyPort" locked="false">8888</parameter>

Step 4 - Creating a Proxy Service

Once the above configurations are done and the WSO2 ESB server is restarted, you can create a simple Passthrough Proxy service to test the scenario.
Note that as the endpoint, I am using a backend where I'm referring to from a host name called appserver.wso2.com. This was the hostname which we added to the squid.conf file above under the acl section.

<proxy name="SSLTunnelingProxy"
          transports="https http"
          startOnLoad="true"
          trace="disable">
      <description/>
      <target>
         <inSequence>
            <send>
               <endpoint>
                  <address uri="https://appserver.wso2.com/services/SimpleStockQuoteService"/>
               </endpoint>
            </send>
         </inSequence>
         <outSequence>
            <send/>
         </outSequence>
      </target>
   </proxy>

Step 5 - Setting up the backend
 
For this scenario, you can take an WSO2 Application Server instance and start it up with ports 443 and 80 as default ports. Then, set the HostName parameter in carbon.xml to appserver.wso2.com. Deploy the SimpleStockQuoteService and you are ready to go.

Step 6 - Invoking the Proxy Service

Using a preferred client of yours you can test the scenario. If the message is sent through the Proxy server, you should see logs as shown below in /var/logs/squid/access.log file.

1493112155.126  49234 127.0.0.1 TCP_MISS/200 2335 CONNECT appserver.wso2.com:443 - HIER_DIRECT/192.168.53.176 -
1493112888.241      0 10.100.7.144 TCP_DENIED_REPLY/403 3429 CONNECT appserver.wso2.com:443 - HIER_NONE/- text/html


Step 7 - Configure authentication in Squid

To enable authentication in Squid, the following configuration needs to be done.

Add the following to the squid.conf profile under  TAG: auth_param

auth_param basic program /usr/lib/squid3/ncsa_auth /etc/squid3/passwords
auth_param basic realm proxy

Note: In ubuntu 12 - the file should be ncsa_auth & in ubuntu 14 - the file is basic_ncsa_auth.

Next, under the TAG: acl add the following

acl authenticated proxy_auth REQUIRED

The following should be added under the TAG: http_access.

http_access allow authenticated

Step 8 - Setting up a user

As a pre-requisite Apache2 Utils will have to be installed.

sudo apt-get install apache2-utils

To create a new user, use the following command.

sudo htpasswd -c /etc/squid3/passwords <>

When it prompts for the password, specify a password of your choice.

Reason for "PasswordInvalidAsk Password Feature is disabled" error when adding through RemoteUserStoreManager

When trying to add users from RemoteUserStoreManager it returned the following SOAP fault.


<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
       <soapenv:Body>
        <soapenv:Fault>
            <faultcode>soapenv:Server</faultcode>
            <faultstring>PasswordInvalidAsk Password Feature is disabled</faultstring>
            <detail/>
        </soapenv:Fault>
    </soapenv:Body>
</soapenv:Envelope>


The reason for this issue is that I have forgotten to add the element in the SOAP message.  Once this element was added, I was able to successfully create the user.

Why doesn't my WSO2 server restart as expected when the Windows instance restarts when WSO2 server is installed as a service

When the Startup Type of a Carbon server that is configured as a Windows service is set to Manual and when the instance that the WSO2 server is running on restarts, the WSO2 Server will not get restarted along with it.

For the WSO2 server to be restarted when the Windows instance restarts, you need to set Startup Type to Automatic instead as shown below. See attachment below.

To change this setting, go to Control Panel\All Control Panel Items\Administrative Tools\Services and change the type from Manual to Automatic.