How does thread switching happen in WSO2 ESB (Switching between ServerWorker & IO reactor threads)

One of the most important things we need to understand when working with WSO2 ESB is it's threading model. There are two types of threads that does the main functionality within the ESB.

a) SynapseWorker threads
b) IO Reactor Threads

Out of these two threads, the SynapseWorker threads handles whatever the operations related to message mediation. On the other hand, the IO reactor threads are available for event handling such as receiving message headers, receiving message body, etc.

Inside the Synapse engine, the message would go through many classes where transformations that are required are done to the incoming message. The classes involved in this flow are explained in a previous post which I have written.

Once the message mediation is done, it is passed onto the Axis2Engine where it's receive() method is called. Inside this method, the type of the receiver is read from the context. In this example, since we are sending the message to a Proxy service, the receiver would be set to ProxyServiceMessageReceiver. So the message context would be passed on to the receive method of the ProxyServiceMessageReceiver class.
This is the point where the incoming message is passed onto the Synapse Engine.

After the mediation flow completes for the incoming message inside the mediation engine, it would be handed over to the PassThroughHttpSender where the outbound HTTP requests can be sent.

The PassThroughHttpSender implements the TransportSender interface. The initialisation of the PassThroughHttpSender happens at the server startup and an instance of the NHttpClientEventHandler is created by the name TargetHandler.

In addition to this, when the PassThroughHttpSender is initialised, it creates an instance of the DefaultConnectingIOReactor as well.
Along with this an instance of the DeliveryAgent is also formed which allows storing of messages for later delivery.
If a connection is available to send the message it would send the message right away and if not, it would put the message to a message queue to be delivered to the backend whenever a connection is available. This implementation is available inside the submit() method of the DeliveryAgent class.

When the message comes in the PassThroughHttpSender it hits it's invoke() method. After removing unwanted headers sent through the request, it would then check the messageContext for the endpoint value sent in the request message.
If an endpoint is passed in from the request, the submit() method of the DeliveryAgent class is called. If no endpoint is sent, it would call the submitResponse() method.

In this scenario, since we have specified an endpoint value, I would explain the flow which continues from the submit() method of the DeliveryAgent class.

Inside the submit() method, it would add the incoming message to a queue as shown below.

Once the message is added to the queue, it would call the getConnection() method of the TargetConnections class. This would return a connection if already available or won't return anything and will notify the delivery agent when a connection is available.

In a scenario where a connection is available, the connected() method of the TargetHandler is called and also a new TargetContext is created and the Protocol state is set to REQUEST_READY. which means that the connection is at the initial stage and is ready to receive a request.

Next the connected() method of the DeliveryAgent class is invoked where it would check the queue for any messages and pass onto the tryNextMessage() method.

Inside this method, the TargetConext will be updated with the status REQUEST_READY. At the same time, the TargetContext will be set with the messageContext and then the message will be sent to the submitRequest() method.

When the submitRequest() method is called, it would create a TargetRequest and attach it to the TargetContext.

Next is the invocation of the requestReady() method of the TargetHandler class where the HTTP headers are written to the TargetRequest.

Then the outputReady() of the TargetHandler is called;  where the write() method of the TargetRequest is hit.

In this method, it reads the data that was written before to the pipe and writes it to the wire. Once this is done, the Protocol status is also updated to REQUEST_DONE.

There you go! Now you know how the request messages are being passed on from the Worker threads to the IO threads within WSO2 ESB.