Friday, October 30, 2015

Working with PostgreSQL 9.3 & WSO2 products

As you all know, WSO2 products support a wide range of RDBMs. In this post, I will explain how you can install PostgreSQL 9.3 on your Linux machine and what commands you have to run to create users/databases and grant permission for databases etc. In the latter part of the post, I will briefly explain on the configuration that needs to be done to connect WSO2 products to a PostgreSQL database.
Installing PostgreSQL

You can install using the apt-get command on Linux as below.

Step 1 - To get information on the newest versions of packages and dependencies
sudo apt-get update


Step 2 - To install PostgreSQL v9.3

sudo apt-get install postgresql-9.3 

Configure PostgreSQL password

To setup a password to login to PostgreSQL, issue the below commands.

sudo -u postgres psql template1

ALTER USER postgres with encrypted password 'postgres';

Note: To quit from the console, press Ctrl + D

Additional configuration for PostgreSQL

Step 1 - Configure md5 Authentication

1. Open the pg_hba.conf

sudo vim /etc/postgresql/9.3/main/pg_hba.conf

2. Find the below line

local all postgres

and change it to

local all postgres md5

Step 2 - Enable TCP/IP connections

1.  Open the postgresql.conf file

sudo vim /etc/postgresql/9.3/main/postgresql.conf

2. Locate the line and uncomment it

#listen_addresses = 'localhost'

Creating a user, a database and then granting permission for the database.

Step 1 - Creating a Linux user

CREATE USER apim WITH PASSWORD 'apim';

Note: Give the password as 'apim'

Step 2 - Login as super user

sudo su - postgres

If successfully logged in, you should get a prompt as below

postgres@ubuntu2:~$

Step 3 - Then connect to the database serve

psql -d template1 -U postgres

Step 4 - Create a user

CREATE USER apim WITH PASSWORD 'apim';

Step 5 - Create a database

CREATE DATABASE apim;


Step 6 - Grant permission for the user to the database


GRANT ALL PRIVILEGES ON DATABASE apim to apim;
 

Now that you have created the user, database and granted permission, you can provide these credentials and try to connect your WSO2 server with this newly created database.

Configuring WSO2 Products with PostgreSQL

Step 1 - Configuring master-datasources.xml

Open the master-datasources.xml of the product you have selected and provide the configuration as below

        <datasource>
            <name>WSO2AM_DB</name>
            <description>The datasource used for API Manager database</description>
            <jndiConfig>
                <name>jdbc/WSO2AM_DB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:postgresql://localhost:5432/apim</url>
                    <username>apim</username>
                    <password>apim</password>
                    <defaultAutoCommit>false</defaultAutoCommit>
                    <driverClassName>org.postgresql.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>

Step 2 - Downloading the correct DB driver

You have to download the correct database driver from the PostgreSQL site.  Once downloaded, drop it to $CARBON_HOME/repository/component/lib.

Step 3 - Starting up the server

Now start up the server using the -Dsetup command and it will create the relevant tables in the database.

Friday, August 14, 2015

Enabing E-mail User Login for WSO2 Products

This post explains different ways e-mail login can be enabled and how users/tenants can login to WSO2 products.

Pre-requisites

Users, tenants and their e-mail addresses that will be used for this scenario are as follows.

Super Admin User Name - admin
A user of Super Admin - adminUser
Email of Super Admin user - admin@yahoo.com
Email of a user of Super Admin - adminUser@gmail.com
Tenant Domain - tenantdomain.com
Tenant Admin - admin@tenantdomain.com
Tenant User - tenantDomainUser@tenantdomain.com
Tenant Admin Email - admin@hotmail.com
Tenant User Email - tenantDomainUser@aol.com

How to create tenants

When creating tenants, you have to give the tenant Admin Username as something like admin@gmail.com & not as admin

Scenario 1

Configuration that needs to be done

carbon.xml

<EnableEmailUserName>true</EnableEmailUserName>

user-mgt.xml

For JDBC User Stores

<Property name="UsernameWithEmailJavaScriptRegEx">[a-zA-Z0-9@._-|//]{3,30}$</Property>

For LDAP based User Stores

<Property name="UserNameSearchFilter">(&(objectClass=person)(|(mail=?)(uid=?)))</Property>

& Comment  out the following

<!--Property name="UserDNPattern">uid={0},ou=Users,dc=wso2,dc=org</Property-->


So when you do the above configuration, you can login from the following types of users

- admin
- admin@yahoo.com
- admin@yahoo.com@carbon.super
- adminUser
- adminUser@gmail.com
- adminUser@gmail.com@carbon.super
- admin@hotmail.com@tenantdomain.com
- tenantDomainUser@aol.com@tenantdomain.com

You cannot login as

- admin@tenantdomain.com
- tenantDomainUser@tenantdomain.com


Senario 2 - Without configuring  EnableEmailUserName property in carbon.xml

Configuration that needs to be done

carbon.xml

<EnableEmailUserName>false</EnableEmailUserName>

user-mgt.xml

Same as in Scenario 1 above

You should be able to login from the below users/email addresses

- admin
- admin@yahoo.com@carbon.super
- adminUser
- adminUser@gmail.com@carbon.super
- admin@hotmail.com@tenantdomain.com
- tenantDomainUser@aol.com@tenantdomain.com
- tenantDomainUser@tenantdomain.com

Cannot login from

- admin@yahoo.com
- adminUser@gmail.com
- admin@tenantdomain.com

To create users with email addresses, you need to change the following properties of the LDAP user store configuration.

<Property name="UserNameAttribute">mail</Property>
<Property name="UsernameJavaRegEx">[a-zA-Z0-9@._-|//]{3,30}$</Property>
<Property name="UserNameSearchFilter">(&(objectClass=person)(mail=?))</Property>

After configuring your server with the above configs, you should be able to add users with email addresses as well as with uids.
For more information, go through the detailed blog written by Asela Pathberiya.

Tuesday, July 21, 2015

Common SVN related issues faced with WSO2 products and how they can be solved

Issue 1

TID: [0] [ESB] [2015-07-21 14:49:55,145] ERROR {org.wso2.carbon.deployment.synchronizer.subversion.SVNBasedArtifactRepository} -  Error while attempting to create the directory: http://xx.xx.xx.xx/svn/wso2/-1234 {org.wso2.carbon.deployment.synchronizer.subversion.SVNBasedArtifactRepository}
org.tigris.subversion.svnclientadapter.SVNClientException: org.tigris.subversion.javahl.ClientException: svn: authentication cancelled
    at org.tigris.subversion.svnclientadapter.javahl.AbstractJhlClientAdapter.mkdir(AbstractJhlClientAdapter.java:2524)
    at org.wso2.carbon.deployment.synchronizer.subversion.SVNBasedArtifactRepository.checkRemoteDirectory(SVNBasedArtifactRepository.java:240)


Reason : The user is not authenticated to write to the provided SVN location i.e.:- http://xx.xx.xx.xx/svn/wso2/ . When you see this type of an error, verify the credentials you have given under the svn configuration in the carbon.xml

    <DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>false</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://svnrepo.example.com/repos/</SvnUrl>
        <SvnUser>username</SvnUser>
        <SvnPassword>password</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>


Issue 2

TID: [0] [ESB] [2015-07-21 14:56:49,089] ERROR {org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask} -  Deployment synchronization commit for tenant -1234 failed {org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask}
java.lang.RuntimeException: org.wso2.carbon.deployment.synchronizer.DeploymentSynchronizerException: A repository synchronizer has not been engaged for the file path: /home/wso2/products/wso2esb-4.9.0/repository/deployment/server/
    at org.wso2.carbon.deployment.synchronizer.internal.DeploymentSynchronizerServiceImpl.commit(DeploymentSynchronizerServiceImpl.java:116)
    at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.deploymentSyncCommit(CarbonDeploymentSchedulerTask.java:207)
    at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:128)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)


Reasons:

Even though you see this exception, the actual cause of the issue could be something else. So note that when you see this exception, you will have to go up the wso2carbon.log and see whether you see any related exceptions near the server startup logs.
(I) SVN version mismatch between local server and the SVN server (Carbon 4.2.0 products support SVN 1.6 only.

Solution - Use the SVN kit jar 1.6 in Carbon server

See https://docs.wso2.com/display/CLUSTER420/SVN-based+Deployment+Synchronizer)

(II) If you have configured your server with a different SVN version than what's in the SVN server and even if you use the correct svnkit jar at the Carbon server side later, the issue will not get resolved

Solution - Remove all the .svn files under $CARBON_HOME/repository/deployment/server folder

(III) A similar issue can be observed when the SVN server is not reachable.

Issue 3

 
[2015-08-28 11:22:27,406] ERROR {org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask} - Deployment synchronization update for tenant -1234 failed java.lang.RuntimeException: org.wso2.carbon.deployment.synchronizer.DeploymentSynchronizerException: No Repository found for type svn at org.wso2.carbon.deployment.synchronizer.internal.DeploymentSynchronizerServiceImpl.update(DeploymentSynchronizerServiceImpl.java:98) at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.deploymentSyncUpdate(CarbonDeploymentSchedulerTask.java:179) at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:137) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.wso2.carbon.deployment.synchronizer.DeploymentSynchronizerException: No Repository found for type svn at org.wso2.carbon.deployment.synchronizer.repository.CarbonRepositoryUtils.getDeploymentSyncConfigurationFromConf(CarbonRepositoryUtils.java:167) at org.wso2.carbon.deployment.synchronizer.repository.CarbonRepositoryUtils.getActiveSynchronizerConfiguration(CarbonRepositoryUtils.java:97) at org.wso2.carbon.deployment.synchronizer.internal.DeploymentSynchronizerServiceImpl.update(DeploymentSynchronizerServiceImpl.java:66) ... 9 more 

Reason:

You will notice this issue when the svn kit (i.e. for latest versions of Carbon i.e. 4.4.x the jar version would be svnkit-all-1.8.7.wso2v1.jar) jar is not available in $CARBON_HOME/repository/components/dropins folder

Sometimes dropping the svn-kit-all-1.8.7.wso2v1.jar would not solve the problem. In such situations, verify whether the trilead-ssh2-1.0.0-build215.jar is also available under $CARBON_HOME/repository/components/lib folder.

Thursday, July 16, 2015

Solving the famous "java.sql.SQLException: Total number of available connections are less than the total number of closed connections" issue

While starting up your WSO2 product after configuring a registry mount, you may have come across the below issue.

Caused by: java.sql.SQLException: Total number of available connections are less than the total number of closed connections
    at org.wso2.carbon.registry.core.jdbc.dataaccess.JDBCDatabaseTransaction$ManagedRegistryConnection.close(JDBCDatabaseTransaction.java:1349)
    at org.wso2.carbon.registry.core.jdbc.dataaccess.JDBCTransactionManager.endTransaction(JDBCTransactionManager.java:178)
    ... 46 more


When you see the above exception, the first thing you have to do is verify the mount configuration of the registry.xml.

See below. In this config, if you accidentally refer to the dbConfig name of the local DB in the mount, you will get the exception mentioned in the $subject.The correct dbConfig name you should refer to is, wso2Mount where wso2Mount is pointed to an external DB.

E.g. :-

     <currentDBConfig>wso2registry</currentDBConfig>
    <readOnly>false</readOnly>
    <enableCache>true</enableCache>
    <registryRoot>/</registryRoot>

    <dbConfig name="wso2registry">
        <dataSource>jdbc/WSO2CarbonDB</dataSource>
    </dbConfig>

    <dbConfig name="wso2Mount">
        <dataSource>jdbc/WSO2RegDB</dataSource>
    </dbConfig>

    <remoteInstance url="https://localhost:9443/registry">
        <id>instanceid</id>
        <dbConfig>wso2Mount</dbConfig>
        <readOnly>false</readOnly>
        <enableCache>true</enableCache>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>instanceid</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

Friday, July 10, 2015

Dealing with "HTTP 502 Bad Gateway"

Recently, while configuring WSO2 products with NginX, we were struggling to connect to the management console of the product since it was returning "HTTP 502 Bad Gateway" error. After doing some research, found out that it was a problem with SELinux.

There are two SELinux boolean settings available by default. One of them is httpd_can_network_connect which allows httpd to connect to anything on the network.

So to enable this, I executed the below command and everything worked well.

sudo setsebool -P httpd_can_network_connect 1
However, there are other similar variables that can be enabled other than this and they are explained in detail in [1].

[1] - https://www.axivo.com/resources/selinux-booleans-explained.22/

Thursday, July 9, 2015

How to print the results summary when JMeter is running on headless mode

When you are running JMeter in GUI mode, you can easily view the results through the Summary report or through the Aggregate report.

But how do you view the summary if you are running JMeter in headless mode?

This can be configured through the jmeter.properties file which resides in $JMETER_HOME/bin folder.

Note that this is enabled by default in latest versions of JMeter. E.g.:- Version 2.13

#--------------------------------------------------------------------------- # Summariser - Generate Summary Results - configuration (mainly applies to non-GUI mode) #--------------------------------------------------------------------------- #

# Define the following property to automatically start a summariser with that name
# (applies to non-GUI mode only)
summariser.name=summary
#
# interval between summaries (in seconds) default 3 minutes
summariser.interval=180
#
# Write messages to log file
summariser.log=true
#
# Write messages to System.out
summariser.out=true

Monday, May 11, 2015

Reason for 'javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?' while working with WSO2 products

While configuring WSO2 API manage to work with WSO2 IS as the Auth Manager, while trying to login to API Store/API Publisher, noticed the following exception.


javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection? at com.sun.net.ssl.internal.ssl.InputRecord.handleUnknownRecord(Unknown Source) at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source) at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source) The reason for this was that we have configured a HTTP port for an HTTPS URL.

  E.g.:-
<authmanager>
        <!--
            Server URL of the Authentication service
        -->
        <serverurl>https://localhost:9763/services/</serverurl>
        <!--
            Admin username for the Authentication manager.
        -->
        <username>admin</username>
        <!--
            Admin password for the Authentication manager.
        -->
        <password>admin</password>
    </authmanager>