Friday, September 19, 2014

How to solve the famous token regeneration issue in an API-M cluster

In a API Manager clustered environment (in my case, I have a publisher, a store, two gateway nodes and two key manager nodes fronted by a WSO2 ELB 2.1.1), while regenerating tokens, if you come across an error saying Error in getting new accessToken with an exception as below at Key Manager node, then this is due to a configuration issue.

TID: [0] [AM] [2014-09-19 05:41:28,321]  INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -  'Administrator@carbon.super [-1234]' logged in at [2014-09-19 05:41:28,321-0400] {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil}
TID: [0] [AM] [2014-09-19 05:41:28,537] ERROR {org.wso2.carbon.apimgt.keymgt.service.APIKeyMgtSubscriberService} -  Error in getting new accessToken {org.wso2.carbon.apimgt.keymgt.service.APIKeyMgtSubscriberService}
TID: [0] [AM] [2014-09-19 05:41:28,538] ERROR {org.apache.axis2.rpc.receivers.RPCMessageReceiver} -  Error in getting new accessToken {org.apache.axis2.rpc.receivers.RPCMessageReceiver}
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:619)
    at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
    at org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117)
    at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
    at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
    at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:853)
Caused by:
org.wso2.carbon.apimgt.keymgt.APIKeyMgtException: Error in getting new accessToken
    at org.wso2.carbon.apimgt.keymgt.service.APIKeyMgtSubscriberService.renewAccessToken(APIKeyMgtSubscriberService.java:281)
    ... 45 more
Caused by:
java.lang.RuntimeException: Token revoke failed : HTTP error code : 404
    at org.wso2.carbon.apimgt.keymgt.service.APIKeyMgtSubscriberService.renewAccessToken(APIKeyMgtSubscriberService.java:252)
    ... 45 more


This is what you have to do to solve this issue.

1. In you Gateway nodes, you need to change the host and the port values of the below APIs that resides under $APIM_HOME/repository/deployment/server/synapse-configs/default/api                                                      _TokenAPI_.xml                                                                                                                                         _AuthorizeAPI_.xml                                                                                                                                       _RevokeAPI_.xml                                                                                                                                        
If you get a HTTP 302 error at Key manager side while regenerating the token, make sure to check the RevokeURL of the api-manager.xml of the Key Manager node to see if it is pointing to the NIO port of the Gateway Node.