Announcement

**Isomorphic** · 2 Jul 2013, 03:48

We created a testcase where a queue of two updates is sent via RESTHandler to a SQLDataSource that implements a DMI which blindly dereferences a value from the DSRequest and then invokes execute() on the DSRequest. The second of these updates does not provide the value that the DMI uses, resulting in an NPE.

On the face of it, this test is exactly the same as yours, and it does indeed cause the first update to be rolled back. You have extra software in the stack here of which we have no visibility - the DMI itself, class LegendRestHandler, and some kind of interaction with Spring, amongst other things, so we suggest you start looking there. If you still believe this is a bug in SmartClient, please provide a minimal, runnable case that demonstrates it.

**claudiobosticco** · 2 Jul 2013, 04:59

SmartClient Version: v8.3p_2013-06-22/PowerEdition Deployment (built 2013-06-22)

ok, I've stripped down my test case, removed my LegendRESTHandler class, so I'm invoking RESTHandler directly, and (as before) I'm not using Spring for the DMI, so I hope Spring isn't interfering with my test case

This is my dataSource:

Code:

<DataSource xmlns:fmt="urn:jsptld:/WEB-INF/fmt.tld" xmlns="http://www.smartclient.com/schema"
            ID="JPC_LISTINI"
            tableName="JPC_LISTINI"
            schema="DBSALES"
            dbName="dbJpcEP"
            serverType="sql"
            requiresAuthentication="false"
        >
	<fields>
        <field sqlType="decimal" primaryKey="true" name="ID_REC" type="sequence" hidden="true">
            <sequenceName>SEQUENCE_ID_REC</sequenceName>
        </field>
        <field sqlType="varchar" sqlLength="200" name="DESCRIZIONE" length="200" type="text">
            <title>
                <fmt:message key="descrizione" bundle="${i18n}"/>
            </title>
        </field>
        <field sqlType="varchar" sqlLength="20" name="NOME" length="20" type="text">
        </field>
		<field foreignKey="JPC_STAGIONI.ID_REC" sqlType="decimal" sqlLength="0" name="ID_STAGIONI_FK" type="integer">
            <title>
                <fmt:message key="stagione" bundle="${i18n}"/>
            </title>
		</field>
        <field sqlType="decimal" sqlLength="0" name="LISTINO_TYPE" type="integer"/>
        <field sqlType="decimal" sqlLength="0" name="LISTINO_CODE" type="integer"/>
        <field sqlType="varchar" sqlLength="3" name="TIPO_LISTINO" type="text"/>
        <field sqlType="timestamp" sqlLength="7" name="DATA_INIZIO_STAGIONE" type="date" includeFrom="JPC_STAGIONI.DATA_INIZIO"/>
        <field sqlType="timestamp" sqlLength="7" name="DATA_FINE_STAGIONE" type="date" includeFrom="JPC_STAGIONI.DATA_FINE"/>
        <field sqlType="varchar" sqlLength="20" name="DESCRIZIONE_STAGIONE" length="20" type="text" includeFrom="JPC_STAGIONI.DESCRIZIONE">
            <title>
                <fmt:message key="stagione" bundle="${i18n}"/>
            </title>
		</field>
        <field name="DATA_INIZIO" type="datetime" required="false"/>
        <field name="DATA_FINE" type="datetime" required="false"/>

	</fields>
    <operationBindings>
        <operationBinding operationType="fetch">
            <selectClause>$defaultSelectClause</selectClause>
            <tableClause>$defaultTableClause</tableClause>
            <whereClause>$defaultWhereClause</whereClause>
            <orderClause>DATA_FINE_STAGIONE DESC, DATA_FINE DESC</orderClause>
            <criteria fieldName="TIPO_LISTINO" operator="equals" value="LEG"></criteria>
        </operationBinding>
        <operationBinding operationType="add">
            <serverObject lookupStyle="new" className="com.juve.legend.eventi.ListinoAddDMI"/>
            <values fieldName="TIPO_LISTINO" value="LEG"></values>
        </operationBinding>
        <operationBinding operationType="update">
            <serverObject lookupStyle="new" className="com.juve.legend.eventi.ListinoUpdateDMI"/>
            <criteria fieldName="TIPO_LISTINO" operator="equals" value="LEG"></criteria>
        </operationBinding>
        <operationBinding operationType="remove">
            <criteria fieldName="TIPO_LISTINO" operator="equals" value="LEG"></criteria>
        </operationBinding>
        <operationBinding operationType="clientExport">
            <criteria fieldName="TIPO_LISTINO" operator="equals" value="LEG"></criteria>
        </operationBinding>
    </operationBindings>
</DataSource>

This is the code I'm using to invoke the RESTHandler:

Code:

    public static void main(String[] args) throws Exception {
        HttpClient httpClient = new HttpClient();
        String localUrl = "http://localhost:8080/Legend/isomorphic/RESTHandler";
        PostMethod post = new PostMethod(localUrl);
        String jsonContent = "{" +
                "    \"transaction\": {" +
                "        \"operations\": [" +
                "            {" +
                "                \"dataSource\": \"JPC_LISTINI\", " +
                "                \"operationType\": \"update\", " +
                "                \"data\": {                    " +
                "                    \"ID_REC\": 475305," +
                "                    \"DESCRIZIONE\": \"zz giornata di campionato\"," +
                "                    \"ID_STAGIONI_FK\": 182001, " +
                "                    \"DATA_INIZIO\": \"2013-09-22T11:07:13\", " +
                "                    \"DATA_FINE\": \"2013-09-22T13:07:13\"          " +
                "                }            " +
                "            }," +
                "            {               " +
                "                \"dataSource\": \"JPC_LISTINI\"," +
                "                \"operationType\": \"update\"," +
                "                \"data\": {                 " +
                "                    \"ID_REC\": 182002," +
                "                    \"DESCRIZIONE\": \"Listino JPC 2013/2014 --\"    " +
                "                }        " +
                "            }" +
                "        ]" +
                "    }" +
                "}";
        post.setRequestEntity(new StringRequestEntity(jsonContent, "application/json", "UTF-8"));
        int responseCode = httpClient.executeMethod(post);
        if (responseCode == 200) {
            String responseBodyAsString = post.getResponseBodyAsString();
            System.out.println(responseBodyAsString);
        }
    }

the DMI update which gives a NPE is:

Code:

    public DSResponse update(DSRequest dsRequest, RPCManager rpcManager) throws Exception {
        dsRequest.getFieldValue("DATA_INIZIO").toString(); // will give NPE for 2nd request in queue
        return dsRequest.execute();
    }

log:

Code:

2013-07-02 14:57:59,454 INFO  RequestContext URL: '/Legend/isomorphic/RESTHandler', User-Agent: 'Jakarta Commons-HttpClient/3.1': Unsupported WITHOUT Accept-Encoding header 
2013-07-02 14:57:59,455 DEBUG FilterChainProxy Converted URL to lowercase, from: '/isomorphic/resthandler'; to: '/isomorphic/resthandler' 
2013-07-02 14:57:59,455 DEBUG FilterChainProxy Candidate is: '/isomorphic/resthandler'; pattern is /images/*; matched=false 
2013-07-02 14:57:59,455 DEBUG FilterChainProxy Converted URL to lowercase, from: '/isomorphic/resthandler'; to: '/isomorphic/resthandler' 
2013-07-02 14:57:59,455 DEBUG FilterChainProxy Candidate is: '/isomorphic/resthandler'; pattern is /isomorphic/resthandler*; matched=true 
2013-07-02 14:57:59,455 DEBUG FilterChainProxy /isomorphic/RESTHandler has an empty filter list 
2013-07-02 14:57:59,456 DEBUG RestRequestParser Parsing json object: '{    "transaction": {        "operations": [            {                "dataSource": "JPC_LISTINI",                 "operationType": "update",                 "data": {                                        "ID_REC": 475305,                    "DESCRIZIONE": "zz giornata di campionato",                    "ID_STAGIONI_FK": 182001,                     "DATA_INIZIO": "2013-09-22T11:07:13",                     "DATA_FINE": "2013-09-22T13:07:13"                          }                        },            {                               "dataSource": "JPC_LISTINI",                "operationType": "update",                "data": {                                     "ID_REC": 182002,                    "DESCRIZIONE": "Listino JPC 2013/2014 --"                    }                    }        ]    }}' 
2013-07-02 14:57:59,466 DEBUG HttpSessionEventPublisher Publishing event: org.springframework.security.web.session.HttpSessionCreatedEvent[source=org.apache.catalina.session.StandardSessionFacade@62182f06] 
2013-07-02 14:57:59,467 INFO  RESTHandler Performing 2 operation(s) 
2013-07-02 14:58:15,073 INFO  SQLDriver [builtinApplication.JPC_LISTINI_update] Executing SQL update on 'dbJpcEP': UPDATE DBSALES.JPC_LISTINI SET DATA_FINE=TO_DATE('2013-09-22 15:07:13','YYYY-MM-DD HH24:MI:SS'), DATA_INIZIO=TO_DATE('2013-09-22 13:07:13','YYYY-MM-DD HH24:MI:SS'), DESCRIZIONE='zz giornata di campionato', ID_STAGIONI_FK=182001 WHERE (JPC_LISTINI.ID_REC=475305) 
2013-07-02 14:58:15,078 INFO  SQLDriver [builtinApplication.JPC_LISTINI_update, builtinApplication.null] Executing SQL query on 'dbJpcEP': SELECT JPC_LISTINI.DATA_FINE, JPC_LISTINI.DATA_INIZIO, JPC_LISTINI.DESCRIZIONE, JPC_LISTINI.ID_REC, JPC_LISTINI.ID_STAGIONI_FK, JPC_LISTINI.LISTINO_CODE, JPC_LISTINI.LISTINO_TYPE, JPC_LISTINI.NOME, JPC_LISTINI.TIPO_LISTINO, JPC_STAGIONI.DATA_INIZIO AS DATA_INIZIO_STAGIONE, JPC_STAGIONI.DATA_FINE AS DATA_FINE_STAGIONE, JPC_STAGIONI.DESCRIZIONE AS DESCRIZIONE_STAGIONE FROM DBSALES.JPC_LISTINI, DBSALES.JPC_STAGIONI WHERE (JPC_LISTINI.ID_REC=475305 AND JPC_LISTINI.TIPO_LISTINO='LEG') AND JPC_LISTINI.ID_STAGIONI_FK = JPC_STAGIONI.ID_REC ORDER BY DATA_FINE_STAGIONE DESC, DATA_FINE DESC 
2013-07-02 14:58:15,081 INFO  DSResponse [builtinApplication.JPC_LISTINI_update, builtinApplication.null] DSResponse: List with 1 items 
2013-07-02 14:58:15,081 INFO  DSResponse [builtinApplication.JPC_LISTINI_update] DSResponse: List with 1 items 
2013-07-02 14:58:43,492 INFO  DefaultServicesManagerImpl Reloading registered services. 
2013-07-02 14:58:43,492 DEBUG JpaTemplate Creating new EntityManager for JpaTemplate execution 
2013-07-02 14:58:43,492 DEBUG SQL select registered0_.id as id0_, registered0_.allowedToProxy as allowedT2_0_, registered0_.anonymousAccess as anonymou3_0_, registered0_.description as descript4_0_, registered0_.enabled as enabled0_, registered0_.evaluation_order as evaluation6_0_, registered0_.ignoreAttributes as ignoreAt7_0_, registered0_.name as name0_, registered0_.serviceId as serviceId0_, registered0_.ssoEnabled as ssoEnabled0_, registered0_.theme as theme0_ from RegisteredServiceImpl registered0_ 
Hibernate: select registered0_.id as id0_, registered0_.allowedToProxy as allowedT2_0_, registered0_.anonymousAccess as anonymou3_0_, registered0_.description as descript4_0_, registered0_.enabled as enabled0_, registered0_.evaluation_order as evaluation6_0_, registered0_.ignoreAttributes as ignoreAt7_0_, registered0_.name as name0_, registered0_.serviceId as serviceId0_, registered0_.ssoEnabled as ssoEnabled0_, registered0_.theme as theme0_ from RegisteredServiceImpl registered0_
2013-07-02 14:58:43,492 WARN  RequestContext dsRequest.execute() failed:  
java.lang.NullPointerException
	at com.juve.legend.eventi.ListinoUpdateDMI.update(ListinoUpdateDMI.java:19)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:972)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:416)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2030)
	at com.isomorphic.servlet.RESTHandler.handleDSRequest(RESTHandler.java:341)
	at com.isomorphic.servlet.RESTHandler.processRequest(RESTHandler.java:319)
	at com.isomorphic.servlet.RESTHandler.doPost(RESTHandler.java:259)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.juve.utils.Log4jSessionFilter.doFilter(Log4jSessionFilter.java:65)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:163)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:246)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.isomorphic.js.JSSyntaxScannerFilter.doFilter(JSSyntaxScannerFilter.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:76)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:722)
2013-07-02 14:58:43,495 DEBUG RPCManager Content type for RPC transaction: text/html; charset=UTF-8 
2013-07-02 14:58:43,496 DEBUG JpaTemplate Closing new EntityManager after JPA template execution 
2013-07-02 14:58:43,496 DEBUG EntityManagerFactoryUtils Closing JPA EntityManager 
2013-07-02 14:58:43,496 INFO  DefaultServicesManagerImpl Loaded 0 services. 
2013-07-02 14:58:43,500 DEBUG RPCManager DMI response, dropExtraFields: true 
2013-07-02 14:58:43,502 DEBUG RPCManager non-DMI response, dropExtraFields: false

but no rollback!

please let me know if my test case is valid, this time.
If you want I would try your test case, too.

**Isomorphic** · 4 Jul 2013, 06:01

OK, the problem here is that the framework has marked the second DSRequest as having started, but it never gets as far as determining that it should join the automatic transaction because it fails in DMI code. The logic for deciding whether or not a database transaction needs to be rolled back assumes that a DSRequest that is marked "started" but not marked "partOfTransaction" is not part of the transaction - ie, it is a separate request that came in the same queue but was excluded from the transaction for some reason (there are many legitimate reasons why a given request might be excluded from the transaction).

In this case, this is an invalid assumption - the request has not been marked "partOfTransaction" because DMI code failed before that step, not because it is really outisde the transaction scope. So one way to fix this would be to call

Code:

dsRequest.setPartOfTransaction(true);

at the top of your DMI.

However, we feel that the automatic behavior could be better in this case. So, we have changed the system so that it tracks whether "partOfTransaction" has been explicitly set one way or the other; when we come to error processing and discover a request that was started but never had its transactional flag set explicitly, we check whether that request would have joined the queue if the DMI had been bypassed.

This gives the correct automatic behavior in most "normal" cases of a DMI on a built-in DataSource type - where the DMI method is there to augment the built-in functionality, and goes on to call dsRequest.execute(). For cases where a DMI does not call dsRequest.execute() and thus would never have been part of the transaction, this heuristic will be incorrect, and it will be necessary to call setPartOfTransaction(false) to override it.

This change will be present in 8,3 and 9.0 builds as of tomorrow, 7/5

Regards,
Isomorphic Software Support

**claudiobosticco** · 5 Jul 2013, 02:06

SmartClient Version: v8.3p_2013-07-05/PowerEdition Deployment (built 2013-07-05)

thank you very much, also for the detailed explanation, the fix is working.

Announcement

[bug ?] RESTHandler transactions

Comment

Comment

Comment

Comment