Disable automatic transaction merge in 8.2

[Osirium1]

I am using the 8.2 release build.

I have a subclassed RestDataSource. Having dropped SC8.2 into my application I am now seeing the log saying:

"Multiple RPCRequests with params attribute in one transaction - merging"

And the request is indeed sent out as a transaction. This never happened with the previous sc_18_10 power 8.1 nightly I was using.

I never want my requests to be queued as my server can't handle a queue. Is there a flag that I can set to prevent transactions from being auto generated?

Many thanks

Osirium

[bakosa]

I'm having the same problem, did you manage to solve this?

[Osirium1]

No solution yet.

I have also found jsonPrefix and jsonSuffix are getting set on the DataSource even though I want them to be null.

And finally, I modified my REST server to process transactions and return the response as outlined in the documentation. However I then got:

WARN:RPCManager:Found a REST request that appears to be in JSON format, but the response was not wrapped in start/end markers - aborting

I looked in the LGPL code but didn't get very far

[fazleyrabbi]

I do not get why it is not working for just one of my larger datasources... that is with more than 27 fields... it seems to work with smaller number of fields.... :( :(

[Isomorphic]

We've added a new flag to turn off transaction merging, RestDataSource.disableQueuing. This will be available in tomorrow's builds.

There is a problem with jsonPrefix/suffix where if you want no prefix or suffix you need to call the setters after calling getOrCreateJSObj() to force some initialization work to happen. We'll fix this too, however, be sure you've read the docs on these properties - unprefixed JSON responses are a security vulnerability.

[billyboingo]

I'm having similiar issues with SmartGWT v3.0, and I see this in (hosted mode) both Chrome 16.0.912.63 m and Firefox 7.0.1. Here is a snippet of the logs from the Developer concole:

15:57:40.187:XRP1[E]:DEBUG:RPCManager:Grabbed prompt from first request that defined one: Finding Records that match your criteria...
15:57:40.236:XRP1[E]:WARN:RPCManager:Multiple RPCRequests with params attribute in one transaction - merging
15:57:40.237:XRP1[E]:INFO:RPCManager:sendQueue[1]: 2 RPCRequest(s); transport: xmlHttpRequest; target: /api/datasource
15:57:40.238:XRP1[E]:DEBUG:RPCManager:XMLHttpRequest POST to /api/datasource?isc_dataFormat=json contentType: application/json with body -->{ transaction: { transactionNum: 1, operations: [{
    "dataSource":"contacts", 
    "operationType":"fetch", 
    "startRow":0, 
    "endRow":75, 
    "textMatchStyle":"substring", 
    "componentId":"isc_ListGrid_0", 
    "data":{
    }, 
    "oldValues":null
}, {
    "dataSource":"location", 
    "operationType":"fetch", 
    "data":null, 
    "oldValues":null
}]}}<--
15:57:40.509:XRP0:INFO:RPCManager:transaction 1 arrived after 272ms
15:57:40.509:XRP0:DEBUG:RPCManager:Result string for transaction 1: "[{"response":{"status":0,"queueStatus":0,"data":[{"id":1,"number":246,"name":"Paradise ","locationId":null}],"startRow":0,"endRow":1,"totalRows":1,"errors":null}},{"response":{"status":0,"queueStatus":0,"data":[],"startRow":0,"endRow":0,"totalRows":0,"errors":null}}]"
15:57:40.510:XRP0:WARN:RPCManager:Found a REST request that appears to be in JSON format, but the response was not wrapped in start/end markers - aborting

As far as I can tell, I'm matching the return format specified in the RestDataSource documentation. Am I missing something?

I'm using Jackson to do the JSON formatting, and it always worked fine until I starting trying to use queueing.

[Isomorphic]

No start/end markers - see RestDataSource.jsonPrefix.

[billyboingo]

No start/end markers - see RestDataSource.jsonPrefix.

I must be missing something. Here is part of my datasource code:

setOperationBindings(fetch, add, update, remove);
setDataFormat(DSDataFormat.JSON);

setDataURL(DATA_URL);
setID(name);
		
setJsonPrefix("while(1);");
setJsonSuffix("//");

But I still get:

17:04:04.615:XRP9:INFO:RPCManager:transaction 1 arrived after 208ms
17:04:04.615:XRP9:DEBUG:RPCManager:Result string for transaction 1: "while(1);[{"response":{"status":0,"queueStatus":0,"data":[{"id":1,"number":246,"name":"Paradise ","locationId":null}],"startRow":0,"endRow":1,"totalRows":1,"errors":null}},{"response":{"status":0,"queueStatus":0,"data":[],"startRow":0,"endRow":0,"totalRows":0,"errors":null}}]//"
17:04:04.615:XRP9:WARN:RPCManager:Found a REST request that appears to be in JSON format, but the response was not wrapped in start/end markers - aborting

I tried following this:

...if you want no prefix or suffix you need to call the setters after calling getOrCreateJSObj() to force some initialization work to happen...

with:

getOrCreateJsObj();
setJsonPrefix("while(1);");
setJsonSuffix("//");

but that results in a dialog (at least on firefox):

Error :Cannot change configuration property 'jsonPrefix' to while(1); after the component has been created.

Anymore help would be appreciated.

[Isomorphic]

We're still planning to fix the problem with jsonPrefix not being settable. You can use setAttribute("jsonPrefix", "newPrefix") after getOrCreateJSObj instead.

[billyboingo]

Interestingly, this works fine for single request/response. I set the prefix/suffix as you described, my response is wrapped and everyone is happy. But as soon as it is a transaction, nothing works - I am still getting the complaint about the response not being wrapped.

[billyboingo]

In a desperate attempt to get transactions to work, I tracked down the default prefix and suffix and wrapped my responses with those. Still the same story:

11:59:58.654:XRP7:INFO:RPCManager:transaction 1 arrived after 269ms
11:59:58.654:XRP7:DEBUG:RPCManager:Result string for transaction 1: "//isc_JSONResponseStart-->[{"response":{"status":0,"queueStatus":0,"data":[{"id":1,"number":246,"name":"Paradise ","locationId":null}],"startRow":0,"endRow":1,"totalRows":1,"errors":null}},{"response":{"status":0,"queueStatus":0,"data":[],"startRow":0,"endRow":0,"totalRows":0,"errors":null}}]//isc_JSONResponseEnd"
11:59:58.654:XRP7:WARN:RPCManager:Found a REST request that appears to be in JSON format, but the response was not wrapped in start/end markers - aborting

Any ideas?

Just one last thing, the javadocs are a bit misleading on this one:

setJsonPrefix

public void setJsonPrefix(String jsonPrefix)
                   throws IllegalStateException
Allows you to specify an arbitrary prefix string to apply to all json format responses sent from the server to this application.
The inclusion of such a prefix ensures your code is not directly executable outside of your application, as a preventative measure against javascript hijacking.

Only applies to responses formatted as json objects. Does not apply to responses returned via scriptInclude type transport.
Note: If the prefix / suffix served by your backend is not a constant, you can use dataFormat:"custom" instead and explicitly parse the prefix out as part of transformResponse().

Note : This is an advanced setting

Parameters:
jsonPrefix - jsonPrefix Default value is null

The same is true of setJsonSuffix(), yet the default IS NOT null...

[lucasm]

I'm using SmartGwt 3.0p build 31-Mar-2012 (yesterday's build). The solution offered of using setAttribute("jsonPrefix", "newPrefix) does not work. First of all, this function doesn't exist. It requires a third paramter: boolean allowPostCreate. Below is a simple test case and here's the error produced:

"Error: 'Unable to get value of the property 'errors': object is null or undefined' in .../sc/modules/ISC_DataBindings.js at line 1274...'

This test case contains two listGrids for drag n drop records. Grid2 has a criteria set ("SurveyorId", 2). I am attempting to set JsonPrefix and JsonSuffix as empty.

package com.lucas.smartgwt.test.client;

import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.core.client.GWT;
import com.google.gwt.user.client.ui.RootPanel;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.Criteria;
import com.smartgwt.client.data.OperationBinding;
import com.smartgwt.client.data.RestDataSource;
import com.smartgwt.client.data.fields.DataSourceIntegerField;
import com.smartgwt.client.data.fields.DataSourceTextField;
import com.smartgwt.client.types.DSDataFormat;
import com.smartgwt.client.types.DSOperationType;
import com.smartgwt.client.types.DSProtocol;
import com.smartgwt.client.util.KeyCallback;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.layout.VLayout;

public class Test implements EntryPoint {

	@Override
	public void onModuleLoad() {

		final String URL = "php/rest.php"; 

		// Set database table
		final RestDataSource dataSource = new RestDataSource();
		dataSource.setID("TestDataSource");

		// Setup primary key column
		DataSourceIntegerField idField = new DataSourceIntegerField("Id", "Id");
		idField.setPrimaryKey(true);

		// Setup other columns
		DataSourceTextField primaryField   = new DataSourceTextField("Primary",    "Primary Name"  );
		DataSourceTextField secondaryField = new DataSourceTextField("Secondary",  "Secondary Name");
		DataSourceTextField surveyorField  = new DataSourceTextField("SurveyorId", "Surveyor Id"   );

		// Hidden fields
		idField.setHidden(true);

		// Set columns
		dataSource.setFields(idField, primaryField, secondaryField, surveyorField);  

		// Fetch binding
		OperationBinding fetch = new OperationBinding();
		fetch.setOperationType(DSOperationType.FETCH);
		fetch.setDataProtocol(DSProtocol.POSTMESSAGE);
		fetch.setDataURL(URL);

		// Add binding
		OperationBinding add = new OperationBinding();
		add.setOperationType(DSOperationType.ADD);
		add.setDataProtocol(DSProtocol.POSTMESSAGE);
		add.setDataURL(URL);

		// Remove binding
		OperationBinding remove = new OperationBinding();
		remove.setOperationType(DSOperationType.REMOVE);
		remove.setDataProtocol(DSProtocol.POSTMESSAGE);
		remove.setDataURL(URL);

		// Update binding
		OperationBinding update = new OperationBinding();
		update.setOperationType(DSOperationType.UPDATE);
		update.setDataProtocol(DSProtocol.POSTMESSAGE);
		update.setDataURL(URL);

		// Set bindings
		dataSource.setOperationBindings(fetch, add, remove, update);

		// Specify as Javascript Object Notation
		dataSource.setDataFormat(DSDataFormat.JSON);

		// Create components
		final VLayout layout = new VLayout();
		final ListGrid grid1 = new ListGrid();
		final ListGrid grid2 = new ListGrid();

		// Set datasource to grids
		grid1.setDataSource(dataSource);
		grid2.setDataSource(dataSource);
		
		// Set surveyor criteria to grid 2
		Criteria criteria = new Criteria();
		criteria.addCriteria("SurveyorId", 2);
		grid2.setCriteria(criteria);
		
		// Set dragging records from grid1 to grid
		grid1.setCanDragRecordsOut(true);
		grid2.setCanAcceptDroppedRecords(true);
		
		// Create JS Object and set json encapsulation
		dataSource.getOrCreateJsObj();
		dataSource.setAttribute("jsonPrefix", "", true);
		dataSource.setJsonSuffix("");
		dataSource.setDisableQueuing(false);
		
		// Layout
		layout.setWidth100();
		layout.setHeight100();
		grid1.setWidth(500);
		grid1.setHeight(300);
		grid2.setWidth(500);
		grid2.setHeight(300);
		layout.addMember(grid1);
		layout.addMember(grid2);


		// Create shortcut for developers console
		if (!GWT.isScript()) { 
			KeyIdentifier debugKey = new KeyIdentifier(); 
			debugKey.setCtrlKey(true); 
			debugKey.setKeyName("D"); 
			Page.registerKey(debugKey, new KeyCallback() { 
				public void execute(String keyName) { 
					SC.showConsole(); 
				}
			});
		}
		
		// Add to root layout
		RootPanel.get().add(layout);

		// Fetch data
		grid1.fetchData();
		grid2.fetchData(criteria);
	}
}

Ok. What about leaving the JSON encapsulation as default values you ask? These are:

Prefix: <SCRIPT>//'\"]]>>isc_JSONResponseStart>>
Suffix: //isc_JSONResponseEnd

I remove the setJsonSuffix/setJsonPrefix lines. It works for single dsrequests, but transactions I still get the following error:

"WARN:RPCManager:Found a REST request that appears to be in JSON format, but the response was not wrapped as configured by the jsonPrefix and jsonSuffix properties - aborting"

Here is the JSON response in question, just so you can see it's properly formatted:

<SCRIPT>//'\"]]>>isc_JSONResponseStart>>[
{
   "response":
   {
      "status":0,
      "startRow":0,
      "endRow":1,
      "totalRows":1,
      "data":[
      {
         "Id":3,
         "Primary":"Primary3",
         "Secondary":"Secondary3",
         "SurveyorId":2
      }],
      "queueStatus":0
   }
},
{
   "response":
   {
      "status":0,
      "startRow":0,
      "endRow":1,
      "totalRows":1,
      "data":[
      {
         "Id":4,
         "Primary":"Primary4",
         "Secondary":"Secondary4",
         "SurveyorId":2
      }],
      "queueStatus":0
   }
}]//isc_JSONResponseEnd

[Isomorphic]

This solution was obsoleted since the dedicated setJsonPrefix / suffix setters now work.

We have automated test cases showing the jsonPrefix/suffix functionality is also working properly, whether the default prefixes are used or not; one problem with your response is that there should not be a <script> tag.

[lucasm]

Thank you. It works now.