Dev console: RPCRequest/DSRequest confusion, data mixup?

[csillag]

Hi,

While looking at the RPC manager in the Dev Console, in the request section, there is RPCRequest and DSRequest.

What is the exact relationship between these two, and what is the workflow here? My understanding until now was that DSRequest is the higher-level representation, describing the intended purpose of the action, and RPCRequest is the lower-level representation, which was actually sent.

However, what I am seeing now seems to contradict this.

I have a form, having only three fields (login, username, email).

Saving a new record using this form generates two calls: there one for validation, and then one for the actual add. In the Developer Console, here is what I see about the second call:

DSRequest:

{
    "dataSource":"app_users_auto", 
    "operationType":"add", 
    "operationId":"registerUser", 
    "componentId":"isc_DynamicForm_2", 
    "data":{
        "USER_LOGIN":"tomp", 
        "USER_NAME":"TOMP", 
        "USER_EMAIL":"t@o.mpa", 
        "REGISTRATION_HOST_NAME":"127.0.0.1", 
        "REGISTRATION_IP":"127.0.0.1", 
        "ADMINISTRATOR":false, 
        "USER_ID":13, 
        "ALLOWED":true, 
        "REGISTRATION_DATE":"2011-02-24T05:22:03", 
        "USER_PASSWORD":"U5HofkkcGIZWDpG1F59t6vLOjUg="
    }, 
    "callback":{
        "target":[DynamicForm ID:isc_DynamicForm_2], 
        "methodName":"saveEditorReply"
    }, 
    "willHandleError":true, 
    "showPrompt":true, 
    "prompt":"Saving form...", 
    "oldValues":{
    }, 
    "clientContext":{
    }, 
    "requestId":"app_users_auto$6272"
}

RPCRequest looks like this:

{
    "actionURL":"http://127.0.0.1:8888/cogdemo/sc/IDACall", 
    "showPrompt":false, 
    "prompt":"Validating...", 
    "transport":"xmlHttpRequest", 
    "promptStyle":"cursor", 
    "bypassCache":true, 
    "data":{
        "values":{
            "USER_LOGIN":"tomp", 
            "USER_NAME":"TOMP", 
            "USER_EMAIL":"t@o.mpa"
        }, 
        "operationConfig":{
            "dataSource":"app_users_auto", 
            "repo":null, 
            "operationType":"validate"
        }, 
        "validationMode":"full", 
        "appID":"builtinApplication", 
        "operation":"app_users_auto_validate", 
        "oldValues":{
            "USER_LOGIN":"tomp", 
            "USER_NAME":"TOMP", 
            "USER_EMAIL":"t@o.mpa"
        }
    }
}

To have a full picture, here is the request as percieved by the server side:

=== 2011-02-24 05:22:03,220 [0-24] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    values:{
        USER_LOGIN:"tomp",
        USER_NAME:"TOMP",
        USER_EMAIL:"t@o.mpa"
    },
    operationConfig:{
        dataSource:"app_users_auto",
        operationType:"add"
    },
    componentId:"isc_DynamicForm_2",
    appID:"builtinApplication",
    operation:"registerUser",
    oldValues:{
    },
    criteria:{
    }
}

The 3rd log (which was gathered from the server) is correct, but as you can see, the DSRequest shown by the RPC manager seems to contain several fields which were definitly not given by the user (like registration IP, date and password); these were generated and added on the server side, so they could not be part of the DSRequest! My guess would be that these fields are added here from the DSResponse, but is this intended? If yes, then what's the purpose?

* * *

On the other hand, the (supposedly) matching RPCRequest only contains what was actually given, which is nice, but it also says "operationType":"validate", which is weird for an add operation.

In fact, the "RPCRequest" part of the second (add) calls matches byte-to-byte to the RPCRequest part of the second (validate) call. It looks like it's just a copy of the first call, instead of the RPCRequest of the second call. Is this intended? If yes, then what's the puspose?

Thank you for your help:

Csillag

ps. I am using SmartGWT Power edition, nightly build from 02.23

[csillag]

In fact, the "RPCRequest" part of the second (add) calls matches byte-to-byte to the RPCRequest part of the second (validate) call. It looks like it's just a copy of the first call, instead of the RPCRequest of the second call.

Oops.. typo error. I try again:

The RPCRequest of the second (add) call matches byte-to-byte to the RPCRequest of the first (validate) call.

It looks like it's just a copy of the first call, instead of the actual RPCRequest of the second call.

[csillag]

I need to reiterate.

When clicking on the second (add) request in the RPC history, the content of the RPCRequest tab (in the Request section in the RPC manager in the Dev Console) is simply not refreshed; it keeps the previous content.

So what see is the RPCRequest representation of whatever call I was previously examining.

The interesting thing is that this only happens for this one type of DSRequest; other DSRequest have their RPCRequest representation shown properly.

(So this seems to be a bug.)

Thank you for your help:

Csillag

[csillag]

Please see also this bug report about an other (but possibly related) problem with the dev console's RPC manager.

[Isomorphic]

In what sample, with what series of requests, in this problem reproducible?

[csillag]

SQL dump:

CREATE TABLE IF NOT EXISTS `test_users` (
  `USER_ID` int(10) NOT NULL AUTO_INCREMENT,
  `USER_LOGIN` varchar(50) NOT NULL COMMENT 'Login',
  `USER_NAME` varchar(128) NOT NULL COMMENT 'Full name',
  `USER_EMAIL` varchar(60) NOT NULL COMMENT 'Email',
  `ADMINISTRATOR` varchar(6) NOT NULL DEFAULT 'False' COMMENT 'Administrator',
  `ALLOWED` varchar(6) DEFAULT 'true' COMMENT 'Allowed',
  PRIMARY KEY (`USER_ID`),
  UNIQUE KEY `Login` (`USER_LOGIN`)
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 AUTO_INCREMENT=17 ;

Datasource definition:

<?xml version="1.0" encoding="UTF-8"?>
<DataSource ID="users" autoDeriveSchema="true" serverType="sql" tableName="test_users" useParentFieldOrder="false" validateRelatedRecords="true">
    <fields>
        <field canEdit="false" hidden="true" name="USER_ID" title="User Id" type="integer"/>
        <field length="50" name="USER_LOGIN" required="true" title="Login" type="text">
            <validators>
                <validator errorMessage="We already have a user with this login. Please chose a different one!" type="isUnique"/>
            </validators>
        </field>
        <field length="128" name="USER_NAME" required="true" title="Full name" type="text"/>
        <field length="60" name="USER_EMAIL" required="true" title="Email" type="text">
            <validators>
                <validator errorMessage="Does not seem to be a valid email address. Please check!" expression="^([a-zA-Z0-9_.\-+])+@(([a-zA-Z0-9\-])+\.)+[a-zA-Z0-9]{2,4}$" type="regexp"/>
            </validators>
        </field>
        <field hidden="true" name="ADMINISTRATOR" title="Administrator" type="boolean"/>
        <field hidden="true" name="ALLOWED" title="Allowed" type="boolean"/>
    </fields>
</DataSource>

Client-side Java code:

public class SubmitTester implements EntryPoint {

	final static String DS_NAME="users";
	
	@Override public void onModuleLoad() {
		DataSource.load(DS_NAME, new Function() { 
			@Override public void execute() { run(); }
		}, false);
	}

	public static void run() {
		new DynamicForm() {{
			setDataSource(DataSource.get(DS_NAME));
			draw();
			editNewRecord();
			getItem("USER_LOGIN").setValue("login" + new Date().getTime());
			getItem("USER_NAME").setValue("name");
			getItem("USER_EMAIL").setValue("a@b.com");
			submit();
			hide();
		}};
		SC.showConsole();
	}
}

Then use can use the Dev console (opened by the testcase) to analyze the events in the RPC manager. There are three calls:
- DataSource load
- Validation
- Add

As you can see, the third DSRequest will show data that was not entered into the form, like the ID, which is generated on the server side, in the DB, using auto-increment.

* * *

In my original post, I have described two different problems (wrong data shown in DSRequest, and RPCManager not refreshed); this test case is for the first error.

Do you need anything else?

[davidj6]

Could you re-run your test and post the response from the validate call?

[csillag]

Could you re-run your test and post the response from the validate call?

The response for the a successful validation is:

[
    {
        queueStatus:0, 
        isDSResponse:true, 
        invalidateCache:false, 
        status:0, 
        data:null
    }
]

[davidj6]

I'm not seeing your situation so far. As currently designed, the "validate" call should be triggered just prior to the add and that's what you are seeing. The validate call is formatted almost identically to the add operation so that's also expected. Where I'm confused is that I cannot visualize what you are really seeing that's an issue.

Can you send a screen shot of your dev console showing the 3 requests and the add response? The screen shot shows the timeline of the requests/responses. Also, please include the request/response for the validate and add operations together. I know you sent some but this will clarify exactly what is being sent and received.

As for the extra "validate" call, this will be removed in an upcoming release - the add operation performs the same validation on the server already.

[csillag]

I'm not seeing your situation so far. As currently designed, the "validate" call should be triggered just prior to the add and that's what you are seeing.

It is.

The validate call is formatted almost identically to the add operation so that's also expected. Where I'm confused is that I cannot visualize what you are really seeing that's an issue.

I am seeing a DSRequest that's just not true; it's containing data that the client has not sent.

Can you send a screen shot of your dev console showing the 3 requests and the add response? The screen shot shows the timeline of the requests/responses. Also, please include the request/response for the validate and add operations together. I know you sent some but this will clarify exactly what is being sent and received.

OK, will do.

[csillag]

I have attached the screen shot. You can see the three requests, and (some of) the request/response of the add request.

Here os all request/response data:

1. Request 1: validation

1.1 DSRequest:

{
    "dataSource":"users", 
    "operationType":"validate", 
    "data":{
        "USER_LOGIN":"login1298927788805", 
        "USER_NAME":"name", 
        "USER_EMAIL":"a@b.com"
    }, 
    "willHandleError":true, 
    "showPrompt":false, 
    "prompt":"Validating...", 
    "oldValues":{
        "USER_LOGIN":"login1298927788805", 
        "USER_NAME":"name", 
        "USER_EMAIL":"a@b.com"
    }, 
    "clientContext":{
        "component":[DynamicForm ID:isc_SubmitTester_2_0], 
        "fieldName":"USER_LOGIN", 
        "rowNum":null, 
        "pendingFields":[
            "USER_LOGIN"
        ]
    }, 
    "requestId":"users$6270"
}

1.2 DSResponse:

[
    {
        queueStatus:0, 
        isDSResponse:true, 
        invalidateCache:false, 
        status:0, 
        data:null
    }
]

2. Request 2: add

2.1.a DSRequest (according to client RPC manager):

{
    "dataSource":"users", 
    "operationType":"add", 
    "componentId":"isc_SubmitTester_2_0", 
    "data":{
        "USER_LOGIN":"login1298927788805", 
        "USER_NAME":"name", 
        "USER_EMAIL":"a@b.com", 
        "ADMINISTRATOR":false, 
        "USER_ID":19, 
        "ALLOWED":true
    }, 
    "callback":{
        "target":[DynamicForm ID:isc_SubmitTester_2_0], 
        "methodName":"saveEditorReply"
    }, 
    "showPrompt":true, 
    "prompt":"Saving form...", 
    "oldValues":{
    }, 
    "clientContext":{
    }, 
    "requestId":"users$6271"
}

2.1.b. RPCRequest (accoring to client RPC manager):
(I am including this to show that the prviously shown DSRequest is displayed with "data" that was not even in the RPC request):

{
    "actionURL":"http://127.0.0.1:8888/cogdemo/sc/IDACall", 
    "showPrompt":true, 
    "prompt":"Saving form...", 
    "transport":"xmlHttpRequest", 
    "promptStyle":"cursor", 
    "bypassCache":true, 
    "data":{
        "values":{
            "USER_LOGIN":"login1298927788805", 
            "USER_NAME":"name", 
            "USER_EMAIL":"a@b.com"
        }, 
        "operationConfig":{
            "dataSource":"users", 
            "repo":null, 
            "operationType":"add"
        }, 
        "componentId":"isc_SubmitTester_2_0", 
        "appID":"builtinApplication", 
        "operation":"users_add", 
        "oldValues":{
        }
    }
}

2.1.c. DSRequest (according to server log):

=== 2011-02-28 21:16:28,877 [0-15] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    values:{
        USER_LOGIN:"login1298927788805",
        USER_NAME:"name",
        USER_EMAIL:"a@b.com"
    },
    operationConfig:{
        dataSource:"users",
        operationType:"add"
    },
    componentId:"isc_SubmitTester_2_0",
    appID:"builtinApplication",
    operation:"users_add",
    oldValues:{
    },
    criteria:{
    }
}

2.2. DSResponse:

[
    {
        queueStatus:0, 
        isDSResponse:true, 
        invalidateCache:false, 
        status:0, 
        data:[
            {
                USER_EMAIL:"a@b.com", 
                USER_LOGIN:"login1298927788805", 
                ADMINISTRATOR:false, 
                USER_ID:19, 
                USER_NAME:"name", 
                ALLOWED:true
            }
        ]
    }
]

Compare 2.1.a and 2.1.c, and observe that 2.1.a constains bogus stuff that was not sent.

[davidj6]

The extra validation call has been removed and the DSRequest will no longer include values sent back from the server. Both of these fixes will be in the next nightly build.

[csillag]

Tested with the 03.03. nightly, and it works great now.

Thank you.

Csillag