Go Back   SmartClient Forums > Technical Q&A
Wiki Register Search Today's Posts Mark Forums Read

Reply
 
Thread Tools Search this Thread
  #1  
Old 26th May 2008, 11:48
schlonz schlonz is offline
Registered Developer
 
Join Date: May 2008
Posts: 10
Default Problem withjson-rpc for datasources and error handling

Hi,

I am trying to get smartclient talk to a json-rpc 2.0 (draft) compatible server. Unfortunately I am running into troubles with getting back error messages.

Can you please:
a) Confirm that I am on the right track regarding a way to talk to jsonrpc and do not follow a stupid way since there are better ones?
b) Maybe tell me a way how to get errors presented to the user?
c) Maybe tell me a way how to solve the http status issue?


Also, if I am right, error handling of http errors is currently not in place. Think it would be great if it would be. :)

Thanks a lot in advance for your help!

The details:

First hurdle is that the preferred way to contact json-rpc is with a POST - also for fetching. Also the data structures are quite a bit different to what RestDataSource expects. Instead of overwriting nearly everything in RestDataSource I started to write my own data source JsonRPCDataSource based on RestDataSource.

So far fetch works fine but as soon as an error occures I run into troubles. JsonRPC over http maps errors also into the http status. And if smartclient gets a e.g. http status 500 the code does not even reach my transformResponse function but fails earlier in DataSource.js in the _handleJSONTextReply function at the line

Code:
if (evalText.match(/^\s*\{/)) {
with an error

Code:
evalText.match is not a function
This is what the developer console says to it:

Code:
20:10:00.343:XRP6:WARN:Log:Error:
	'Object doesn't support this property or method'
	in http://localhost:8001/
	at line 297
    DataSource.$379(rpcResponse=>Obj, jsonText=>Array[1], rpcRequest=>Obj)
    Class.fireCallback(_1=>Obj, _2=>"rpcResponse,data,rpcRequest", _3=>Array[3], _4=>[JsonRPCDataSource ID:processListDS], _5=>undef) on [Class Class]
    Class.fireCallback(_1=>Obj, _2=>"rpcResponse,data,rpcRequest", _3=>Array[3], _4=>undef)
        "return isc.Class.fireCallback(_1,_2,_3,this,_4)"
    RPCManager.fireReplyCallback(_1=>Obj, _2=>Obj, _3=>Obj, _4=>Array[1])
    RPCManager.fireReplyCallbacks(_1=>Obj, _2=>Obj)
    RPCManager.performOperationReply(_1=>Obj, _2=>Obj)
    RPCManager.$39d(_1=>2)
    RPCManager.performTransactionReply(_1=>2, _2=>Array[1], _3=>undef)
    callback(transactionNum=>2, results=>Obj, wd=>undef)
        "isc.RPCManager.performTransactionReply(transactionNum,results,wd)"
    ** recursed on Class.fireCallback
For the user all this is hidden. No feedback. Just the "Finding records that match your criteria..." window blocking everything else. The only thing a user can do is to hit the browser's refresh button.

Well, I came to the conclusion that currently without rewriting smartclient code there is no way how to deal with http error codes. Just for testing I changed the backend to always send http 200 status codes - also for errors.

My transformResponse deals with the different data structure (I think :)) and should map them to the dsResponse structure. But I never see an error message in the user interface. Just "No items to show." in the ListGrid.

To shorten things I tried to force an error by rewriting my transformResponse this way:

Code:
    transformResponse : function (dsResponse, dsRequest, data) {
        dsResponse.status = isc.DSResponse.STATUS_VALIDATION_ERROR;
        dsResponse.errors = {   pid:[
                      {errorMessage:"First error on field1"},
                      {errorMessage:"Second error on field1"}
                  ]
              };
        dsResponse.data = {   pid:[
                      {errorMessage:"First error on field1"},
                      {errorMessage:"Second error on field1"}
                  ]
              };
        return dsResponse;}
pid is a field in my ListGrid and data source. Still no difference.

The console is well aware of the error:

Code:
21:43:08.515:XRP0:WARN:RPCManager:Error performing rpcRequest: error: VALIDATION_ERROR, response: {data: Obj,
startRow: 0,
status: -4,
endRow: 0,
totalRows: 0,
httpResponseCode: 200,
transactionNum: 2,
clientContext: Obj,
context: Obj,
errors: Obj}
Regards,
Erich

PS: Just in case, the respective json-rpc specifications can be found under:
http://groups.google.com/group/json-...c-1-2-proposal
http://groups.google.com/group/json-...-rpc-over-http


edit: Maybe I should also mention that I am using:
dataFormat: "json"
and
dataProtocol:"postMessage"

Last edited by schlonz; 26th May 2008 at 12:00.. Reason: added two details at the end of the posting
Reply With Quote
  #2  
Old 26th May 2008, 12:02
Isomorphic Isomorphic is offline
Administrator
 
Join Date: May 2006
Posts: 38,393
Default

Hi schlonz,

Starting with DataSource rather than RestDataSource is the right approach.

The fact that you can't get to transformResponse() with a 500 server error using dataFormat:"json" is a bug. We'll correct it and add automated tests so it cannot recur - in the meantime, use dataFormat:"custom" and you'll get the plain text of your response as dsResponse.data in transformResponse(), which you can then eval() yourself.

Your error reporting code is close - take a look at this example to see this working properly. Both your dsResponse.error and dsResponse.data have an extra level of structure ("pid") which doesn't match the intended format of those dsResponse properties.

Also, a failure to fetch data shouldn't be reported as a validation error, which specifically means a failed attempt to save. For a total failure to fetch, use any negative value for dsResponse.status other than VALIDATION_ERROR, which has special meaning. Don't set dsResponse.errors (which again is specifically for validation errors), instead set dsResponse.data to an error message. This will trigger an error dialog by default, which you can override via overriding DataSource.handleError() or RPCManager.handleError()
Reply With Quote
  #3  
Old 26th May 2008, 15:24
schlonz schlonz is offline
Registered Developer
 
Join Date: May 2008
Posts: 10
Default

Hi Isomorphic,

First thanks a lot for your fast reply!

Quote:
Originally Posted by Isomorphic
Hi schlonz,

Starting with DataSource rather than RestDataSource is the right approach.
Sorry, that is what I meant. My data source extends DataSource.


Quote:
The fact that you can't get to transformResponse() with a 500 server error using dataFormat:"json" is a bug. We'll correct it and add automated tests so it cannot recur - in the meantime, use dataFormat:"custom" and you'll get the plain text of your response as dsResponse.data in transformResponse(), which you can then eval() yourself.
Regarding the bug: good to hear! dataFormat: custom does not show the problem as it does with json. That helps a lot!

Quote:
Your error reporting code is close - take a look at this example to see this working properly. Both your dsResponse.error and dsResponse.data have an extra level of structure ("pid") which doesn't match the intended format of those dsResponse properties.
You are right. My code was indeed wrong.

Quote:
Also, a failure to fetch data shouldn't be reported as a validation error, which specifically means a failed attempt to save. For a total failure to fetch, use any negative value for dsResponse.status other than VALIDATION_ERROR, which has special meaning. Don't set dsResponse.errors (which again is specifically for validation errors), instead set dsResponse.data to an error message. This will trigger an error dialog by default, which you can override via overriding DataSource.handleError() or RPCManager.handleError()
Hmmm. I fear I do not get it. If I do so I run into the next problem. You should be able to easily reproduce it: Just copy and paste the code below into the 'JS' tab in the URL you mentioned above, press 'Try it' and then on the View tab press 'Save'. The code is the same as the original one just, the status is set to another number, the error message is saved in dsResponse.data instead of .errror and it is using a ListGrid instead of the DynamicForm.

Result here with Firefox 2.0.0.14 as well as with Internet Explorer 7.0.5730.11 under Windows XP SP2: 'Finding records that match your criteria...' pops up, vanishes, ppops up, ... endless until I close the browser or refresh, etc.

Trying the same with a DynamicForm works btw. Does that mean a ListGrid expects a different type of error message/handling?

Code:
isc.DataSource.create({
    ID: "users",
    dataFormat: "json",
    dataURL: "/isomorphic/system/reference/inlineExamples/dataIntegration/json/serverValidationErrors/serverResponse.js",

    fields: [
        {name: "userName", title: "Username", type: "text", required: true, length: 50},
        {name: "firstName", title: "First Name", type: "text", required: true, length: 50},
        {name: "lastName", title: "Last Name", type: "text", required: true, length: 50},
        {name: "email", title: "Email", type: "text", required: true, length: 100},
        {name: "password", title: "Password", type: "password", required: true, length: 20}
    ],
    transformResponse : function (dsResponse, dsRequest, jsonData) {
        var status = isc.XMLTools.selectObjects(jsonData, "/response/status");
        if (status != "success") {
            dsResponse.status = -1234;
            var errors = isc.XMLTools.selectObjects(jsonData, "/response/errors");
            dsResponse.data = errors;
        }
    }
});


isc.ListGrid.create({
    ID: "boundForm",
    dataSource: "users",
    fields:[{name: "userName"}]
});


isc.VLayout.create({
    members: [
        boundForm,
        isc.IButton.create({
            title: "Save",
            click : "boundForm.fetchData();"
        })
    ]
});
Isomorphic, thank you for your patience!

Regards,
Erich
Reply With Quote
  #4  
Old 26th May 2008, 15:52
Isomorphic Isomorphic is offline
Administrator
 
Join Date: May 2006
Posts: 38,393
Default

Error handling is uniform between the ListGrid and DynamicForm (both go through a DataSource).

The first problem in your code is that you set dsResponse.data to the errors, which is an Object, not a String. So this will not cause the warning dialog, as indicated in the docs for RPCManager.handleError()

The second is more subtle. You have changed from an "add" operation type to a "fetch" operationType and SmartClient's databinding system is doing it's best to recognize this response structure as a valid respond to "fetch", which expects an Array of matching records. So the entire response structure is interpreted as one record, placed in dsResponse.data, and dsResponse.endRow and dsResponse.totalRows are defaulted accordingly. But then the data is swapped for a single String without updating totalRows or endRow, with the result that you are advertising that one record is available but there was a transient error fetching it - so SmartClient tries again.

Enabling the ResultSet log category gives you something pretty close to a play-by-play account of the problem.

This would be a correct way of updating a dsResponse for a "fetch" that encountered an unrecoverable error:

Code:
            dsResponse.status = -1;
            dsResponse.data = "Data not available";
            dsResponse.totalRows = dsResponse.endRow = 0;
Reply With Quote
  #5  
Old 27th May 2008, 10:11
schlonz schlonz is offline
Registered Developer
 
Join Date: May 2008
Posts: 10
Default

Hi Isomorphic,

I think I am starting to understand the mechanism. Already changed my code and it works great.

Thank you for your quick and competent answers. I am impressed. :)

Regards,
Erich
Reply With Quote
  #6  
Old 27th May 2008, 12:36
ledifni ledifni is offline
Registered Developer
 
Join Date: Jan 2008
Posts: 112
Default

I like what you're doing here. Any chance Isomorphic could add a JsonRpcDataSource to SmartClient? I put a post in the Wishlist forum.
Reply With Quote
  #7  
Old 27th May 2008, 14:48
Isomorphic Isomorphic is offline
Administrator
 
Join Date: May 2006
Posts: 38,393
Default

Yeah we'd be happy to - Schlonz do you think you'll end up with something general-purpose enough to post here, that Isomorphic could use as a starting point?
Reply With Quote
  #8  
Old 28th May 2008, 08:58
schlonz schlonz is offline
Registered Developer
 
Join Date: May 2008
Posts: 10
Default

Sure! That was my plan from the beginning on. I would also love to find other people to have a look at the code and maybe improve it.

Currently the code consists just of a few lines supporting only the most basic use case. Give me 1-2 weeks to improve it and I will post it.

But beware: I am a lousy Javascript developer (simply not much experience with this language) and my experience with smartclient consists of 4 evenings.
So please keep your expectations low! Given my experience someone more experienced with both technologies will have to review the code in any case before it can be used in production!

Currently I am trying to make the data source JSON-RPC 2.0 compliant. Version 2.0 is only a draft so far. Would certainly be good if it supported 1.0 as well.


Regards,
Erich
Reply With Quote
  #9  
Old 29th May 2008, 07:31
ledifni ledifni is offline
Registered Developer
 
Join Date: Jan 2008
Posts: 112
Default

I hesitate to offer too much time for this right now, since my schedule tends to be a little hairy these days and I can't always keep up non-paying commitments -- but what time I have I'd be happy to offer. Can you think of a convenient way for us to exchange code so we can collaborate on this? If you like, you can email me at jodakim@hotmail.com (my spam-magnet email address) and I'll reply with my GMail address so we can send attachments back and forth.
Reply With Quote
  #10  
Old 29th May 2008, 14:30
schlonz schlonz is offline
Registered Developer
 
Join Date: May 2008
Posts: 10
Default

Hi Isomorphic, ledifni,

ledifni, you got mail.

Both of you: You know the community here better. What do you think would be the best approach to share the code? Posting it into the forum as non-attachment is probably not really what we want.

I could put it on google code or similar. Any better suggestions?

For me putting it on google code would mean it's there open for improvements until eventually isomorphic integrates it into SmartClient.


The code grows and is getting into a state where it can be used for at least some purposes. Currently it handles json-rpc 2.0 dataProtocol:"postMessage" for http POST (tested by me) and dataProtocol:"getParams" for a http GET (not tested and probably failing - my backend does not support GET so far).

Furthermore dataFormat "json" works if your backend returns errors as http status code = 200. I guess we better wait for the fix from isomorphic here.

As a work around dataFormat "custom" is also supported if your backend uses status codes != 200.

I have not looked into security so far.

The support for dataFormat "custom" is a dirty hack: it reuses (copy & pasted & modified) functions from DataSource.js.

It uses the http://www.JSON.org/json2.js library (public domain) and a base64 library (BSD License) from - ouch - Tibco GI. Sorry for that. ;)
I was not in the mood to rewrite these things. But I guess these licenses should not give us any troubles with the LGPL license of SmartClient.

Currently I see only one hurdle to fully support JSON-RPC: If the back end uses http status codes != 200 AFAIK smarclient does not forward the body of such messages to transformResponse. A pitty since there we could find more meaningful information.


Regards,
Erich
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search


© 2010,2011 Isomorphic Software. All Rights Reserved