version 3.0: includeFrom() : what's the status?

[damend]

Hi,

using 3.0, 6. Okt 2011
I am looking forward to use the includeFrom() in my SQLDatasource,
but seems not to be working in showcase.

This would influence our decision to switch from 2.5 to 3.0 very soon.

Can you tell us, when it will be working?
If it's expected to be longer, we'll consider to solve it by using existing functionalities.

Will includeFrom work like it's only reusing the datasource field definitions,
or will it be a operation which is executed on datasource?

Thanks,

[Isomorphic]

Can you elaborate on "seems not to be working"?

[damend]

An exception is thrown in the showcase indicating that the field is not taken from the includeFrom datasource, but from the same datasource in the generated SQL query.

[damend]

In my case I need to decide dynamically if includeFrom has to be used or not(go to same datasource or to another one).

Can this be done, because these informations are for DataSourceField and not criteria, correct?

[Isomorphic]

Well, don't keep us in suspense, what's the exception?

If it's not clear, claimed bugs against 3.0 should be reported with the same information we always ask for.

Yes, includeFrom can be specified dynamically, vs dsRequest.outputs and listGrid.includeFrom - this is shown in a sample as well.

[damend]

Here is the error-message when running the showcase.war 3.0 in tomcat:

Column not found: CITY.CONTINENT in statement [SELECT LIMIT 0 54 city.cityId, city.cityName, city.continent, city.countryId FROM city WHERE ('1'='1')]


You'll get the same !

[Isomorphic]

It looks like this is an issue that has just been resolved in the 3.0 codebase.
Please try with the next nightly build (Oct 14 or greater) and let us know if you continue to see the problem

Thanks
Isomorphic Software

[Segmeno]

Hi,

we are currently facing the same issue. I just downloaded the nightly build 'SmartGWT/3.x/EnterpriseEval/2011-12-15' and tested the includeFrom functionality.

Datasource definition:

<DataSource ID="tbSecRole" tableName="tbSecRole">
    <fields>
        <field name="RoleID" type="integer" hidden="true" primaryKey="true"/>
        <field name="Role" type="text" required="true" length="80"/>
        <field name="CreatedBy" type="integer" foreignKey="tbSecUser.UserID" />
        <field includeFrom="tbSecUser.UserName"/>
    </fields>
</DataSource>

statement (from log):

SELECT tbSecRole.CreatedBy, tbSecRole.Role, tbSecRole.RoleID, tbSecRole.UserName FROM dbo.tbSecRole WHERE ('1'='1')

The user name is still picked from the wrong table...

Here the stack trace:

=== 2011-12-15 14:25:23,962 [ec-6] WARN  RequestContext - dsRequest.execute() failed: 
java.sql.SQLException: Invalid column name 'UserName'.
	at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)
	at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2820)
	at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2258)
	at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:632)
	at net.sourceforge.jtds.jdbc.MSCursorResultSet.processOutput(MSCursorResultSet.java:943)
	at net.sourceforge.jtds.jdbc.MSCursorResultSet.cursorCreate(MSCursorResultSet.java:541)
	at net.sourceforge.jtds.jdbc.MSCursorResultSet.<init>(MSCursorResultSet.java:154)
	at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQLQuery(JtdsStatement.java:424)
	at net.sourceforge.jtds.jdbc.JtdsStatement.executeQuery(JtdsStatement.java:1304)
	at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
	at com.isomorphic.sql.SQLDataSource.executeWindowedSelect(SQLDataSource.java:1965)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1305)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:293)
	at com.isomorphic.sql.SQLDataSource.executeFetch(SQLDataSource.java:237)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1291)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:721)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:1948)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:199)
	at com.ascom.camtool.gwt.server.security.SecureIDACall.handleDSRequest(SecureIDACall.java:119)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:156)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:121)
	at com.ascom.camtool.gwt.server.security.SecureIDACall.processRequest(SecureIDACall.java:99)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:73)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:259)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:964)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:304)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

[Isomorphic]

Do you have a DataSource tbSecUser, with a field UserName? includeFrom requires a related DataSource and field, it is not just a table and column name.

If you think you've set things up correctly, provide the rest of the required information: the other DataSource, and the complete server log for the request (never omit this info).

[Segmeno]

Yes, I know that I need a datasource + field to make it work.
I just realized that this behaviour is caused by the way we load the datasources.
We wanted to use user specific datasources to implement our own security. So user A has different fields and permissions than user B for the same datasource. In doing so the DataSource-xml can be modified on-the-fly
and then created one time when the user logs in.
Therefore we use our own DataSourceLoader which handles the xml user specific and stores the created datasources in our own Container:

ds = DataSource.fromXML(modifiedXmlData);
SecureDataSourceContainer.register(ds);

Later in the IDACall we wanted to use our created datasource object for the DSRequest-handling:

ds = SecureDataSourceContainer.get(dsRequest.getDataSourceName())
dsRequest.setDataSource(ds);
dsRequest.execute();

I`m aware that this way we do not use your DataSourceManager or PoolManager. And that will probably the reason why the includeFrom functionality is not working for us.

So the question is: Is there a way to add DataSource-objects or *.ds.xml Strings to the PoolManager instead of having it handle the DataSource-creation?
Or can we enrich the dsRequest with more information to make this work? Seems as if the missing 'uploadedFiles' within the DSRequest are causing trouble...

[Isomorphic]

Overall, you probably want to throw away all this custom security stuff and use Declarative Security, which handles the same things and would be much much simpler.

However as far as the general problem: clearly when you use "includeFrom" we need to be able to find the related DataSource by the ID you specified. The right way to make this ID dynamic/user-specific is to use DataSource.addDynamicDSGenerator(), embed a user-specific DataSource ID into the includeFrom attribute, and make sure your DSGenerator can provide the right DataSource when passed this user-specific DataSource ID.

[ashkan]

you have mentioned that we can use declarative security however the problem with declarative security is that we have to hard code the role names in the xml file and that's what we want to avoid .
basically what we had with last version was loading the original ds file and then getting roles and permissions for the logged in user and modify the loaded data source based on that information .
how is it possible to get this level of dynamicity (not hard coding the roles) with your declarative security ?
thx

[Isomorphic]

We've just explained how the whole .DS.XML can be dynamic. Beyond that, read the QuickStart overview on Declarative Security, focusing on the idea that roles can be used as just names of capabilities.

[Segmeno]

Thanks for the hint about the DynamicDSGenerator.
The includeFrom is working perfectly for the fetch operation but for the 'add' I am still having that 'Invalid column name' problem.

I`m pretty sure that I did everything correct. The addOperation is working if I omit the 'includeFrom'. The serverlog and both ds.xml are attached. If you need more input please let me know.

I appreciate your help.

[Isomorphic]

You seem to be passing UserName on the "add", which doesn't make any sense. It should probably be ignored automatically and we'll look into doing this, but just removing it from the request will fix the issue.