[Isomorphic]

Again, progressiveLoading can be set on a per-operationBinding basis.

So you can have it on for the first request, then set operationId to use a different operationBinding that does not have progressiveLoading enabled.

[aluainsha]

Again thanks for your patience on this.

We've tried turning on progressiveLoading (assuming that somehow we already know the total number of rows in the resultSet) so that a "select count(*)" would NOT be issued automatically.

Unfortunately, with this method we also got some interference from the heuristic. Our settings of startRow and endRow are often not honored -- SmartGWT always seems to add 1 or 20 rows to the number of rows that we instruct it to fetch.

Is there a way with progressiveLoading==true that we can control the exact startRow and endRow?

Not sure if this is DB related ... anyway, we are using MySQL.

Thanks a lot!

[Isomorphic]

operationBinding.skipRowCount could be used as an alternative to progressiveLoading.

Use the same pattern of issuing a normal request first (to have the row count executed and have totalRows be correct), then switching to an operationBinding where skipRowCount has been set.

[aluainsha]

You can set startRow and endRow to fetch ranges of rows at a time. You can see in the generated SQL that these values are being passed to the database as part of the SQL (the SQL to do this differs by DB).

This approach means the query is being repeated of course, as opposed to streaming results from the server, which is what we use to do very large CSV exports. If you're interested in having a supported & documented API to do streaming in the same way we do CSV exports, consider Feature Sponsorship.

Hi, you mentioned that you do streaming with very large CSV. Is that automatically used when a SmartGWT do an exportData() on the client side? (Or this is a separately API?)

The reason I am asking this is that we found that we would get Java's OutOfMemoryError if we tried do a client-side exportData() and when the result sets were large (more than 150 MB or so).

We are thinking of disallowing our SmartGWT clients to do exportData() directly on the client side and force them to use our fetch-N-row-at-a-time server-side implementation (which runs slower, but can cap the use of memory).

com.alcatel_lucent.aware.nara.server.DynDsIDACall top-level exception
java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2760)
	at java.util.Arrays.copyOf(Arrays.java:2734)
	at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
	at java.util.ArrayList.add(ArrayList.java:351)
	at com.isomorphic.sql.SQLTransform.toListOfMapsOrBeans(SQLTransform.java:710)
	at com.isomorphic.sql.SQLTransform.toListOfMapsOrBeans(SQLTransform.java:517)
	at com.isomorphic.sql.SQLTransform.toListOfMapsOrBeans(SQLTransform.java:511)
	at com.isomorphic.sql.SQLTransform.toListOfMapsOrBeans(SQLTransform.java:486)
	at com.isomorphic.sql.SQLDriver.getTransformedResults(SQLDriver.java:601)
	at com.isomorphic.sql.SQLDriver.executeQuery(SQLDriver.java:872)
	at com.isomorphic.sql.SQLDataSource.executeNativeQuery(SQLDataSource.java:415)
	at com.isomorphic.sql.SQLDataSource.executeNativeQuery(SQLDataSource.java:409)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1402)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:314)
	at com.isomorphic.sql.SQLDataSource.executeFetch(SQLDataSource.java:258)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1367)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:726)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2033)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:216)
	at com.alcatel_lucent.aware.nara.server.DynDsIDACall.handleDSRequest(DynDsIDACall.java:39)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:173)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:138)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:74)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:259)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

Our environment:

1. SmartClient Version: v8.3p_2013-01-15/PowerEdition Deployment (built 2013-01-15)

2. Browser: Firefox

3. Server log: (see above)

[Isomorphic]

We're not sure what you mean about a "client-side exportData()" - exportData() sends a request to the server, and that's where the export is performed.

Set dsRequest.streamResults to enable streaming of results.

Note that this helps for CSV exports, but is ineffective for Excel exports, because the version of Apache POI currently in use does not support streaming output. We plan to address this in the future, but if it's critical for you, use the Feature Sponsorship program to get a time commitment.

[Isomorphic]

Note for anybody who ends up on this thread while searching for SmartClient/SmartGWT streaming features. As of SmartClient version 10.0 / SmartGWT version 5.0, we support fully streamed exports to Excel when using the XLSX format (also known as OOXML). We stream directly from the database to the Excel document, which allows you to export huge datasets without using up large amounts of server memory.