How to refresh ListGrids after DataSource.updateCaches() with no server roundtrip

[d.cavestro]

Hi,
I have a problem propagating to my ListGrids some changes happened at the server side.

Before showing the edit form for a record in my DataSource I trigger a fetch - by id - and subsequently an updateCaches() on my datasource, BUT my listgrids still continue showing the old record data.

Provided that:
* The fetch by id is done in order to ensure the editing of the most current data.
* The above mentioned listgrids are filtered or full views on the same datasource instance.
* I'm not talking about an UPDATE operation (this works fine), but only about the integration of my fetch by id with the listgrids local cache.

I found that only calling invalidateCache() on listgrids seems to actually refresh their data, but this always lead to another roundtrip with the server (in other words another fetch).
I am using my datasource - actually a RestDataSource - as a virtual table, and the server always returns ALL the rows for all data fetches, cause I want to use only client-side filtering. The only exception to this logic is the fetch by id, served with only the record whose pk matches the passed id param.

The question is:
how can I notify the listgrids of underlying data changes without triggering a fetch to the server?

I'd expect that updateCaches() triggers a "refresh" of each listgrid resultset and consequently a "refresh" of the widget. What did I miss?


At client side I'm using
SmartClient Version: 8.0/LGPL Development Only (built 2010-05-18)
GWT 2.0.4
Firefox 3.6.9

at server side Grails 1.3.4


Kind regards
Davide

[Isomorphic]

If you're fetching a single record that needs updating, this should be passed to updateCaches() as an "update" dsResponse.

[d.cavestro]

Thank you Isomorphic,
as you suggested - since I'm not actually sending an UPDATE operation - I've used the specific signature updateCaches (DSRequest, DSResponse), hence "forging" the DSRequest operationType to seem like an update operation.
IT WORKS!

One last thing: maybe it's my fault, but if it is not written somewhere don't you think it should be documented in a more accessible way? (I know you are working on the SmartGWT complete referente, but in the interim the FAQ or a best-practice could be more appropriate than this thread alone...)

On the other hand I have to say that you are always extremely fast to give give GOOD and FREE support, which is not a little thing :-)

Thank you
Davide

[d.cavestro]

Hi Isomorphic,
please let me ask you another question:
what about calling updateCaches() for a fetch operation that returns more than a record? (but at limit the entire record set)

This could be the logical evolution for integrating old local data with the fresh one coming from the server...
Is it feasible with the "trick" exposed above?
Moreover: in this scenario the difference between an updateCaches() with the entire data set (all rows still present at the server side) and an invalidateCache() would ONLY be that the first retain even "removed" records while the latter not?

If you're fetching a single record that needs updating, this should be passed to updateCaches() as an "update" dsResponse.

[Isomorphic]

Don't really follow, but consider that you can pass a dsResponse with invalidateCache set to updateCaches(), and see also DataSource.setCacheAllData() for write-through caching.

[d.cavestro]

Sorry Isomorphic,
it's my fault, I've written gibberish, like a crashed server:-)

Don't really follow, but consider that you can pass a dsResponse with invalidateCache set to updateCaches(), and see also DataSource.setCacheAllData() for write-through caching.

I will try to be more precise:
as you suggested I've used the method updateCaches() after a fetch by id (getting a single row).

I guess updateCaches () has 2 main effects:
* integrates fresh copy of existing data from server with the local - existing - copy (similar to an update operation... insomuch as we have to simulate an UPDATE operation setting the operationType on the request)
* forces every visual components bound to the target datasource to "refresh" their appearance, hence showing the fresh copy.

This works perfectly as long as I am fetching a single record that needs updating (as you previously said), but doesn't seem to be the right strategy when I need to re-fetch the entire DataSource (in a high level of concurrency the chance that some records have been changed, added or removed is too high, so I want that changes reflected to the visual components as soon as possible).

In that case, I could use invalidateCache() on the DataSource and then fetch all data, but I never obtain the second - and only appliable for this scenario - effect that an updateCaches() call instead would give: triggering the refresh/redesign of every visual components bound to that datasource.

So the question is: how can I force every visual component bound to a certain datasource to resynch with its model when I cannot use using updateCaches()?

I don't want to operate directly on the visual components (mainly listgrids): they are sharing the same DataSource also to centralize that kind of management.

Directly setting the data into the visual components (or their resultsets) isn't so attractive, cause I guess it leads to disconnected data, hence to the need to directly manage every call to the server for data changes...

I've tried setting cacheAllData to true: that way the datasource seems to automatically switch to local-only (from Developer Console I see my fetches with URL "[local only]", so frustrating any subsequent automatic refresh.
A simple call to invalidateCache() on the datasource clearly is not enough.

This is a snippet from my attempts to assemble an automatically refreshed DataSource

final int cacheTimeout = 30;//high frequency for test purposes
//setCacheAllData (true);
//setAutoCacheAllData (true);
//setCacheMaxAge (cacheTimeout);

/*
 * local data automatic refresh
 */
final Timer t = new Timer () {
	private int errorCount = 0;
	public void run () {
		try {
			SC.logWarn ("Automatically fetching data for cache timeout");
			invalidateCache();
			fetchData (null, new DSCallback() {
				
				@Override
				public void execute (DSResponse response, Object rawData, DSRequest request) {
					SC.logWarn ("Data automatically fetched for cache timeout");
					/* 
					 * HERE I would need to call updateCaches or something else to pass
					 * the fresh data to visual components resultsets
					 */
				}
			});
		} catch (final Exception e) {
			errorCount++;
			if (errorCount>3) {
				SC.logWarn ("Disabling automatic refresh for datasource "+getID ()+", since it just failed for 3 times");
				cancel ();
			}
		}
	}
};

t.scheduleRepeating (cacheTimeout*1000);

Hope I made myself somewhat clear this time...

Kind regards
Davide

[d.cavestro]

Am I still gibberishing?
Isomorphic?

[davidj6]

What Isomorphic was saying is that when you get all your new records back, you create a DSResponse for the call to updateCaches() with your new records as the data AND set the invalidateCache property to true in the DSResponse. This lets the dataSource know the new "response" replaces any current data.

[d.cavestro]

What Isomorphic was saying is that when you get all your new records back, you create a DSResponse for the call to updateCaches() with your new records as the data AND set the invalidateCache property to true in the DSResponse. This lets the dataSource know the new "response" replaces any current data.

Ok, now I've got it.

So I leave the following snippet to other users trying to do automatic cache refresh

//refresh every 5 minutes
final int cacheTimeout = 300;
setCacheMaxAge (cacheTimeout);

final Timer t = new Timer () {
    private int errorCount = 0;

    @Override
    public void run () {
        try {
            GWT.log ("Automatically fetching data for cache timeout");
            refreshData ();
        } catch (final Exception e) {
            errorCount++;
            if (errorCount > 3) {
                SC.logWarn ("Disabling automatic refresh for datasource " + getID ()
                    + ", since it just failed for 3 times");
                    cancel ();
            }
        }
    }

    /**
     * Fetches fresh data for this DataSource AND notify related databound components to refresh
     */
    private void refreshData () {
        fetchData (null, new DSCallback () {
            @Override
            public void execute (final DSResponse response, final Object rawData, final DSRequest request) {
                refreshCache (response, request);
            }
        });
    }

    /**
     * Integrates the local cache with fresh data for this 
     * DataSource coming form the specified response
     * 
     * @param response the response carrying fresh data
     * @param request the original request
     */
    private void refreshCache (final DSResponse response, final DSRequest request) {
        GWT.log ("Data automatically fetched for cache timeout");

        //a fake response to carry "commands"
        final DSResponse carrier = new DSResponse ();
        //request invalidation for all previously cached data
        carrier.setInvalidateCache (true);
        //copy the fresh data to the "fake" response
        carrier.setData (response.getData ());

        request.setOperationType (DSOperationType.UPDATE); // API not very clear
        // integrates local cache with data passed with carrier AND notify related databound components to refresh
        updateCaches (carrier, request);
    }
};

t.scheduleRepeating (cacheTimeout * 1000);

where obviously the refreshData() method can be exposed and called everywhere on appropriate conditions (even in response to server "pushes").
Hope the snippet could be useful to someone else.


Many thanks to Isomorphic and davidj6.
Cheers
Davide

PS: I suggest that you integrate the updateCaches() JavaDoc and possibily the FAQ/best practices to inform users about the availability of these tricks (I mean - for instance - the combination of seInvalidateCache and UPDATE operation with fresh data)