Cubegrid performance issue.

[Emilien]

1. SmartClient Version: v9.1p_2014-12-05/PowerEdition Deployment (built 2014-12-05)
2. Google chrome : Version 39.0.2171.95 m

Hi !

I have a performance problem with sparse matrix cubegrid (no record behind every cells).
Performances are really really bad when their is incomplete data, in pure client (edit value) and server (saveEdits), it can go from 1 to 100.

Maybe you know some tricks or bug fix to improve thoses performances or must I fullfil the matrix data to avoid thoses performances issues ? (that will consume more network resources).

I can't easily provide you a sample but it use a very simple cube grid without any kind of options (hilites, formatter etc) and only numbers on 3 dimensions (2 cols, 1 row).

Thx for your help !

[Isomorphic]

Yes, you should provide a runnable sample. We don't have other reports of performance issues in this mode, which suggests you are doing something slightly different. Sample code will allow us to see what it is.

[Emilien]

I have a sample with local data (cubeGrid.setData(...)) it seems their is no performances problems, no differences whatever the filling rate is.

I just actually cannot clean the edited values without rebuilding the entire cube.

The sample is in attachment, I'm trying to make it working with a server.
(I was unable to work with a local datasource with set clientonly, it seems to try to fetch data from the server)

[Isomorphic]

What's the purpose of this sample? You said it doesn't have a performance problem, but you've also moved away from the mode that would actually save to the server, so it seems to be a pointless sample to look at.

As far as getting rid of editValues, you haven't really explained what the problem is, but we see no calls to clearEditValue(s) and the only call to discardAllEdits() is commented out, so this code definitely would not be expected to successfully get rid of editValues.

[Emilien]

I succeed to make it working with a server side as the one we have in our environment, but it seems to have no performance problem, I'm looking for the difference to reproduce the performance problem. I will keep you in touch for this.

I tried to use the discardAllEdits method but it doesn't clear the edit value of the cube (in the example) and their is no clearEditValues method (their is only clearEditValue(int row, int col)

[Emilien]

I didn't succeed to reproduce the problem yet, but I took snapshot of the cpu profile of the operation:

The 900 cells custom screenshot is the profile of my initial cubegrid. The setEditValue method spend a long time on refresh method, I don't know why (150s).

The 10kcells is the one in the example above (I succeed to make it work with the same server side as my custom cube) here the time seems to be correct as the refresh method take a very small part of the edit value process.

Can you give me some help about possible differences ?
Maybe the original source of the cube could help me to understand the root cause of the problem.

Thx !

[Isomorphic]

Sorry, we can't do much with this. We can see that you're calling setEditValue() a lot and causing a lot of refreshing. But we can't see anything about why you would call it so many times (that's not normal) nor do we have any way of figuring out whether it's unusually slow.

To analyze whether there is a framework issue here, again we need code that we can run.

[Emilien]

I call it because I try to update all the value with the setEditValue method in a loop on selected cells.

for (int[] cellCoord : cubeGrid.getCellSelection().getSelectedCells()) {
        final int row = cellCoord[0];
        final int col = cellCoord[1];

        cubeGrid.setEditValue(row, col, random.nextInt(1000));
}

The problem is the inner call of the "refresh" method that appear in my custom case and NOT in the example where I tried to reproduce it.

It's a standard usecase that we need in our business. I can't yet provide you a running example as the one I tried to made for you doesn't seems to have the problem. I'm not able to find why for know, that why I need some info from you if you have some ideas. I will keep looking at finding the difference but I can't spent too much time actually on this.

Have you an idea for why the "refresh" method is internally called by the setEditValue in one case and not in the other one ?

[Isomorphic]

It's fine to call setEditValue(). We don't see a reason why it would be slow, as long as you aren't calling it 1000s of times in a single loop, which doesn't make sense with your described use case anyway.

As far as why it would immediately refresh, perhaps in one case, you have made some other changes to the grid which marked it as pending a redraw (eg, a direct call to markForRedraw(), or some action that led to one). In that case, a call to setEditValue() would not attempt an incremental refresh since a full refresh is already scheduled to happen.

[Emilien]

I tried a manual refresh of the grid before the loop but it change nothing.

The loop with the setEditValue do nothing more than I shown to you.

I tried to remove all the custom settings in my initial case but the refresh is still called. It could be possible that the container of the cube (or one of them) force the cubegrid to be refreshed after those editValues ?

I found a debug version of the javascript code of the CubeGrid (not obfuscated) but I was unable to find same version for the list grid (and above the Layout code). Do you know where I can find them ?

Thx in advance, and thx for the previous response too :)

Edit : Ok I found on google code the unobfucated code of the listgrid :

This is the important part for the redraw. As you said a redraw is asked in some way, I just need to understand in which way :)

    if (suppressDisplay || !this.isDrawn() || !this.body) return;
    
	// values shown in the cells for this row are now stale (whether each cell is being edited
	// or not), so refresh.
    
	// if totalRows changed, it indicates that that a new edit row is being created at the end
	// of the body, hence we need to redraw to add the row to the DOM
    var shouldRedrawBody =  (addedRow || this.body.isDirty());
        
    if (shouldRedrawBody) {
        var editorShowing = this.isEditingRecord(rowNum, colNum) && this._editRowForm != null
    
    	
        if (editorShowing) this._updateEditItemValues();

        this.body.markForRedraw(
        	
        );
        
    } else {        
        this._displayNewEditValues(rowNum, colNum, changedFields);
    }

[Isomorphic]

See the Debugging overview for how to use debug modules. However, we can't really recommend taking this approach - you can't have very much code in your app, it would seem a lot of easier to just figure out the difference between your two versions than to dive into a very complex codebase that supports a lot of modes and features you aren't using.

We're not sure what you mean about the "container of the cube" forcing a refresh, but note that you can turn on diagnostic categories like "redraws" and "redrawTrace" in the Developer Console to see some information about where redraws are coming from.

Also you mentioned a "manual refresh of the grid" - we recommended trying a call to markForRedraw() and we don't know whether you did that or something else - if you did something else, try markForRedraw().

[Emilien]

I found a workaround :

    public native void setEditValueNoRedraw(int row, int col, String value) /*-{
        var self = this.@com.smartgwt.client.widgets.BaseWidget::getOrCreateJsObj()();
        self.setEditValue(row, col, value, true);
     }-*/;

And I call markForRedraw after that. No more call to the refresh is done.
Note the previous quotted code was an old version from 2009 and not the current one we are using (4.1)

Edit : I tryed the markForRedraw before the loop without success, the refresh method is still called in the setEditValueMethod.

[Isomorphic]

Great that you found an approach that works in your app.

Just for clarity's sake, we need to remind you that you are calling an undocumented unsupported API, which could have unintended side effects, and your code may break in the future and it would not be considered a framework issue.

To get a supportable solution, we still need a way to reproduce the problem, so we can understand whether it's a usage issue or framework issue and figure out the right code to change.

[pablo99]

I am also experiencing performance issues when calling setEditValue(). Yes I am calling it in a loop, which could potentially be 1000 iterations. Is there anyway to disable the redraw after every call?