Large valueMaps and performance issues

[senordhuff]

Hello,

We have a need to create a very large valueMap that could contain thousands of records. The key here is obviously performance, and the contents of the valueMap are dynamic and could change frequently as the user add,edits, and deletes records in the applicaiton. So, I was reading your documentation about optionDataSource and saw the following text:

Unlike the similar use of +{pickList.optionDataSource,optionDataSource} for pickLists, the entire set of related records is fetched, without paging, so this is appropriate only for smaller valueMaps. For very large valueMap situations, such as an accountId field that should show an accountName where there are thousands of accounts, the recommended approach is: 

declare two fields in the DataSource, eg "accountId" and "accountName". When fetching records for display in a grid, have your server send back values for both fields. 
in a grid, show only the "accountId" field, and use formatCellValue on the accountId field to display the accountName, eg: 
     formatCellValue:"record.accountName"
 
This approach is superior to, say, having the server return only the display value, because inline editing, the filterEditor, and related forms will be using the appropriate fieldName when saving and filtering. The inline editors and filter fields can be configured to use databound picklists via field.editorProperties and field.filterEditorProperties respectively.

I'm not clear on why this offers superior performance for large valueMaps? And, will this method make it possible for me to edit the valueMap without significant performance impact after the user has already loaded the valueMap?

I am currently calling fetchData on my datasource and then, in the callback, creating an client-side valueMap from the fetch. However, with thousands of records, I am getting "script running slowly" messages because of the callback that populates the valueMap and whenever I try to edit the valueMap. So, I'm trying to figure out the best way to manage this valueMap.

Thanks for the help.

[Isomorphic]

Hi senordhuff,

The approach outlined means that the valueMap is never loaded in its entirety into the browser. Instead, each loaded record contains the valueMapping for that one record, as a pair of fields within the record. These fields would typically be populated by a SQL join or similar backend operation. Make sense?

[senordhuff]

Sorry, I don't understand completely. If each record contains the valueMapping just for that one record, how does the user choose from the other thousands of options available in the valueMap when editing the record?

[Isomorphic]

You can still use pickList.optionDataSource via listGridField.editorProperties to use a large valueMap with the pickList during inline editing (with data paging). listGridField.optionDataSource is an independent property.

[senordhuff]

So, how do you specify with pickList.optionDataSource whether you want the valueMap to use paging or not?

[Isomorphic]

Paging is on by default for pickList.optionDataSource. If you don't want paging, you can always ignore the startRow and endRow fields of the DSRequest and simply return all rows.