problem with custom picker

[paulf]

I have a custom picker that does a name-based search on foreign key fields. The foreign key field has a displayField associated with it in the datasource definition. So for example the following Experiment datasource has foreign key to Reference. The Reference field has a custom picker called "objectFinder".

<DataSource
    ID="ExperimentDS"
    serverType="generic"
>
    <fields>
<field name="id" title="id" type="text" isIdentity="false" length="36"  hidden="true" primaryKey="true" />
        <field name="reference" title="reference" type="text" valueType="custom" editorType="objectFinder" requiresServer="true" displayField="referenceName" />
        <field name="referenceName" title="reference name" type="text" hidden="true"  />
        <field name="externalId" title="external id" type="text" isIdentity="false" length="64" />
        <field name="description" title="description" type="text" isIdentity="false" length="512" />
        </fields>
    </DataSource>

When I use the custom picker in a filterEditor, the item 'referenceName' is being handed to the fetchData() operation instead of reference. On the server, the log shows the filter criteria as {referenceName: "REF123"}.

The primary key value for the reference is correct but the field name is not.

When I execute getValues() on the ListGrid filterEditor in the console I get this:

Evaluator: result of 'isc_ListGrid_66.filterEditor.getValues()' (32ms):
{project: undef,
reference: "REF.33713",
externalId: undef,
referenceName: "REF.33713"}


Here is the code that sets the selection on the custom picker:

// set the specified value and dismiss the picker dialog
setSelection : function (record) {
    var valueMap = {};
    var value = record.value;
    valueMap[record.id] = value;
    this.setValueMap(valueMap);
    this.setValue(record.id);
    this.list.setData([]);
    this.dialog.hide();
}

When I Log.logInfo() the values for the valueMap and setValue(), all is correct.

Any thoughts as to why "referenceName" field is being handed to the fetch operation instead of "reference".

Thanks,
-pf

[Isomorphic]

The user is only seeing values from the displayField, so what the user is typing in is a value for the displayField, not the valueField. So the criteria are on the referenceName field in the case.

[paulf]

I understand that the user only sees displayField values. The question is why is referenceName being handed to the fetch operation? In other similar situations, such as when there is a optionDataSource defined for a field, the fetch operation gets handed the valueField and not the display field. It seems that this is the normal case.

Furthermore, in the custom picker setSelection() method, I'm explicitly setting a valueMap() before doing a setValue(). Isn't the point of a valueMapped field that the valueField is used for the fetch and not the displayField?

-pf

[Isomorphic]

It's because it's a ComboBox. Ask yourself the question: what field does the user's typed in value apply to? Clearly the displayField (referenceName).

Saving is always based on the valueField.

[paulf]

I'm having trouble parsing your last response.

Just to be clear, the field in question is not a ComboBox, it's a custom editor as I mentioned.

My question is not about the operation of the custom editor. It works fine. My question is about the operation of the ListGrid filterEditor that uses the cusom editor. The custom editor gets correct value and sets the valueMap and value accordingly.

I'm trying to figure out if there any correspondence between this:

Evaluator: result of 'isc_ListGrid_66.filterEditor.getValues()' (32ms):
{reference: "REF.33713",
externalId: undef,
referenceName: "REF.33713"}

(remember 'referenceName' is the displayField for 'reference'.)

and what I'm seeing in the server log:

-------------------------------------------------
43500 INFO com.daylight.port.shared.dmi.GenObjectDMI - processing GenObjectDMI.fetchRows() using metatype: Experiment
43500 INFO com.daylight.port.shared.dmi.GenObjectDMI - fetch criteria: reference: REF.8027
...
43625 INFO com.daylight.port.shared.dmi.GenObjectDMI - processing GenObjectDMI.fetchRows() using metatype: Experiment
43625 INFO com.daylight.port.shared.dmi.GenObjectDMI - fetch criteria: referenceName: REF.8027
----------------------------------------------

My basic question is why is the fetch being executed twice? Once for the value field (reference - which is correct) and one for the displayField (refeerenceName - which is not correct).


Do I need to manually override the filterData() method on the ListGrid to strip out the values for any displayFields?

-pf

[paulf]

I think the problem would be solved if I could find someway to prevent the filterEditor.getValues() from returning a value for the displayField (referenceName) and just return a value for the valueField (reference). That's kind of what I was getting at when I mentioned the typical behavior for valueMapped fields. (see previous thread)

Isn't this correct, and if so suggestions?

-pf

[paulf]

A related question is when using a displayField to provide values for another field in a ListGrid, should the displayField be a hidden field in the ListGrid, or does it just need to be a field on the underlying datasource?

-pf

[Isomorphic]

We really don't have enough information to comment.

You mention that you're using a custom editor but haven't explained anything about it (extends ComboBox? TextItem? something else?).

Nor do we understand why you consider criteria on the displayField to be "incorrect". Again there's the argument that the user is inputting a value that is clearly a search string on the displayField. By design we form criteria against the displayField, and this seems clearly correct, ComboBox or not. If, for example, the ComboBox didn't do this, the backend code would have to compensate criteria being submitted against the wrong field.

But if you need to change this for some reason, one place would be DataSource.tranformRequest().

[Isomorphic]

On your related question: yes, it's typical that in a ListGrid, the valueField is visible and has a displayField set, and the field used as a displayField is not shown.

[paulf]

The custom picker (ObjectFinder) is an extension of TextItem. It allows a user to find an object by typing in a name, getting a list of records back, then selecting a single record from the displayed values. The id of that record becomes the foreign key value for a criteria object. In other words, it's another way of doing exactly what optionDataSource does for smaller lists of foreign keys.

An example use would be as follows:

The user has opened an ListGrid for an Experiment table. The Experiment datasource has foreign key to Reference. The user wants to filter the experiment grid based on a single reference. There are 1000s of references so we can't use an optionDataSource to do the selection. Instead, the picker for the reference field uses a ObjectFinder.

The ObjectFinder is essentially a Dialog with an embedded ListGrid with its own filterEditor. The selectRecord() method on the ObjectFinder listGrid sets the value for the 'reference' field in the experiment grid.

The user types in a reference name (title) into the ObjectFinder's filterEditor, then selects a single reference from a list of displayed values in the ObjectFinder listGrid. The primary key of the reference becomes the foreign key for the experiment search. We just need to be able to filter the experiment table based on the reference field.

(select * from experiment where reference_id = 'xxx')

So this is why 'referenceName' just gets in the way from a filtering standpoint. It is necessary only to display the reference in the Experiment grid.

So you see, you already do something almost identical to this when filtering based on optionDataSources. We just need to be able to mimic the same behavoir with the caveat that the the custom picker allows the user to select a single foreign key value from tables that have lots of records.

Does that help?

-pf

[Isomorphic]

Before getting into any of these other details - you mention 1000s of references as a reason optionDataSource can't be used. It seems like you might be confused about ListGridField.optionDataSource, which causes advance loading of the entire related DataSource, vs FormItem/SelectItem.optionDataSource, which uses load on demand.

The typical way that you would handle this use case is explained in the docs for ListGridField.optionDataSource: use a valueField and displayField for the Records displayed in the ListGrid, then use an optionDataSource with a SelectItem or ComboBox for the filterEditor, so you can search against the related table and use load on demand to avoid loading the whole set of records.

[paulf]

We did try the loadOnDemand/optionDataSource approach and it broke down for the following reasons:

1) Some of the foreign key tables have millions of records. In these cases we found it is necessary to have the user manually invoke the filter. Even with delayed filtering on the first couple of keystrokes we still would get too many records back. It is necessary for us to have the user control when the filtering occurs.
2) The server will handle wildcard searching on both the front and back of the filter string (i.e. *phosphorylate*).

This is why we decided to write our own custom picker.

So to get back to the original question.
--------------------------------------------------------
I think the problem would be solved if I could find someway to prevent the custom picker filterEditor.getValues() from returning a value for the displayField (referenceName) and just return a value for the valueField (reference). That's kind of what I was getting at when I mentioned the typical behavior for valueMapped fields. (see previous thread)

Isn't this correct, and if so suggestions?
------------------------------------------

Just to clarify/reinforce the question.

There are two ListGrids we are talking about here. Let's call one the parent ListGrid and the other is the ListGrid associated with the custom picker.

The typical beavior of a valueMapped field (as I read it) is to create a correspondence between valueField and the displayField. The displayField is what gets shown while the valueField is what gets returned as the value of the field. Let's also say that this field is associated with the parent ListGrid.

For our custom picker we are trying to do nothing more than replicate the standard behavior of something like a comboBox. The picker itself is implemented as an embedded ListGrid with (showFiltereditor: true). The user types in a string, (possibly with widlcards), hits the filterEditor icon (of the picker, NOT the parent ListGrid) and gets back a set of records in the custom picker ListGrid. All we are trying do to is grab the id of the selected record in picker's valueMap and use it to set the value of the associated parent grid field.

Is this clear?

-pf

[Isomorphic]

On #2, have you seen textMatchStyle?

On #1, you could return stubbed results in this case, eg, a record that displays as "Please type a longer query string to retrieve search results". This request would not have to leave the client (DataSource.transformRequest() + clientCustom data protocol would be one way to return this result client-side).

Regardless, with your approach that involves a separate ListGrid, that explains the reference to the filterEditor (at last!) but there is no documented way to call filterEditor.getValues() directly, what exactly does your code look like here? And why do you care to get this value from the filterEditor when, once the filter has been applied, every record in the grid has it?

[paulf]

We will certianly look into trying a client custom data protocol. Not sure if this will work but we would always prefer to use supported functionality rather than write our own.

In any case, here is the code for the custom picker. I based the basic structure of it on the custom picker you provided in the examples. 'metatype' (the class name) and searchFieldName are typically passed in a variety of ways but most often as editorProperties: .

isc.ClassFactory.defineClass("ObjectFinderItem", isc.TextItem);

// instance properties and methods
isc.ObjectFinderItem.addProperties({
	icons:[{}], // could specify a different image here
	dialogHeight: 320,
	dialogWidth: 260,
	dialog: null,
	list: null,
	searchFieldName: null,
	metatype: null,
	
    // (this logic could alternatively go on the 'click' handler of the icon object)
	iconClick : function (form, item, icon) {
		// get global coordinates of the clicked picker icon
		var iconRect = this.getIconPageRect(icon);
		
		this.list = isc.ListGrid.create({dataSource: "genericObject", 
						autoFetchData: false,
						showHeader: false,
						recordClick: this.getID() + ".setSelection(record)",
						showFilterEditor: true,
						height: "100%",
						width: "100%",
						objectFinder: this,
						fields: [{name: "id"},
						         {name: "valueField"},
								{name: "value", width: 240}
						],
						filterData: function(args, callback, requestProperties) {
							// args will just have value; add in metatype and valueField
							var newargs = {value: args.value, 
											valueField: this.objectFinder.searchFieldName,
											metatype: this.objectFinder.metatype};
							
							this.Super("filterData", [newargs, callback, requestProperties]);
						}
		});

		this.dialog = isc.Dialog.create({
						autoCenter: false,
						isModal: true,
						dismissOnOutsideClick: true,
						// These suppress the Dialog header
						showHeader: false,
						edgeImage:"[SKIN]/Menu/m.png",
						edgeSize:10, edgeTop:17, edgeBottom:17,
						edgeCenterBackgroundColor:"#F7F7F7",
						
						showToolbar: false,
						width: this.dialogWidth,
						height: this.dialogHeight,
						items:[this.list]
		});
		this.list.hideField("id");
		this.list.hideField("valueField");
		
		if (this.metatype != null) {
			Log.logInfo("metatype from filterEditorProperties: " + this.metatype);
		}
		if (this.metatype == null) {
			// First try picking it up as a form field attribute
			this.metatype = item.metatype;
			Log.logInfo("metatype from item: " + this.metatype);
		} 
		// Next see if form.metatype is non null		if (this.metatype == null) {
			this.metatype = item.form.metatype;
			Log.logInfo("metatype from item.form: " + this.metatype);
		}
		// Finally try getting it as an operator (SearchBuilder)
		if (this.metatype == null) {
			this.metatype = item.form.getValue("operator");
			Log.logInfo("metatype from item.form.getValue('operator'): " + this.metatype);
		}
		// searchFieldName will be name or externalId or title or, ...
		if (this.searchFieldName == null) {
			this.searchFieldName = MYAPP.objectFinderFields[this.metatype];
		}
		if (this.searchFieldName == null) {
			this.searchFieldName = "name";
		}
		// show the picker dialog
		var pageBottom = MYAPP.getPageBottom();
		// parents.length == 0 will be true if 'form' is a ListGrid filterEditor
//		if (parents.length != 0) {
//			var topElement = parents[parents.length - 1];
//			pageBottom = topElement.getPageBottom();
//		}
		var finderBottom = item.getPageTop() + this.dialogHeight;
		// -10 maybe corresponds to dialog.edgeSize above?
		var realLeft = item.getPageLeft() - 10;
		if (pageBottom > finderBottom) {
			this.showDialog(realLeft, item.getPageTop() - 2);
		} else {this.showDialog(realLeft, item.getPageTop() - this.dialogHeight);
		}
	},
	
	// show the picker dialog at the specified position
	showDialog : function (left, top) {
		this.dialog.moveTo(left, top);
		this.dialog.show();
	},
	
	// set the specified value and dismiss the picker dialog
	setSelection : function (record) {
	    var valueMap = {};
		var value = record.value;
		valueMap[record.id] = value;
		this.setValueMap(valueMap);
    	this.setValue(record.id);
    	this.list.setData([]);
		this.dialog.hide();
	}
});

[Isomorphic]

The use of the clientCustom dataProtocol is what would allow you to re-use the existing functionality instead of launching a separate picker. Bear in mind, you can switch the dataProtocol to clientCustom for an individual request, meaning that you can intercept requests where the search string is too short and provide a special response, while allowing other requests through.

As far as your picker code - do you still have something you need help with here? If so, did you notice this?

And why do you care to get this value from the filterEditor when, once the filter has been applied, every record in the grid has it?

Beyond this, you also already have an override in place (filterData) where you could modify the criteria if you wanted to? Not to mention transformRequest(), or a DMI. So you seem to have multiple ways of solving your issue - let us know if you still need help.