Validator called on the wrong field

[agalataud]

Hi,

We're currently running SmartGWT Power edition 4.1 (v9.1p_2014-03-16/PowerEdition Deployment 2014-03-16).
One of our test site is suffering a problem that we're not able to reproduce on all other sites, neither in gwt dev mode.

The problem is with a form validation on a datasource that contains 2 fields with 2 validators each. When validating the form, regexp validator of the 2nd field is applied to the 1st.

This is clear when we test 2 invalid values handled by each regexp validator: error messages from field 1 validator and from field 2 validator are shown. See screenshot in attachment.

Definition of the validators in ds.xml:

<field name="CODE" type="text" title="$code" escapeHTML="true" length="16" required="true">
  <validators>
	<validator type="isUnique" criteriaFields="DELETED" requiresServer="true" errorMessage="This code is already used."/>
	<validator type="regexp" expression="^[A-Za-z0-9-_//,\.]*$" errorMessage="Invalid characters : space, (&gt;&lt;\ and accented characters"/>
  </validators>
</field>
<field name="NAME" type="text" title="$name" escapeHTML="true" required="true" length="32">
  <validators>
	<validator type="isUnique" criteriaFields="DELETED" requiresServer="true" errorMessage="This name is already used."/>
	<validator type="regexp" expression="^[^&gt;&lt;/\\]*$" errorMessage="Invalid characters (&gt;&lt;/\"/>
  </validators>
</field>

Is there something incorrect in the validators declaration?

Thanks for your help in advance
Regards
Antoine

[Isomorphic]

No, there's nothing obviously wrong with the validator definitions.

Sorry, we've never seen an issue like this and we have no theory as to how a framework issue could cause this - there's nothing more we can do unless you can provide a way to reproduce it.

[Blama]

Hi agalataud,

the error is shown for field1 ("CODE") together with the error message from field1 ("CODE"). So no mismatch there IMO.

Please note that your regex has no valid syntax (at least in Notepad++).

Most likely it should be ^[A-Za-z0-9\-_//,\.]*$ or ^[A-Za-z0-9\-_//,\\\.]*$ in order to match your test-string "TEST/\".

Best regards,
Blama

[agalataud]

Hi,

We still have the problem from time to time. I found the issue disappears when application is restarted, making me think validators retrieved from ds.xml on server side get messed up at some point. When problem occurs, all clients are impacted, no matter if we clean client cache.

So far, I haven't been able to reproduce it in development mode, only on a compiled and deployed application. There is nothing in logs like an exception or something related to an error when DS are loading.

Following is the DS declaration for which the problem has appeared today:

<DataSource
    ID="cif_document"
	useAnsiJoins="true"
	serverType="sql"
	tableName="CIF_DOCUMENT"
	serverConstructor="com.fircosoft.cdb.fatca.server.DocumentDataSource"
	audited="true"
	titleField="REFERENCE"
>
    <fields>
    	<field name="ID" type="sequence" sequenceName="SQ_CIF_DOCUMENT" hidden="true" primaryKey="true" />
        <field name="CUSTOMER_ID" type="integer" hidden="true" />
        <field name="TYPE_ID" type="integer" foreignKey="cif_document_type.ID" hidden="true" required="true" />
        <field name="TYPE" type="text" includeFrom="cif_document_type.NAME" />
        <field name="REFERENCE" type="text" required="true" length="64"/>
        <field name="COUNTRY_ID" type="integer" foreignKey="geo_country.ID" hidden="true" required="true" />
        <field name="COUNTRY" type="text" includeFrom="geo_country.NAME" />
        <field name="ISSUE_DATE" type="date" />
        <field name="EXPIRY_DATE" type="date" />
        <field name="DESCRIPTION" type="text" length="1024"/>
        <field name="ATTACHMENT_ID" type="integer" foreignKey="attachment.ID" hidden="true" joinType="outer" />
        <field name="ATTACHMENT_CREATED" type="creatorTimestamp" includeFrom="attachment.CREATED" />
        <field name="ATTACHMENT_CREATED_BY" type="integer" includeFrom="attachment.CREATED_BY" />
        <field name="ATTACHMENT_FILENAME" type="text" includeFrom="attachment.ATTACHMENT_FILENAME" />
        <field name="ATTACHMENT_FILESIZE" type="integer" includeFrom="attachment.ATTACHMENT_FILESIZE" />
        <field name="CREATED" type="creatorTimestamp" title="$created" />
        <field name="CREATED_BY" type="integer" title="$createdBy" />
        <field name="UPDATED" type="modifierTimestamp" title="$updated" />
        <field name="UPDATED_BY" type="integer" title="$updatedBy" />
    </fields>

A custom validator is applied on DESCRIPTION field, while no such validator is declared in the DS (but in another DS).
See screenshot attached.

Regards
Antoine

[Blama]

Hi Antoine,

that's one for Isomorphic.

But what you can do, as the regex is a client side validator: When you experience the error on a client in FF, hit F12 and examine the reply of the DataSourceLoader-Servlet called at startup.
You might have to use FF with F12-tools open all the time or use some other way to get to the reply-content. If you see a reply for your DS-Id not matching your .ds.xml, there is definitely something wrong.

What could be the reason is caching. I remember getting weird errors like yours (more not matching field definitions than validators) a long time ago. I think these were introduced by the browser caching the DataSourceLoader result which then in return did not match what the server thought the client was using.
So please make sure that your GET to DataSourceLoader is always a real HTTP 200, no 304 and not served from cache.

Best regards,
Blama

[agalataud]

I double checked that what is loaded on client is not loaded from cache.
Just now, I've been able to reproduce the problem on a dev environment, using an i18n language pack containing lot of accented and special characters (french).

The key-value pair in this bundle is:

validator.code=Charactères invalides, espace, et charactères accentués

In the logs, I get something that looks valid:

=== 2014-06-10 14:34:53,878 [0-11] INFO  Validation - [builtinApplication.businessUnit_validate] Validation error: [
    {
        CODE:{
            errorMessage:"Charactères invalides, espace, et charactères accentués"
        }
    }
]

However in the screen, the message is: "Charactères invalides, espace, et charactères accentués.
Charactères invalides"

Part of the value is repeated. In some cases, I have the value repeated 3 times, fully.

Could it be that values in i18n bundles are not XML/JS escaped in SmartGWT when dealing with validators?

[Isomorphic]

There was an issue, fixed a couple of months ago, where certain special characters might not appear in a hover. Retest with the latest and you should see this corrected.

However as far as the initial, seemingly unrelated report of validators being applied to the wrong field, this report is still unique amongst all users ever, and we don't have a speculation as to the cause.

[agalataud]

Hi

We're currently running with nightly build 2014-06-14.

Here is a clear example of duplicated localization with a standard validator.
We have 2 languages supported, french and english.
When using SmartGWT required validator, localization is performed by SmartGWT using its own resources.

As you can see in first screenshot, both english and french messages appear.

I tried to debug in JS a bit (2nd screenshot), and watching some values in isc_DynamicForm_validate, I've found that field variable contains all translations of validator message (reported as _21.$79u in watch list) but error reported by validator execution contains only the right translation (here french - see _26.errors).
This may be a dumb shortcut but, why validator execution results aren't used to fill tooltip?

[Isomorphic]

Sorry, we really can't do anything further with just another screenshot of an issue happening in your app, and unfortunately, the JavaScript state you're inspecting is just 1-to-1 with the screenshot, and doesn't suggest anything about how your application gets into this state.

What we would suggest is coordinating with this other developer from your team, who is attempting to work on "dynamic localization" for your application.

Whatever changes and/or subsystem have been introduced with this goal in mind are almost certainly responsible in some way for the weird behavior you're seeing, which has never been reported by another user, and is virtually certain to result from some kind of misuse of the GWT or SmartGWT localization APIs.

[agalataud]

I've been able to reproduce the duplicate validator message problem ('Field required' and 'Champ requis' both displayed) in a test project I started from scratch.
Basically only a form, 1 data source and french and english language packs.

Is there a way to send you the .zip of the eclipse project?

Also find attached a screenshot showing the pb in this test case.

Regards
Antoine

[Isomorphic]

In general, do not use a complete project as a test case. Many, many things can be wrong with a project that are not flaws in SmartGWT.

Instead, just show the modifications needs for a standard SDK to reproduce the problem - generally this means just one .java file with an onModuleLoad() definition, which can replace one of the .java files from a sample project such as BuiltInDS.

[agalataud]

I will provide what you want tomorrow.

Just to let you know my disappointment with support on this issue and other recent issues some colleagues and I reported.
I find it unbelievable that you refuse a sample project with nothing more than what you're asking (a java, a ds.xml and 2 properties files). What's the point of asking seperate uploads?
Since I reported this issue (1 month ago), I haven't seen any effort on your side to try at least a simple test case.
Same scenario with other issues reported: I had to insist with several posts, finally giving you test cases of almost 20 lines i believe you could have produced in 10'minutes.

Is your method based on discouraging people?

[Isomorphic]

No, this is our method for saving you time.

Typically, when a user tries to use a complete Java project as a test case, it contains many, many issues: hardcoded paths, need for databases to be present and certain database tables to exist, third-party library dependencies, only works with certain IDEs, etc.

A huge amount of back and forth is then required to get the project to actually run - and historically the result has been that the problem wasn't with SmartGWT after all, so all the effort spent working through project setup issues was pointless.

Creating test cases by modifying the sample projects is just a much more effective process. It naturally isolates the problem to SmartGWT and there is zero wasted effort on either side.

As a specific example, you mention your test project includes a .ds.xml file. Setting this up was likely a waste of time; starting from the builtinDS sample project you would have had a working DataSource with database table and sample data already present, and could have focused on just adding whatever settings are required to replicate the problem.

Since I reported this issue (1 month ago), I haven't seen any effort on your side to try at least a simple test case.

?? where is this assumption coming from?

You realize that what you're currently claiming is that validators just show mixed-language messages any time they fail? You can trivially verify, as we did, that this doesn't happen in framework samples, since you can switch the locale in the Showcase and just try it.

More generally, on your other bugs, we either tried the corresponding Showcase example, ran the corresponding automated test just in case, or indeed, wrote some quick test code to check on your claimed behavior.

The problem is that there is typically some unusual detail on your end that is actually causing the problem, and this is true for both framework issues and errors in your code. Once we've tried to replicate a problem and not been able to, we need to see your actual code to be able to make further progress.

[agalataud]

When providing a test case, I need to ensure it's as simple as possible, but also as close as possible to my situation.
In the end I need the bug affecting the feature I'm using to be fixed.
Moreover, I guess that even if I provide a .ds.xml referring to a SQL table, it doesn't take much time to mock this up.
Finally, I don't have in mind all the features of the framework, so finding an identical-but-simpler feature to demonstrate isn't necessarily the easiest approach for the reporter. Again, I think you're on the good side to apply such approach.

About the bug itself: here I provide a .ds.xml because it's how I reproduce the bug and I believe the use of JSTL-like internalization (fmt:) in it could be part of the problem.

[Isomorphic]

When providing a test case, I need to ensure it's as simple as possible, but also as close as possible to my situation.

No, nothing of the kind is required. The sole requirements for a test case are that it reproduces a bug and is simple.

When you see a problem in your app and you believe it's a framework issue, the most efficient thing is just to grab a sample and modify it to show the problem.

There is no reason to try to closely replicate your environment. That's just wasted time.

That all said, this is an incomplete test case (no SQL Schema, no resource bundle), but since you're having so much trouble here, we'll make the mods necessary to the sample project and show you what an appropriate test case would look like (dead simple, and easier than what you've done).