[bug] ComboBoxItem and SpecialValues forces to fetch all result set instead of 75.

[antanas.arvasevicius]

Hi, Isomorphic,
I've noticed that if I use specialValues then ComboBoxItem / SelectItem will trigger full fetch from server instead of 75 records at a time, is it a documented behavior or a bug? Thanks.

[antanas.arvasevicius]

Hi, how about this bug?

[Blama]

Hi Antanas,

I tried it out of curiosity in this sample and don't see the behavior you describe (1st fetch 0..78, following 78..153).

Best regards
Blama

[antanas.arvasevicius]

Hi, I'll try to provide a small test case

[antanas.arvasevicius]

Here you go, proof of concept:

how to proceed:
1) click drop down. you'll see fetchData in console log
2) scroll to the bottom. you'll see that no additional fetches is happening.
--------

Just remove "specialValues" and you'll see how should pagination work - multiple fetches will occur.

This bug is really a server cpu eater so we completely removed "specialValues" usage in our code.

    var testData = new Array(500).map(function (v, i) { return { id: ""+(i+1), name: 'title_' + (i+1) }});

    var ds = isc.DataSource.create({
        ID: "ds",
        clientOnly: true,
        fields: [{name: 'id', primaryKey: true}, {name: 'name'}],
        testData: testData,
        fetchData : function (criteria, callback, requestProperties) {
            console.log('fetchData: ', criteria, requestProperties);
            this.performDSOperation("fetch", criteria, callback, requestProperties);
        },
    });

    isc.DynamicForm.create({
        autoDraw: true,
        dataSource: "ds",
        useAllDataSourceFields: false,
        fields: [
            {
                title: "Select",
                name: "name",
                editorType: "ComboBoxItem",
                required: true,
                foreignDataSource: "ds",
                valueField: "id",
                foreignDisplayField: "name",
                specialValues: {
                    "-1": "Select None",
                    special: "Some Special"
                }
            }
        ]
    });

tested on latest version 10.x

[antanas.arvasevicius]

I'm really suspicious about SmartClient as a product, it seems it really not run in production for a long time, or the usages were pretty trivial. And as it started developing in 1998 and now hundreds of years passed since then and still bugs bugs bugs everythere. Do we use that product as not intended to or what I don't know. Much younger frameworks seems more stable than this.

[Isomorphic]

It's important to read the docs thoroughly - the specialValues docs refer repeatedly to separateSpecialValues, which reads:

If true, specialValues special values such as the empty value will be shown in a separate non-scrolling area, in the separateValuesList. Aside from making these values more easily accessible, showing them in a separate list allows data paging to be used, which is disabled if the separateValues are shown in the normal drop-down list along with other values.

We're going to make this even more prominent (since you might not be the only person to miss it). But just to note, this is not a bug, your other post today is not a bug (bad usage instead), and while you did find a bug a while ago (thanks for that), you certainly have no basis for claiming stability issues relative to other software. We're confident that a comparison to a framework with similar features would show a much much lower bug count for SmartClient.

"Similar features" is very important - Tiny libraries like JQuery or Angular have very few capabilities and very few bugs, just as no bugs have been reported against SmartClient's low-level functions in years. But try to add UI components that replicate even part of a SmartClient UI and your bug count goes way up, and in this kind of direct comparison, we think it's clear that SmartClient has less bugs, even as we support more sophisticated features. Note in particular that one of the issues you filed today relied on two features (Adaptive Filtering and Relative Date Advanced Criteria) no other framework even offers.

As far as usage, SmartClient is the framework that is used for the most complex applications, and has been since first release 15 years ago. A list of customers and some of their use cases is here. We wish we could share the far more complex stuff we do with some customers, but business critical applications are typically confidential: no pharma company wants to share the innovations it's made to its drug discovery pipeline, no defense company can share the software they use for multi-billion dollar bid estimates, etc. So even though we're showing a list of the world's largest and most sophisticated companies, realize, even that doesn't really convey the scope of what actually runs on our tech.

[antanas.arvasevicius]

Hi,
understood. By looking in the docs of "specialValues" property there is not a single word about paged fetching disabling then used "specialValues".
How do you suppose I would find that doc block about paging in help as that help is entirely for a different property ("separateSpecialValues" not a "specialValues")?
I would suggest you to consider set that "separateSpecialValues" to true by default, because paged fetching is such a great feature and it just gets disabled unintentionally by using some non critical property.

I suggest everybody should add this code to the top of their apps:

isc.PickList.addProperties({separateSpecialValues: true})

[antanas.arvasevicius]

As for framework, I do not even try to compare SmartClient to other frameworks as jQuery or Angular or whatever, it just not comparable. Maybe we could compare this to ExtJS, but it is almost 10 years away from SmartClient's first development so this fact says more than enough.
By what I said yesterday, is that so strange that product has run so many years and we still hit on some bugs which our case seems was pretty trivial and it's strange that no one reported that kind of bugs. But there is a bright side: then we see strange behavior in our app we are pretty sure that 99% is our fault, not a framework in itself so it's really super stable in it's API.
I can say thanks your all team for such a great framework, and good luck in future. Hope you'll drop all legacy compatibilities in new version of smartclient any IE 6 - 9, and old firefox, and concentrate on new technologies, as now html5 and browsers a pretty stable and conform to standards. Also I strongly feel that you must consider to use TypeScript as your base language as our codebase now it's 150k+ LOC and it is just pleasure to work with it , all intellisence, all compilation type checking, no any jsdoc blocks and linters. We can use any ES6/7 features and compile to ES5/3 target. full recompilation tooks just 8 seconds for our codebase. Nice thing that it's transparently compatible to javascript legacy code. no hidden magic as in (Google GWT). also await/async support which allows async programming write in style of sequential code no need to callback in callback.
Just try to prototype something, many IDE's support it out of the box (as IntelliJ ir VSCODE) and you could remove any magic which you've done to support Object Oriented programming in SmartClient :) (except mixins i think, but this also become available in future)

[Isomorphic]

Yes, that's exactly what we meant by making it "more prominent" - we're adding an additional sentence to the docs of specialValues to mention that separateSpecialValues is required to maintain paging. To be fair though, the existing doc mentions separateSpecialValues prominently, twice, and in our doc viewer, you don't even have to click to see related docs (just hover). It's often tricky to figure out how much should be mentioned immediately vs left for related docs. In this instance, having seen at least one person missed the related docs and ended up reporting a bug, we're going to go ahead and cover this behavior inline as we just described.

Note, changing a framework default in the way you mentioned can be risky. Higher level features of SmartClient use lower-level features, so an advanced dialog in the framework might use specialValues and make assumptions that they are not displayed separately. Safer practice, if you prefer a different default behavior in your app, is to create a subclass of SelectItem or ComboBoxItem that you use pervasively in lieu of the default control.

[antanas.arvasevicius]

Yes, it was our fault to not subclassing default components and use these instead of default one. Lesson for a future.

[Isomorphic]

On your other message - it's great that you understand the difference in scope of what we provide. On the bugs you found - basically at any given time it is still possible to find bugs in recently introduced features, or in areas of the framework where we had to revisit old, well-tested code due to some nasty new browser bug.

Your bug today - unnecessary fetches with RelativeDates in the criteria - is an example of the former. RelativeDate support went in just two releases ago, and we squashed a bunch of bugs discovered shortly after release, but the bug you found was the kind that most application developers would not notice: the ultimate displayed results were correct, but you noticed that SmartClient should have avoided the fetch.

Your other bug - CanvasItem inline editing in a grid with show/hide cycles - was an example of the latter: a regression in a stable subsystem caused when we fixed something else (in this case the root cause was workarounds for nasty IE9 CSS3 bugs).

To minimize this, we create a suite of automated tests with every new feature, and every time a bug is fixed, an automated test goes in to ensure there's never a regression in that area again.

Despite the ludicrously large test suite we've accrued over the years, we still get a steady stream of valid bugs, just because the usage of our product is so broad and so deep - but the telltale sign of progress is that *most* bugs these days involve multiple interacting subsystems, several non-default settings, and someone doing something slightly non-mainstream.