Filtering ListGrid with FilterBuilder - Time greater than equal to N seconds ago

[danaconway]

I'm using SmartClient_v121p_2020-04-14_PowerEdition. I have a ListGrid backed by a client only DataSource. As I receive new data, I update the DataSource. I have a requirement to filter this data by "Last X Seconds", so I created a FilterBuilder to be able to do this. The problem that I am having is that when I select the options in the FilterBuilder to filter by, let's say the last 10 seconds, it initially filters the data that already exists in the grid, but any new data being added does not get filtered. If I change the number of seconds in the FilterBuilder and re-apply the filter, it starts working. I tried to reproduce this in the Feature Explorer with the code below, but I cannot. However, I can consistently reproduce this in my project. I was wondering if there was perhaps a known bug in the version that I am using and/or if there is a workaround.

var formatString = "MM/dd/yyyy HH:mm:ss";
isc.DateUtil.setInputFormat("MDY");
isc.DateUtil.setNormalDatetimeDisplayFormat(formatString);
isc.DateUtil.setShortDatetimeDisplayFormat(formatString);
isc.DateUtil.setNormalDisplayFormat(formatString);
isc.DateUtil.setShortDisplayFormat(formatString);

var ds = isc.DataSource.create({
ID: "myDS",
fields: [
{name: "display_time", type: "datetime", canFilter: false},
{name: "time", type: "datetime", timeUnitOptions: ["hour", "minute", "second"]},
{name: "id", type: "text"}
],
dataFormat: "json",
clientOnly: true
});

isc.FilterBuilder.create({
ID:"advancedFilter",
dataSource:"myDS"
});

isc.ListGrid.create({
ID: "myList",
width:700, height:224, alternateRecordStyles:true,
dataSource: "myDS", autoFetchData: true,
fields:[
{name:"display_time", align: "center",
formatCellValue: function(value) {
var validDateTime = isc.DateUtil.parseInput(value);
if( validDateTime !== null ) {
return isc.Time.toTime(validDateTime, "to24HourTime", true);
}
return value;
}},
{name:"time", width: 200, align: "center", hidden: true},
{name:"id", align: "center"}
]});

isc.IButton.create({
ID:"filterButton",
title:"Filter",
click : function () {
myList.filterData(advancedFilter.getCriteria());
}
})

isc.IButton.create({
ID:"clearFilterButton",
title:"Clear",
click : function () {
myList.clearCriteria();
}
})

isc.VStack.create({
membersMargin:10,
members:[ advancedFilter, filterButton, clearFilterButton, myList ]
})

function sleep(milliseconds) {
return new Promise(resolve => setTimeout(resolve, milliseconds));
}

async function sendData() {

for( var i = 0; i < 30; i++ ) {
var dt = new Date();
dt.setMilliseconds(0);
var msg = {display_time: dt, time: dt, id: i};
ds.addData(msg);
await sleep(3000);
}
}


sendData();

Thanks in advance!

[Isomorphic]

No, sorry, there is no known bug in this area (we pretty much immediately all fix all known bugs).

In this test case you have added data correctly and filtering works and is automatic. In your real app, you must be adding data in some not-quite-valid way, where the ListGrid doesn't know the data has been updated, and so can't automatically refilter. Or perhaps you overrode a method and broke automatic refiltering. That's about all the speculation we can do given a test case that shows things working as expected.

[danaconway]

Thank you. I realized that I left out some information here. After defining my FilterBuilder and ListGrid, I perform a database query to to get historical data to initialize the grid with. Then, I subscribe to a live stream of data, which is added to the same grid. The code is something like this:

async function getInitData() {
    // Get data from db
    var data = await getData();

    // Make sure date is formatted correctly for filtering
    for( var i = 0; i < data.length; i++ )  {
       let d = new Date( data[i].time );
       d.setMilliseconds(0);

       data[i].time = d;
       data[i].display_time = d;
    }

    myDS.setCacheData(data);
    myList.invalidateCache();

    // Then start receiving live data and using myDS.addData(newData)....
}

If there is no initial data in the database, the filtering works fine. Or, if I only subscribe to live data and leave out the initialization from the database, the filtering is fine. It's only when there is initial data from the database that the filtering issue happens. But, I can get the filtering to start working after changing the number of seconds and re-applying the filter once or sometimes twice. Also, the data from the database is filtered out correctly; it's only the new data coming in that is not filtered correctly.

[Isomorphic]

Note that, by design and as documented, lg.fetchData() / filterData() does nothing if you pass criteria that is the same as the grid already has. So repeated calls with unchanged data should do nothing.

The problem is that new data should by automatically filtered as well, and in the test case you've shown us, it is indeed filtered as expected.

So the running theory is that you have other overrides on your grid that are breaking automatic filtering of new data. We can't do anything further until we have a test case showing this weird behavior.