DataSource includeFrom fields are not working properly

[eschwegs]

After updating smartGWT libraries, generated SQL involving include fields is not working. JOINs don't get generated and included field references in the where clause are defaulting to Test_Coverage_Metric.null (where Test_Coverage_Metric is the main dataSource's tableName)

i.e.
Working SQL (differences from broken SQL marked in bold text)

SmartClient Version: v12.1p_2020-07-31/PowerEdition Deployment (built 2020-07-31)

SELECT
Test_Coverage_Metric.FK_Test_Coverage_Report, SUM(Test_Coverage_Metric.Covered) AS Covered, SUM(Test_Coverage_Metric.Total) AS Total, SUM(cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total, 0)) , 0) as decimal(30, 2) )) AS PercentCovered, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group
FROM
Test_Coverage_Metric
JOIN
Test_Coverage_Report
ON Test_Coverage_Metric.FK_Test_Coverage_Report = Test_Coverage_Report.PK_Test_Coverage_Report
JOIN
Test_Project_Stream
ON Test_Coverage_Report.FK_Test_Project_Stream = Test_Project_Stream.PK_Test_Project_Stream
LEFT OUTER JOIN
Test_Session_Stream
ON Test_Project_Stream.FK_Test_Session_Stream = Test_Session_Stream.PK_Test_Session_Stream
LEFT OUTER JOIN
Test_Project
ON Test_Project_Stream.FK_Test_Project = Test_Project.PK_Test_Project
JOIN
Test_Bench
ON Test_Project.FK_Test_Bench = Test_Bench.PK_Test_Bench
JOIN
Test_Coverage_Category_Term
ON Test_Coverage_Metric.FK_Test_Coverage_Category_Term = Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term
JOIN
Test_Coverage_Group_Term
ON Test_Coverage_Metric.FK_Test_Coverage_Group_Term = Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term
WHERE
(
(
(
Test_Coverage_Group_Term.Coverage_Group IN (
'AVS', 'Core', 'H264', 'H265', 'MPEG2', 'Platform', 'Total', 'VP9'
)
)
AND Test_Coverage_Group_Term.Coverage_Group IS NOT NULL
)
AND (
(
Test_Coverage_Metric.FK_Test_Coverage_Report IN (
1621, 1622
)
)
AND Test_Coverage_Metric.FK_Test_Coverage_Report IS NOT NULL
)
AND (
(
Test_Coverage_Category_Term.Coverage_Category IN (
'Statement', 'Decision', 'All Skips', 'Debug tools', 'Defensive coding', 'Incomplete feature', 'Unreachable code', 'Unmeasurable'
)
)
AND Test_Coverage_Category_Term.Coverage_Category IS NOT NULL
)
AND (
Test_Bench.BenchName = 'Avd Bench'
AND Test_Bench.BenchName IS NOT NULL
)
AND '1'='1'
AND '1'='1'
AND (
Test_Bench.BenchName = 'Avd Bench'
AND Test_Bench.BenchName IS NOT NULL
)
AND (
Test_Project.PK_Test_Project = 18
AND Test_Project.PK_Test_Project IS NOT NULL
)
AND (
(
Test_Session_Stream.PK_Test_Session_Stream IN (
77, 78, 98, 110, 107, 109, 108
)
)
AND Test_Session_Stream.PK_Test_Session_Stream IS NOT NULL
)
)
GROUP BY
Test_Coverage_Metric.FK_Test_Coverage_Report, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group

Broken SQL (differences from working SQL marked bold text)

SmartClient Version: v12.1p_2021-07-04/PowerEdition Deployment (built 2021-07-04)

SELECT
Test_Coverage_Metric.FK_Test_Coverage_Report, SUM(Test_Coverage_Metric.Covered) AS Covered, SUM(Test_Coverage_Metric.Total) AS Total, SUM(cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total, 0)) , 0) as decimal(30, 2) )) AS PercentCovered, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group
FROM
Test_Coverage_Metric
/* include field joins are missing */
JOIN
Test_Coverage_Category_Term
ON Test_Coverage_Metric.FK_Test_Coverage_Category_Term = Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term
JOIN
Test_Coverage_Group_Term
ON Test_Coverage_Metric.FK_Test_Coverage_Group_Term = Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term
WHERE
(
(
(
Test_Coverage_Group_Term.Coverage_Group IN (
'AVS', 'Core', 'H264', 'H265', 'MPEG2', 'Platform', 'Total', 'VP9'
)
)
AND Test_Coverage_Group_Term.Coverage_Group IS NOT NULL
)
AND (
(
Test_Coverage_Metric.FK_Test_Coverage_Report IN (
1621, 1622
)
)
AND Test_Coverage_Metric.FK_Test_Coverage_Report IS NOT NULL
)
AND (
(
Test_Coverage_Category_Term.Coverage_Category IN (
'Statement', 'Decision', 'All Skips', 'Debug tools', 'Defensive coding', 'Incomplete feature', 'Unreachable code', 'Unmeasurable'
)
)
AND Test_Coverage_Category_Term.Coverage_Category IS NOT NULL
)
AND (
Test_Coverage_Metric.null = 'Avd Bench'
AND Test_Coverage_Metric.null IS NOT NULL
)
AND '1'='1'
AND '1'='1'
AND (
Test_Coverage_Metric.null = 'Avd Bench'
AND Test_Coverage_Metric.null IS NOT NULL
)
AND (
Test_Coverage_Metric.null = 18
AND Test_Coverage_Metric.null IS NOT NULL
)
AND (
(
Test_Coverage_Metric.null IN (
77, 78, 98, 110, 107, 109, 108
)
)
AND Test_Coverage_Metric.null IS NOT NULL
)
)
GROUP BY
Test_Coverage_Metric.FK_Test_Coverage_Report, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group

For reference. This is the datasource

<DataSource
ID="TestCoverageMetricView"
serverType="sql"
defaultTextMatchStyle="exactCase"
tableName="Test_Coverage_Metric"
>
<fields>
<!--
! Test_Coverage_Metric table definition
-->
<field name="PK_Test_Coverage_Metric" type="sequence" width="100" primaryKey="true"
title="PK_Test_Coverage_Metric"
hidden="true"
/>
<field name="FK_Test_Coverage_Report" type="integer" width="100" foreignKey="Test_Coverage_Report.PK_Test_Coverage_Report"
title="Coverage Report"
hidden="true"
displayField="Title" valueField="PK_Test_Coverage_Report"
optionDataSource="TestCoverageMetricView"
filter_outputs="PK_Test_Coverage_Report,Title,Generation_Date"
filter_sortSpecifier="-Generation_Date"
filter_groups="PK_Test_Coverage_Report,Title,Generation_Date"
filter_pickListProp_sortSpecifier="-Generation_Date"
filter_pickListProp_groupByField="PK_Test_Coverage_Report,Title,Generation_Date"
/>
<field name="FK_Test_Coverage_Group_Term" type="integer" width="100" foreignKey="Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term"
title="FK_Test_Coverage_Group_Term"
hidden="true"
displayField="Coverage_Group" valueField="PK_Test_Coverage_Group_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="PK_Test_Coverage_Group_Term,Coverage_Group"
filter_sortSpecifier="Coverage_Group"
filter_groups="PK_Test_Coverage_Group_Term,Coverage_Group"
filter_pickListProp_sortSpecifier="Coverage_Group"
filter_pickListProp_groupByField="PK_Test_Coverage_Group_Term,Coverage_Group"
/>


<field name="FK_Test_Coverage_Category_Term" type="integer" width="100" foreignKey="Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term"
title="Coverage Category FK"
hidden="true"
/>

<field name="Covered" type="integer" width="100" summaryFunctionType="sum"
title="Covered Statements"
hidden="false"
prompt="Number of code statements covered"
/>
<field name="Total" type="integer" width="100" summaryFunctionType="sum"
title="Total Statements"
hidden="false"
prompt="Number of total statements"
/>

<field name="PercentCovered" type="float" width="100" summaryFunctionType="sum"
title="Percent Covered"
hidden="false"
customSelectExpression="cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total,0)) , 0) as decimal(30,2) )"
prompt="Percentage of statements covered"
/>

<!-- aggregate txt : i.e. 86% (86/100) -->
<field name="Percent_Covered_Total" type="text" width="150"
title="Aggregate"
hidden="true"
customSelectExpression="cast( cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total,0)) , 0) as decimal(30,2) ) as varchar) + '% (' + cast(Covered as varchar) + '/' + cast(Total as varchar) + ')' "
prompt="Text represinting an aggregate of Percent%(covered/total) i.e. 86% (86/100) "
/>

<!-- Damn it, this does not work with charting
<field name="RowCnt" type="integer" customSelectExpression="DISTINCT PK_Test_Coverage_Metric" summaryFunctionType="count"
title="Row Count"
hidden="true"
/>
-->

<!--
! Test_Coverage_Report table field
-->
<field includeFrom="Test_Coverage_Report.PK_Test_Coverage_Report"
title="Report"
hidden="false"
prompt="Coverage report"
displayField="Title" valueField="PK_Test_Coverage_Report"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Report"
filter_outputs="PK_Test_Coverage_Report,Title,Generation_Date"
filter_sortSpecifier="-Generation_Date"
filter_groups="PK_Test_Coverage_Report,Title,Generation_Date"
filter_pickListProp_sortSpecifier= "-Generation_Date"
filter_pickListFields="Title,Generation_Date"
/>

<field includeFrom="Test_Coverage_Report.Title"
title="Report Name"
prompt="Coverage report name"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Report"
filter_outputs="Title,Generation_Date"
filter_sortSpecifier="-Generation_Date"
filter_groups="Title,Generation_Date"
filter_pickListProp_sortSpecifier= "-Generation_Date"
filter_pickListFields="Title,Generation_Date"
/>
<field includeFrom="Test_Coverage_Report.FK_Test_Project_Stream"
prompt="Session stream"
hidden="true"
/>
<field includeFrom="Test_Coverage_Report.Description"
prompt="Coverage report description"
optionDataSource="TestCoverageMetricView"
xxxxoptionDataSource="Test_Coverage_Report"
filter_outputs="Description"
filter_sortSpecifier="Description"
filter_groups="Description"
filter_pickListProp_sortSpecifier= "Description"
filter_pickListFields="Description"
/>
<field includeFrom="Test_Coverage_Report.Comment"
prompt="Coverage Report Comment"
displayField="Comment" valueField="Comment"
optionDataSource="TestCoverageMetricView"
filter_outputs="Comment"
filter_sortSpecifier="Comment"
filter_groups="Comment"
filter_pickListProp_sortSpecifier="Comment"
/>
<field includeFrom="Test_Coverage_Report.Generation_Date"
prompt="Date in which the coverage report was generated"
xxx_na_valueField="Generation_Date" displayField="Generation_Date"
xxx_na_optionDataSource="TestCoverageMetricView"
xxx_na_filter_outputs="Generation_Date"
xxx_na_filter_sortSpecifier="Generation_Date"
xxx_na_filter_groups="Generation_Date"
xxx_na_filter_pickListProp_sortSpecifier="Generation_Date"
/>
<field includeFrom="Test_Coverage_Report.Revision"
prompt="Coverage report firmware revision"
displayField="Revision" valueField="Revision"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Report"
filter_outputs="Revision"
filter_sortSpecifier="Revision"
filter_groups="Revision"
filter_pickListProp_sortSpecifier="Revision"
xxxxxfilter_pickListProp_groupByField="Revision"
/>

<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Session_Stream.PK_Test_Session_Stream"
title="Session Stream"
hidden="false"
prompt="Session stream"
displayField="Stream_Name" valueField="PK_Test_Session_Stream"
optionDataSource="TestCoverageMetricView"
filter_outputs="Stream_Name,PK_Test_Session_Stream"
filter_sortSpecifier="Stream_Name"
filter_groups="Stream_Name,PK_Test_Session_Stream"
filter_pickListProp_sortSpecifier= "Stream_Name"
xxxfilter_pickListProp_groupByField="Stream_Name"

/>
<field name="Stream_Name" includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Session_Stream.Name"
title="Stream Name"
hidden="true"
prompt="A session stream name"
/>

<!--
! Project
-->
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.PK_Test_Project"
title="Project"
hidden="false"
prompt="Project"
valueField="PK_Test_Project" displayField="ProjName"
optionDataSource="TestCoverageMetricView"
filter_outputs="PK_Test_Project,ProjName"
filter_sortSpecifier="ProjName"
filter_groups="PK_Test_Project,ProjName"
filter_pickListProp_sortSpecifier= "ProjName"
filter_pickListProp_groupByField="PK_Test_Project,ProjName"

/>
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.ProjName"
hidden="false"
prompt="Project name"
valueField="ProjName" displayField="ProjName"
optionDataSource="TestCoverageMetricView"
filter_outputs="ProjName"
filter_sortSpecifier="ProjName"
filter_groups="ProjName"
filter_pickListProp_sortSpecifier= "ProjName"
ffffilter_pickListProp_groupByField="ProjName"

/>

<!-- Bench -->
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.Test_Bench.PK_Test_Bench"
title="Bench"
hidden="true"
prompt="Bench"
/>
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.Test_Bench.BenchName"
title="Bench Name"
hidden="true"
prompt="Bench name"
/>

<!--
! Test_Coverage_Category_Term table fields
-->
<field includeFrom="Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term"
title="Coverage Category"
hidden="false"
prompt="Coverage Category"
displayField="Coverage_Category" valueField="PK_Test_Coverage_Category_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="Coverage_Category,PK_Test_Coverage_Category_Term"
filter_sortSpecifier="Coverage_Category"
filter_groups="Coverage_Category"
filter_pickListProp_sortSpecifier= "Coverage_Category"
filter_pickListProp_groupByField="Coverage_Group,PK_Test_Coverage_Group_Term"

/>
<field includeFrom="Test_Coverage_Category_Term.Coverage_Category"
title="Coverage Category Name"
hidden="false"
prompt="Coverage Category i.e. Statement Skip"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Category_Term"
filter_outputs="Coverage_Category"
filter_sortSpecifier="Coverage_Category"
filter_groups="Coverage_Category"
filter_pickListProp_sortSpecifier="Coverage_Category"
/>
<field includeFrom="Test_Coverage_Category_Term.isSkipCategory"
hidden="true"
/>
<field includeFrom="Test_Coverage_Category_Term.parentId"
hidden="true"
/>


<!--
! Test_Coverage_Group_Term table fields
-->
<field includeFrom="Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term"
title="Coverage Group"
hidden="false"
prompt="Coverage Group (Bench defined code grouping)"
displayField="Coverage_Group" valueField="PK_Test_Coverage_Group_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="Coverage_Group,PK_Test_Coverage_Group_Term"
filter_sortSpecifier="Coverage_Group"
filter_groups="Coverage_Group"
filter_pickListProp_sortSpecifier= "Coverage_Group"
filter_pickListProp_groupByField="Coverage_Group,PK_Test_Coverage_Group_Term"
/>
<field includeFrom="Test_Coverage_Group_Term.Coverage_Group"
title="Coverage Group Name"
hidden="false"
prompt="Coverage group name"
xxxoptionDataSource="Test_Coverage_Group_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="Coverage_Group"
filter_sortSpecifier="Coverage_Group"
filter_groups="Coverage_Group"
filter_pickListProp_sortSpecifier="Coverage_Group"

/>



</fields>

</DataSource>

And here is the relavent section of one of the data sources that gets included

<DataSource ID="Test_Bench" serverType="sql" tableName="Test_Bench">

<fields>
<field name="PK_Test_Bench" title="Bench ID" type="sequence" primaryKey ="true" canEdit="false" hidden="true" />
<field name="BenchName" title="Bench" type="text" lenght="63" width="150" canEdit="false" />
... other fields defined ...
</fields>

<operationBinding operationType="fetch" >
<orderClause>
CT_on_create DESC
</orderClause>
</operationBinding>

</DataSource>

Let me know if you need more info.

[Isomorphic]

The DataSources came through with broken HTML quoting, can you fix those?

We also need to see a sample request that would generate this SQL, so we see how values are specified. Ideally, this is server-side code that runs on it's own (a client-side request could be going through DMI logic that does various things, including eg dsRequest.addToTemplateContext(), but also many other things that would influence behavior).

[eschwegs]

For reference. This is the datasource:

<DataSource
ID="TestCoverageMetricView"
serverType="sql"
defaultTextMatchStyle="exactCase"
tableName="Test_Coverage_Metric"
>
<fields>
<!--
! Test_Coverage_Metric table definition
-->
<field name="PK_Test_Coverage_Metric" type="sequence" width="100" primaryKey="true"
title="PK_Test_Coverage_Metric"
hidden="true"
/>
<field name="FK_Test_Coverage_Report" type="integer" width="100" foreignKey="Test_Coverage_Report.PK_Test_Coverage_Report"
title="Coverage Report"
hidden="true"
displayField="Title" valueField="PK_Test_Coverage_Report"
optionDataSource="TestCoverageMetricView"
filter_outputs="PK_Test_Coverage_Report,Title,Generation_Date"
filter_sortSpecifier="-Generation_Date"
filter_groups="PK_Test_Coverage_Report,Title,Generation_Date"
filter_pickListProp_sortSpecifier="-Generation_Date"
filter_pickListProp_groupByField="PK_Test_Coverage_Report,Title,Generation_Date"
/>
<field name="FK_Test_Coverage_Group_Term" type="integer" width="100" foreignKey="Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term"
title="FK_Test_Coverage_Group_Term"
hidden="true"
displayField="Coverage_Group" valueField="PK_Test_Coverage_Group_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="PK_Test_Coverage_Group_Term,Coverage_Group"
filter_sortSpecifier="Coverage_Group"
filter_groups="PK_Test_Coverage_Group_Term,Coverage_Group"
filter_pickListProp_sortSpecifier="Coverage_Group"
filter_pickListProp_groupByField="PK_Test_Coverage_Group_Term,Coverage_Group"
/>


<field name="FK_Test_Coverage_Category_Term" type="integer" width="100" foreignKey="Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term"
title="Coverage Category FK"
hidden="true"
/>

<field name="Covered" type="integer" width="100" summaryFunctionType="sum"
title="Covered Statements"
hidden="false"
prompt="Number of code statements covered"
/>
<field name="Total" type="integer" width="100" summaryFunctionType="sum"
title="Total Statements"
hidden="false"
prompt="Number of total statements"
/>

<field name="PercentCovered" type="float" width="100" summaryFunctionType="sum"
title="Percent Covered"
hidden="false"
customSelectExpression="cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total,0)) , 0) as decimal(30,2) )"
prompt="Percentage of statements covered"
/>

<!-- aggregate txt : i.e. 86% (86/100) -->
<field name="Percent_Covered_Total" type="text" width="150"
title="Aggregate"
hidden="true"
customSelectExpression="cast( cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total,0)) , 0) as decimal(30,2) ) as varchar) + '% (' + cast(Covered as varchar) + '/' + cast(Total as varchar) + ')' "
prompt="Text represinting an aggregate of Percent%(covered/total) i.e. 86% (86/100) "
/>

<!-- Damn it, this does not work with charting
<field name="RowCnt" type="integer" customSelectExpression="DISTINCT PK_Test_Coverage_Metric" summaryFunctionType="count"
title="Row Count"
hidden="true"
/>
-->

<!--
! Test_Coverage_Report table field
-->
<field includeFrom="Test_Coverage_Report.PK_Test_Coverage_Report"
title="Report"
hidden="false"
prompt="Coverage report"
displayField="Title" valueField="PK_Test_Coverage_Report"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Report"
filter_outputs="PK_Test_Coverage_Report,Title,Generation_Date"
filter_sortSpecifier="-Generation_Date"
filter_groups="PK_Test_Coverage_Report,Title,Generation_Date"
filter_pickListProp_sortSpecifier= "-Generation_Date"
filter_pickListFields="Title,Generation_Date"
/>

<field includeFrom="Test_Coverage_Report.Title"
title="Report Name"
prompt="Coverage report name"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Report"
filter_outputs="Title,Generation_Date"
filter_sortSpecifier="-Generation_Date"
filter_groups="Title,Generation_Date"
filter_pickListProp_sortSpecifier= "-Generation_Date"
filter_pickListFields="Title,Generation_Date"
/>
<field includeFrom="Test_Coverage_Report.FK_Test_Project_Stream"
prompt="Session stream"
hidden="true"
/>
<field includeFrom="Test_Coverage_Report.Description"
prompt="Coverage report description"
optionDataSource="TestCoverageMetricView"
xxxxoptionDataSource="Test_Coverage_Report"
filter_outputs="Description"
filter_sortSpecifier="Description"
filter_groups="Description"
filter_pickListProp_sortSpecifier= "Description"
filter_pickListFields="Description"
/>
<field includeFrom="Test_Coverage_Report.Comment"
prompt="Coverage Report Comment"
displayField="Comment" valueField="Comment"
optionDataSource="TestCoverageMetricView"
filter_outputs="Comment"
filter_sortSpecifier="Comment"
filter_groups="Comment"
filter_pickListProp_sortSpecifier="Comment"
/>
<field includeFrom="Test_Coverage_Report.Generation_Date"
prompt="Date in which the coverage report was generated"
/>
<field includeFrom="Test_Coverage_Report.Revision"
prompt="Coverage report firmware revision"
displayField="Revision" valueField="Revision"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Report"
filter_outputs="Revision"
filter_sortSpecifier="Revision"
filter_groups="Revision"
filter_pickListProp_sortSpecifier="Revision"
xxxxxfilter_pickListProp_groupByField="Revision"
/>

<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Session_Stream.PK_Test_Session_Stream"
title="Session Stream"
hidden="false"
prompt="Session stream"
displayField="Stream_Name" valueField="PK_Test_Session_Stream"
optionDataSource="TestCoverageMetricView"
filter_outputs="Stream_Name,PK_Test_Session_Stream"
filter_sortSpecifier="Stream_Name"
filter_groups="Stream_Name,PK_Test_Session_Stream"
filter_pickListProp_sortSpecifier= "Stream_Name"
xxxfilter_pickListProp_groupByField="Stream_Name"

/>
<field name="Stream_Name" includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Session_Stream.Name"
title="Stream Name"
hidden="true"
prompt="A session stream name"
/>

<!--
! Project
-->
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.PK_Test_Project"
title="Project"
hidden="false"
prompt="Project"
valueField="PK_Test_Project" displayField="ProjName"
optionDataSource="TestCoverageMetricView"
filter_outputs="PK_Test_Project,ProjName"
filter_sortSpecifier="ProjName"
filter_groups="PK_Test_Project,ProjName"
filter_pickListProp_sortSpecifier= "ProjName"
filter_pickListProp_groupByField="PK_Test_Project,ProjName"

/>
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.ProjName"
hidden="false"
prompt="Project name"
valueField="ProjName" displayField="ProjName"
optionDataSource="TestCoverageMetricView"
filter_outputs="ProjName"
filter_sortSpecifier="ProjName"
filter_groups="ProjName"
filter_pickListProp_sortSpecifier= "ProjName"
ffffilter_pickListProp_groupByField="ProjName"

/>

<!-- Bench -->
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.Test_Bench.PK_Test_Bench"
title="Bench"
hidden="true"
prompt="Bench"
/>
<field includeFrom="Test_Coverage_Report.Test_Project_Stream.Test_Project.Test_Bench.BenchName"
title="Bench Name"
hidden="true"
prompt="Bench name"
/>

<!--
! Test_Coverage_Category_Term table fields
-->
<field includeFrom="Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term"
title="Coverage Category"
hidden="false"
prompt="Coverage Category"
displayField="Coverage_Category" valueField="PK_Test_Coverage_Category_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="Coverage_Category,PK_Test_Coverage_Category_Term"
filter_sortSpecifier="Coverage_Category"
filter_groups="Coverage_Category"
filter_pickListProp_sortSpecifier= "Coverage_Category"
filter_pickListProp_groupByField="Coverage_Group,PK_Test_Coverage_Group_Term"

/>
<field includeFrom="Test_Coverage_Category_Term.Coverage_Category"
title="Coverage Category Name"
hidden="false"
prompt="Coverage Category i.e. Statement Skip"
optionDataSource="TestCoverageMetricView"
xxxoptionDataSource="Test_Coverage_Category_Term"
filter_outputs="Coverage_Category"
filter_sortSpecifier="Coverage_Category"
filter_groups="Coverage_Category"
filter_pickListProp_sortSpecifier="Coverage_Category"
/>
<field includeFrom="Test_Coverage_Category_Term.isSkipCategory"
hidden="true"
/>
<field includeFrom="Test_Coverage_Category_Term.parentId"
hidden="true"
/>


<!--
! Test_Coverage_Group_Term table fields
-->
<field includeFrom="Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term"
title="Coverage Group"
hidden="false"
prompt="Coverage Group (Bench defined code grouping)"
displayField="Coverage_Group" valueField="PK_Test_Coverage_Group_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="Coverage_Group,PK_Test_Coverage_Group_Term"
filter_sortSpecifier="Coverage_Group"
filter_groups="Coverage_Group"
filter_pickListProp_sortSpecifier= "Coverage_Group"
filter_pickListProp_groupByField="Coverage_Group,PK_Test_Coverage_Group_Term"
/>
<field includeFrom="Test_Coverage_Group_Term.Coverage_Group"
title="Coverage Group Name"
hidden="false"
prompt="Coverage group name"
xxxoptionDataSource="Test_Coverage_Group_Term"
optionDataSource="TestCoverageMetricView"
filter_outputs="Coverage_Group"
filter_sortSpecifier="Coverage_Group"
filter_groups="Coverage_Group"
filter_pickListProp_sortSpecifier="Coverage_Group"

/>

</fields>

</DataSource>

Here is one of the data sources that gets included via a long chain of included datasources. It contains the BenchName field used in the query.

<DataSource ID="Test_Bench" serverType="sql" tableName="Test_Bench">

<fields>
<field name="PK_Test_Bench" title="Bench ID" type="sequence" primaryKey ="true" canEdit="false" hidden="true" />
<field name="BenchName" title="Bench" type="text" lenght="63" width="150" canEdit="false" />
<field name="Revision" title="Revision" type="text" length="31" canEdit="false" />
<field name="State" title="State" type="text" length="63" prompt="One of the following states: Active, Inactive, Obsolete" >
<valueMap>
<value>Active</value>
<value>Inactive</value>
<value>Obsolete</value>
</valueMap>
</field>
<field name="CT_on_create" title="Created" type="datetime" canEdit="false" />
<field name="Logo_Path" title="Logo Path" type="text" length="1023" width="600" height="30" prompt="Bench logo - full path (URL) to image" browserSpellCheck="false" />
<field name="Description" title="Description" type="text" length="1023" width="600" height="40" prompt="Description of the Bench" />
<field name="HomeURL" title="Home URL" type="text" length="255" width="600" height="30" prompt="URL to the Bench's home on the web" browserSpellCheck="false"/>
<field name="HomeSite" title="Home Site" type="text" length="48" prompt="Geographic location where the bench is maintained"
sortField="SiteDescription"
valueField="SiteDescription"
optionDataSource="ISO_UIProjectSite"
/>

<field name="FK_BenchLead" title="Lead" type="integer" foreignKey="Users.PK_Users_Userid" relatedTableAlias="LeadTBL" joinType="outer" prompt="Bench Project Lead."
displayField="User_BenchLead"
foreignDisplayField="FullName"
valueField="PK_Users_Userid"
sortByField="FullName"
editorType="ComboBoxItem"
/>
<field name="FK_BenchMaintenance" title="Maintainer" type="integer" foreignKey="Users.PK_Users_Userid" relatedTableAlias="maintTBL" joinType="outer" prompt="Bench Project Maintainer."
displayField="User_Maint" foreignDisplayField="FullName" editorType="ComboBoxItem"
/>
<field name="FK_Alternate" title="Alternate" type="integer" foreignKey="Users.PK_Users_Userid" relatedTableAlias="altTBL" displayField="User_Alt" joinType="outer" prompt="Bench Project Alternate."
foreignDisplayField="FullName" editorType="ComboBoxItem"
/>

<field name="WvNotifyLeadEn" title="Notify Lead for Waivers" type="boolean" length="1" width="30" prompt="Enable notifications for the Bench Lead when waiver states change." />
<field name="WvNotifyMaintEn" title="Notify Maintianer for Waivers" type="boolean" length="1" width="30" prompt="Enable notifications for the Bench Maintainer when waiver states change." />
<field name="WvNotifyAltEn" title="Notify Alternate for Waivers" type="boolean" length="1" width="30" prompt="Enable notifications for the Bench Alternate when waiver states change." />
<field name="ExtraWaiverEmails" title="Extra Notification Waiver Emails" type="text" length="1023" width="600" height="50" prompt=" An additional list of emails to notify when a waiver state changes. Use to add those who are not one of the bench rolls (Lead, Maintainer, Alternate)"
hint="i.e. [&quot;Mailing.List@Broadcom.com&quot;, &quot;project.manager@Broadcom.com&quot;]" showHintInField="true" browserSpellCheck="false" >
<validators>
<validator type="regexp" validateOnChange="true" clientOnly="true" errorMessage="Must be in the syntax of a Javascript Object list of emails. i.e. [&quot;me@company.com&quot;, &quot;you@your_company.com&quot;]"
expression_validEmailAddressExample="^[a-zA-Z0-9.!#$%&amp;'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$"
expression="^ *\[ *(&quot;[a-zA-Z0-9.!#$%&amp;'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*&quot​( *, *(&quot;[a-zA-Z0-9.!#$%&amp;'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*&quot​)* *\] *$"
/>
</validators>
</field>


<field name="BenchRollsWaiverEmails" canEdit="false" hidden="true" customSelectExpression=" '['
+ CASE WHEN LeadTBL.Email IS NOT NULL AND WvNotifyLeadEn = 1 THEN '&quot;' + LeadTBL.Email + '&quot;' ELSE '' END
+ CASE WHEN maintTBL.Email IS NOT NULL AND WvNotifyMaintEn = 1 AND LeadTBL.Email IS NOT NULL AND WvNotifyLeadEn = 1 THEN ', ' ELSE '' END
+ CASE WHEN maintTBL.Email IS NULL OR WvNotifyMaintEn = 0 THEN '' ELSE '&quot;' + maintTBL.Email + '&quot;' END
+ CASE WHEN altTBL.Email IS NOT NULL AND WvNotifyAltEn = 1 AND ((maintTBL.Email IS NOT NULL AND WvNotifyMaintEn = 1 )OR (LeadTBL.Email IS NOT NULL AND WvNotifyLeadEn = 1)) THEN ', ' ELSE '' END
+ CASE WHEN altTBL.Email IS NULL OR WvNotifyAltEn = 0 THEN '' ELSE '&quot;' + altTBL.Email + '&quot;' END
+']' "
/>

<!-- related and display fields from other data sources -->

<field name="User_BenchLead" includeFrom="Users.FullName" includeVia="FK_BenchLead" hidden="true" />
<field name="User_Maint" includeFrom="Users.FullName" includeVia="FK_BenchMaintenance" hidden="true" />
<field name="User_Alt" includeFrom="Users.FullName" includeVia="FK_Alternate" hidden="true" />
<field name="BenchName_Revision" type="text" customSelectExpression="BenchName + ' ' + Revision " hidden="true" />

<field name="Email_BenchLead" includeFrom="Users.Email" title="BenchLead's Email" includeVia="FK_BenchLead" hidden="true" />
<field name="Email_Maint" includeFrom="Users.Email" title="Maintainer's Email" includeVia="FK_BenchMaintenance" hidden="true" />
<field name="Email_Alt" includeFrom="Users.Email" title="Alternate's Email" includeVia="FK_Alternate" hidden="true" />

</fields>

<operationBinding operationType="fetch" >
<orderClause>
CT_on_create DESC
</orderClause>
</operationBinding>

</DataSource>

There is no server side DMI logic , constructor or other server side logic that I have added to introduce template contexts or anything. Here are the logs to show the request. It should show how values are specified, or the end result of that anyway. I think that should be more then enough.

=== 2021-07-08 08:05:17,221 [-189] INFO RequestContext - URL: '/barr/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36': Safari with Accept-Encoding header
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Origin:http://10.75.224.44:39707
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Cookie:isc_cState=ready; GLog=%7B%0A%20%20%20%20trackRPC%3Atrue%2C%20%0A%20%20%20%20isc_pageURL%3A%22http%3A//10.75.224.44%3A39707/BARR.html%3FnoVersionCheck%26debugLevel%3D0%26autoLogin%3Dguin8%22%2C%20%0A%20%20%20%20isc_pageGUID%3A%228CD1178B-BFF7-46A6-B0B4-1CA6DCE6A139%22%2C%20%0A%20%20%20%20priorityDefaults%3A%7B%0A%20%20%20%20%20%20%20%20sgwtInternal%3A1%0A%20%20%20%20%7D%2C%20%0A%20%20%20%20defaultPriority%3A3%2C%20%0A%20%20%20%20left%3A1%2C%20%0A%20%20%20%20top%3A0%2C%20%0A%20%20%20%20width%3A1512%2C%20%0A%20%20%20%20height%3A753%0A%7D; JSESSIONID=1lgxbsa1p25ptwetus737t92a
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Accept:*/*
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Connection:keep-alive
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Referer:http://10.75.224.44:39707/BARR.html?noVersionCheck&debugLevel=0&autoLogin=guin8
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Host:10.75.224.44:39707
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Accept-Language:en-US,en;q=0.9
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Content-Length:6161
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - Header Name:Value pair: Content-Type:application/x-www-form-urlencoded; charset=UTF-8
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - session exists: 1lgxbsa1p25ptwetus737t92a
=== 2021-07-08 08:05:17,222 [-189] DEBUG IDACall - remote user: es941797
=== 2021-07-08 08:05:17,224 [-189] DEBUG XML - Parsed XML from (in memory stream): 1ms
=== 2021-07-08 08:05:17,224 [-189] DEBUG PoolableDataSourceFactory - Tried to create DataSource of type 'transaction' but null was returned
=== 2021-07-08 08:05:17,224 [-189] DEBUG PoolableDataSourceFactory - Created DataSource null of type 'transaction' in the pooling flow
=== 2021-07-08 08:05:17,224 [-189] DEBUG PoolableDataSourceFactory - Created DataSource 1140 of type 'Object' and assigned it to thread qtp1798269610-189
=== 2021-07-08 08:05:17,224 [-189] DEBUG PoolableDataSourceFactory - Created DataSource 1140 of type 'Object' in the pooling flow
=== 2021-07-08 08:05:17,224 [-189] DEBUG PoolableDataSourceFactory - Activated DataSource 1140 of type 'Object'
=== 2021-07-08 08:05:17,225 [-189] DEBUG PoolableDataSourceFactory - Created DataSource 1141 of type 'List' and assigned it to thread qtp1798269610-189
=== 2021-07-08 08:05:17,225 [-189] DEBUG PoolableDataSourceFactory - Created DataSource 1141 of type 'List' in the pooling flow
=== 2021-07-08 08:05:17,225 [-189] DEBUG PoolableDataSourceFactory - Activated DataSource 1141 of type 'List'
=== 2021-07-08 08:05:17,225 [-189] DEBUG PoolableDataSourceFactory - Tried to create DataSource of type 'elem' but null was returned
=== 2021-07-08 08:05:17,225 [-189] DEBUG PoolableDataSourceFactory - Created DataSource null of type 'elem' in the pooling flow
=== 2021-07-08 08:05:17,225 [-189] DEBUG RPCManager - Processing 1 requests.
=== 2021-07-08 08:05:17,226 [-189] DEBUG PoolableDataSourceFactory - Activated DataSource 244 of type 'TestCoverageMetricView'
=== 2021-07-08 08:05:17,226 [-189] DEBUG DSRequest - Caching instance 244 of DS 'TestCoverageMetricView' from DSRequest.getDataSource()
=== 2021-07-08 08:05:17,226 [-189] DEBUG DSRequest - Caching instance 244 of DS TestCoverageMetricView
=== 2021-07-08 08:05:17,226 [-189] DEBUG RPCManager - Request #1 (DSRequest) payload: {
criteria:{
operator:"and",
_constructor:"AdvancedCriteria",
criteria:[
{
fieldName:"Coverage_Group",
operator:"inSet",
value:[
"AVS",
"Core",
"H264",
"H265",
"MPEG2",
"Platform",
"Total",
"VP9"
]
},
{
fieldName:"FK_Test_Coverage_Report",
operator:"inSet",
value:[
"1621",
"1622"
]
},
{
fieldName:"Coverage_Category",
operator:"inSet",
value:[
"Statement",
"Decision",
"All Skips",
"Debug tools",
"Defensive coding",
"Incomplete feature",
"Unreachable code",
"Unmeasurable"
]
},
{
fieldName:"BenchName",
operator:"equals",
value:"Avd Bench"
},
{
fieldName:"BenchName",
operator:"equals",
value:"Avd Bench"
},
{
fieldName:"PK_Test_Project",
operator:"equals",
value:"18"
},
{
fieldName:"PK_Test_Session_Stream",
operator:"inSet",
value:[
77,
78,
98,
110,
107,
109,
108
]
}
]
},
operationConfig:{
dataSource:"TestCoverageMetricView",
repo:null,
operationType:"fetch",
textMatchStyle:"exactCase"
},
groupBy:[
"FK_Test_Coverage_Report",
"Coverage_Category",
"Coverage_Group"
],
summaryFunctions:{
PercentCovered:"sum",
Covered:"sum",
Total:"sum"
},
appID:"builtinApplication",
operation:"TestCoverageMetricView_fetch",
oldValues:{
operator:"and",
_constructor:"AdvancedCriteria",
criteria:[
{
fieldName:"Coverage_Group",
operator:"inSet",
value:[
"AVS",
"Core",
"H264",
"H265",
"MPEG2",
"Platform",
"Total",
"VP9"
]
},
{
fieldName:"FK_Test_Coverage_Report",
operator:"inSet",
value:[
"1621",
"1622"
]
},
{
fieldName:"Coverage_Category",
operator:"inSet",
value:[
"Statement",
"Decision",
"All Skips",
"Debug tools",
"Defensive coding",
"Incomplete feature",
"Unreachable code",
"Unmeasurable"
]
},
{
fieldName:"BenchName",
operator:"equals",
value:"Avd Bench"
},
{
fieldName:"BenchName",
operator:"equals",
value:"Avd Bench"
},
{
fieldName:"PK_Test_Project",
operator:"equals",
value:"18"
},
{
fieldName:"PK_Test_Session_Stream",
operator:"inSet",
value:[
77,
78,
98,
110,
107,
109,
108
]
}
]
}
}
=== 2021-07-08 08:05:17,226 [-189] INFO IDACall - Performing 1 operation(s) [UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36]
=== 2021-07-08 08:05:17,227 [-189] INFO DSRequest - Executing TestCoverageMetricView.fetch rows: 0->-1 with
criteria: {
operator:"and",
_constructor:"AdvancedCriteria",
criteria:[
{
fieldName:"Coverage_Group",
operator:"inSet",
value:["AVS", "Core", "H264", "H265", "MPEG2", "Platform", "Total", "VP9"]
},
{
fieldName:"FK_Test_Coverage_Report",
operator:"inSet",
value:["1621", "1622"]
},
{
fieldName:"Coverage_Category",
operator:"inSet",
value:["Statement", "Decision", "All Skips", "Debug tools", "Defensive coding",
"Incomplete feature","Unreachable code", "Unmeasurable"]
},
{fieldName:"BenchName", operator:"equals", value:"Avd Bench"},
{fieldName:"BenchName", operator:"equals", value:"Avd Bench"},
{fieldName:"PK_Test_Project", operator:"equals", value:"18"},
{
fieldName:"PK_Test_Session_Stream",
operator:"inSet",
value:[77, 78, 98, 110, 107, 109, 108]
}
]
}
=== 2021-07-08 08:05:17,227 [-189] DEBUG PoolableDataSourceFactory - Activated DataSource 294 of type 'Test_Coverage_Category_Term'
=== 2021-07-08 08:05:17,227 [-189] DEBUG Relation - Caching instance of toDS 'Test_Coverage_Category_Term' in the DSRequest map
=== 2021-07-08 08:05:17,227 [-189] DEBUG PoolableDataSourceFactory - Activated DataSource 297 of type 'Test_Coverage_Group_Term'
=== 2021-07-08 08:05:17,227 [-189] DEBUG Relation - Caching instance of toDS 'Test_Coverage_Group_Term' in the DSRequest map
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - DataSource TestCoverageMetricView is not in the pre-checked list, processing...
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - Processing security checks for DataSource Test_Coverage_Category_Term, field PK_Test_Coverage_Category_Term
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - DataSource Test_Coverage_Category_Term is not in the pre-checked list, processing...
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - Processing security checks for DataSource Test_Coverage_Category_Term, field Coverage_Category
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - DataSource Test_Coverage_Category_Term is not in the pre-checked list, processing...
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - Processing security checks for DataSource Test_Coverage_Group_Term, field PK_Test_Coverage_Group_Term
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - DataSource Test_Coverage_Group_Term is not in the pre-checked list, processing...
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - Processing security checks for DataSource Test_Coverage_Group_Term, field Coverage_Group
=== 2021-07-08 08:05:17,227 [-189] DEBUG DeclarativeSecurity - DataSource Test_Coverage_Group_Term is not in the pre-checked list, processing...
=== 2021-07-08 08:05:17,227 [-189] DEBUG AppBase - No userTypes defined, allowing anyone access to all operations for this application
=== 2021-07-08 08:05:17,227 [-189] DEBUG AppBase - No public zero-argument method named '_TestCoverageMetricView_fetch' found, performing generic datasource operation
=== 2021-07-08 08:05:17,229 [-189] INFO SQLDataSource - derived query:
SELECT
$defaultSelectClause
FROM
$defaultTableClause $defaultAnsiJoinClause
WHERE
$defaultWhereClause
GROUP BY
Test_Coverage_Metric.FK_Test_Coverage_Report, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group
=== 2021-07-08 08:05:17,229 [-189] DEBUG SQLDataSource - 244: Executing SQL query on 'PROD_COPY': SELECT Test_Coverage_Metric.FK_Test_Coverage_Report, SUM(Test_Coverage_Metric.Covered) AS Covered, SUM(Test_Coverage_Metric.Total) AS Total, SUM(cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total,0)) , 0) as decimal(30,2) )) AS PercentCovered, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group FROM Test_Coverage_Metric JOIN Test_Coverage_Category_Term ON Test_Coverage_Metric.FK_Test_Coverage_Category_Term = Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term JOIN Test_Coverage_Group_Term ON Test_Coverage_Metric.FK_Test_Coverage_Group_Term = Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term WHERE (((Test_Coverage_Group_Term.Coverage_Group IN ('AVS', 'Core', 'H264', 'H265', 'MPEG2', 'Platform', 'Total', 'VP9')) AND Test_Coverage_Group_Term.Coverage_Group IS NOT NULL) AND ((Test_Coverage_Metric.FK_Test_Coverage_Report IN (1621, 1622)) AND Test_Coverage_Metric.FK_Test_Coverage_Report IS NOT NULL) AND ((Test_Coverage_Category_Term.Coverage_Category IN ('Statement', 'Decision', 'All Skips', 'Debug tools', 'Defensive coding', 'Incomplete feature', 'Unreachable code', 'Unmeasurable')) AND Test_Coverage_Category_Term.Coverage_Category IS NOT NULL) AND (Test_Coverage_Metric.null = 'Avd Bench' AND Test_Coverage_Metric.null IS NOT NULL) AND (Test_Coverage_Metric.null = 'Avd Bench' AND Test_Coverage_Metric.null IS NOT NULL) AND (Test_Coverage_Metric.null = 18 AND Test_Coverage_Metric.null IS NOT NULL) AND ((Test_Coverage_Metric.null IN (77, 78, 98, 110, 107, 109, 108)) AND Test_Coverage_Metric.null IS NOT NULL)) GROUP BY Test_Coverage_Metric.FK_Test_Coverage_Report, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group
=== 2021-07-08 08:05:17,252 [-189] DEBUG PoolableSQLConnectionFactory - makeObject() created an unpooled Connection '85326423'
=== 2021-07-08 08:05:17,252 [-189] DEBUG SQLConnectionManager - Borrowed connection '85326423'
=== 2021-07-08 08:05:17,253 [-189] DEBUG SQLTransaction - Started new PROD_COPY transaction "85326423"
=== 2021-07-08 08:05:17,253 [-189] DEBUG SQLDataSource - Setting DSRequest as being part of a transaction
=== 2021-07-08 08:05:17,253 [-189] INFO SQLDriver - Executing SQL query on 'PROD_COPY' using connection '85326423':
SELECT
Test_Coverage_Metric.FK_Test_Coverage_Report, SUM(Test_Coverage_Metric.Covered) AS Covered, SUM(Test_Coverage_Metric.Total) AS Total, SUM(cast( ISNULL( (Test_Coverage_Metric.Covered * 100.0 / NULLIF(Test_Coverage_Metric.Total, 0)) , 0) as decimal(30, 2) )) AS PercentCovered, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group
FROM
Test_Coverage_Metric
JOIN
Test_Coverage_Category_Term
ON Test_Coverage_Metric.FK_Test_Coverage_Category_Term = Test_Coverage_Category_Term.PK_Test_Coverage_Category_Term
JOIN
Test_Coverage_Group_Term
ON Test_Coverage_Metric.FK_Test_Coverage_Group_Term = Test_Coverage_Group_Term.PK_Test_Coverage_Group_Term
WHERE
(
(
(
Test_Coverage_Group_Term.Coverage_Group IN (
'AVS', 'Core', 'H264', 'H265', 'MPEG2', 'Platform', 'Total', 'VP9'
)
)
AND Test_Coverage_Group_Term.Coverage_Group IS NOT NULL
)
AND (
(
Test_Coverage_Metric.FK_Test_Coverage_Report IN (
1621, 1622
)
)
AND Test_Coverage_Metric.FK_Test_Coverage_Report IS NOT NULL
)
AND (
(
Test_Coverage_Category_Term.Coverage_Category IN (
'Statement', 'Decision', 'All Skips', 'Debug tools', 'Defensive coding', 'Incomplete feature', 'Unreachable code', 'Unmeasurable'
)
)
AND Test_Coverage_Category_Term.Coverage_Category IS NOT NULL
)
AND (
Test_Coverage_Metric.null = 'Avd Bench'
AND Test_Coverage_Metric.null IS NOT NULL
)
AND (
Test_Coverage_Metric.null = 'Avd Bench'
AND Test_Coverage_Metric.null IS NOT NULL
)
AND (
Test_Coverage_Metric.null = 18
AND Test_Coverage_Metric.null IS NOT NULL
)
AND (
(
Test_Coverage_Metric.null IN (
77, 78, 98, 110, 107, 109, 108
)
)
AND Test_Coverage_Metric.null IS NOT NULL
)
)
GROUP BY
Test_Coverage_Metric.FK_Test_Coverage_Report, Test_Coverage_Category_Term.Coverage_Category, Test_Coverage_Group_Term.Coverage_Group
=== 2021-07-08 08:05:17,255 [-189] DEBUG DSRequest - freeOnExecute is false for request of type fetch on DataSource TestCoverageMetricView - not freeing resources!
=== 2021-07-08 08:05:17,256 [-189] WARN RequestContext - dsRequest.execute() failed:
com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near the keyword 'null'.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:259)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1547)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:857)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:757)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7344)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2713)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:224)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:204)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:659)
at com.isomorphic.sql.SQLDriver.getTransformedResults(SQLDriver.java:1112)
at com.isomorphic.sql.SQLDriver.executeQuery(SQLDriver.java:1532)
at com.isomorphic.sql.SQLDataSource.executeNativeQuery(SQLDataSource.java:586)
at com.isomorphic.sql.SQLDataSource.executeNativeQuery(SQLDataSource.java:580)
at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1833)
at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:441)
at com.isomorphic.sql.SQLDataSource.executeFetch(SQLDataSource.java:386)
at com.isomorphic.datasource.DataSource.execute(DataSource.java:2499)
at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:658)
at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:555)
at com.isomorphic.application.AppBase.execute(AppBase.java:498)
at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:3027)
at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:226)
at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:183)
at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:148)
at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:119)
at com.isomorphic.servlet.IDACall.doPost(IDACall.java:79)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:178)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at beer.server.NoCache.doFilter(NoCache.java:31)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:263)
at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:91)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at beer.server.DevelopmentAuthenticationFilter.doFilter(DevelopmentAuthenticationFilter.java:49)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:224)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)

[eschwegs]

I was able to stumble on to the source of the problem. You can tell me if this is intended functionality change with wonky side effects or a bug. For me, it is fatal for a lot of my current code.

I previously was able to get CubeGrid data by setting the groups and summaryFunctions in the requestProperties. With the 2021-07-05 libraries, the Where clause was being generated with DataSource includeFrom fields that were not being resolved properly. I was able to make the symptom disappear by also setting the outputs. I suspect other issues are also happening without the outputs also being explicitly specified as I also saw screwy behavior experimenting by adding just groupBy and not the summary field names. Previously smartGWT took care of deriving the outputs on it's own.

i.e.
// set up requestProperties
requestProperties.setGroupBy(groupBy.toArray(new String[groupBy.size()]));
requestProperties.setSummaryFunctions(summaryFunctions);


// debug Now required for 12.1 2021-07-05 smartGWT library ???
Set<String> outputs = new HashSet<>();
outputs.addAll(groupBy);
outputs.addAll(summaryFunctions.keySet());
requestProperties.setOutputs(outputs.stream().collect(Collectors.joining(",")));

[Blama]

Hi Isomorphic, Hi eschwegs,

this thread recently changed where+outputs in order to fix a bug. Perhaps it's related.

Hope this helps & Best regards
Blama

[Isomorphic]

eschwegs setting outputs manually is not required, so this seems to be a bug.

Despite the issue is more or less clear, we do not yet understand what's causing it exactly and in order to fix it in a correct way we need to reproduce it. Your provided information is not complete enough, so please:
- provide a completely standalone use case (ideally a server-side code that runs on its own) or
- strip your setup to reduce the amount of datasources involved to the bare minimum that would still showcase the issue (single *missed* join is enough to get going) or
- provide all of the missing parts of your current setup (at least all datasources involved)

[Isomorphic]

Just to note: we took your resources, set it up in a project, stubbed things out until we could run as much of it as was feasible, and that didn't reproduce the issue. That's why we need more details to proceed.

[eschwegs]

I'll try and simplify it. There is a lot going on in my description as several data sources are being included. I was hoping you would be able to see it on your end so I wouldn't have to create a stand alone example.

[eschwegs]

I was able to simplify the example to 2 data sources, one including from the other. I have comments in the code that instruct you to either uncomment or comment the code that sets the outputs. That makes it pass or fail when the request originates from the client or when executed from a server side SQLDataSource.executeFetch(...) call. Ironically, my first attempt to put it all together in the main data source as a groovy script fails regardless. Thus, I'll include all three methods as they may be telling. Each example is executed from a button.

DataSources:

<DataSource ID="ExampleFieldResolutionBug" serverType="sql" tableName="Test_Coverage_Metric" >
<fields>
<field name="PK_Test_Coverage_Metric" type="sequence" primaryKey="true" />
<field name="FK_Test_Coverage_Report" type="integer" foreignKey="ExampleIncludeBug.PK_Test_Coverage_Report" />
<field includeFrom="ExampleIncludeBug.PK_Test_Coverage_Report" />
</fields>

<operationBinding operationType="fetch" operationId="bugExample">
<script language="groovy">
import com.isomorphic.criteria.AdvancedCriteria;
import com.isomorphic.criteria.DefaultOperators;
import com.isomorphic.criteria.criterion.SimpleCriterion;
import com.isomorphic.datasource.DSRequest;

// get SessionToolsView Session Tools records
DSRequest myDsRequest = new DSRequest("ExampleFieldResolutionBug", "fetch");

AdvancedCriteria crit = new AdvancedCriteria("and",
new SimpleCriterion("FK_Test_Coverage_Report", DefaultOperators.Equals.getID(), "1621"),
new SimpleCriterion("PK_Test_Coverage_Report", DefaultOperators.Equals.getID(), "1621")
);
myDsRequest.setAdvancedCriteria(crit);

// Commenting these 2 line out or leaving them in seems to make no difference, we see the null failure reguardless
//List outputs = Arrays.asList("FK_Test_Coverage_Report");
//dsRequest.setOutputs(outputs);

List groupBy = Arrays.asList("FK_Test_Coverage_Report");
myDsRequest.setGroupBy(groupBy);

return myDsRequest.execute();
</script>
</operationBinding>

</DataSource>

<DataSource ID="ExampleIncludeBug" serverType="sql" tableName="Test_Coverage_Report">
<fields>
<field name="PK_Test_Coverage_Report" type="sequence" primaryKey="true" />
</fields>
</DataSource>

a random datasource I had a DMI code lying around to add to.

<DataSource ID="randomDS" serverType="sql" tableName="unimportant" serverConstructor="barr.server.randomDMI" >
...
</DataSource

SQLDataSource snippet:

public class randomDMI extends SQLDataSource {
@Override
public DSResponse executeFetch(DSRequest dsrequest) throws Exception {

   if (opID.equals("bugExample")) {

      // get SessionToolsView Session Tools records
      DSRequest myDsRequest = new DSRequest("ExampleFieldResolutionBug", "fetch");

      AdvancedCriteria crit = new AdvancedCriteria("and", new Criterion[] {
         new SimpleCriterion("FK_Test_Coverage_Report", DefaultOperators.Equals.getID(), "1621"),
         new SimpleCriterion("PK_Test_Coverage_Report", DefaultOperators.Equals.getID(), "1621"),
      });
      myDsRequest.setAdvancedCriteria(crit);

      List<String> outputs = Arrays.asList("FK_Test_Coverage_Report");

      // Commenting this line out makes it such that the PK_Test_Coverate_Report field does not get resolved in the criteria
      //myDsRequest.setOutputs(outputs);

      List<String> groupBy = Arrays.asList("FK_Test_Coverage_Report");
      myDsRequest.setGroupBy(groupBy);

      return myDsRequest.execute();
   } 
}
}

Layout snippet

Button startGWTLibBugButton = new Button("smartGWT 202107-07-05 client generated fetch");
startGWTLibBugButton.setAutoFit(true);
startGWTLibBugButton.addClickHandler((ClickEvent event) -> {

DSRequest requestProperties = new DSRequest();

String[] groupBy = {"FK_Test_Coverage_Report"};
requestProperties.setGroupBy(groupBy);

// Commenting this line out makes it such that the PK_Test_Coverate_Report field does not get resolved in the criteria
//requestProperties.setOutputs("FK_Test_Coverage_Report");

AdvancedCriteria criteria = new AdvancedCriteria(OperatorId.AND, new Criterion[] {
new Criterion("FK_Test_Coverage_Report", OperatorId.EQUALS, "1621"),
new Criterion("PK_Test_Coverage_Report", OperatorId.NOT_NULL), //includeFrom="Test_Coverage_Report.PK_Test_Coverage_Report"
});

DataSource.get("ExampleFieldResolutionBug").fetchData(criteria, null, requestProperties);

});
yourLayout.addMember(startGWTLibBugButton);

Button startGWTLibBugButton2 = new Button("smartGWT 202107-07-05 server side DMI generated fetch");
startGWTLibBugButton2.setAutoFit(true);
startGWTLibBugButton2.addClickHandler((ClickEvent event) -> {

DSRequest requestProperties = new DSRequest();
requestProperties.setOperationId("bugExample");
DataSource.get("randomDS").fetchData(null, null, requestProperties);

});
yourLayout.addMember(startGWTLibBugButton2);

Button startGWTLibBugButton3 = new Button("smartGWT 202107-07-05 groovy script generated fetch");
startGWTLibBugButton3.setAutoFit(true);
startGWTLibBugButton3.addClickHandler((ClickEvent event) -> {
DSRequest requestProperties = new DSRequest();
requestProperties.setOperationId("bugExampleWithOutputs");
DataSource.get("ExampleFieldResolutionBug").fetchData(null, null, requestProperties);
});
yourLayout.addMember(startGWTLibBugButton3);

Hope this helps

[Isomorphic]

Thank you for the information! This is fixed now and will be available for download in nightly builds since July 15 (today). Let us know please how it worked for you.

[Blama]

Hi Isomorphic,

did you create a automated test for my outputs issue linked before or did test it manually? Not that it regresses because of this fix. I assume they are related?

Thank you & Best regards
Blama

[Isomorphic]

These issues seem related, but they actually aren't. Anyway they both are covered in automated tests, so they will not be accidentally regressed in the future.

[Blama]

Perfect, thank you.

[eschwegs]

Thanks, I've tested it on my end and now things work.