Announcement

**Isomorphic** · 1 Apr 2020, 11:52

You've got various wrong attempts in your .pdf but what you've posted here is actually correct.

You report that things are now working as expected in forms, but somehow the grid is not doing what you expect. There are a few ways you could break this:

1. you specified fields on the ListGrid that override the correct DS definition

2. the data could be bad - no actual values for retro_desc for the values of retro_id in the "table" DataSource

3. you've got clientOnly="false" - that's an unnecessary setting but if you had messed up another aspect of your environment (deleted or didn't install isomorphic/system/schema), then this setting might actually break the includeFrom functionality by making the DataSource actually clientOnly. See if there are requests to the server in the RPC tab of the Developer Console. If there aren't, that's the problem, remove the useless setting.

**kgunderson** · 2 Apr 2020, 05:08

...but what I've posted is not working as expected. The form almost works as expected; the retro_id contents show up in the field and the retro_desc shows up in the dropdown selection of the SelectItem. The listgrid does not work as expected; the retro_id shows up in the listgrid column, and per the showcase and with the datasource above, I expect the retro_desc.

1. No. The listgridProperties is as simple as can be.

Code:

 isc.ClassFactory.defineClass("CatastrophesPanel", BaseBodyPanel).addProperties({
      panelTitle: "Catastrophes",
      
      myListGridProperties: {
          dataSource: "rscReacctCatastrophes",
          autoFetchData: true
      },
        formProperties: {
          dataSource: "rscReacctCatastrophes",
          fields: [
              {name: "cat_desc", width: 380},
              {name: "retro_id", width: 363},
              {name: "alt_retro_id", width: 363},
              {name: "is_marked"}
          ]
      },
      newButtonProperties: {
          click : function() {
              this.panel.form.getItem("cat_desc").setCanEdit(true);
              this.panel.newClicked();
          }
      },
      editButtonProperties: {
          click : function() {
              this.panel.form.getItem("cat_desc").setCanEdit(false);
              this.panel.editClicked();
          }
      },
      saveButtonProperties: {
          click: function() {
              this.panel.form.getItem("cat_desc").setCanEdit(false);
              this.panel.saveClicked();
          }
      },
      cancelButtonProperties: {
          click: function() {
              this.panel.form.getItem("cat_desc").setCanEdit(false);
              this.panel.cancelClicked();
          }
      },  
      InitWidget : function() {
          this.Super("initWidget", arguments);
      }
  });

2. No. Otherwise the form would be off too, correct?
3. No. See #2.... and there are requests to the server. I will remove and retest though.

The 'current state' in the pdf is the one that comes the closest to the showcase. The listgrid is still incorrect; the retro_id shows up in the grid column. The form works as expected; the retro_desc shows up in the field and in the dropdown of the SelectItem/ComboBox.

The current state datasource looks like this:

Code:

 <DataSource
     ID="rscReacctCatastrophes"
     serverType="sql"
     serverConstructor="com.reacctsystems.reacct.srv.CatastrophesDataSource"
     appserverDomain="web"
     generatedBy="v11.104.10639"
     clientOnly="false"
     testFileName="test_data/Catastrophes.data.csv"
  >
     <fields>
        <field name="cat_id" type="integer" format="########0" title="Catastrophe" primaryKey="true" hidden="true" autoGenerated="true" />
        <field name="cat_desc" type="text" length="200" title="Description" />
        <field name="retro_id" type="integer" format="######0" title="RetroID" editorType="SelectItem" foreignKey="rscReacctProgramYearsThin.retro_id" foreignDisplayField="retro_desc" />
        <field name="alt_retro_id" type="integer" format="######0" title="AltRetroID" detail="true" editorType="SelectItem" foreignKey="rscReacctProgramYearsThin.retro_id" foreignDisplayField="retro_desc" />
        <field name="is_marked" type="checkbox" title="Marked?" />
     </fields>
  </DataSource>

Thanks for your willingness to help!

Regards,
K.

**kgunderson** · 2 Apr 2020, 05:26

#3 There was no change when I removed the clientOnly and testFileName attributes.

This is what I am seeing. Below is a picture of the screen with the 'current state' datasource (notice the retro_id in the listgrid column, and the retro_desc in the form... the dropdown of the selectitem is correct too):

Here's a picture of the screen with the 'showcase' datasource (both the listgrid and the form are now broken... the dropdown of the selectitem is correct):

Where I want to get to is to have the retro_desc in the listgrid column, the retro_desc in the formfield, and the retro_desc in the dropdown. The closest I've come is with the current state datasource which gives me two out of the three.

Regards,
K.

**Isomorphic** · 2 Apr 2020, 10:56

Get rid of your current state and return to what you posted here, which as we've covered is correct (and matches the docs, and the Showcase).

Then, take a look at the server log and the response for the request from the grid. You should see a join being performed on the server, which populates the hidden retro_desc field, and you should see the data coming back in the RPC tab with retro_desc populated. Then the grid will show it.

If you don't, we need to look at how you broke the server behavior. You do have a serverConstructor declared, so maybe there's something in there that turned off the default includeFrom behavior?

**kgunderson** · 2 Apr 2020, 12:25

Here's the datasource (ala showcase):

Code:

 <DataSource
     ID="rscReacctCatastrophes"
     serverType="sql"
     serverConstructor="com.reacctsystems.reacct.srv.CatastrophesDataSource"
     appserverDomain="web"
     generatedBy="v11.104.10639"
  >
     <fields>
        <field name="cat_id" type="integer" format="########0" title="Catastrophe" primaryKey="true" hidden="true" autoGenerated="true" />
        <field name="cat_desc" type="text" length="200" title="Description" />
        <field name="retro_id" type="integer" format="######0" title="RetroID" editorType="SelectItem" foreignKey="rscReacctProgramYearsThin.retro_id" displayField="retro_desc" />
        <field includeFrom="rscReacctProgramYearsThin.retro_desc" hidden="true" />
        <field name="is_marked" type="checkbox" title="Marked?" />
     </fields>
  </DataSource>

...to be complete, here is the rscReacctProgramYearsThin datasource:

Code:

 <DataSource
     ID="rscReacctProgramYearsThin"
     serverType="sql"
     serverConstructor="com.reacctsystems.reacct.srv.ProgramYearsThinDataSource"
     appserverDomain="web"
     generatedBy="v11.104.10683"
  >
     <fields>
        <field name="retro_id" type="integer" title="RetroID" primaryKey="true" />
        <field name="retro_desc" type="text" length="200" title="RetroIDDesc" />
     </fields>
  </DataSource>

Here's the incorrect screen (the retro_id is being incorrectly shown in the listgrid and form fields; the retro_desc is correctly shown in the selectitem dropdown):

Here's the two server fetch requests. The java server code parses the request and makes an appserver call.

Code:

=== 2020-04-02 16:08:34,271 [ec-9] DEBUG RPCManager - Processing 2 requests.
=== 2020-04-02 16:08:34,272 [ec-9] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
    },
    operationConfig:{
        dataSource:"rscReacctCatastrophes",
        repo:null,
        operationType:"fetch",
        textMatchStyle:"substring"
    },
    startRow:0,
    endRow:83,
    componentId:"isc_CatastrophesPanel_0_myListGrid",
    appID:"builtinApplication",
    operation:"rscReacctCatastrophes_fetch",
    oldValues:{
    }
}
=== 2020-04-02 16:08:34,273 [ec-9] DEBUG RPCManager - Request #2 (DSRequest) payload: {
    criteria:{
    },
    operationConfig:{
        dataSource:"rscReacctProgramYearsThin",
        repo:null,
        operationType:"fetch",
        textMatchStyle:"startsWith"
    },
    componentId:"isc_PickListMenu_0",
    appID:"builtinApplication",
    operation:"rscReacctProgramYearsThin_fetch",
    oldValues:{
    }
}
=== 2020-04-02 16:08:34,273 [ec-9] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2020-04-02 16:08:34,273 [ec-9] DEBUG DeclarativeSecurity - DataSource rscReacctCatastrophes is not in the pre-checked list, processing...
=== 2020-04-02 16:08:34,274 [ec-9] DEBUG DeclarativeSecurity - Processing security checks for DataSource rscReacctProgramYearsThin, field retro_id
=== 2020-04-02 16:08:34,274 [ec-9] DEBUG DeclarativeSecurity - DataSource rscReacctProgramYearsThin is not in the pre-checked list, processing...
=== 2020-04-02 16:08:34,275 [ec-9] DEBUG DeclarativeSecurity - Processing security checks for DataSource rscReacctProgramYearsThin, field retro_desc
=== 2020-04-02 16:08:34,275 [ec-9] DEBUG DeclarativeSecurity - DataSource rscReacctProgramYearsThin is not in the pre-checked list, processing...
=== 2020-04-02 16:08:34,276 [ec-9] DEBUG AppBase - [builtinApplication.rscReacctCatastrophes_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2020-04-02 16:08:34,276 [ec-9] DEBUG AppBase - [builtinApplication.rscReacctCatastrophes_fetch] No public zero-argument method named '_rscReacctCatastrophes_fetch' found, performing generic datasource operation
02-Apr-2020 16:08:34.276 INFO [http-nio-28080-exec-9] com.reacctsystems.reacct.srv.GenericDataSource.executeFetch Request ID: rscReacctCatastrophes Domain: web
02-Apr-2020 16:08:34.280 INFO [http-nio-28080-exec-9] com.reacctsystems.reacct.srv.GenericDataSource.executeFetch URL: AppServerDC://localhost:20999/rsc_hcc Service: rsc_hcc
=== 2020-04-02 16:08:34,292 [ec-9] INFO  DSResponse - DSResponse: List with 83 items
=== 2020-04-02 16:08:34,292 [ec-9] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2020-04-02 16:08:34,293 [ec-9] DEBUG DeclarativeSecurity - DataSource rscReacctProgramYearsThin is not in the pre-checked list, processing...
=== 2020-04-02 16:08:34,294 [ec-9] DEBUG AppBase - [builtinApplication.rscReacctProgramYearsThin_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2020-04-02 16:08:34,294 [ec-9] DEBUG AppBase - [builtinApplication.rscReacctProgramYearsThin_fetch] No public zero-argument method named '_rscReacctProgramYearsThin_fetch' found, performing generic datasource operation
02-Apr-2020 16:08:34.295 INFO [http-nio-28080-exec-9] com.reacctsystems.reacct.srv.GenericDataSource.executeFetch Request ID: rscReacctProgramYearsThin Domain: web
02-Apr-2020 16:08:34.295 INFO [http-nio-28080-exec-9] com.reacctsystems.reacct.srv.GenericDataSource.executeFetch URL: AppServerDC://localhost:20999/rsc_hcc Service: rsc_hcc
=== 2020-04-02 16:08:34,310 [ec-9] INFO  DSResponse - DSResponse: List with 95 items
=== 2020-04-02 16:08:34,311 [ec-9] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2020-04-02 16:08:34,311 [ec-9] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2020-04-02 16:08:34,315 [ec-9] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2020-04-02 16:08:34,327 [ec-9] INFO  Compression - /reacct/isomorphic/IDACall: 12620 -> 2558 bytes

**Isomorphic** · 2 Apr 2020, 12:35

Wait, what? What does this mean:

Code:

Here's the two server fetch requests. The java server code parses the request and makes an appserver call.

You declared your DataSource to be type "sql". Are you using our SQL connector, or are you actually just providing the data yourself?

**kgunderson** · 2 Apr 2020, 13:18

We are providing the data ourselves through an appserver layer. The CatastrophesDataSource.java extends our GenericDataSource that then extends com.isomorphic.datasource.SQLDataSource. GenericDataSource defines executeFetch(), executeAdd(), et al.

The GenericDataSource parses the DSRequest and makes an OpenEdge appserver call returning a ResultSet which then gets parsed and formed into a DSResponse.

Regards,
K.

**Isomorphic** · 2 Apr 2020, 13:50

OK :) No mysteries here at all then!

If you look at the docs for includeFrom, our DataSources implement it by either a SQL join (SQL, JPA, Hibernate) or via doing a request to a related DataSource, but you're just wiping out all the default functionality, so of course, nothing happens...

Since you wiped out all the default functionality, you need to implement the behavior: fill in the retro_desc included field by whatever means you can. Hopefully your API to OpenEdge allows you to ask for that data to be done efficiently in a single request (similar to a SQL join), otherwise, you'll have to implement it as we describe for the fallback strategy: do a separate request and merge the data.

**kgunderson** · 2 Apr 2020, 17:28

0. The tomcat console correctly shows two requests from the ListGrid with non-zero returned records: rscReacctAppCatastrophes and rscReacctAppProgramyearsThin.
1. What do you mean by "or via doing a request to a related DataSource"? I thought I was doing that. Do you have documentation for that?
2. Going back to the PDF and using the "current state" datasource below, I am 2/3s of the way there with the same two datasources. The formfield and selectitem dropdown populate correctly. The listgrid column does not. So all the data is on the client.

Code:

 <DataSource
     ID="rscReacctCatastrophes"
     serverType="sql"
     serverConstructor="com.reacctsystems.reacct.srv.CatastrophesDataSource"
     appserverDomain="web"
     generatedBy="v11.104.10639"
 >     <fields>
        <field name="cat_id" type="integer" format="########0" title="Catastrophe" primaryKey="true" hidden="true" autoGenerated="true" />
        <field name="cat_desc" type="text" length="200" title="Description" />
        <field name="retro_id" type="integer" format="######0" title="RetroID" editorType="SelectItem" foreignKey="rscReacctProgramYearsThin.retro_id" foreignDisplayField="retro_desc" />
       <field name="is_marked" type="checkbox" title="Marked?" />     </fields>
  </DataSource>

Still out of luck?

Regards,
K.

**Isomorphic** · 2 Apr 2020, 20:54

So again: includeFrom is a feature of the framework for SQL/HB/JPA DataSources. You are completely replacing the logic of those DataSources, which means, the features that those DataSources provide by using SQL cannot operate at all, because while you technically extended a SQLDataSource, you have no SQL tables and are not allowing it to perform SQL queries at all. So obviously, it cannot provide the features it documents, such as includeFrom.

So you are not 2/3 of the way there - you are at zero. What you need to do is provide an equivalent implementation of the includeFrom feature, that is, the equivalent of a SQL join, based on the OpenEdge APIs you have available.

When we mentioned "or via doing a request to a related DataSource", this is already covered in the includeFrom docs. That's just a standard way of doing the equivalent of a SQL join, if you instead just have two entities you can fetch from and can't do a true join.

We're not really sure what part is not being understood, so, we have to start here: do you know what a SQL join does? When we say that includeFrom does a SQL join to get data from a related table, do just that part make sense to you, in terms of the data that would be retrieved and that the client needs? If so, then you should be able to look at the OpenEdge API, and look for a way to request a join.

**kgunderson** · 3 Apr 2020, 09:32

I am familiar with SQL joins.

When you say "our DataSources implement it by either a SQL join (SQL, JPA, Hibernate) or via doing a request to a related DataSource", I understood that to mean I can either go the route of implementing the includeFrom (SQL Join) in our java server code OR via doing a request to a related DataSource. That is to say, there are two methods SmartClient uses to handle the includeFrom functionality. I wanted to explore the later... is that something I can control in the datasource json file? Maybe flip a switch and Isomorphic uses a related-datasource method instead of a join. What would it mean to put "generic" as a serverType? Ultimately... can I avoid making changes to the java server code? It's pretty lean right now and performance is really good.

To clarify, our java server code extends SQLDataSource and your documentation says SQLDataSource uses either SQL Joins OR a request to a related datasource. It was unclear to me that your implementation of the SQLDataSource did not utilize that second method to achieve the includeFrom functionality when, say, there was a serverConstructor involved. That is NOT in the includeFrom documentation. I was asking for clarity, hoping for a "try this" from Isomorphic or, better yet, a small chunk of code.

I am not at zero. Without any changes to our java server code to manage the includeFrom and SQL Join, and using the foreignKey without the includeFrom, I can WITH THE SAME 'CURRENT STATE' DATASOURCE have a correctly functioning SelectItem form field. I was asking for clarity on why it was working for one object and not another.... again, exploring that, looking for a "try this" from isomorphic.

Still looking for friendly suggestions on moving forward.

Regards,
K.

**Isomorphic** · 3 Apr 2020, 17:24

The SelectItem works because it doesn't need the join to be functional. See the docs for optionDataSource and fetchMissingValues for how this subsystem works. Your includeFrom field could be entirely absent and it would still work.

No, using serverConstructor doesn't inherently wipe out functionality. As we cover in the QuickStart Guide, serverConstructor provides a means of subclassing the built-in data connectors (SQLDataSource et al). When you subclass and override methods, it doesn't matter if you're using a SQLDataSource or any other Java class, if you override a method on a class and don't call Super then you've wiped out the functionality of the superclass.

So, it doesn't really matter whether SQLDataSource would have performed this join via a true SQL join or via two requests, none of that code is active because you overrode the fetch operation completely.

It doesn't look like you're using any of the functionality of the SQLDataSource - you're not providing a DB, or tables, or allowing it to do queries - nothing. So we'd recommend serverType="generic" (or just omitting the type) as saying serverType="sql" is basically just misleading. You had us thinking you were actually using SQL, for one.

As we previously covered, our advice (and it's very good advice!) is to look through the OpenEdge docs to see how you can request a join. This is Oracle, they understand performance, it's almost certainly there.

There is no special sauce in our implementation of a crude in-memory join for implementing includeFrom when a true SQL join isn't available. It's just the brute force method of fetching from one DS, getting all the PK values, fetching display values from the other DS, merging results. Extremely inefficient, turns every fetch into two *serial* operations, but the best you can do when a true join isn't available.

You definitely don't want to be doing this as it could literally halve (or worse) your server scalability. So again, that's why our advice was, and continues to be: look up how to request a join from OpenEdge.

Announcement

foreignKey datasource fields....

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment