DataSourceField.fetch Log Message

[aderosso]

When our application is starting up (loading modules and dataSources), we see this log message for a couple different fields:

2024-05-07 17:19:09.012 INFO com.isomorphic.datasource.DSRequest:235 - Executing DataSourceField.fetch rows: 0->-1 with criteria: {name:"poNumber"}

I'm not sure what's causing this, or really what a "DataSourceField.fetch" is. We see the usual message from DSRequest "Executing [Datasource nam].fetch" message, but not "DataSourceField.fetch". Any idea what this is?

[Isomorphic]

This is odd, because the DataSourceField DataSource is an internal DataSource used only for deserializing .ds.xml files. However the contents of the request include data that is clearly from your application.

First step would be to set a breakpoint and look at the call stack to see how your application is triggering such a request.

[aderosso]

I figured this out, we see this while a datasource is being loaded (datasources.jsp with sc:loadDS tag). This has something to do with fields in our datasources where the type is a simpleType. We see one log message per simlyType type if that makes sense.

We have 6 different simpleTypes defined in our application, we get 6 log messages, one for the first time that simpleType is encountered in one of our datasources.

This isn't causing an issue, I just wanted to figure out what this meant.

[Isomorphic]

That's interesting, but still doesn't explain a DSRequest against the internal DataSourceField DataSource using application-specific data - that is not an expected result of using SimpleTypes. So even though there's no obvious problem caused by this, we would still recommend the approach above, of using a breakpoint to figure out where the request is coming from.

[aderosso]

Sorry, should have mentioned, I did figure this out using a break point. The request is from our bootstrap page with including the datasources.jsp file:

<script src="./datasources.jsp" type="application/javascript"></script>

Our datasources.jsp file looks like this:

<%@ page contentType="application/javascript; charset=UTF-8" pageEncoding="UTF-8" %>
    <%@ taglib uri="http://www.smartclient.com/taglib" prefix="sc" %>

    <%-- Simple Types for DS First --%>
    <sc:loadDS ID="codeType"/>
    <sc:loadDS ID="currencyType"/>
    <sc:loadDS ID="measurementType"/>
    <sc:loadDS ID="qtyType"/>
    <sc:loadDS ID="requiredText"/>
    <sc:loadDS ID="textType"/>

    <sc:loadDS ID="AllProjectMaterial"/>
    <sc:loadDS ID="AlternateHardware"/>
    <sc:loadDS ID="AlternateHardwareItem"/>
    <sc:loadDS ID="AlternateHardwareItemAttribute"/>
    ...

That's how I figured this out. I put the breakpoint within the DSRequest class, it stopped 1 time for each simple type, the first time that simple type was encountered in our datasources. Each stop from the breakpoint was from within the same request from ./datasources.jsp.

[Isomorphic]

Well, that means the fetch is related to the SimpleType, but it's still inexplicable that this results in a fetch against the internal DataSourceField schema, with application-specific data. We would recommend inspecting the call stack from the breakpoint: you will likely find some application code in the callstack, maybe some kind of dubious override of internals, which is initiating this fetch - perhaps because it tries to do something for all DataSources, and doesn't realize this is a SimpleType being loaded.

Secondly, why do you have a datasources.jsp like this? It seems to do the same thing that dataSourceLoader would do, only this way, it would be a bit slower. Are you pre-compiling or something? Note that doing so would break Declarative Security, which delivers different DataSources to different users based on their roles.

[aderosso]

Here's the filtered stack trace:

execute:3008, DSRequest (com.isomorphic.datasource)
70 hidden frames
doFilterInternal:49, MDCFilter (com.assaabloy.protech.server.web.filter)
9 hidden frames
doFilter:45, LoginRequiredFilter (com.assaabloy.protech.server.authentication)
87 hidden frames

Here's the full stack trace:

execute:3008, DSRequest (com.isomorphic.datasource)
fetchById:8357, DataSource (com.isomorphic.datasource)
fetchById:8306, DataSource (com.isomorphic.datasource)
fetchStoredRecord:188, StoredRecordHandler (com.isomorphic.velocity)
size:234, StoredRecordHandler (com.isomorphic.velocity)
putMapEntries:503, HashMap (java.util)
<init>:492, HashMap (java.util)
prepareCombinedRecord:91, EditedRecordHandler (com.isomorphic.velocity)
put:177, EditedRecordHandler (com.isomorphic.velocity)
setResultingValue:331, ValidationContext (com.isomorphic.datasource)
validateField:193, DefaultValidators (com.isomorphic.util)
validateValue:119, SimpleType (com.isomorphic.datasource)
create:85, SimpleType (com.isomorphic.datasource)
validateFieldValue:2760, BasicDataSource (com.isomorphic.datasource)
validateFieldValue:2648, BasicDataSource (com.isomorphic.datasource)
elementAsRecord:2478, BasicDataSource (com.isomorphic.datasource)
toRecord:1769, BasicDataSource (com.isomorphic.datasource)
toRecords:1753, BasicDataSource (com.isomorphic.datasource)
toRecords:1714, BasicDataSource (com.isomorphic.datasource)
create:2646, DataSource (com.isomorphic.datasource)
validateFieldValue:2760, BasicDataSource (com.isomorphic.datasource)
validateFieldValue:2648, BasicDataSource (com.isomorphic.datasource)
elementAsRecord:2356, BasicDataSource (com.isomorphic.datasource)
toRecord:1769, BasicDataSource (com.isomorphic.datasource)
toRecords:1728, BasicDataSource (com.isomorphic.datasource)
recordsFromXML:2483, DataSource (com.isomorphic.datasource)
toDSRecords:531, XML (com.isomorphic.xml)
toDSRecords:517, XML (com.isomorphic.xml)
loadObjectFromFile:717, XML$RecordsFromXMLCache (com.isomorphic.xml)
getObjectFromFile:167, ProcessedFileCache (com.isomorphic.store)
loadCacheableDSRecords:729, XML (com.isomorphic.xml)
loadInstance:218, DataStructCache (com.isomorphic.store)
loadDS:593, DataSource (com.isomorphic.datasource)
forName:562, DataSource (com.isomorphic.datasource)
makeUnpooledObject:127, PoolableDataSourceFactory (com.isomorphic.datasource)
makeObject:152, PoolableDataSourceFactory (com.isomorphic.datasource)
borrowObject:1220, GenericKeyedObjectPool (org.apache.commons.pool.impl)
borrowObject:90, PoolManager (com.isomorphic.pool)
getDataSource:204, DataSourceManager (com.isomorphic.datasource)
getDataSource:121, DataSourceManager (com.isomorphic.datasource)
get:249, DataSourceManager (com.isomorphic.datasource)
getIncludeFromInfo:9175, DataSource (com.isomorphic.datasource)
init:670, BasicDataSource (com.isomorphic.datasource)
init:302, SQLDataSource (com.isomorphic.sql)
initialize:1076, DataSource (com.isomorphic.datasource)
fromConfig:248, BasicDataSource (com.isomorphic.datasource)
fromConfig:1047, DataSource (com.isomorphic.datasource)
loadDS:660, DataSource (com.isomorphic.datasource)
forName:562, DataSource (com.isomorphic.datasource)
makeUnpooledObject:127, PoolableDataSourceFactory (com.isomorphic.datasource)
makeObject:152, PoolableDataSourceFactory (com.isomorphic.datasource)
borrowObject:1220, GenericKeyedObjectPool (org.apache.commons.pool.impl)
borrowObject:90, PoolManager (com.isomorphic.pool)
getDataSource:204, DataSourceManager (com.isomorphic.datasource)
getDataSource:121, DataSourceManager (com.isomorphic.datasource)
outputDSAsJS:131, LoadDSTag (com.isomorphic.taglib)
outputJS:103, LoadDSTag (com.isomorphic.taglib)
doStartTag:59, LoadDSTag (com.isomorphic.taglib)
_jspx_meth_sc_005floadDS_005f16:23, datasources_jsp (org.apache.jsp.titan)
_jspService:23, datasources_jsp (org.apache.jsp.titan)
service:70, HttpJspBase (org.apache.jasper.runtime)
service:590, HttpServlet (javax.servlet.http)
service:466, JspServletWrapper (org.apache.jasper.servlet)
_serviceJspFile:379, JspServlet (org.apache.jasper.servlet)
serviceJspFile:40014, JspServlet (org.apache.jasper.servlet)
service:327, JspServlet (org.apache.jasper.servlet)
service:101, JettyJspServlet (org.eclipse.jetty.jsp)
service:590, HttpServlet (javax.servlet.http)
service:1419, ServletHolder$NotAsync (org.eclipse.jetty.servlet)
handle:764, ServletHolder (org.eclipse.jetty.servlet)
doFilter:1665, ServletHandler$ChainEnd (org.eclipse.jetty.servlet)
doFilterInternal:49, MDCFilter (com.assaabloy.protech.server.web.filter)
doFilter:117, OncePerRequestFilter (org.springframework.web.filter)
doFilter:202, FilterHolder (org.eclipse.jetty.servlet)
doFilter:1635, ServletHandler$Chain (org.eclipse.jetty.servlet)
doFilter:352, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:100, AuthorizationFilter (org.springframework.security.web.access.intercept)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:126, ExceptionTranslationFilter (org.springframework.security.web.access)
doFilter:120, ExceptionTranslationFilter (org.springframework.security.web.access)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:45, LoginRequiredFilter (com.assaabloy.protech.server.authentication)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:131, SessionManagementFilter (org.springframework.security.web.session)
doFilter:85, SessionManagementFilter (org.springframework.security.web.session)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:100, AnonymousAuthenticationFilter (org.springframework.security.web.authentication)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:164, SecurityContextHolderAwareRequestFilter (org.springframework.security.web.servletapi)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:63, RequestCacheAwareFilter (org.springframework.security.web.savedrequest)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilterInternal:58, DefaultLogoutPageGeneratingFilter (org.springframework.security.web.authentication.ui)
doFilter:117, OncePerRequestFilter (org.springframework.web.filter)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:238, DefaultLoginPageGeneratingFilter (org.springframework.security.web.authentication.ui)
doFilter:224, DefaultLoginPageGeneratingFilter (org.springframework.security.web.authentication.ui)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:227, AbstractAuthenticationProcessingFilter (org.springframework.security.web.authentication)
doFilter:221, AbstractAuthenticationProcessingFilter (org.springframework.security.web.authentication)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilterInternal:187, OAuth2AuthorizationRequestRedirectFilter (org.springframework.security.oauth2.client.web)
doFilter:117, OncePerRequestFilter (org.springframework.web.filter)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:107, LogoutFilter (org.springframework.security.web.authentication.logout)
doFilter:93, LogoutFilter (org.springframework.security.web.authentication.logout)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doHeadersAfter:90, HeaderWriterFilter (org.springframework.security.web.header)
doFilterInternal:75, HeaderWriterFilter (org.springframework.security.web.header)
doFilter:117, OncePerRequestFilter (org.springframework.web.filter)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilter:117, SecurityContextPersistenceFilter (org.springframework.security.web.context)
doFilter:87, SecurityContextPersistenceFilter (org.springframework.security.web.context)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilterInternal:62, WebAsyncManagerIntegrationFilter (org.springframework.security.web.context.request.async)
doFilter:117, OncePerRequestFilter (org.springframework.web.filter)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilterInternal:42, DisableEncodeUrlFilter (org.springframework.security.web.session)
doFilter:117, OncePerRequestFilter (org.springframework.web.filter)
doFilter:361, FilterChainProxy$VirtualFilterChain (org.springframework.security.web)
doFilterInternal:225, FilterChainProxy (org.springframework.security.web)
doFilter:190, FilterChainProxy (org.springframework.security.web)
invokeDelegate:354, DelegatingFilterProxy (org.springframework.web.filter)
doFilter:267, DelegatingFilterProxy (org.springframework.web.filter)
doFilter:202, FilterHolder (org.eclipse.jetty.servlet)
doFilter:1635, ServletHandler$Chain (org.eclipse.jetty.servlet)
_doFilter:263, CompressionFilter (com.isomorphic.servlet)
doFilter:91, BaseFilter (com.isomorphic.servlet)
doFilter:210, FilterHolder (org.eclipse.jetty.servlet)
doFilter:1635, ServletHandler$Chain (org.eclipse.jetty.servlet)
doFilter:170, WebSocketUpgradeFilter (org.eclipse.jetty.websocket.servlet)
doFilter:202, FilterHolder (org.eclipse.jetty.servlet)
doFilter:1635, ServletHandler$Chain (org.eclipse.jetty.servlet)
doHandle:527, ServletHandler (org.eclipse.jetty.servlet)
handle:131, ScopedHandler (org.eclipse.jetty.server.handler)
handle:598, SecurityHandler (org.eclipse.jetty.security)
handle:122, HandlerWrapper (org.eclipse.jetty.server.handler)
nextHandle:223, ScopedHandler (org.eclipse.jetty.server.handler)
doHandle:1580, SessionHandler (org.eclipse.jetty.server.session)
nextHandle:221, ScopedHandler (org.eclipse.jetty.server.handler)
__doHandle:1384, ContextHandler (org.eclipse.jetty.server.handler)
doHandle:40020, ContextHandler (org.eclipse.jetty.server.handler)
nextScope:176, ScopedHandler (org.eclipse.jetty.server.handler)
doScope:484, ServletHandler (org.eclipse.jetty.servlet)
doScope:1553, SessionHandler (org.eclipse.jetty.server.session)
nextScope:174, ScopedHandler (org.eclipse.jetty.server.handler)
doScope:1306, ContextHandler (org.eclipse.jetty.server.handler)
handle:129, ScopedHandler (org.eclipse.jetty.server.handler)
handle:149, ContextHandlerCollection (org.eclipse.jetty.server.handler)
handle:51, HandlerList (org.eclipse.jetty.server.handler)
handle:122, HandlerWrapper (org.eclipse.jetty.server.handler)
handle:563, Server (org.eclipse.jetty.server)
dispatch:1598, HttpChannel$RequestDispatchable (org.eclipse.jetty.server)
dispatch:753, HttpChannel (org.eclipse.jetty.server)
handle:501, HttpChannel (org.eclipse.jetty.server)
onFillable:287, HttpConnection (org.eclipse.jetty.server)
succeeded:314, AbstractConnection$ReadCallback (org.eclipse.jetty.io)
fillable:100, FillInterest (org.eclipse.jetty.io)
run:53, SelectableChannelEndPoint$1 (org.eclipse.jetty.io)
runTask:421, AdaptiveExecutionStrategy (org.eclipse.jetty.util.thread.strategy)
consumeTask:390, AdaptiveExecutionStrategy (org.eclipse.jetty.util.thread.strategy)
tryProduce:277, AdaptiveExecutionStrategy (org.eclipse.jetty.util.thread.strategy)
run:199, AdaptiveExecutionStrategy (org.eclipse.jetty.util.thread.strategy)
run:411, ReservedThreadExecutor$ReservedThread (org.eclipse.jetty.util.thread)
runJob:969, QueuedThreadPool (org.eclipse.jetty.util.thread)
doRunJob:1194, QueuedThreadPool$Runner (org.eclipse.jetty.util.thread)
run:1149, QueuedThreadPool$Runner (org.eclipse.jetty.util.thread)
runWith:1596, Thread (java.lang)
run:1583, Thread (java.lang)

We have datasources defined this way because that's how your archetype generates projects: https://smartclient.com/smartgwt/jav...enSupport.html. We were under the assumption that was best practices. We don't have any precompiling or different data sources per users. Please let us know what the best practice would be for loading data sources.

[Isomorphic]

Can you share one of the SimpleType files? It seems like it either has a validator in it that requires a DSRequest to be performed (such as hasRelatedRecord) or you may have some expanded properties in your SimpleType definition that erroneously trigger validation that requires a DSRequest.

Edited to add: specifically, somehow "poName" gets in there, so whichever SimpleType references that data value would be the best one to see.

[aderosso]

Here's all of them:

<SimpleType name="codeType" inheritsFrom="integer" defaultOperator="equals">
    <validOperators>
        <value>equals</value>
        <value>notEqual</value>
        <value>inSet</value>
        <value>notInSet</value>
        <value>isBlank</value>
        <value>notBlank</value>
    </validOperators>
</SimpleType>

<SimpleType name="currencyType" inheritsFrom="localeCurrency" defaultOperator="equals" precision="12" decimalPrecision="2" javaClass="java.math.BigDecimal">
    <validOperators>
        <value>equals</value>
        <value>notEqual</value>
        <value>greaterThan</value>
        <value>lessThan</value>
        <value>greaterOrEqual</value>
        <value>lessOrEqual</value>
    </validOperators>
</SimpleType>

<SimpleType name="measurementType" inheritsFrom="any" defaultOperator="equals" measurementField="true">
    <validOperators>
        <value>equals</value>
        <value>notEqual</value>
        <value>greaterThan</value>
        <value>lessThan</value>
        <value>greaterOrEqual</value>
        <value>lessOrEqual</value>
        <value>between</value>
        <value>betweenInclusive</value>
    </validOperators>
</SimpleType>

<SimpleType name="qtyType" inheritsFrom="any" defaultOperator="equals">
    <validOperators>
        <value>equals</value>
        <value>notEqual</value>
        <value>greaterThan</value>
        <value>lessThan</value>
        <value>greaterOrEqual</value>
        <value>lessOrEqual</value>
        <value>between</value>
        <value>betweenInclusive</value>
    </validOperators>
</SimpleType>

<SimpleType name="requiredText" inheritsFrom="text" xmlns:fmt="WEB-INF">
    <fmt:bundle basename="com.assaabloy.protech.shared.Messages"/>
    <validators>
        <validator type="regexp" expression="\S">
            <errorMessage>
                <fmt:message key="valueCannotBeBlank"/>
            </errorMessage>
        </validator>
    </validators>
</SimpleType>


<SimpleType name="textType" inheritsFrom="text" defaultOperator="iEquals">
    <validOperators>
        <value>iEquals</value>
        <value>iNotEqual</value>
        <value>iContains</value>
        <value>iNotContains</value>
        <value>iStartsWith</value>
        <value>iNotStartsWith</value>
        <value>iEndsWith</value>
        <value>iNotEndsWith</value>
    </validOperators>
</SimpleType>

[Isomorphic]

OK, last question: is one of these SimpleTypes attached to a field "poName"? Which one if so?

[aderosso]

Yes:

<field name="poNumber" nativeName="po_number" type="requiredText" length="25" required="true">
    <validators>
        <validator type="isUnique" criteriaFields="companyId,poNumber,revisionNumber">
            <errorMessage>PO Number already exists</errorMessage>
        </validator>
    </validators>
    <title>
        <fmt:message key="poNumber"/>
    </title>
</field>

[Blama]

Hi aderosso

We don't have any precompiling or different data sources per users. Please let us know what the best practice would be for loading data sources.

on your smaller topic in this thread, see here: https://forums.smartclient.com/forum...er-call-result

I'd use a call to DataSourceLoader that may be cached with cache breaker query params for:

• SmartClient version
• User
• User language
• Build date (for changed fmt-strings)
• If you are using DynamicDSGenerator: Last config change in your DB that might result in different .ds.xml returned

In general this should result in pretty stable DataSourceLoader-URLs and should make loading DataSourceLoader from the browser a rare event.

Best regards
Blama

[Blama]

Hi aderosso,

seeing your #11 and the fmt-tag in a different line than title: Please see this thread. You might have whitespace in your DataSource field titles clientside that you don't want.

Best regards
Blama

[aderosso]

Thank you Blama! We've also been having issues with changes to datasources when running locally not being picked up, I'm assuming this is stemming from the same issue. We're switching back to the datasourceLoader.

[Isomorphic]

We'll be taking a look at the odd SimpleType-related fetch, but we do think it's likely harmless.

Separately, you mentioned that the archetype shows a datasources.jsp file instead of DataSourceLoader calls - the archetype actually shows both approaches, and it's mostly a matter of style.

If your app has multiple entrypoints which are set up as different .jsps/URLs endpoints, and all of those entrypoints need the identical set of DataSources loaded, you can avoid duplication by having them all point to a shared datasources.jsp. However, it's far more common that if your app has multiple entrypoints, it still all boils down to the same .jsp bootstrap file, in which case having a separate datasources.jsp file doesn't reduce duplication.

Another concern is just ease of editing: if you maintain a datasources.jsp, that file will have one line per DataSource, as opposed to adding them all into the parameters from a single <script> tag. That one-line tag can get a little hard to scan and edit once you get above ~30 DataSources.