Hi Isomorphic,
those now sat quite a while. Do you have an opinion on those?
Especially the "queueStatus is present multiple times" is a bug IMHO. As I gave a customer access to the API now, I expect the question why this is to come up very soon now.
Thank you & Best regards
Blama
Announcement
Collapse
No announcement yet.
X
-
Hi Isomorphic,
issue #3, same request:
Code:package com.smartgwt.sample.server.listener; import java.io.Reader; import java.io.StringReader; import java.io.StringWriter; import java.util.List; import java.util.Map; import javax.servlet.http.HttpServletRequest; import com.isomorphic.datasource.DSRequest; import com.isomorphic.datasource.DSResponse; import com.isomorphic.datasource.DataSource; import com.isomorphic.tools.DataImport; import com.isomorphic.tools.DataImport.ImportFormat; public class EmployeesUpload { public DSResponse upload(DSRequest dsRequest, HttpServletRequest servletRequest) throws Exception { @SuppressWarnings("unchecked") Map<String, Object> requestValues = dsRequest.getValues(); Reader jsonReader = new StringReader(convertToCSV(requestValues)); DataImport dataImporter = new DataImport(ImportFormat.CSV, ";"); dataImporter.setPopulateDisplayFields(true); @SuppressWarnings("unchecked") List<Map<String, Object>> dataImporterResult = dataImporter.importDataSourceRecords(jsonReader, "employeesUpload"); DSRequest addRequest = new DSRequest("employeesUpload", DataSource.OP_ADD, dsRequest.getRPCManager()); addRequest.setValues(dataImporterResult); DSResponse addResponse = addRequest.execute(); return addResponse; } /* private String convertToJSON(Map<String, Object> requestValues) { // json-simple-1.1.1.jar // https://code.google.com/archive/p/json-simple/ JSONObject jsonFinalObject = new JSONObject(); for (Map.Entry<String, Object> entry : requestValues.entrySet()) { jsonFinalObject.put(entry.getKey(), entry.getValue()); } return jsonFinalObject.toJSONString(); } private String convertToXML(Map<String, Object> requestValues) { StringWriter sw = new StringWriter(); sw.append("<record>\r\n"); for (Map.Entry<String, Object> entry : requestValues.entrySet()) { sw.append("\t<" + entry.getKey() + ">" + entry.getValue() + "</" + entry.getKey() + ">\r\n"); } sw.append("</record>"); return sw.toString(); } */ private String convertToCSV(Map<String, Object> requestValues) { StringWriter csvHeader = new StringWriter(); StringWriter csvData = new StringWriter(); for (Map.Entry<String, Object> entry : requestValues.entrySet()) { String cleanKey = entry.getKey().replace(";", "").replace("\"", ""); String cleanValue = entry.getValue().toString().replace(";", "").replace("\"", ""); csvHeader.append(cleanKey + ";"); csvData.append(cleanValue + ";"); } return csvHeader.toString() + "\r\n" + csvData.toString(); } }
Code:<response> <status> 0 </status> <queueStatus> 0 </queueStatus> <startRow> -1 </startRow> <endRow> -1 </endRow> <totalRows> -1 </totalRows> <queueStatus> 0 </queueStatus> <queueStatus> 0 </queueStatus> <data> <record> <Salary> 12345.0 </Salary> <ReportsTo> 4 </ReportsTo> <Gender> male </Gender> <EmployeeId> 100002 </EmployeeId> <Job> Developer </Job> <ReportsToName> Charles Madigen </ReportsToName> </record> </data> </response>
- As you can see, the queueStatus is now present three times.
- Generally speaking this is working, but IMHO the workaround with CSV should not be necessary. DataImport-methods should have an overload that take Map<String, Object> and do the conversion displayName->ID.
- When using JSON or XML, what is the expected structure? Could you add it to the docs? You can see my convertToXML() and convertToJSON(), but the DataImport-docs don't say anything about the enclosing tag or similar.
Additionally, XML is mentioned as possibility besides XML/CSV in the class description, but never in the methods.
DataImport contains methods that can be used to import data from a test data file to the List of Maps format commonly used for DataSource records (importToRows()), or directly into a DataSource (importToDataSource()).
By default the input file is expected to contain comma-delimited data like a .csv file, but JSON and XML are also supported.
Imported data may be transformed during import, for details search SmartClient Reference for "dataSourceField.importStrategy".- As enhancement: It would be great if it were possible to directly add data to a DataSource via a ARC request using the String-to-ID lookup and not to have to call DataImport oneself in a boilerplate method. If this is not a good idea as default for all ADDs for fields with importStrategy="display", then perhaps as additional attribute for an ADD or UPDATE operation binding.
Blama
Leave a comment:
-
Hi Isomorphic,
issue #2 (minor), same setup, but
Code:public DSResponse upload(DSRequest dsRequest, HttpServletRequest servletRequest) throws Exception { return new DSResponse(dsRequest.getDataSource()).setSuccess(); }
Code:<response> <status> 0 </status> <queueStatus> 0 </queueStatus> <startRow> -1 </startRow> <endRow> -1 </endRow> <totalRows> -1 </totalRows> <queueStatus> 0 </queueStatus> <data /> </response>
Best regards
Blama
Leave a comment:
-
6.1p DataImport importToDataSource issue in combination with RESTHandler
Hi Isomorphic,
I'm currently creating an API to my application and hit some bumps here in DataImport I use in combination with RESTHandler (latest 6.1p, v11.1p_2017-11-08).
The issues/questions are mainly DataImport-related and the whole testcase is serverside only (FF26 Dev mode in Eclipse).
employeesUpload.ds.xml:
Code:<DataSource xmlns="lmscompany/ds" xmlns:fmt="lmscompany/fmt" ID="employeesUpload" serverType="sql" tableName="employeeTable" recordName="employee" useAnsiJoins="true"> <fmt:bundle basename="com.smartgwt.sample.server.listener.DSXMLResources-utf8" encoding="utf-8" /> <fields> <field name="EmployeeId" title="Employee ID" type="integer" primaryKey="true" required="true"/> <field name="Name" uploadFieldName="Name" title="Name" type="text" length="128"> <validators> <validator type="isUnique" caseSensitive="true" /> </validators> </field> <field name="ReportsTo" uploadFieldName="ReportsTo" displayField="ReportsToName" required="true" importStrategy="display" foreignKey="employees.EmployeeId" relatedTableAlias="relatedReportsTo"> <title><fmt:message key="ReportsTo" /></title> <validators> <validator type="hasRelatedRecord" errorMessage="Unknown ReportsTo" /> </validators> </field> <field name="ReportsToName" includeFrom="employees.Name" includeVia="ReportsTo" /> <field name="Job" uploadFieldName="Job" title="Title" type="text" length="128"> <validators> <validator type="isOneOf" /> </validators> <valueMap> <value>Developer</value> <value>IT-Infrastructure</value> <value>Other</value> </valueMap> </field> <field name="Gender" uploadFieldName="Gender" title="Gender" type="text" length="7"> <validators> <validator type="isOneOf" /> </validators> <valueMap> <value>male</value> <value>female</value> </valueMap> </field> <field name="Salary" uploadFieldName="Salary" title="Salary" type="integer"> </field> </fields> <serverObject lookupStyle="new" className="com.smartgwt.sample.server.listener.EmployeesUpload" /> <operationBindings> <operationBinding operationType="custom" operationId="upload" serverMethod="upload" /> </operationBindings> </DataSource>
Code:ReportsTo = MyManager
Code:<servlet> <servlet-name>Upload</servlet-name> <servlet-class>com.smartgwt.sample.server.listener.Upload</servlet-class> </servlet> <servlet-mapping> <servlet-name>Upload</servlet-name> <url-pattern>/Upload</url-pattern> </servlet-mapping>
Code:package com.smartgwt.sample.server.listener; import java.io.Reader; import java.io.StringReader; import java.io.StringWriter; import java.util.Map; import javax.servlet.http.HttpServletRequest; import org.json.simple.JSONObject; import com.isomorphic.datasource.DSRequest; import com.isomorphic.datasource.DSResponse; import com.isomorphic.tools.DataImport; import com.isomorphic.tools.DataImport.ImportFormat; public class EmployeesUpload { public DSResponse upload(DSRequest dsRequest, HttpServletRequest servletRequest) throws Exception { @SuppressWarnings("unchecked") Map<String, Object> requestValues = dsRequest.getValues(); Reader jsonReader = new StringReader(convertToCSV(requestValues)); DataImport dataImporter = new DataImport(ImportFormat.CSV, ";"); dataImporter.setPopulateDisplayFields(true); long i = dataImporter.importToDataSource(jsonReader, "employeesUpload"); if (i == 1) return new DSResponse(dsRequest.getDataSource()).setSuccess(); else return new DSResponse(dsRequest.getDataSource()).setFailure("Not successfully imported"); } private String convertToCSV(Map<String, Object> requestValues) { StringWriter csvHeader = new StringWriter(); StringWriter csvData = new StringWriter(); for (Map.Entry<String, Object> entry : requestValues.entrySet()) { String cleanKey = entry.getKey().replace(";", "").replace("\"", ""); String cleanValue = entry.getValue().toString().replace(";", "").replace("\"", ""); csvHeader.append(cleanKey + ";"); csvData.append(cleanValue + ";"); } return csvHeader.toString() + "\r\n" + csvData.toString(); } }
Code:POST to http://127.0.0.1:8888/builtinds/sc/RESTHandler Body: <request> <dataSource>employeesUpload</dataSource> <operationType>custom</operationType> <operationId>upload</operationId> <data> <EmployeeId>100001</EmployeeId> <Name>New colleague</Name> <ReportsTo>Charles Madigen</ReportsTo> <Job>Developer</Job> <Gender>male</Gender> <Salary>12345</Salary> </data> </request>
Code:<response> <status> -1 </status> <queueStatus> -1 </queueStatus> <data> Not successfully imported </data> </response>
Code:=== 2017-11-08 21:18:08,342 [7-31] INFO RequestContext - URL: '/builtinds/sc/RESTHandler', User-Agent: 'null': Unsupported WITHOUT Accept-Encoding header === 2017-11-08 21:18:08,343 [7-31] DEBUG RESTHandler - Defaulting response data format to xml === 2017-11-08 21:18:08,343 [7-31] DEBUG RestRequestParser - Parsing xml object: '<request> <dataSource>employeesUpload</dataSource> <operationType>custom</operationType> <operationId>upload</operationId> <data> <EmployeeId>100001</EmployeeId> <Name>New colleague</Name> <ReportsTo>Charles Madigen</ReportsTo> <Job>Developer</Job> <Gender>male</Gender> <Salary>12345</Salary> </data> </request>' === 2017-11-08 21:18:08,345 [7-31] DEBUG XML - Parsed XML from (in memory stream): 2ms === 2017-11-08 21:18:08,347 [7-31] INFO RESTHandler - Performing 1 operation(s) === 2017-11-08 21:18:08,347 [7-31] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null === 2017-11-08 21:18:08,347 [7-31] DEBUG DeclarativeSecurity - DataSource employeesUpload is not in the pre-checked list, processing... === 2017-11-08 21:18:08,348 [7-31] DEBUG LocaleMessage - Returning Properties from cache for props file com.smartgwt.sample.server.listener.DSXMLResources-utf8_de_DE === 2017-11-08 21:18:08,348 [7-31] DEBUG LocaleMessage - Returning Properties from cache for props file com.smartgwt.sample.server.listener.DSXMLResources-utf8_de_DE === 2017-11-08 21:18:08,350 [7-31] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null === 2017-11-08 21:18:08,350 [7-31] DEBUG DeclarativeSecurity - Request is not a client request, ignoring security checks. === 2017-11-08 21:18:08,351 [7-31] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null === 2017-11-08 21:18:08,351 [7-31] DEBUG DeclarativeSecurity - Request is not a client request, ignoring security checks. === 2017-11-08 21:18:08,351 [7-31] DEBUG AppBase - [builtinApplication.null] No userTypes defined, allowing anyone access to all operations for this application === 2017-11-08 21:18:08,351 [7-31] DEBUG AppBase - [builtinApplication.null] No public zero-argument method named '_null' found, performing generic datasource operation === 2017-11-08 21:18:08,351 [7-31] INFO SQLDataSource - [builtinApplication.null] Performing fetch operation with criteria: {EmployeeId:"Charles Madigen"} values: {EmployeeId:"Charles Madigen"} === 2017-11-08 21:18:08,352 [7-31] DEBUG SQLDataSource - [builtinApplication.null] DataSource 51 acquired SQLDriver instance 1994064315 during initialization [B]=== 2017-11-08 21:18:08,352 [7-31] WARN SQLWhereClause - [builtinApplication.null] Got non-numeric value 'Charles Madigen' for numeric column 'EmployeeId', creating literal false expression: java.lang.NumberFormatException: For input string: "Charle"[/B] === 2017-11-08 21:18:08,352 [7-31] INFO SQLDataSource - [builtinApplication.null] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause [B]=== 2017-11-08 21:18:08,353 [7-31] INFO SQLDataSource - [builtinApplication.null] 51: Executing SQL query on 'HSQLDB': SELECT employeeTable.userOrder, employeeTable.Name, employeeTable.EmployeeId, employeeTable.ReportsTo, employeeTable.Job, employeeTable.Email, employeeTable.EmployeeType, employeeTable.EmployeeStatus, employeeTable.Salary, employeeTable.OrgUnit, employeeTable.Gender, employeeTable.MaritalStatus FROM employeeTable WHERE ('0'='1')[/B] === 2017-11-08 21:18:08,353 [7-31] DEBUG SQLConnectionManager - [builtinApplication.null] Borrowed connection '998793675' === 2017-11-08 21:18:08,353 [7-31] INFO SQLDriver - [builtinApplication.null] Executing SQL query on 'HSQLDB' using connection '998793675': SELECT employeeTable.userOrder, employeeTable.Name, employeeTable.EmployeeId, employeeTable.ReportsTo, employeeTable.Job, employeeTable.Email, employeeTable.EmployeeType, employeeTable.EmployeeStatus, employeeTable.Salary, employeeTable.OrgUnit, employeeTable.Gender, employeeTable.MaritalStatus FROM employeeTable WHERE ('0'='1') === 2017-11-08 21:18:08,354 [7-31] INFO DSResponse - DSResponse: List with 0 items === 2017-11-08 21:18:08,354 [7-31] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 51 === 2017-11-08 21:18:08,354 [7-31] DEBUG SQLDriver - Freeing SQLDriver dbConnection 998793675 for SQLDriver instance 1994064315 === 2017-11-08 21:18:08,354 [7-31] DEBUG SQLConnectionManager - About to close connection with hashcode "998793675" === 2017-11-08 21:18:08,354 [7-31] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 51 === 2017-11-08 21:18:08,354 [7-31] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 51 === 2017-11-08 21:18:08,354 [7-31] DEBUG ValidationContext - Adding validation errors at path '/employeesUpload/ReportsTo/ReportsTo': {errorMessage=Unknown ReportsTo} === 2017-11-08 21:18:08,355 [7-31] INFO Validation - Validation error: [ { ReportsTo:{ errorMessage:"Unknown ReportsTo" } } ] === 2017-11-08 21:18:08,355 [7-31] DEBUG DataSourceDMI - Freeing resources in DataSourceDMI... === 2017-11-08 21:18:08,355 [7-31] DEBUG DataSourceDMI - Freeing QueueResources in DataSourceDMI === 2017-11-08 21:18:08,355 [7-31] DEBUG DataSourceDMI - Freeing resources in DataSourceDMI... === 2017-11-08 21:18:08,355 [7-31] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8 === 2017-11-08 21:18:08,355 [7-31] DEBUG RPCManager - DMI response, dropExtraFields: false === 2017-11-08 21:18:08,356 [7-31] WARN RequestContext - Content type has already been set to: text/html; charset=UTF-8 - setting to: text/xml === 2017-11-08 21:18:08,356 [7-31] DEBUG DataSourceDMI - Freeing QueueResources in DataSourceDMI
Best regards
BlamaTags: None
Leave a comment: