TypeScript integration

[MrHighTech]

hello Isomorphic and all,

I am currently playing with TypeScript and I actually see value with it. There are a couple of older posts here about using the referenceDocs(js/xml).

I am working on that and will contribute it to DefinitelyTyped when it is completed. I think a first level integration should be available this week for testing, but it might take a bit more to have a nice integration with the create methods (Interfaces will help there).

The reference docs are pretty good, but I am having a problem with inheritance. Looking at the documentation for ListGrid for example lists it as extending VLayout. But the field inheritsFrom is not present (only there in a couple of places).

The implementsInterfaces are also present, but I haven't checked them thoroughly yet.

I was under the impression the online documentation is generated form the referenceDocs... Any ideas on how to get a reliable hierarchy from the the file ?

Thanks!

[Isomorphic]

Thanks for taking this on, we look forward to your results.

Since the SmartClient Reference is actually a SmartClient application, it derives certain information directly from SmartClient at runtime rather than relying on the referenceDocs.xml file to contain it.

However, this particular circumstance may be the sole one you actually run into when implementing TypeScript, and it probably doesn't matter anyway - we generally put in the @inheritsFrom tag in JSDoc if the inheritance chain is important. In the case of ListGrid inheriting from VLayout, really the only valid way to use the superclass functionality is via listGrid.gridComponents; direct calls to addMember() or removeMember() would be messing with internals.

[MrHighTech]

Thanks for a quick reply!

I will use the extends when it is available for now then. We'll see how things turn out. Keep you guys posted.

[kylemwhite]

MrHighTech,

How's your SmartClient TypeScript project progressing? I am now working on a project using SmartClient and (maybe) TypeScript. Having the SmartClient type definitions would sure be nice. If you've got a good start, maybe I can assist.

[antanas.arvasevicius]

Hi, I'm also using TypeScript, but we've made interface declarations only for api which we are currently using in project. There is more problems as how to integrate TypeScript class system to SmartClient`s system.
For Example if we'll need a subclass of ListGrid

class MyListGrid extends isc.ListGrid {
}

it won`t work.

Maybe is it possible to call some internals of SmartClient to emulate initialization of dummy classes, e.g. we could have a isc.Component which will call isc.[component].create({}, THIS); and then SmartClient will attach all its behavior to that specific object by prototype, and we'll had an interface declaration of that specific class so all IDE autocompletion would work.

pseudo code:

class Component {
   constructor(properties) {

      // TODO: needed new SmartClient API
      // Class.createOnGivenInstance(this, properties, a, b, c, d) 
      // which will create and initialize all methods on a given 
      // object instead of creating new. 

     window['isc'][this.componentClass()].createOnGivenInstance(this, properties);


   }
}

class ListGrid extends Component implements isc.IListGrid {
    // @override
    protect componentClass() { return 'ListGrid'; }
}

[kylemwhite]

Isomorphic and all,

Since nothing appeared on DefinitelyTyped, I decided to give this a go myself. I now see the issues that MrHighTech and others have run into. I.e. the referenceDocs.xml file is not completely accurate and missing things like inheritsFrom attributes. I have been able to manually code just the parts that I am using as I suspect a lot of others have, but this approach would not be appropriate for a project hosted on DefinitelyTyped or NuGet etc.

Since isc is such a huge library, code generation is really the only practical way to produce a complete TypeScript type definition library and the referenceDocs.xml file has a perfectly good schema to include the correct types and interfaces including inheritance and documentation. I've actually generated a file (that compiles and works) of just the types but not the interfaces (classes) because of the missing inheritsFrom attribute issue. Even with the types, there are many inconsistencies which can be overcome with custom rules, although it would be nice if we didn't have to do that. To keep the library up-to-date, the generation would need to be automated, thus making complete and accurate source information even more important.

With my generated types and manually-created interfaces, my TypeScript coding experience in Visual Studio is very pleasant and I've found and fixed tons of bugs. It's very similar to coding in Java with SmartGWT but I can now code everything in Visual Studio and not worry about using another IDE like Eclipse or NetBeans. I suspect this approach, combined with Isomorphic's sample of .NET and MVC, is very attractive to other .NET developers as well.

I would be happy to continue the work to generate the rest of the isc TypeScript library and submit it to DefinitelyTyped if we can get a complete and accurate source for code generation so that the library can be made as robust as possible. The source could be a revised referenceDocs.xml file or whatever it is generated from (a database?) And I would be happy to contribute the necessary information to help improve the quality of the source.

TypeScript is rapidly gaining in popularity (over-taking coffeescript), especially (I suspect) among .NET developers such as myself. It's adoption is no longer 'tepid' as previously stated in another post in 2013. I am willing to contribute to the community but need support from the source (and probably support from the community for tests, documentation etc.) Or, if Isomorphic wants to do it all, that would be even better.

Thanks,

Kyle

[Isomorphic]

Hi Kyle, thanks for stepping up to help with TypeScript. It's true that it's popularity is starting to rise.

In the latest release of SmartClient (11.1), the inheritance chain is now explicitly listed in referenceDocs.xml (there may still be gaps - let us know), so that particular issue should be solved. Note that a possible approach is just to generate the TypeScript interface definitions in a browser-based application running SmartClient, which allows runtime information like the inheritance chain to be checked, and is part of why our own SmartClient Reference is a little more complete than referenceDocs.xml (aside from inheritance, it can look up things like default values on the fly).

Is there any other information that you've noticed that's missing from referenceDocs.xml and that is needed for TypeScript generation?

As far as targeting a different format than referencesDocs.xml, is there any particular format you have in mind?

[kylemwhite]

I've just downloaded 11.1d and indeed the referenceDoc.xml file has (at least one) inheritsFrom attribute that didn't exist before.... looks promising. I am working on parsing the file (my old code didn't work, has the schema changed?) so I can generate the types and interfaces. I'll re-post when I've made some significant progress.

I am much more proficient at C# than JavaScript or even TypeScript at this point so it probably makes sense for me to stick with that for the typedef generator. I think the referenceDocs will contain all the information I need once fixed up.

Yes, there are other issues with the referenceDocs which could be improved... both for the typedef generator and for the SmartClient reference (IMHO). Many types (type and valueType attributes) are specified in the referenceDocs with an English description vs. an actual JavaScript type. For example integer, Integer, int, float, and Float are all things that we understand, but JavaScript (and TS) only knows number. The fact that the number should be an int or float should be in the description field but not the type field. Other examples include: Array of T and Array[] of T which should just be T[]. xxClassName should just be string (with the appropriate description of what that string should contain of course) etc. I'll probably have more as I progress. I don't know how easy it is for you to update the referenceDocs, which is why I mentioned the DB but parsing the referenceDocs is fine with me (once I generate the XSD and the proxy classes). Also, I'll need to parse out the link codes (${isc.DocUtils....}) into just text or maybe JSDoc link codes. If the raw data was available, that would make it easier than parsing out the codes (although I can't say I know what that would look like).
Of course I can write custom rules to transform int to number etc. but it would be cleaner if the referenceDocs was consistent and we didn't have to do that.

In the latest version I downloaded today, the SmartClient Reference page doesn't seem to be working. I can open the developer console but when I click the Reference Docs tab, it just gets stuck with a dialog saying "Loading Documentation modules". There is a JS error that says, "Uncaught TypeError: Cannot read property 'getCurrentHistoryId' of undefined. Tested in Chrome and IE... same thing. Hopefully that can be fixed soon as the Reference Docs greatly helps me with the typedef generator.

Thanks for the support.

Kyle

[kylemwhite]

Ok, I've got a github repository going and I've uploaded two files: isc.types.d.ts and isc.classes.d.ts. These are both 100% code-generated from the referenceDocs.xml file (with a bunch of rules to fix the previously mentioned type issues). The types file compiles but the classes file does not. So far, I'm generating only a small subset of classes including Class, BaseWidget, Canvas, Layout, VLayout, ListGrid, Tabset and Tab. I'm only doing the properties, no methods yet and only on the instance (not the static class). If I tried to include everything, the files will be huge so I'll need to figure out how to break them up. I'm NOT a TypeScript expert. This effort is to help me with my first TypeScript project which is also a SmartClient project. If there are TypeScript gurus out there that would like to advise on file structure etc. please let me know.

By looking at the generated code, it becomes obvious (to me anyway) the issues faced with the referenceDocs data. I'm sure we can get it cleaned up but it will take a few iterations. In many cases, I bailed and just used any. This is a good way to get it going but eventually I'd like to get everything properly typed.

https://github.com/kylemwhite/isc

[Isomorphic]

[quote]For example integer, Integer, int, float, and Float are all things that we understand ... [/url]

It's true that TypeScript / JavaScript lacks the richness of SC's type system, but this is definitely not a reason to dumb down our own reference documentation, as that just removes extremely valuable information that developers need. The right place to "dumb down" the types is in the script that translates from SC's type information to TypeScript's less rich type system. This also allows improvements to be made as TypeScript gets richer.

That script should, for example, translate "float" to the JavaScript Number type, but also add a note to the docs for the target attribute or method parameter saying that floating point numbers are expected or allowed. We take this same kind of approach when generating SmartGWT's Java API from SC's reference for cases where Java's type system is also too weak (for example, no way to subclass or constrain the expected values of Strings).

Given this, is there anything you see that needs fixing in the referenceDocs.xml file?

Note: we're not reproducing an issue with the SmartClient Reference with the latest available 11.1 build (4-18-2017). You may need to clear cache.

[kylemwhite]

It's true that TypeScript / JavaScript lacks the richness of SC's type system

Did I miss something? What is the SC type system? It's JavaScript, there is no type system, that's why we have things like GWT and TypeScript. There are, of course, the custom types documented in the referenceDocs and those can easily be defined in TypeScript and referenced where appropriate. In fact, the type definitions part is already working and uploaded to the git repository. But, as far as enforcing fundamental types like float, integer etc. SC can't do that anymore than any other JavaScript library (at compile time). TypeScript can, however, at least enforce a number vs. a string vs. a custom type at COMPILE time, and even provide intellisense (depending upon the IDE of course).

but this is definitely not a reason to dumb down our own reference documentation, as that just removes extremely valuable information that developers need

Agree to disagree on this one. I see this an opportunity to make the documentation better, not dumbed down. The description field (IMHO) is the proper place to further express the constraints that a property or parameter should adhere to. It could be as simple as "a positive integer representing the number of minutes before timing out" or as complicated as "a prime number ending in 5 on Tuesdays or 3 on weekends". No language can enforce (at compile time) every constraint that a programmer might wish for. But if there's a schema that specifies a 'type' for a particular language, then it seems to me the type field should contain the actual language type, not a description of what that parameter represents, that goes in the description field.

This issue has actually bugged me in the docs even before I started using TypeScript. Many times I would encounter something like CSSStyleName and wonder what that is, then looking further at the docs I see that it is really just a 'string' and since in JS, there is no type enforcement, it's not like I could create a new CSSStyleName object and pass it in. It's just a string that should, hopefully, match up to a style name in a CSS file. I wished it would have just said the type is 'string' and the description would explain what the string represents. Of course I come from an OOP background, not JavaScipt so that may affect my thinking.

Now, however, with TypeScript, we can actually define a type called CSSStyleName (which is really still just a string, after I fix it) but the intellisense will have documentation explaining what kind of string it is so the developer will get that information immediately without having to look it up in the docs.


Intellisense in Visual Studio (yes, I know I still need to remove the encoding in the description)

Those are my opinions, take 'em or leave 'em. As for fundamental types, yes it is very easy to map Integer, int, float, Float, positiveInteger (and whatever else I find) to number. I am already putting the original type field in the comments (see the isc.classes.d.ts file in the repository) and could also include them in the JSdoc field (although it might appear to be redundant in some cases if that information is already in the description field).

for example, no way to subclass or constrain the expected values of Strings

The Good news: TypeScript DOES allow you to constrain the expected values of strings. See the PreserveOpenState type in isc.types.d.ts. The only valid values are "never", "whenUnique" or "always" and TypeScript will enforce this. This works because the <values> tags in the referenceDocs actually contain the real string values.



The Bad news: Many of these string 'enums' do not contain the actual strings in the <values> tag. They appear to contain a reference to a constant or something although in most cases, I cannot find the definition of the constant. For example NavigationMode has the values TableView.WHOLE_RECORD and TableView.NAVICON_ONLY which really become "wholeRecord" and "naviconOnly" (according to the Docs but I don't know how it figured that out). I don't know how to get the real values from the values in the <values> tag. If they all follow the same pattern like, "remove the period and everything before it, strip the underscore and convert to camelCase", then I can write a transform function to do that. Or maybe the docs could be modified to just contain the actual values? Please advise.

That's it for now. Please have a look at the git repository as I think it will make some of the issues clearer, especially if viewed by someone familiar with TypeScript.

[Isomorphic]

No language can enforce (at compile time) every constraint that a programmer might wish for.

Definitely. But this doesn't mean that constraints should never be expressed via type. For example, no one would think Java could be improved by having all parameters be Object and just having developers document what's expected. The trend is actually the reverse, as some modern languages incorporate parameter constraints, code contracts, or compilable annotations that express constraints on basic types.

Not to mention that, of course, systems like XML Schema allow such constraints for data values, and in many ways, the distinction between data values and parameter values is blurring (isn't an XML element passed to a WSDL web service basically a method parameter?). In fact, SC's type system allows you to attach not just constraints but also behaviors to atomic types - SimpleTypes in SmartClient can extend each other, and add value constraints, parsing and formatting rules, default editors to use, etc.

So, we definitely will not be changing our reference to show just Number where we currently show int, float, etc as it clearly provides useful information in a compact, easily understood format. Similarly for things like CSSStyleName - we find this a lot better than having docs throughout the system link to an explanation of what to pass, and the fact that the information is structurally part of the doc makes it much easier to build tools for editing component properties.

However we would like to make referenceDocs.xml, or a derivative format, easier to process for TypeScript *and other possible targets* (recall, we actually generate full Java interfaces from this information).

Currently, for the handful of types like CSSStyleName which are basically strings, for translation to Java APIs we just have a list of them in the translation tool, and the list is:

<class name="Callback"/>
<class name="ListGridGroupState"/>
<class name="RelativeDateString"/>
<class name="CSSClass" />
<class name="CSSText" />
<class name="CSSColor" />
<class name="CSSClassName" />
<class name="CSSStyleName" />
<class name="Color" />
<class name="SCClassName" />
<class name="SCImgURL" />
<class name="DataPath" />
<class name="DateInputFormat" />
<class name="ListGridFieldState" />
<class name="ListGridSelectionState" />
<class name="ListGridSelectedState" />
<class name="ListGridSortState" />
<class name="ListGridViewState" />
<class name="TreeGridOpenState" />
<class name="TreeGridViewState" />
<class name="FormItemBaseStyle" />
<class name="VelocityExpression" />
<class name="XPathExpression" />
<class name="HTMLString" />
<class name="DetailViewerViewState" />
<class name="FormatString" />

For now, we'd suggest just using this same list in the TypeScript translation script. We'll look at adding something like adding a ' baseType:"String" ' property to referenceDocs.xml so this list doesn't need to be hardcoded into the translation script, but we're very focused on the 11.1 / 6.1 release right now, so there will be some delay.

For the NavigationMode enum, those values are the names of constants, so you can actually evaluate "TableView.NAVICON_ONLY" to get the string "naviconOnly". However, we went through and eliminated these as part of making our docs more easily translate to Java. We can fix this one, let us know if you find others.

[kylemwhite]

recall, we actually generate full Java interfaces from this information

I kinda figured that but I'm finding so many inconsistencies that you must have a bunch of rules built in code like I do to make everything work. I think that by making the referenceDocs (or a derivative format) more consistent will improve SmartGWT, TypeScript, the documentation and possibly SmartClient itself.

Currently, for the handful of types like CSSStyleName which are basically strings, for translation to Java APIs we just have a list of them in the translation tool, and the list is:

Ok, it looks like I am re-inventing the your translation tool because I've got a similar list going. Your's is more complete of course so I'll just put that in there. Thanks.

We'll look at adding something like adding a ' baseType:"String" ' property to referenceDocs.xml

That is fantastic!(although I would use lowercase string). The same field could be used for number. And I understand about focusing on 11.1. I'm looking forward to the Tahoe skin.

We can fix this one, let us know if you find others.

There's a bunch in the 11.0, Next week, I'll be using 11.1 and let you know when I find more.

I made a lot of progress today but nothing to upload yet.

[Isomorphic]

We don't plan to add "baseType" declarations for int, float, etc as that's a small, static, self-explanatory list. However it would be great to hear about any further issues you discover that make it difficult to translate to a TypeScript API definition (or, like the string-based types, require information to be maintained inside the translation script).

[kylemwhite]

It's general inconsistencies in the way that properties are defined that makes it tough. Here are some Examples:

ListGridField.includeInRecordSummaryFields: array of fieldNames <-- fieldNames is pluralized. Also the way of specifying arrays is inconsistent. Would prefer just FieldName[] or string[].

DataSourceField.filterEditorType: FormItem className <-- Again, this is really a string, Hopefully the baseType will fix that.

Menu.data: Array of MenuItem | Array[] of Record | Tree | RecordList <-- Array is specified in two different ways for the same property. Or is specified with |

Validator.dependentFields: String[]<-- Another way an array is specified

UserTask.targetVM: ValuesManager or String <-- Or is specified with or.

LayoutProperties.placeHolderProperties: canvas properties <-- Canvas is not capitalized

If I had my way, all arrays would be specified as T[] or Array<T> if it really uses an Array object and Or would always be specified with a pipe |. This maps nicely to (future)JavaScript and TypeScript without any transformation. Also, it would be nice if things were all Capitalized correctly and not pluralized. These things, of course, are all nit-picky details for a human to figure out but require extra rules when read by a program.

If you're considering modifying the schema, one change that might make it easier for all kinds of code/document generation is converting the type attribute to a tag and allow multiple type tags per property. This would allow any code-generator to easily parse union types and format them in the appropriate (using |, or OR or a comma-seperated list like This,That, or TheOther) way either for documentation or for code. For example

<docItem defaultValue="null" type="attr" definingClass="class:Menu" ref="attr:Menu.data" valueType="Array of MenuItem | Array[] of Record | Tree | RecordList" description="An array of menuItem objects, specifying the menu items this menu should show.&amp;#010&amp;#010 Data may also be set to a ${isc.DocUtils.linkForRef('class:Tree')} in which case a hierarchy of menus and&amp;#010 submenus will automatically be generated to match the tree structure.  See also&amp;#010 ${isc.DocUtils.linkForRef('attr:Menu.dataSource')} for dynamically fetching menuItems and submenus from a&amp;#010 hierarchical DataSource." flags="IRW" name="data">
    <groups>data</groups>
    <setter>setData</setter>
    <examples>${isc.DocUtils.linkForExampleId('fullMenu')}</examples>
</docItem>

could be something like

<docItem defaultValue="null" type="attr" definingClass="class:Menu" ref="attr:Menu.data" flags="IRW" name="data">
    <valueType type="MenuItem[]" description="An array of MenuItem objects, specifying the menu items this menu should show">
    <valueType type="Record[]" description="An array of Records, specifying the menu items this menu should show">
    <valueType type="Tree" description="a Tree in which case a hierarchy of menus and submenus will automatically be generated to match the tree structure">
    <valueType type="RecordList" description="A RecordList object because...">
<valueType type="Record[]" description="An array of Records, specifying the menu items this menu should show">
    <groups>data</groups>
    <setter>setData</setter>
    <examples>${isc.DocUtils.linkForExampleId('fullMenu')}</examples>
</docItem>