14.1p VoiceAssist feedback

[Blama]

Hi Isomorphic,

I just tried Tripe-Control VoiceAssist here (v14.1p_2025-11-09) in the search field, just to try it, and the audio recognition is working great for me in Windows 11 Chromium 142. Pretty cool feature. There seems to be a bug, though. I say "Bermuda", and it's filled into the field, but then on Enter-Key the field is emptied again, and in the Developer Console docs it says "17:15:41.315:WARN:Log:Canceling value dictation". I also assume that the log level here is a bit too aggressive:

*17:15:10.845:INFO:Log:initialized
*17:15:11.035:WARN:Log:New Class ID: 'EditContext' collides with ID of existing object with value 'function EditContext() { [native code] }'. Existing object will be replaced.
This conflict would be avoided by disabling ISC Simple Names mode. See documentation for further information.
*17:15:11.248:INFO:Log:Google Maps APIs loaded correctly
*17:15:11.473:INFO:Log:isc.Page is loaded
17:15:11.924:WARN:VoiceAssist:Speech key handler installed for 'Control' key
17:15:15.043:WARN:Log:Feature enabled: true
17:15:17.142:WARN:Log:Starting value dictation - isc_DynamicForm_0_TextItem_isc_OID_4
17:15:18.844:WARN:Log:Value (interim): ber
17:15:18.942:WARN:Log:Value (interim): berm
17:15:19.152:WARN:Log:Value (interim): bermud
17:15:19.180:WARN:Log:Value (interim): Bermuda
17:15:19.833:WARN:Log:Canceling value dictation

As written in the docs it does not work in Firefox 144, but it also does not work in Edge 142, which is unexpected.

Trying this in my non-default browsers I wasn't logged in at smartclient.com and then could not use VoiceAssist, as it showed your "Ask to be enabled for AI samples popup".
If this is using the browser's SpeechRecognition API, why does it require your AI package? Should not everything be handled by the browser/locally?
Also, the close button in the "VoiceAssist activated"-Notify is a bit mislocated (see attachment).

Best regards
Blama

PS: I'm pretty sure I did not see a "new feature"-blog post about this, and it seems to exist since 14.0. As it is (again) a pretty cool feature, I think a post about it would be good.

[Blama]

PS: In Edge 142 I get the browser "Do you want to allow" popup (see picture) and then a red dot in the tab title in the browser UI for a few seconds after a Ctrl-double tap. But no text is being recognised.

17:24:54.743:WARN:Log:Starting value dictation - isc_DynamicForm_0_TextItem_isc_OID_6
17:24:56.746:TMR5:WARN:Log:Dictated value:
17:24:56.747:TMR5:WARN:Log:Stopping value dictation

[Blama]

Actually, it is working sporadically. I then get a "Bermuda, bermuda", when I definitely only said it once. So perhaps it is more of an Edge problem.

[Isomorphic]

hi Blama, thanks for the feedback.

This is a new feature, despite being ported back some way because, as you say, it's a bit cool. :) So, there will be a blog for this - we just didn't get to it yet.

By way of detail - SpeechRecognition is a browser thing, but speech-to-text doesn't happen in the browser - Chromium uses a proprietary Google service, which is why it works in Chrome, but not in various Chromium-offshoots like Brave, where it is intentionally not implemented for security reasons - ie, sending voice-recordings to a third-party server. In the specific case of Brave, the SpeechRecognition class is present, but it's not fully implemented so there's no way to "turn it on" - you can try to start a recording without errors, but it doesn't register speech or fire any events and stops after a couple of seconds.

Edge now uses a proprietary Microsoft Azure service, so differences in what gets recorded are likely down to that engine. We weren't aware of any differences in processing between Google's and Microsoft's services, but we'll do some testing with recent Edge and let you know if we find anything we can improve.

On the issues:

1) as you say - value-dictation doesn't need AI, so we'll make sure we don't perform that check - note that if you DO have AI, there are some niceties, like you can do voice-filter by holding down Control and saying "show me Calendar samples", for example.

2) we don't see the issue where a value dictated to a filterEditor field gets cleared - even on our 14.0 online showcase we see this working. It could be related to the incorrect requirement for the AI to be present, if the recording is cancelled. We'll look a bit deeper, but please provide an exact sample/steps if it isn't just 'any Country sample, say "Bermuda"'!

3) the logging has been fixed back to 14.1, for the moment - set VoiceAssist to DEBUG if you want to see all the detailed output

4) We'll fix the misplaced close icon in one of the Notifies and get back to you

[Isomorphic]

hi Blama,

A quick follow-up.

We've made some changes - from tomorrow's builds, dated November 20:

1) you shouldn't see any AI-related issues when dictating item-values

2) we've made a change to pre-emptively ask for microphone permissions the first time you triple-tap Control to enable the feature - if it's not allowed, you'll get an isc.say() to that effect - if the browser pops the "Allow Microphone" popup, you'll get the same isc.say() if you disallow the permissions - if you allow the permissions, you'll then see the "VoiceAssist enabled" type Notify, and that means we know it'll work later when you actually try to start a recording

3) the Azure Speech API that Edge uses is *considerably* slower than the Google service - both in the initial handshake and in providing each interim-transcription - we've made a quick change to wait longer for Edge to handshake, and that fixes the issue where it would stop recording before it got around to starting! Note that this slowness from Azure Speech also means that you'll need to wait until you see your spoken words appear on-screen before you double-tap Control to end dictation - otherwise, your dictation will be cut short. We're looking at improving this latter.

4) in Edge, we actually *did* eventually see the issue where if you said, for example, "monaco", the filter value would end up being "Monaco. monaco". The duplication was a non-Chrome bug that's been fixed, and the "Monaco.", with the trailing ".", was caused by the grammar features of the Azure Speech service correcting "monaco" into "Monaco." But that breaks value-dictation in most cases, certainly in the case of filter-values, where punctuation will either break the search or, in some cases, invoke unexpected filtering behaviors. So we've prevented it from adding punctuation at the end.

We're looking at the issue with the close button in the initial Notify moving after initial draw() - it's a Notify issue rather than VoiceAssist per-se - we'll update when we have more to say.