Can Willow Speak?

[ggalt]

I'm working on getting Anylist (my preferred grocery list app) to work under Home Assistant and Willow using this reverse engineered API. My question is, how do I send commands back to Willow so that Willow will respond (e.g., "Alexa, is 'lettuce' on my grocery list?" and getting a yes or no answer)?

[kristiankielhofner]

Willow can speak with TTS.

If you're using Home Assistant and have TTS responses configured (with a Willow Inference Server URL) Willow we speak whatever HA returns from the pipeline.

I haven't tried it yet but the latest release of HA supports returning text from services.

[ggalt]

Thanks! This is exactly what I was looking for. I am just learning to navigate both Home Assistant and Willow, so I really appreciate the direction.

[myxor]

Did anyone get this working with current HA versions?

[nikito]

Yes, running with latest HA and TTS is working here. You do have to do some config changes in willow to enable it however, it is not enabled by default. Thanks and Best Regards, Nick Bento

…

On Wed, Aug 2, 2023 at 2:50 PM Marco H ***@***.***> wrote: Did anyone get this working with current HA versions? — Reply to this email directly, view it on GitHub <#226 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTDKPWESQ5RR4IKNEMUNDXTKOPFANCNFSM6AAAAAA2C2JSNY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[myxor]

I am thinking of a (REST) endpoint on Willow which can be called from HA with some String which then will be read by Willow via TTS. Is this possible?

[murtoz]

Yes, running with latest HA and TTS is working here. You do have to do some config changes in willow to enable it however, it is not enabled by default.

Any chance you could give a high level overview of steps/config changes needed to get TTS working on willow

[nikito]

It would depend on what version of Willow you are using. If you have WAS running and are using the WAS willow builds you can do this pretty easily in the WAS UI. Thanks and Best Regards, Nick Bento

…

On Tue, Aug 8, 2023 at 9:09 AM murtoz ***@***.***> wrote: Yes, running with latest HA and TTS is working here. You do have to do some config changes in willow to enable it however, it is not enabled by default. Any chance you could give a high level overview of steps/config changes needed to get TTS working on willow — Reply to this email directly, view it on GitHub <#226 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTDKOLZTZOC2VL4J6RNVDXUI3CBANCNFSM6AAAAAA2C2JSNY> . You are receiving this because you commented.Message ID: ***@***.***>

[murtoz]

Thanks Nick, WAS was the missing link - I only just got willow up and running this morning following the instructions from the willow repository, and had completely missed WAS. Will get it set up now and have a play.

[nikito]

Note that WAS only works if willow is built from feature/was branch; the main branch of willow does not yet work with WAS. Once WAS is configured and running, you can use it to re-flash your ESP32 with a WAS compatible build. :) Thanks and Best Regards, Nick Bento

…

On Tue, Aug 8, 2023 at 10:16 AM murtoz ***@***.***> wrote: Thanks Nick, WAS was the missing link - I only just got willow up and running this morning following the instructions from the willow repository, and had completely missed WAS. Will get it set up now and have a play. — Reply to this email directly, view it on GitHub <#226 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTDKJ7K7TPICM3ZRAB5JDXUJC3TANCNFSM6AAAAAA2C2JSNY> . You are receiving this because you commented.Message ID: ***@***.***>

[rlust]

Looking for instructions on how to allow HA to use the ESP32-Box to speak. I am running WAS on ESP and it is working well.

[nikito]

Replying here as my email reply seems to have missed stuff. To enable TTS you have to turn it on in the WAS Config area, and supply the URL to the TTS endpoint:

[rlust]

Thanks!

Is it possible to send speech from Home Assistant?

[nikito]

Yes, if you have assist configured there are built in commands that speak a reply. Otherwise you'd need to write your own intent scripts that return speech. I believe this is documented in the home assistant documentation. 🙂

[thinkbig1979]

Hi all, the OP asked if Willow can speak, and the focus very quickly shifted to HA, and TTS responses to commands.
However, I'd like to zoom out here a bit.

I don't use HA, but I do automate stuff, and I would very much like to be able to use my willow devices to produce spoken output whenever I want. If I set an alarm for 10AM, I would love to be able to send a text to an endpoint at 10AM, and have my Willow device say "This is your reminder to go to the dentist. Dumbass!"
So not just as a response to a command (which I would also like to do outside of HA, but can't find any documentation on), and not just to repeat my command (which is what it currently does if I have TTS set as "Audio Response Type" in WAS).

Essentially, what would be great is to have just an endpoint that I can send a bit of text, along with the ID of the device I want it to output speech on. Perhaps I have missed it somewhere, but is this already possible?

[thinkbig1979]

Nice! Looking forward to trying it out! :-)

[zolakk]

Ok. I have the command endpoint set up to OpenHAB so I guess i'll take the OpenHAB configuration side over to their forum and see if I can sort that end out. Excited about the progress on this project though, it's super cool!

[kristiankielhofner]

From what I remember the OpenHAB speech/intent stuff is fairly limited. You configure your NLP engine, set a default, and Willow hits the OpenHAB URL. The issue is last I looked it only comes back with HTTP status code. No JSON, no output text, nothing like that. In Willow all we're able to do is show/playback Success or Fail.

[zolakk]

Yeah that lines up with my experiences so far with it. I'm hoping then, absent of getting a proper response back I can utilize the upcoming notification REST endpoint as a workaround if possible. I know I can also switch to Home Assistant to get it working properly but it would be a lot of work I'd like to avoid

[stintel]

This is actually a new feature we are finishing up in Willow/WAS that will be released very soon! :)

Now available in WAS 0.2.0-rc.1 and Willow 0.3.0-rc.1!

[jazzmonger]

I'm working on getting Anylist (my preferred grocery list app) to work under Home Assistant and Willow using this reverse engineered API. My question is, how do I send commands back to Willow so that Willow will respond (e.g., "Alexa, is 'lettuce' on my grocery list?" and getting a yes or no answer)?

Back to the ORIGINAL question...

@ggalt did you make any progress with this? My household has been on Anylist since the early beta days years ago. It literally runs my life. It is the #1 most used app in our family, on ALL platforms. I have 30+ lists and I must have 300+ recipes in it as well as I am also a chef. Shopping with my wife in the same store is now actually a pleasure instead of a fight.
The integration w/ Alexa works very elegantly. Lack of this functionality will be probably the only thing that prevents me from unplugging Alexa permanently. If we can nail this, it would be huge.

[ggalt]

@jazzmonger sorry to be MIA, but between work and the holidays, I've been slammed. I have not yet gotten Anylist into Home Assistant/Willow. I started on a javascript implementation that works from the command line (you can see it here). It's rather basic and focuses only on the grocery list functionality. However, I did find that there has been a somewhat recent addition to Home Assistant that purports to work with Anylist. I haven't had time to try it out. But in either case -- mine or the HA add-on -- the glue to take commands from Willow and pass back something intelligent still needs to be done. I'm hoping that around the end of the year, I might find time to work on this some more. Let me know if you make any progress.

[jazzmonger]

I see that! I 've tried getting it working, but keep getting a 500 error.
kevdliu/hacs-anylist#5

But, assuming that eventually works, I now see you're really after providing meaningful responses via Willow. did you see the latest WIS server release supports TTS notifications on the Box3? I installed it and it works GREAT. Lots of options as well!

rest_command:
  willow_notify:
    url: http://willow.local:8502/api/client?action=notify
    method: POST
    content_type: application/json
    payload: '{"cmd":"notify","data":{"backlight":"{{backlight}}","backlight_max":"{{backlightMax}}","repeat":"{{repeat}}","audio_url":"{{audioUrl}}","volume":"{{volume}}"}}'

I also created a complete Willow Dashboard w/ a ton of near-realtime sensors using the SSH integration.

I've been SUPER impressed w/ Willow so far. Speech recognition is like 99+% accurate. Its about as easy and straightforward as its going to get for a local GPU powered inference server.

[ggalt]

Wow! That is some fantastic work! Have you posted the dashboard code? I'd love to take a look.

On the 500 error . . . could it have been 503, by any chance? Take a look at this issue on @codetheweb's page. It seems the authentication method for Anylist changed a while back. If the HA add-on didn't update (or if you are using an older version of @codetheweb's library), you will get a 503 error.

Thanks for the YAML code. I'll take a look this weekend and see if I can get it working (I HATE Q4!!).

[jazzmonger]

Q4... - I'm in the wine biz, we call it OND. Oct, Nov, Dec and we're SLAMMED.

it is a 500 error. I can see all my lists in HA, I just can add to them. Its probably something simple.

I'll post the dashboard stuff on my GitHub. It wasn't straightforward at all.... took me 3 days and a TON of help from the Linux Discord channel to get all the pieces working.

https://github.com/jazzmonger/Willow-Dashboard/blob/main/README.md