Orca & Cosmic

[a11yChief]

Hey all,

Opening this as something of a discussion area because screen reader accessibility of cosmic is currently in a pretty dire state. There is limited to no feedback from any basic interactions with the system, including opening a terminal, using the app launcher by pressing super key, typing after using the super key, navigating to the desktop, opening the files app, arrowing through an icon view of files, and probably a bunch of other stuff I can't test due to the lack of voice feedback on all of the above.

I caught Brodie Robertson's video recap of the launch call, and noted the team has made efforts to accommodate accessibility. Currently, I'm unable to test any part of the system for more useful feedback around this welcome announcement, besides noting the above.

What are the best next steps from here, to figure out exactly what the issues are and how they can be addressed?

I appreciate this is an Alpha, but sorting it now will make it a lot cheaper to maintain accessibility down the road. It would also make cosmic the ONLY accessible tiling window option for blind / visually impaired people, as far as I know. I've been waiting for this day for about twenty years.

Looking forward to helping where I can :D

P.S: I'm using arch BTW

[ids1024]

Good screen reader support is challenging, but the situation is at least less dire than it appears. Since there's been work but it hasn't been merged yet.

There's currently no accepted way to pass keyboard input to screen readers on Wayland. So Orca's key bindings don't currently work with GTK4 on Gnome Wayland (GTK3 used a legacy mechanism, and the new one is annoyingly X11 only currently). In pop-os/cosmic-comp#759 I've been working on a protocol for this. There are still a few things to fix there but it seems doable enough.

That requires a patched version of libat-spi2-core for Orca to use the protocol, so use on other distros will depend on packaging of that.

I'm not sure if other Wayland compositors will favor this particular way of solving the problem, but I'll probably propose it upstream. Hopefully this at least shows more clearly what is needed. Previous discussion of Wayland protocols or DBus portals for this seems to have stalled without getting as far as a working prototype like this.

For our clients to have accessibility support, Iced needs to integrate it. We've done some work on that a while ago in iced-rs/iced#1849, using the Accesskit library. We hope to update that work soon and try to get that merged upstream in Iced. Though getting all widgets and all apps to provide good accessibility metadata is not a small task either.

[aaron-gh]

There still doesn't appear to be any progress here. I just installed alpha 7 and I was unable to interact with the system at all. I was only able to get to a browser by guessing after pressing the super key. I cannot test further due to this.

[a11yChief]

@aaron-gh it's not that there hasn't been progress; I actually lost track of this thread but @ids1024 mentions work in the background projects (Wayland protocols, UI Libraries etc) that will pave the way for Wayland Only Desktop Environments to be screen reader accessible across the board.

@ids1024 Cheers for the reply, and sorry it's taken so long to spot it. Couple follow ups, but mad props for such a dedicated understanding of what's actually required.

"There's currently no accepted way to pass keyboard input to screen readers on Wayland. So Orca's key bindings don't currently work with GTK4 on Gnome Wayland (GTK3 used a legacy mechanism, and the new one is annoyingly X11 only currently)."
Am I right in thinking this is no longer the case? Been daily driving gnome with Orca since KDE began splitting kwin to go wayland only, and keyboard stuff like control to shut up has been working way more reliably than prior wayland sessions since a couple months ago. If this was your patch then thank you.

"For our clients to have accessibility support, Iced needs to integrate it."
Amen. And more UI Libraries need this attention. How far along is this? Have they accepted anything upstream yet? I'm learning rust in my free time so if there's any targetted testing that you'd like me to conduct on individual UI elements, I'd be more than happy to do it.

What are the remaining boundaries and is there anything VI Linux users can do to speed things along?

Cheers :)

[jackpot51]

We are close to enabling some features that will make things just start working with orca. Cosmic apps right now do not interact correctly but we do have the plumbing with accesskit to fix it and I hope to improve things before the first stable release.

[ids1024]

Yeah, Gnome recently added a DBus protocol for accessibility key bindings. This is different (and probably a bit better) then the version I implemented for COSMIC. pop-os/cosmic-comp#1412 will add support for that to cosmic-comp.

We do still need to update our code for Acceskit integration in Iced, and get that upstreamed.

[aaron-gh]

Yeah, Gnome recently added a DBus protocol for accessibility key bindings. This is different (and probably a bit better) then the version I implemented for COSMIC. pop-os/cosmic-comp#1412 will add support for that to cosmic-comp.

We do still need to update our code for Acceskit integration in Iced, and get that upstreamed.

If what you're addressing here is the patch to enable Orca keys to work, that works extremely well in Gnome Wayland and if it's possible to bring that across to Cosmic that would fix the accessibility keybinds problem I think in most cases.

[aaron-gh]

We are close to enabling some features that will make things just start working with orca. Cosmic apps right now do not interact correctly but we do have the plumbing with accesskit to fix it and I hope to improve things before the first stable release.

I'm a full-time Orca user, so I'm happy to test things as they come. Happy to see this is being worked on; it will be great to have another accessible Desktop option under Wayland

[jackpot51]

I am curious, do you use the default key bindings or have you made changes?

[a11yChief]

I am curious, do you use the default key bindings or have you made changes?

I change them where I can on account of using multiple OSs and screen readers, way easier to have similar shortcuts across Windows Mac and Linux for muscle memory.

[ids1024]

Good to know the keybindings in Orca are working well on Gnome. From the implementation, I think it should be as good or probably better than how it works on X11 (though there could be more improvement), but it's hard to test when I'm not very used to using screen readers.

The cosmic-comp PR to add the protocol probably need a little more work before it's ready to test, but then keybindings should have parity with Gnome.

[aaron-gh]

I am curious, do you use the default key bindings or have you made changes?

I''ve been a Linux user for years so the default Orca bindings make sense to me.
I do use a laptop though so use laptop layout and add some more for things that aren't mapped by default, copying flat review text to the clipboard etc.

[aaron-gh]

Good to know the keybindings in Orca are working well on Gnome. From the implementation, I think it should be as good or probably better than how it works on X11 (though there could be more improvement), but it's hard to test when I'm not very used to using screen readers.

The cosmic-comp PR to add the protocol probably need a little more work before it's ready to test, but then keybindings should have parity with Gnome.

I woulsn't say better than x11; some things still aren't possible under Wayland.

Things like copying to the clipboard with flat review for example, because Wayland requires any application that's doing clipboard opperations to have an open window and Orca doesn't

[ids1024]

Yeah, I mean the keybindings in particular should be implemented in a way that's better than on X (to the extent that makes a difference in practice), while most functionality of Orca doesn't really care if the application is on X11 or Wayland. Some other features may still have issues on Wayland.

[a11yChief]

I woulsn't say better than x11; some things still aren't possible under Wayland.

Things like copying to the clipboard with flat review for example, because Wayland requires any application that's doing clipboard opperations to have an open window and Orca doesn't

Lol yeah I have to open a browser or terminal just to get Orca to output the time with INSERT+T. I can't even open the screen reader preferences with the screen reader running, (INSERT+SPACE) until I have n application open on my screen in the workspace I'm in at the time. Sadly I need to use Wayland as my default because HDR helps me avoid eye-strain.

[a11yChief]

Hey all,

I swear this isn't me having a go, because it's clear you've put a lot of thought into how to build Cosmic to be accessible from day one for screen reader users.

Orca is still silent in the Beta. Could we please have a progress report on where things are at the moment?

Also, if this is unable to ship by the final release, would it be worth System76 putting a report together on just how difficult a task this has been? It would be a good opportunity to showcase the state of Linux accessibility in general.

It's pretty wild that a company with actual backing and an intent to do the right thing hasn't been able to integrate accessibility into their own project, even when the task is fully understood by engineers.

Happy to help in any way I can :D