Wheelchair Joystick Mouse (movement and buttons) using HID emulation

[deinhofer]

Hi,

I am working on the project Wheely-Joystick-Mouse for wheelchair users, so that they would be able to use the primary wheelchair joystick also as a bluetooth mouse for their computer or Smartphone.

I have implemented support for mouse movement dependig on the tilt level of the puck.js using the built-in IMU.
You can also configure various button press patterns via web UI to trigger mouse button actions.

The mouse movement works pretty well, but sometimes there are annoying interruptions in mouse movement.
It looks like some HID reports were lost.
What could be the problem? Should I got down with the frequency of sending HID reports? How is the NRF.sendInterval parameter related?

It's also important that the algorithm works pretty energyefficient, so after some time of inactivity the puck.js should go idle.
So all hints in this regard are pretty welcome!!

Follow the instructions to flash and configure the puck.s as a mouse joystick:
Wheely-Joystick-Mouse Web-UI

How could I improve the firmware?

[gfwilliams]

Hi - this looks like a great idea!

You could actually add this as something you could flash with https://espruino.com/apps to make it even easier?

It looks like some HID reports were lost.

When the Puck is idle it'll go to a low power connection interval (200ms) but when transmitting it'll go to the highest (7.5ms) - NRF.setConnectionInterval will force that but it's probably ok as-is.

But if you're sending too much information the HID command will fail. I see you're trapping errors from HID.* already - maybe if you get an error, add that command to a list that you then try and send again later? (just make sure you limit the length to maybe 10 items?)

But also I see you're using console.log, and this could be trying to push the data you're logging over bluetooth as well? So that would slow things down and I'd maybe add a if (DEBUG) console.log and set a DEBUG=false up the top.

Also I see I think you're polling at 50ms, but the AHRS code is only updating at 12.5hz (~80ms?) - it might be better if the AHRS code had a callback that called your main code, rather than you querying AHRS.getOrientationDegree?

so after some time of inactivity the puck.js should go idle.

You know if you're idle as you check x/y: https://github.com/inclusion-international/Wheely-Joystick-Mouse/blob/main/src/Espruino/main.js#L293

So maybe have a counter that you increment when x and y are 0, then after it gets to some value you turn the accelerometer off.

The Puck has a mode where it can put the accelerometer into a low power mode and call you when it moves (https://www.espruino.com/Puck.js#accelerometer-gyro), or you could just have it wake up with a button press?

[deinhofer]

Hi - this looks like a great idea!

You could actually add this as something you could flash with https://espruino.com/apps to make it even easier?

ok, makes sense, when it's finished.

It looks like some HID reports were lost.

When the Puck is idle it'll go to a low power connection interval (200ms) but when transmitting it'll go to the highest (7.5ms) - NRF.setConnectionInterval will force that but it's probably ok as-is.

ok, so that setting is only important to save power if there are not writes.
by the way, sometimes when writing the firmware to flash, I get the following error:

But if you're sending too much information the HID command will fail. I see you're trapping errors from HID.* already - maybe if you get an error, add that command to a list that you then try and send again later? (just make sure you limit the length to maybe 10 items?)

But also I see you're using console.log, and this could be trying to push the data you're logging over bluetooth as well? So that would slow things down and I'd maybe add a if (DEBUG) console.log and set a DEBUG=false up the top.

ok, thanks. I will do that.

Also I see I think you're polling at 50ms, but the AHRS code is only updating at 12.5hz (~80ms?) - it might be better if the AHRS code had a callback that called your main code, rather than you querying AHRS.getOrientationDegree?

For mouse movement the HID update rate could be as fast as the accelerometer sampling frequency, but there are cases, where a don't want to send data that rapidly, e.g. when sending keys.
But for mouse I will keep almost the same values.

Would it be more power efficient if I directly call the mouse movement from the accelerometer callback?

so after some time of inactivity the puck.js should go idle.

You know if you're idle as you check x/y: https://github.com/inclusion-international/Wheely-Joystick-Mouse/blob/main/src/Espruino/main.js#L293

So maybe have a counter that you increment when x and y are 0, then after it gets to some value you turn the accelerometer off.

The Puck has a mode where it can put the accelerometer into a low power mode and call you when it moves (https://www.espruino.com/Puck.js#accelerometer-gyro), or you could just have it wake up with a button press?
yes, that's what I had in mind as well. Maybe also use the low-power built-in tilt function of the nRF to wake up and then set the frequency to 12.5 again.

[gfwilliams]

Would it be more power efficient if I directly call the mouse movement from the accelerometer callback?

Marginally, but there's not much in it... Calling it direct would stop you having the risk that maybe you skip one accelerometer reading or use one reading twice though

[deinhofer]

Some other things:

Is it possible to empty the HID-buffer in the case of bluetooth connection?
It seems that old packets will automatically transmitted then, e.g. sometimes I am getting old key presses.

[gfwilliams]

Is it possible to empty the HID-buffer in the case of bluetooth connection?

I don't understand I'm afraid... Old packets shouldn't get queued up between connections - I think usually the HID command fails with an exception if you use it when there is no connection active