Project Proposal

Main Objective –

The Physical therapy application essentially motivates the patients to perform exercises prescribed by their physical therapist and gives real-time feedback on the correctness of their movements. In addition, it generates a final report that summarizes their exercise statistics with percentage accuracy (as compared with the actual exercise)

Subtasks of the above main task and the components involved –

Capabilities of Kinect

Depth sensor that indicates which objects are near versus far in the given scene – so that patient could be notified to be at a correct distance from the Kinect Accelerometer on Kinect - The 3 axis accelerometer provides position information for the Kinect. The X and Y values indicate roll and tilt with the Z axis being a check to determine if the device is upside down. Keep in mind that the Accelerometer only measures tilt and NOT orientation (Z axis). Since Kinect cannot measure Orientation, we use Gyroscope to measure the same. IMU Fusion Board (ADXL345 & IMU3000) – is equipped with both gyroscope and digital 3-axis third-party accelerometer , so that the final data set obtained from the IMU includes includes 3-axis gyroscope data, 3-axis accelerometer data, and temperature data. (and therefore we eliminate the need to use a separate gyroscope as well)

Visual feedback to the user –

We combine the data obtained from IMU fusion board and the Kinect, and provide visual feedback to user whether the movements are correct or not. (We don’t need to use actuators now as the user could visualize the real time feedback of the movements, but we might include some vibrator or alert to further notify the user about the movement.)

The central/host machine for processing the data is a beagle board xm that runs OpenNI and is responsible to monitor the sensor fusion data and provide feedback to the user so the two motion images are one for the actual exercise and one which the user is currently doing. Also,if the patient is supposed to perform a particular exercise with a specific number of repetitions, the screen also keeps on updating the count as the patient keeps finishing exercises.

List of Components involved

• Beagle board xm
• IMU fusion board(IMU 3000) X 3
• Flex sensor
• Microsoft Kinect
• mbed

We would build an arm band with all the sensors on it.

Task Division

Shalini will mainly be in charge of the board porting and setup the environment, both IMU board and Beagle board, and part of the sensor/actuator data acquisition and processing, and communication between all boards. Bo Yuan's task will focus on the hardware connection, wiring and the rest of sensor/actuator data acquisition and processing, and communication between all boards. Mingfei Shao will concentrate on the interactive visualization and communication module of the central machine. Yiran Qin mainly in charge of the fusion of data acquired from both IMU boards and Kinect, and implementation of algorithms to determine the correctness of patient movement.

The project is divided into modules so that it could be done in parallel.

• Oct 19 – Nov 1: Board porting, environment setup, hardware wiring, basic movement capture and skeleton drawing, in-depth research on decision making, forwarding/invert forwarding kinematics.
• Nov 2 – Nov 10: Communication set-up, sensor data acquisition and basic processing, precise movement capture and visualization.
• Nov 11 – Nov 20: Precise data acquisition with better filter and complete processing, fully reliable communication, fully visualized user-machine interactive, roughly precise movement determination and margin projection.
• Nov 21 – Nov 27: Further improve the precision of movement determination.
• Nov 27 – Dec 4: Margin for further tuning and extra features.