AR application in physical therapy

Overview

AR Application in Physical Therapy is an interdisciplinary project conducted in collaboration with the Faculty of Physical Therapy at October University for Modern Sciences and Arts (MSA). This innovative project aims to enhance physical therapy practices by integrating advanced technologies, including augmented reality (AR) and machine learning, to improve patient outcomes and provide better training tools for therapists.

In this project, we utilized a combination of benchmarking datasets and data collected within our university clinics, facilitated by the faculty of physical therapy. The primary objective was to create a robust system that can accurately assess and guide physical therapy exercises using cutting-edge techniques.

Techniques Employed

Pose Estimation: Using Mediapipe framework to accurately track and analyze body movements during exercises in real-time with low-latency.
Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs): Implementing deep learning models to process and analyze the data collected from physical therapy sessions.
Signal Processing with Continuous Wavelet Transform (CWT): Converting signals into images to leverage the power of image-based deep learning models. This technique allowed us to apply transfer learning on pre-trained models effectively.
Vision Transformer and Time-Series Transformer: Employing transformer models to handle both visual and temporal aspects of the data, providing more accurate and comprehensive analysis.
Few-Shot Learning: Experimenting with few-shot learning techniques to improve the system’s ability to generalize from a limited amount of data.
Augmented Reality Visualization: Developing an AR environment to visualize a 3D skeleton performing the exercises. This approach ensures patient privacy by not displaying the actual doctor, while still providing clear and accurate guidance.

Key Contributions

Interdisciplinary Collaboration: The project was made possible through the collaborative efforts of the faculties of computer science and physical therapy, combining expertise in technology and clinical practice.
Data Collection and Utilization: Leveraging both existing benchmarking datasets and newly collected data from university clinics to build a comprehensive and effective system.
Advanced Machine Learning Techniques: Integrating various advanced techniques, including pose estimation, CNNs, RNNs, transformers, and few-shot learning, to enhance the accuracy and effectiveness of the physical therapy guidance system.
AR Visualization: Using augmented reality to create an immersive and interactive training environment, improving the learning and execution of physical therapy exercises.
The project resulted into 2 conference publications.

Publications

Physical Rehabilitation Exercises Classification Using Deep Learning Models. 2024 14th International Conference on Electrical Engineering (ICEENG) DOI.
Enhancing Physical Therapy through Transformer-Based Models: A Study on Exercise Classification. 2024 Intelligent Methods, Systems, and Applications (IMSA) DOI.

This project represents a significant advancement in the field of physical therapy, showcasing the potential of interdisciplinary research and the application of modern technologies to improve healthcare outcomes.

Student List

Abdallah Hamdy LinkedIn
Alaa Taie

Main Supervisor

Asoc. Prof. Ayman Ezzat LinkedIn

Did you find this page helpful? Consider sharing it 🙌