1. Abstract

In the field of physical medicine and rehabilitation, biofeedback is a valuable tool, aiding in the recovery of individuals with joint injuries. Biofeedback systems improve rehabilitation by boosting motivation and precision and they provide real-time feedback on exercise performance, enabling self-assessment and progress. Conventional physiotherapy programs often entail high costs and resource allocation. As a cost-effective alternative, self-administered training systems allow patients to continue their rehabilitation at home, though this shift presents challenges related to potential human errors. Recent developments include textile solutions with integrated sensors for comfortable and efficient wear. Therefore, this proposal introduces a low-cost hydraulic soft filament sensor (SFS) integrated into a flexible brace, aimed at monitoring joint angles and guiding users through rehabilitation via a video interface. It incorporates an intelligent calibration technique leveraging a neural network (NN) model, wireless technology, onboard signal processing, a smartphone app, and assessment algorithms using machine-learning models tailored for knee and elbow injuries. This advancement improves the interaction between humans and machines during rehabilitation. Human-subject experiments at Vinmec Times City International Hospital will validate the system’s effectiveness, marking a pioneering effort in physical medicine and rehabilitation at Vinmec and Vinuniversity. This grant proposal not only addresses a critical need in rehabilitation but also offers a unique opportunity for interdisciplinary education and collaboration between the University of Illinois Urbana-Champaign and Vinuniversity.

2. Background

Biofeedback has emerged as a valuable tool in the realm of physical medicine and rehabilitation, contributing significantly to the recovery of individuals dealing with joint injuries and treatments [1]. Biofeedback systems incorporate entertainment to motivate patients and boost their engagement, precision, and exercise frequency, making the rehabilitation process more enjoyable and productive [2]. These systems offer real-time feedback on body position and orientation during exercises, with data presented in diverse forms like numerical values and graphical displays [3]. Users can use these data to assess their performance and make necessary improvements. Traditional manual and physical exercise-based physiotherapy programs in rehabilitation centers often entail substantial expenses and resource allocation [4]. To address these challenges, self-administered training systems have emerged as a promising alternative. These systems allow patients to continue their rehabilitation journey in the comfort of their own homes, significantly reducing costs and enhancing convenience [5]. However, this shift towards home-based rehabilitation also brings challenges, primarily related to the potential for human error that can compromise the effectiveness of rehabilitation protocols. To address this, many of these systems utilize low-cost inertial measurement units (IMUs) for monitoring and guidance [6]. In recent developments, textile solutions with integrated sensors have proven to be a comfortable and efficient means of constructing sensing systems that users can wear seamlessly [7].

Ideal sensors for the application should seamlessly be integrated without hindering motion or adding rigidity [8]. Traditional sensors often fall short of meeting these requirements [9]. Soft sensors have emerged as promising solutions for flexible bodies [10]. They need to match or exceed the body’s operational strain and have a low elastic modulus to avoid interference [11]. Additionally, they should be able to deform in multiple directions in sync with the target’s motion. Soft sensors based on resistance, capacitance, or magnetic fields were common but suffered from signal interference due to complex electrical wiring [12]. A recent breakthrough involves fluid pressure-based mechanisms, offering tactile, force, pressure, and strain sensing. However, existing soft pneumatic sensors faced integration challenges and miniaturization issues. Our recent publication introduced a hydraulic soft sensor with tunable sensitivity, surpassing existing fluidic sensors [13]. This advancement forms the basis for the filament sensor proposed in this project.

Figure 1. Accelerating patient rehabilitation via wearable filament sensor networks.

3. Research Objectives and Clinical Outcomes

3.1. Significance outcomes for key stakeholders

a/ A new low-cost hydraulic soft filament sensor integrated into a flexible brace to monitor a patient’s joint angle and guide the user through a video interface for rehabilitation and an intelligent calibration technique leveraging a deep reinforcement learning model. Remarkably, this flexible brace comes at a fraction of the cost, estimated to be less than $30, as compared to the substantial $4486 expenditure associated with standard rehabilitation after anterior cruciate ligament reconstruction [14]. By enabling patients to carry out their rehabilitation from the comfort of their own homes, this proposed device holds the potential to significantly reduce the need for hospital visits, especially for those residing in rural or densely populated areas. This will be the first sensing system using soft sensors for accelerating patient rehabilitation established at VinUni and in Vietnam.

b/ A wireless technology using Near Field Communication (NFC) and Bluetooth Low Energy (BLE) and an onboard circuit for signal acquisition and processing as well as a smartphone app with specialized exercise videos catering to knee and elbow injuries, leading to more intuitive and natural interactions between humans and machines.

c/ Human-subject experiments to verify the flexible brace with integrated filament sensors at the Motion Lab, Vinmec Times City International Hospital. This will be the first time that patients recovering from surgical procedures or those with mobility impairments are studied with a sensing system for physical medicine and rehabilitation at Vinmec and Vinuniversity.

d/ Accelerated, low-latency, real-time signal processing from the sensor with the supercomputer at NCSA in combination with edge devices deployed at Vinmec.[13, 15]

e/ This grant will uniquely facilitate the training of Ph.D. and M.S. students across various scientific fields and forge new collaborations between the University of Illinois Urbana-Champaign and Vinuniversity.

3.2. Overview of the Filament Sensor with a Wireless Human-Computer Interface

This project revolves around developing a low-cost hydraulic soft filament sensor with wireless technology and a user-friendly human-computer interface to monitor a patient’s joint angle and guide the user through a video interface for physical medicine and rehabilitation. At its core, the filament sensor utilizes a hydraulic pressure-based tubular strain sensor, boasting a flexible micro-sized coil filled with an incompressible fluid. This design allows the sensor to adapt to various forms, from long fibers to skin-like structures. A designed filament sensor, in the shape of a flexible brace, offers continuous monitoring and support by presenting data in the simplest forms such as numeric joint angles, easy-to-understand geometrical representations, or simple bar plots. This thoughtful approach aids patients in the gentle correction of their posture during their rehabilitation and physical therapy routines in the comfort of their own homes. The integration of wireless technology seamlessly connects an onboard circuit to a smartphone app, enhancing its practical applications. Several specialized exercise videos tailored to address specific injuries at the knee and elbow are readily accessible in the developed smartphone app for monitoring, assessing performance, and guiding patients. If the advanced smartphone application identifies any health concerns, it promptly notifies the patient’s doctor while also recommending that the patient schedule a visit to the hospital for a comprehensive check-up. Furthermore, continuous monitoring in post-injury patients not only helps in the early detection and correction of abnormal posture but also guides the education of individuals about maintaining a healthy posture, thereby preventing long-term complications.

3.3. Approach

Aim 1: Develop a new low-cost hydraulic soft filament sensor based on previous publications with low-profile, robust, and sensitive characteristics and a sensor intelligent calibration technique (Dr. Mai Thanh Thai, Prof. Elizabeth T. Hsiao-Wecksler, and Girish Krishnan). (Years 1-3)

Firstly, the investigation centers around the dynamic interaction between a flexible micro-sized coil and an incompressible fluid. This exploration is geared towards optimizing sensor sensitivity, paving the way for more accurate measurements and reliable data acquisition. To enhance robustness and sensitivity, an incompressible fluid is proposed as an alternative to water, as previously mentioned in our publication [13]. This substitution aims to fortify the filament sensor’s performance, making it more adaptable and reliable under various conditions. Furthermore, the project seeks to conceive an efficient method for seamlessly integrating the filament sensor into a flexible brace without compromising its performance as proposed by [16]. This integration is crucial for expanding the practical applications of the sensor, especially in fields where flexibility and mobility are paramount. Given Vietnam’s prominent position as one of the world’s leading garment and textile exporters [17], there is substantial merit in establishing fabric-based systems like the one proposed in this project. In addition, the research pioneers the development of an intelligent calibration technique, leveraging a neural network (NN) model. This innovative approach holds the potential to revolutionize sensor calibration, enabling automatic adjustments and precise tuning, ultimately enhancing the accuracy and reliability of sensor data as presented by [18]. Lastly, VinUni students will have the valuable opportunity to acquire a wide range of skills and knowledge. They will not only gain expertise in fabricating the soft filament sensor and the flexible brace and setting up experiments to thoroughly understand the sensing system at VinUni, but they will also receive comprehensive training in the working principles and cutting-edge designs of intelligent calibration techniques, employing NN models at UIUC.

Aim 2: Develop a wireless technology and an onboard circuit for signal acquisition and processing (Dr. Van Dinh Nguyen and Dr. Mai Thanh Thai). (Years 1-3)

In this aim, we will design a comprehensive onboard circuit, featuring the ARM Cortex-M4 microcontroller as we did in [19], to effectively acquire and precisely filter pressure data from the filament sensor at Vinuniversity. Additionally, we will develop a novel wireless solution, centered around Near Field Communication (NFC) and Bluetooth Low Energy (BLE), that jointly optimizes key aspects such as bandwidth, power efficiency, range capabilities, and alignment with the application requirements. This approach aims to exploit the strengths of NFC and BLE to cater to the unique needs of WBANs, ensuring efficient and tailored wireless communication for vital sign collection and data transfer. To further enhance flexibility and power conservation, we will develop a multimodal communication system, allowing devices to seamlessly switch between the two protocols and WiFi and cellular networks based on proximity and data transfer requirements. This holistic approach is geared toward providing VinUni students with the knowledge and skills required to not only construct electronic circuits but also stay abreast of the latest advancements in wireless technologies.

Aim 3: Build a smartphone app with specialized exercise videos tailored to address specific injuries at the knee and elbow. These videos will be utilized in the user study for Aim 5 (Dr. Mai Thanh Thai, Dr. Van Dinh Nguyen, and Prof. Hung Manh La). (Years 2-4)

The body parts that experienced the highest frequency of injuries were the knees (21.3%), and knee injuries were particularly impactful in terms of their impact on soccer players’ absence from the game [20]. The rehabilitation of a knee injury requires an average of 45 days for Professional Male Soccer Players in the Netherlands. Additionally, elbow injuries (19.6%) ranked as the second most common cause of injury in Major League Baseball over 18 seasons from 1998 to 2015 [21]. A proactive approach to healthcare and rehabilitation is being taken with the creation of customized exercise videos tailored to accelerate the recovery process for individuals dealing with knee and elbow injuries. These videos are specifically designed to target and alleviate the issues afflicting patients, ensuring a personalized approach to recovery. Additionally, our team will develop cutting-edge assessment algorithms using machine-learning models that not only motivate but also accurately evaluate patient progress throughout their rehabilitation journey. To ensure the safety and well-being of our patients, we will implement alert features that promptly notify medical professionals of any potential health concerns that may arise during the rehabilitation process. Our comprehensive approach to injury rehabilitation seeks to provide the highest level of care and support for our patients. With this aim, VinUni students will not only acquire technical skills but also learn how to create a professional smartphone application and a state-of-the-art machine-learning model to assess patient performance.

Aim 4: Accelerating overall model training and inference using supercomputers at NCSA (Dr. Phuong Cao). (Years 3-5)

The sensor design in Aim 1 will be optimized for better fit and accuracy through Markov chain Monte Carlo (MCMC) simulation of different sensor materials, body shapes, and injury types. This simulation follows best practices of NCSA’s Industry partner program that have been successfully employed by Abbvie, Eli Lilly, and Boeing. Furthermore, the energy model in Aim 2 will be trained using a reinforcement learning model to minimize power consumption usage on the filament sensor network. Different deployment scenarios will be concurrently evaluated to find the optimal sensor network arrangement and antenna array orientation angle. Lastly, the smartphone app and the user study will be evaluated on a different, diverse cohort of patients in Illinois, U.S.A., facilitated by NCSA’s health care innovation office, UI Health, Carle Hospital, Mayo Clinic, and Phuong Cao.

Aim 5: Conduct a comprehensive user study with over 30 participants to showcase the enhanced effectiveness of the sensing system in physical medicine and rehabilitation, and scaling up the prototype while incorporating valuable lessons learned for seamless technology transfer (Dr. Mai Thanh Thai, MD. Ngoc Minh Ho, Dr. Van Dinh Nguyen and Elizabeth T. Hsiao-Wecksler). (Years 4-6)

• Phase 1: Utilize a diverse group of 15 healthy individuals selected from Vinuniversity to test and refine our research prototypes under the guidance and oversight of Vinmec’s medical professionals.
• Phase 2: Extend our research by enrolling an additional 15 participants, who are post-injury patients with knee and elbow conditions, from the Motion Lab at the Center of Orthopedics and Sports Medicine, Vinmec Times City International Hospital because Vinmec can access to large patient population for clinical studies.
• Phase 3: In the last phase, we will scale up the prototype to a larger scale leveraging the lessons learned during the development process. This phase involves refining and enhancing the existing prototype based on insights gained from initial testing and user feedback. It will result in a more robust and efficient system that can be deployed on a larger scale to benefit a wider audience. Technology transfer is another key component of this project, aiming to disseminate the expertise and technology developed to various stakeholders (e.g. Vinmec). Through collaboration with Vingroup electronics and biotech manufacturing, academic institutions, medical professionals, and rehabilitation centers, the project seeks to ensure that the knowledge and innovations generated are effectively shared and applied, ultimately contributing to advancements in the field of physical medicine and rehabilitation.

3.4. Recruitment, Training, Mentoring, and Publication Strategies

Students from various scientific backgrounds at VinUni will be recruited and mentored by both VinUni and UIUC PIs. Monthly remote meetings via Zoom or Teams will track their research progress and professional development. Our plans include recruiting two prospective PhD students in 2024, specializing in mechanical engineering for Aim 1 and electrical engineering for Aim 2, from Hanoi University of Science and Technology. In the subsequent years, from 2025 to 2027, we aim to welcome three more PhD students. They will have majors in software engineering, big data engineering, and biomedical engineering, corresponding to Aims 3, 4, and 5, respectively. Additionally, we plan to have three Master’s students, two in mechanical engineering and one in electrical engineering, as well as one postdoc researcher. In December, we will host a special recruitment seminar. The event will feature a distinguished panel, including (1) faculty members from UIUC and VinUni who will discuss the latest research trends, challenges, and opportunities, (2) VinMec doctors who will share their experiences in the rehabilitation field and emphasize the technological needs in the medical sector, (3) past VinUni and Illinois students who will share their valuable insights from internship experiences. This seminar is set to coincide with the SoICT 2023 conference in HCMC, ensuring a diverse audience of students and healthcare experts. The seminar itself will be a hybrid event, with VinUni’s PIs present in person in Hanoi and HCMC, providing an opportunity for direct interaction with students. Simultaneously, we will broadcast the seminar to engage interested VinUni students. Following the seminar, the selection process for prospective students will commence promptly. Lastly, research results will be openly shared with the scientific, medical, and public communities through peer-reviewed journals, national conferences, and public press releases. Sensing system designs and the related software algorithms will be available through collaborative research agreements to facilitate the broad use and expanded development of our sensing approach for physical medicine and rehabilitation.

3.5. Funding Allocation

$20K per a PhD student enrolled at VinUniversity includes $5K of article publication charges, $5K of traveling fees and conference fees, $6K of lab facility and clinical data collection, and $4K of consumable supplies.

3.6. Future development and collaboration

This project opens avenues for potential expansion and securing external funding containing VinIF and Nafosted. Promising avenues include the application of the developed sensing system in Vietnamese rehabilitation centers and marketing the design as a home-based rehabilitation and posture correction product. This project provides valuable chances to collaborate with esteemed faculty from Illinois and VinUni in the fields of robotics [22], supercomputing [23], microsystems [24], biomechanics [25], and data analytics [26]. This research will serve to further deepen the collaborative ties between VinUni, Vinmec, UIUC, and the University of Nevada Reno.

4. References

1. O. M. Giggins, U. M. Persson, B. Caulfield, Journal of neuroengineering and rehabilitation 2013, 10, 1.
2. A. Leardini, G. Lullini, S. Giannini, L. Berti, M. Ortolani, P. Caravaggi, Journal of neuroengineering and rehabilitation 2014, 11, 1.
3. J. McClelland, J. Zeni Jr, R. M. Haley, L. Snyder-Mackler, journal of orthopaedic & sports physical therapy 2012, 42,135.
4. S. Howard-Wilsher, L. Irvine, H. Fan, T. Shakespeare, M. Suhrcke, S. Horton, F. Poland, L. Hooper, F. Song, Disability and Health Journal 2016, 9, 11.
5. L. Prosperini, D. Fortuna, C. Giannì, L. Leonardi, M. R. Marchetti, C. Pozzilli, Neurorehabilitation and neural repair 2013, 27, 516.
6. D. T.-P. Fong, Y.-Y. Chan, Sensors 2010, 10, 11556.
7. T.-W. Shyr, J.-W. Shie, C.-H. Jiang, J.-J. Li, Sensors 2014, 14, 4050.
8. J. Yin, R. Hinchet, H. Shea, C. Majidi, Advanced Functional Materials 2021, 31, 2007428.
9. D. Rus, M. T. Tolley, Nature 2015, 521, 467.
10. S. Mousavi, M. T. Thai, M. Amjadi, D. Howard, S. H. Peng, T. N. Do, C. H. Wang, Journal of Materials Chemistry A 2022, 10, 13673.
11. C. Tawk, G. Alici, Advanced Intelligent Systems 2021, 3, 2000223.
12. G. Gerboni, A. Diodato, G. Ciuti, M. Cianchetti, A. Menciassi, IEEE/ASME Transactions on Mechatronics 2017, 22,1881.
13. J. Davies, M. T. Thai, T. T. Hoang, C. C. Nguyen, P. T. Phan, H. P. Phan, N. H. Lovell, T. N. Do, Advanced Materials Technologies 2023, 8, 2201453.
14. C. J. DeFrancesco, D. G. Lebrun, J. T. Molony Jr, M. R. Heath, P. D. Fabricant, The American Journal of Sports Medicine 2020, 48, 1100.
15. M. T. Thai, P. T. Phan, T. T. Hoang, H. Low, N. H. Lovell, T. N. Do, IEEE Robotics and Automation Letters 2021, 6, 5089.
16. P. T. Phan, M. T. Thai, T. T. Hoang, J. Davies, C. C. Nguyen, H.-P. Phan, N. H. Lovell, T. N. Do, Scientific reports 2022, 12, 11067.
17. R. Nayak, M. Akbari, S. M. Far, Journal of Cleaner Production 2019, 225, 291.
18. J. Davies, M. T. Thai, T. T. Hoang, C. C. Nguyen, P. T. Phan, K. Zhu, D. B. N. Tran, H. M. La, Q. P. Ha, N. H. Lovell, presented at 2023 IEEE International Conference on Robotics and Automation (ICRA) 2023.
19. S. Peng, S. Wu, Y. Yu, Z. Sha, G. Li, T. T. Hoang, M. T. Thai, T. N. Do, D. Chu, C. H. Wang, Journal of Materials Chemistry A 2021, 9, 26788.
20. J. H. Stubbe, A.-M. M. Van Beijsterveldt, S. Van Der Knaap, J. Stege, E. A. Verhagen, W. Van Mechelen, F. J. Backx, Journal of athletic training 2015, 50, 211.
21. S. Conte, C. L. Camp, J. S. Dines, Am J Orthop 2016, 45, 116.
22. S. Sagheb, Y.-J. Mun, N. Ahmadian, B. A. Christie, A. Bajcsy, K. Driggs-Campbell, D. P. Losey, presented at 2023 IEEE International Conference on Robotics and Automation (ICRA) 2023; b) Q. M. Ta, C. C. Cheah, IEEE Transactions on Robotics 2019, 35, 861; c) B. Xu, A. Akhtar, Y. Liu, H. Chen, W. H. Yeo, S. I. Park, B. Boyce, H. Kim, J. Yu, H. Y. Lai, Advanced Materials 2016, 28, 4462.
23. Y. Chen, E. Huerta, J. Duarte, P. Harris, D. S. Katz, M. S. Neubauer, D. Diaz, F. Mokhtar, R. Kansal, S. E. Park, Scientific Data 2022, 9, 31.
24. G. Bahl, J. Zehnpfennig, M. Tomes, T. Carmon, Nature communications 2011, 2, 403.
25. H. Rahman, E. Currier, M. Johnson, R. Goding, A. W. Johnson, M. E. Kersh, Journal of biomechanical engineering 2017, 139, 110801.U. K. Mukherjee, K. K. Sinha, Production and Operations Management 2018, 27, 1816.