Project ExoHaptik: Pflegekraft.

A mixed reality simulator for caregivers with force feedback via exoskeleton

This blog post is about the research project "ExoHaptik:Pflegekraft". In the course of this project we want to take up the classical problems of virtual- and mixed reality and show possible solutions.

The project is based on the following tasks:

Nursing activities make considerable demands on the health of the carers. In addition to psychological stress, caregivers bear an increased risk of developing musculoskeletal disorders (MSDs), especially in the area of the spine. According to the AOK report on absenteeism, professional carers are registered as unfit for work for an average of 25.1 days. 25% of sick days are attributable to MSE and one in five retirements are due to MSE. Both professional caregivers and family carers are exposed to high loads or partial loads when transferring people in need of care, but frequent bending and twisting of the upper body are also named as risk factors for MSE. Especially in private home care, carers are also confronted with unfavourable structural conditions, lack of aids or aids that are unsuitable for the environment.

The aim of the project is to support caring relatives and professional carers in their nursing activities and in maintaining their health. This is achieved on the one hand by a more efficient planning of reconstruction measures and the provision of aids in private homes. On the other hand, the nursing staff is sensitised to their posture by training them in the correct posture for certain activities. The ExoHaptic System, a multi-user mixed reality simulation with force feedback via exoskeleton, is used for the training. The output is via a VR application.

Screenshot VR application from the Proof-of-concept

Nursing staff are involved in the project at an early stage, among other things to test the application in the development process.

Test the application with the help of the nursing staff on site at Lightshape

The housing consultation is to be carried out via an MR HMD with see-through so that an overlapping of the conversions can be seen by the relatives.

In the project, the following three "classical" technical problems of virtual or mixed reality will be addressed:

  1. In most VR systems the haptic perception is missing. The existing systems for haptic feedback are very complex and do not fit the application of care. Therefore, an innovative approach is pursued for this project: an exoskeleton, which is normally only intended to support the movements of the wearer, is used for force feedback. To simulate the resistances and weights acting on the wearer's body, the actuators of the exoskeleton are used in reverse.
  2. Although most VR systems are "one-man-shows", systems have recently been developed that make it possible to couple several head-mounted displays (HMDs). Several users can thus interact simultaneously in the same virtual scene without actually being in the same place. Such a system will be used, extended and adapted for the application area within the project.
  3. Virtual Reality und Augmented Reality are separate worlds, each with specialized output devices. ExoHaptik:Pflegekraft pursues the vision of a Mixed Reality HMD, which implements the continuum between the physical and digital world in one output device.

Lightshapes concrete role in the development of the multi-user HMD application for distributed training classes and expert involvement is described in more detail below.

The focus is on the development of a VR application with a multi-user solution that enables location-independent networking. This will be achieved with the help of a tracking system, based on the HTC Vive Tracking technology for objects and persons for evaluation and adaptation to the necessary functions.

Screenshot VR application from the Proof-of-concept

The necessary functions include:

  • External tracking of the person in the exoskeleton by this system
  • External tracking of the position and the condition of the patient dummy
  • Construction of an object tracking to simulate objects in space in VR
  • Real-time data processing between exoskeleton and VR system, function and operation of the VR system
  • Development and implementation of user interface and user experience (UI/UX) to operate the application (host / client)
  • multi-user scenario
  • Communication solution local/online
  • Investigation of contradictory spatial situations (teleportation) and their solution
  • Several locations with different room sizes and orientation
  • Parametrizable avatars for visual representation of the people involved in VR/AR

In addition, the technical preparation for the usage studies with evaluation function is carried out with the trained nursing staff.

Further information about the project can be found here: www.technik-zum-menschen-bringen.de/projekte/exohaptik

A summary and further information on the potential of AR and VR can be found in this BLOG.

Need Help?
Please Contact us.

We Would love
to hear about your project

Let us know
+49 711 219 578-0