Is it possible to collect ICF-based functioning data through a chatbot: the AIRE project experience

Jaana Paltamaa^(1), Chichaeva Julija^(2), Mälkönen Ida^(2), Mäkelä Antti⁽³⁾

¹ Jamk University of Applied Sciences, School of Health and Social Studies, Research and development, Jyväskylä, Finland.

² Jamk University of Applied Sciences, School of Health and Social Studies, Institute of Rehabilitation, Jyväskylä, Finland.

³ Jamk University of Applied Sciences, School of Technology, Institute of Information Technology, Jyväskylä, Finland.

Abstract

The COVID-19 pandemic showed that the current rehabilitation system needs to evolve towards a more digital environment. Our aim in the AIRE project (https://www.jamk.fi/fi/projekti/aire) was to develop and pilot a digital rehabilitation platform and operating model based on conversational AI, and to promote digital leap in the rehabilitation sector. The overall aim of this use case was to model how to collect ICF-based functioning data using conversational artificial intelligence. The related possibilities are discussed in this article. An ASIA-chatbot based on open-source conversational assistant RASA was developed and piloted. The use case consists of a total of 177 different end-user expressions related to neck and shoulder issues. The inputs of the end-user’s natural language expressions were collected into the intent library. Those intents that were identified as part of functioning were linked to the ICF. This paper presents the procedure and discusses its potential use in rehabilitation. This use case showed that ICF-based functioning data can be collected using conversational AI, but there are still several challenges to its systemic use. For example, how to make buttons in the conversation to be beneficial in ICF-based information collection. The feasibility, acceptability, safety, and effectiveness of various conversational AI formats should also be assessed. The use of the ICF for automated assessment and monitoring of functioning could in the future provide a description of functioning for informed decision-making to optimise the rehabilitation of people with chronic diseases. This paper discusses new application possibilities for digital rehabilitation, ICF and conversational AI.

Keywords: Digital rehabilitation, ICF, Conversational artificial intelligence (AI), chatbot

1. Introduction

The World Health Organization (WHO) has internationally highlighted the development of rehabilitation as an important part of primary health care. The development of rehabilitation is seen as an important part of the Sustainable Development Goals and is significant from the perspective of equality. The need of rehabilitation is predicted to increase in the coming years due to changes in the health and characteristics of the population. (World Health Organization 2023.) The impact of the COVID-19 pandemic across multiple sectors, including rehabilitation is obvious (Thomas et al. 2020) thus providing the opportunity to re-think health systems approaches by the adoption of a biopsychosocial perspective (Leonardi et al. 2020).

The International Classification of Functioning, Disability and Health (ICF) offers a conceptualization of functioning that can underpin assessment and documentation in rehabilitation. The ICF contains a mutually exclusive set of domains of health which are the units of classification and called categories. Together, these categories create the language for describing a person’s health, as well as the lived experience of health. (World Health Organization 2001.)

Disability affects many people and, according to the ICF, human functioning and disability can be experienced, described, and ‘measured’ on a continuum. Health information is also collected by different modes. Information may be collected by means of a technical or clinical test, by a self-report instrument, in terms of expert, or proxy judgment, or by means of a population survey. (Cieza et al. 2019, 2.)

The aims and intended fields of application of the ICF are broad (World Health Organization 2001). In the 20 years since the publication of the ICF, several studies have shown a growing interest in its use in different domains (Leonardi et al., 2022, 2).  ICF was mainly used in clinical practice, policy development and social policy, and in education areas, but the implementation level of the ICF for coding functioning status remained low. Newman-Griffis et al. (2021) and Meskers et al. (2022) have studied natural language processing (NLP) for automated functioning assessment. It was found that ICF presents a valuable framework for organizing the information recorded in health records about patient functioning (Newman-Griffis et al., 2021, 1) and demonstrated feasibility, reliability, and internal validity for the automated recognition of multiple ICF categories and their levels from written notes in a hospital electronic health record (Meskers et al., 2022, 242). In a recent dissertation Nieminen (2023) developed an artificial intelligence algorithm application that identifies disability information from electronic health records in accordance with the ICF framework.

Conversational artificial intelligence (AI) refers to computer programs, which are created to simulate conversations with the human users by utilizing text, speech, or images. The programs may be used by messaging in mobile apps or a website or it can be integrated to the car, television, or other device. As in many other fields, conversational AI is increasingly being applied in health care as well. It enables interactive dialogues and text- or speech-based features make it easy to use for a variety of populations. (Tudor Car et al. 2020.) However, at this moment, there is not fully unambiguous concept or definition concerning solutions of conversational AI. In this paper, when describing a conversational AI solution developed in the AIRE project, the expression “conversational AI” refers to a chatbot.

The main objective of the AIRE project (AI in Rehabilitation 2021–2023) was to develop and pilot a digital rehabilitation platform and operating model based on conversational AI suitable for the field of rehabilitation, as well as promote a digital leap in the sector of rehabilitation. The purpose of these actions is to support and strengthen the rehabilitee’s functioning. One of the developed chatbots was ASIA-chatbot, which could be piloted through this use case and the results are discussed  later in this paper. The overall aim of this use case was to model how to collect ICF-based functioning data using conversational artificial intelligence. This paper presents the procedure and discusses its potential use in rehabilitation.

2. Methods

2.1 Design

This use case uses a small number of pilot participants of students and professionals who anonymously tried the ASIA-chatbot related to neck and shoulder issues. Later on, the participants are referred to as end-users”.  Meaningful functioning concepts were linked to the ICF based on standardised procedure (Cieza et al. 2019). The pilot results will be used to assess the potential of conversational AI in functioning data collection and rehabilitation process.

2.2 Setting

ASIA-chatbot implementation was based on open-source conversational assistant RASA (Bocklisch et al. 2017) Botfront platform. Botfront provides an interface that allows chatbot developers to rapidly build chatbot prototypes and conversation paths. In addition, Botfront includes several operational tools for servicing and updating a chatbot that is published and in use by end-users.

The ASIA-chatbot was a pilot chatbot originally developed for demonstration purposes only. The aims of this chatbot were to provide information and advice for self-care of neck and shoulder problems, to encourage taking care of posture, and to guide the user to services. Contents included information about rehabilitation, self-care, ergonomics, causes of neck and shoulder problems and related symptoms, and acute injury screening.  The chatbot identifies those persons who must contact the healthcare professional urgently e.g., those with sudden or radiating pain, numbness or tingling, or a possible injury.

The ASIA-chatbot was preceded by a chatbot, that discussed remote working and neck and shoulder problems in a more ambiguous way.  ASIA-chatbot was public and available through hyperlink situated in AIRE-project homepage from the beginning of April 2022 to the end of August 2023. It was piloted on volunteers anonymously for example by rehabilitation students in their own digital rehabilitation-related study courses and by rehabilitation specialists at various events. In the development process, feedback and inputs were constantly collected on the basis of which the ASIA-chatbot was further modified.

2.3 ICF-linking

The ICF constitutes a unified and standard language suitable as a reference for the comparability of health information. The ICF linkage rules have been applied to a wide range of use cases, including many questionnaires and clinical assessments, qualitative data collected from open-ended questions during focus groups and patient interviews, and e-mail surveys with health professionals and patients (Cieza et al. 2019, 2-3).

In this use case intents were identified from the inputs of the end-user’s natural language expressions and collected into the intent library. Those intents that were identified as part of functioning were linked to the ICF according to the ICF-linking rules defined by Cieza et al. (2019). ICF-linking was performed by a group of three physiotherapists with expert knowledge of the ICF framework. The meaningful concept within each intent were identified and linked to the most precise ICF category. If a meaningful concept constituted of more than one construct, each construct was linked to the most precise ICF category.

Utilizing the intent library, a proof-of-concept RASA-component was developed that assigned ICF categories to user inputs. The component read in user inputs and the corresponding intents and then added the corresponding ICF information to the conversation flow data structures, which could then be exported and further processed with data analysis and visualization methods (Fig. 1). The structure of the data allowed fine-grained, per-conversation analysis and broad analysis. During the use case individual conversations between the end-user and the chatbot, and how the ICF information occurred in the dialog were analysed as well as the more general analysis of the overall trend of ICF-classified user inputs was made.

Figure 1.  Two examples of the Botfront conversation flow from ASIA-chatbot. End-user‘s input in natural language (user), the intent (in purple “tells or asks”) and related ICF codes (<”…”>).  Picture: Antti Mäkelä

3. Results

The ASIA-chatbot contained a total of 177 different end-user expressions related to neck and shoulder issues. In the intent library they could be grouped into three main categories, namely pain, the cause of the pain and treating issues. Of these, 55 % included mentions of pain and other symptoms. Some (7 %) considered the cause of the pain. The rest of the mentions (38 %) were related to treating issues, for example with medication or massage, or asking for self-care advice (exercise and gymnastics). Mentions that were explicitly related to injury or diagnosis were excluded, as these are issues related to health status in the International Classification of Diseases (ICD).

A total of 16 intentions were specified and were found to relate to functioning. They were linked to a total of seven two-level ICF categories, eight three-level ICF categories and one four-level ICF category (Table 1).  Participants mainly described functioning, which could be linked into six ICF categories. Of these, pain was the most frequent. (Fig.2.)

Table 1. Examples of end-user expressions, intents, meaningful concepts and ICF categories identified in ASIA-chatbot content.

Figure 2. Amount of end-users inputs (n = 301) within six ICF categories occurred in the AIRE-chatbot dialog. Picture: Jaana Paltamaa

4. Discussion

The overall aim of this pilot was to launch a design on how ICF-based functioning data could be collected through conversational AI. Our starting point was different from previous NLP studies (Newman-Griffis et al. 2021; Meskers et al. 2022) or artificial intelligence algorithm application (Nieminen 2023). We have a small case material, but it can be used as a basis to draw some conclusions about the potential of conversational AI. Our first experiences show that chatbots can support a holistic approach to person’s functioning and needs. Chatbots could provide rehabilitees an accessible and easy-to-use digital solution to facilitate person-oriented data collection on functioning and support the individual rehabilitation process.

The use of the ICF reflects a real interest among those working on person’s functioning at the national and organizational level (Leonardi et al. 2022), and we hope that this use case can act as a catalyst for data collection through conversational AI. The user can talk to the chatbot using natural language or navigate through the dialogue using buttons. A chatbot featuring accurate and adaptable natural language is fundamental to human and conversational AI interaction.

ICF highlights that all aspects of functioning are relevant to all people. The ICF framework offers a possibility to explore person reported outcomes (PRO). This use case shows that individuals who used the ASIA-chatbot described mainly body functions. This is understandable because the case was about persons with neck and shoulder issues. They did not cover all aspects of functioning indicated as important for persons with musculoskeletal conditions in relation to a condition-specific standard through the ICF Core Sets (ICF Research Branch 2017). It also seemed that the end-user raised very little about the day-to-day challenges of participation. In rehabilitation, there should be more emphasis upon establishing the person’s needs and long-term goals, usually at the level of activities or participation (Madden et al. 2016; Stucki & Grimby 2007; Wade 2005).

Overall, it was easy to use the ICF linking rules to the end-user expressions. Quality was ensured by having two people with knowledge of the ICF framework responsible for the linking. What was needed was an IT expert who could develop the intent library, a proof-of-concept RASA-component. A prerequisite was regular close cooperation between the different experts and finding a common language.  All in all, we would have liked to see a larger number of participants. This should have been considered and planned earlier during the AIRE project.

Future chatbots could more comprehensively consider all components of person’s functioning according to the ICF, provide information to the professional and thereby improve person’s functioning, long-term goals, and the quality of life. The chatbot should be trained to filter meaningful concepts and link them to ICF categories from user’s natural language. However, during AIRE project was observed that users often face difficulties in effectively describing their problems and formulating relevant questions for the chatbot. For this case integration of buttons in the conversation can be beneficial. The buttons may help with conversation and bring all the angles of the chatbot’s subject area visible. In this case buttons could also be used to observe all the functional abilities according to the ICF-classification. An example of a hypothetical dialog flow is shown in Figure 3.

The use of a chatbot for ICF-related data collection allows service providers to collect data on a wide range of topics that need to be addressed by their services. It might assist service planning at the organisational level or also at the individual level for the professional in planning rehabilitation. The prerequisite is that the information systems can be successfully developed to allow this.

It is essential that the use of the ICF respect the autonomy, dignity, and rights of clients. ICF has eleven ethical provisions, on respect and confidentiality, clinical use of ICF and social use of ICF information (WHO 2001:244–245).  When necessary, data security regulations are considered, it could be possible to collect the preliminary information needed from the client before appointment. If the chatbot had been used to collect information on the functioning of the rehabilitation service users, the information can be used to plan actions. For example, it can be found that service users’ challenges are concentrated in certain ICF categories for which they do not have services. In this case, recruitment of new professionals should be considered.

It should also be checked whether the chatbot is a medical device under the legislation. If the conversational AI application presented here would be classified as a medical device in Finland, it should meet all the requirements of the Medical Devices Regulation. Finnish Coordinating Center for Health Technology Assessment (FinCCHTA) uses the Digi-HTA process (health technology assessment framework for digital healthcare services) to evaluate the feasibility of new technologies in health care. The Finnish Digi-HTA recommendations include clear and beneficial information, but according to Haverinen et al. (2022) their integration into healthcare decision-making processes should be improved.

The COVID-19 pandemic showed the need to develop the current rehabilitation system towards a more digitalized environment. The rapidly changing environment of conversational AI may introduce new possibilities and methodologies for use of chatbots in general, such as more intelligent chatbot platforms or AI-based tooling to facilitate faster development on existing chatbot platforms. Chatbots in the field of rehabilitation are subject to these rapid developments, and future work in this area must carefully observe the emerging AI-technologies.

The scoping review of Tudor Car et al. (2020) highlights that the utilization of conversational AI in rehabilitation services is relatively new and has undergone limited evaluation. Future research should prioritize the assessment of feasibility, acceptability, safety, and effectiveness of various conversational AI formats (text, speech), while also considering to the specific requirements of the target population. (Car et al. 2020.) Moreover, the role of conversational AI as part of healthcare services needs strengthening and health technology assessment should carry out to systematically evaluate the positive and unintended effects of new technologies.

Figure 3. An example of a hypothetical chatbot dialog flow considering ICF and related guides. Picture: Julija Chichaeva

5. Conclusion

The development of AI applications in rehabilitation is a growing field. The described ICF-based chatbot data infrastructure should be developed to facilitate start-up. Developers must consider implementation challenges. In the future, the use of the ICF for the automatic assessment and monitoring of functioning could support informed decision-making to optimise the rehabilitation of individuals with chronic conditions. The findings of this use case provide insight how chatbots could be applied in rehabilitation and open new research possibilities in the areas of ICF and conversational AI.

6. Acknowledgements

The AIRE project (AI in Rehabilitation) was funded by The European Regional Development Fund (EURA 2014/12200/09 02 01 01/2021/PL). REACT-EU instrument (Recovery Assistance for Cohesion and the Territories of Europe) provides additional funding for the ongoing structural funds programmes of the 2014–2020 programme period. The purpose of REACT-EU instrument is to assist in repairing the damage caused by the coronavirus pandemic (Ministry of Economic Affairs and Employment of Finland, 2021).

7. Compliance with Ethical Standards

The data were collected anonymously without any identifying information. Data protection principles were linked in the first page before the use of the ASIA-chatbot and it was pointed out not to write any personal information (name etc..).

All procedures performed in this use case study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Conflict of Interest: The authors declare no conflict of interest.

Corresponding author:

Jaana Paltamaa,
firsname.lastname(a)jamk.fi

ORCID https://orcid.org/0000-0003-2146-3453

RESEARCH GATE https://www.researchgate.net/

8. References

Bocklisch, T., Faulkner, J., Pawlowski, N., & Nichol, A. (2017). Rasa: Open Source Language Understanding and Dialogue Management. https://doi.org/10.48550/arXiv.1712.05181

Cieza, A., Fayed, N, Bickenbach, J. & Prodinger, B. (2019). Refinements of the ICF Linking Rules to strengthen their potential for establishing comparability of health information. Disability and Rehabilitation, 41:5, 574-583. https://doi.org/10.3109/09638288.2016.1145258

Haverinen, J., Turpeinen, M., Falkenbach, P., & Reponen, J. (2022). Implementation of a new Digi-HTA process for digital health technologies in Finland. International Journal of Technology Assessment in Health Care, 38(1), E68. doi:10.1017/S0266462322000502

ICF Research Branch. (2017). ICF Core Set projects. https://www.icf-research-branch.org/icf-core-sets-projects2

Leonardi, M., Lee, H., van der Veen, S., Maribo, T., Cuenot, M., Simon, S., Paltamaa, J., Maart, S., Tucker, C., Besstrashnova, J., Shosmin, A., Cid, D., Almborg, A-H., Anttila H., Yamada, S., Frattura, L., Zavaroni, C., Zhuoying, Q., Martinuzzi, A., … & Kraus de Camargo, O. (2020).  Avoiding the Banality of Evil in Times of COVID-19: Thinking Differently with a Biopsychosocial Perspective for Future Health and Social Policies Development. SN Comprehensive Clinical Medicine, 2, 1758–1760. https://doi.org/10.1007/s42399-020-00486-8

Leonardi, M., Lee, H., Kostanjsek, N., Fornari, A., Raggi, A., Martinuzzi, A., Yáñez, M., Almborg, A-H., Fresk, M., Besstrashnova, Y., Shoshmin, A., Sulyvan Castro, S., Santana Cordeiro, E., Cuenot, M., Haas, C., Maart, S., Maribo. T., Miller, J., Mukaino, M., … & Kraus de Camargo, O. (2022). 20 Years of ICF—International Classification of Functioning, Disability and Health: Uses and Applications around the World. International Journal of Environmental Research and Public Health, 19, 11321. https://doi.org/10.3390/ijerph191811321

Madden, R.H., Lukersmith, S., Millington, M.J., Scarf, C., Fortune, N., Hartley, S. & Llewellyn, G.  (2016). Participatory Monitoring of Community-Based Rehabilitation and other Disability-Inclusive Development Programmes: The Development of a Manual and Menu.  Disability CBR & Inclusive Development, 26(4):26.

Meskers, G., van der Veen, S., Kim, J., Meskers, C., Smit, Q., Verkijk, S., Geleijn, E., Widdershoven, G., Vossen, P., & van der Leeden, M. (2022). Automated recognition of functioning, activity and participation in COVID-19 from electronic patient records by natural language processing: a proof-of-concept. Annals of Medicine, 54:1, 235-243, https://doi.org/10.1080/07853890.2021.2025418

Ministry of Economic Affairs and Employment of Finland. (2021). https://tem.fi/en/-/react-eu-instrument-to-support-regions-affected-by-the-coronavirus-pandemic

Newman-Griffis, D., Camacho Maldonado, J., Ho, P-S., Sacco, M., Jimenez Silva, R., Porcino, J. & Chan, L. (2021). Linking Free Text Documentation of Functioning and Disability to the ICF With Natural Language Processing. Frontiers in Rehabilitation Sciences, 2:742702. https://doi.org/10.3389/fresc.2021.742702

Nieminen, L. (2023). Decision Support for Tailored Biopsychosocial Rehabilitation: In Non-specific Low Back. [Doctoral dissertation, Tampere University]. Tampere University Dissertations 780.) https://urn.fi/URN:ISBN:978-952-03-2851-1

Stucki, G. & Grimby, G. (2007). Organizing Human Functioning and Rehabilitation Research into Distinct Scientific Fields. Part I: Developing a comprehensive structure from the Cell to Society. Journal of Rehabilitation Medicine, May;39(4):293-8. https://doi.org/10.2340/16501977-0050

Thomas, P. (2020). Physiotherapy management for COVID-19 in the acute hospital setting: clinical practice recommendations. Journal of Physiotherapy, Volume 66, Issue 2, April 2020, 73-82. https://doi.org/10.1016/j.jphys.2020.03.011

Tudor Car, L., Dhinagaran, D.A., Kyaw, B.M., Kowatsch, T., Joty, S., Theng, Y.L. &, Atun, R. (2020). Conversational Agents in Health Care: Scoping Review and Conceptual Analysis. Journal of Medical Internet Research, Aug 7;22(8):e17158. https://doi.org/10.2196/17158

Wade, DT. (2005). Describing rehabilitation interventions. Editorial. Clinical Rehabilitation, 19, 811–818. doi:10.1191/0269215505cr923ed

World Health Organization.  (2001). The International Classification of Functioning, Disability and Health: ICF; WHO: Geneva, Switzerland, 2001. World Health Organization. (2023). Rehabilitation. https://www.who.int/health-topics/rehabilitation

Permanent address for this publication:

URN: http://urn.fi/urn:nbn:fi:jamk-issn-2954-1069-9