What are the potential benefits, considerations, and risks of AI for Research Funding Organisations (RFOs)?

Guest post by Amanda Jane Blatch-Jones, Senior Research Fellow – Research on Research, School of Healthcare Enterprise and Innovation University of Southampton.

What does the evidence say about the current and emerging use of AI? What are the potential benefits of AI for Research Funding Organisations (RFOs)? What are the considerations and risks of AI for RFOs? Read on to learn about the findings from a scoping review that aimed to answer these questions!

We are living in a world where Artificial Intelligence (AI) is increasingly around us, whether that is at home or in the workplace. With the rapid growth of AI being used to resolve problems, increase efficiency, enhance decision-making and reduce repetitive tasks, we are witnessing how AI can help tackle not only environmental issues such as climate change but also in the healthcare setting, particularly for personalised medicines and disease prevention.

Although we are witnessing technological advances, the application and readiness of AI remain to be harnessed in the context of funding and managing health and social care research.  With more and more tools becoming available and the use of AI and Machine Learning (ML) in research rapidly growing, there is a need for funding organisations and governments to respond to the increased use of AL, particularly GenAI (e.g., ChatGPT). This has called into question the benefits, challenges and use of these tools in academic writing and research-focused activities. Much of the current evidence focuses on the use of AI in grant applications and peer review, with limited evidence on how AI could enhance research management practices for funding organisations.

There is a potential for different AI tools (not just GenAI) to have a beneficial impact, for example through automation and integration into research management, risk assessments, outcome analyses, data interrogation, and automating evidence to facilitate reporting. With increasing demands on funding organisations to be accountable, transparent and open, there is a need to assess and consider the role of AI and whether it could harness research management practices.

Reviewing and assessing the current AI literature 

With the increasing interest in AI capabilities (such as predictive analytics, ML, automation and large language modelling) we identified a need to understand the current state of the evidence base. We were interested to understand if reliable tools are already available to improve operational efficiency and enhance business insights. We suspected that there is a knowledge gap in how these potential solutions and tools could be used to inform funding organisations’ research management processes and practices. We recognised a key challenge in the rapid speed at which advancements in this space are occurring, particularly around the capabilities of these digital advancements and being able to fully assess the performance, fairness, potential bias and maturity of AI technologies. (1)

To assess the existing evidence, we undertook a scoping review to assess the breadth and complexity of the topic of AI. We included several different types of literature (including journal articles, commentaries, editorials, and grey literature) and various study designs (including randomised controlled trials, observations, case-control studies, and cross-sectional studies) that would not usually be included in more traditional systematic reviews.

Our scoping review focused on three questions: ‘What does the evidence say about the current and emerging use of AI?’; ‘What are the potential benefits of AI for Research Funding Organisations (RFOs)?’ and ‘What are the considerations and risks of AI for RFOs?’

Find out more about the methods and a full account of the findings from the review.

What did we discover?

Figure 1: Mapping of the evidence across the utility and potential, consideration and risks, and shared AI efforts for research funding organisations

Our scoping review illuminated our understanding in many ways. It helped us to understand the current utility and potential opportunity AI may offer us; things we need to consider and the potential risks; and cultural issues as we evolve and change in response to developments in AI.

Key observations included:

• The opportunities, potential and capabilities of AI for funding organisations are vast; however, it must be recognised that the utility and applicability of AI data-driven tools are still being evaluated. (2)
• There is the potential for AI technology to reduce management and administrative burden by taking on resource-intensive tasks that are often constraints on organisations and inhibit the development of the organisation and its staff. However, increasing productivity and freeing up time to dedicate and redirect focus and attention to value-added activities, requires complex decision-making and personalised human interaction. (2-5)
• There are concerns over current data quality and data management practices and the implications for machine performance and the value and fairness of AI decisions. (6) The overall risk with using systems that are inaccurate, biased, or compromise human rights (e.g., privacy, accountability) is that users and leaders will lose trust in AI and not look to harness potentially useful technologies. (7)
• For public funding organisations, there is additional concern regarding societal perception and impact, accountability and regulatory compliance, including the need to follow the positions of public administrations on AI ethics and use.
• Although AI has the potential to reduce research bureaucracy, which is particularly relevant to funding organisations, the advancement of technology can also result in disruption to how funding organisations operate and function which may not necessarily result in fundamental changes. (3, 4, 6-9)
• Integrating AI technologies requires new initiatives to modernise existing research data management practices; whilst that may come at a cost, it will inevitably produce greater efficiencies and optimise reduction in operational costs resulting in longer-term efficiency gains for research funding organisations. (5, 10, 11)
• The facilitation of shared AI efforts through collaborations is important in achieving AI readiness and successful implementation. Although funding organisations could benefit from AI-driven solutions to improve research management practices, there is a need to address AI governance and ethics, and the associated risks to the research funding landscape. (12-16)
• The lack of understanding of the value, impact and sustainability of AI adoption remains a significant gap in the evidence. There is a need for evaluation studies to understand whether AI technologies/AI-enabled innovations can achieve the intended goals, and help to build trust in the use, applicability, and adoption for the application of AI. (17, 18)  

Future considerations and innovations for change

To drive innovation in AI and benefit all sectors there needs to be collaboration and more interdisciplinary focus on AI announcements from all sectors, organisations, and industries.

There are various avenues to consider when reflecting on the potential implementation of AI technologies and several initiatives are being established such as the Responsible Artificial Intelligence UK and the European Commission’s Joint Research Centre’s Artificial Intelligence Watch. Both are working towards the integration of and experimentation in AI, particularly when there is a need to enhance and strengthen data management and administrative practices in research.

References

1.         Stoykova S, Shakev N. Artificial Intelligence for Management Information Systems: Opportunities, Challenges, and Future Directions. Algorithms. 2023;16(8):357. https://doi.org/10.3390/a16080357.

2.         Valle-Cruz D, García-Contreras R. Towards AI-driven transformation and smart data management: Emerging technological change in the public sector value chain. Public Policy and Administration. 2023:22. https://doi.org/10.1177/09520767231188401.

3.         van Noordt C, Misuraca G. Artificial intelligence for the public sector: results of landscaping the use of AI in government across the European Union. Government Information Quarterly. 2022a;39(3):13. https://doi.org/10.1016/j.giq.2022.101714.

4.         van Noordt C, Misuraca G. Exploratory Insights on Artificial Intelligence for Government in Europe. Social Science Computer Review. 2022b;40(2):426-44. https://doi.org/10.1177/0894439320980449.

5.         Sheeder F. Compliance in the Age of Artificial Intelligence. Journal of Health Care Compliance. 2023:47-50. [Available from: https://www.proquest.com/docview/2899183756/fulltext/F4F6E4B238594297PQ/1?accountid=13963&sourcetype=Trade%20Journals [Last accessed: 03/09/2024]].

6.         Yu P, Xu H, Hu X, Deng C. Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration. Healthcare. 2023;11(20):2776. https://doi.org/10.3390/healthcare11202776.

7.         Vrabie C, editor Artificial Intelligence Promises to Public Organizations and Smart Cities. 14th PLAIS EuroSymposium on Digital Transformation (PLAIS); 2022 Dec 15; Sopot, POLAND. CHAM: Springer International Publishing Ag; 2022. https://doi.org/10.1007/978-3-031-23012-7_1.

8.         Sheeder F. Compliance in the Age of Artificial Intelligence. Journal of Health Care Compliance. 2023;25(6):47-50.

9.         S&P Global Market Intelligence. 2023 Global Trends in AI Report. 2023. [Available from: https://www.prnewswire.com/news-releases/new-study-reveals-data-management-is-a-top-challenge-in-the-ai-revolution-301901556.html [Last accessed: 03/09/2024)].

10.       Leong N. New Study Reveals Data Management Is a Top Challenge in the AI Revolution. 2023. [Available from: https://www.proquest.com/docview/2851109882?accountid=13963&sourcetype=Wire%20Feeds [Last accessed: 03/09/2024]].

11.       Tito M. A comparative analysis of good enterprise data management practices: insights from literature and artificial intelligence perspectives for business efficiency and effectiveness. University of Oulu, Faculty of Medicine 2023.[Available from: https://oulurepo.oulu.fi/handle/10024/42423 [Last accessed: 03/09/2024]].

12.       Born. 10 Ways Funders Can Address Generative AI Now Stanford Social Innovation review2023 [Available from: https://ssir.org/articles/entry/10_ways_funders_can_address_generative_ai_now# [Last accessed: 03/09/2024].

13.       Bentum S. Digital Transformation Strategies for Applied Science Domains [Ph.D.]. United States — Indiana: Indiana University; 2023.

14.       Allan B, Oldfield R, Doutriaux C, Lewis K, Ahrens J, et al. Assessment of data-management infrastructure needs for production use of advanced machine learning and artificial intelligence: Tri-Lab Milestone (8554). Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA.; 2023.[Available from: https://www.osti.gov/biblio/2212844 [Last accessed: 03/09/2024]].

15.       van Noordt C, Medaglia R, Tangi L. Policy initiatives for Artificial Intelligence-enabled government: An analysis of national strategies in Europe. Public Policy and Administration. 2023b:39. https://doi.org/10.1177/09520767231198411.

16.       van Noordt C, Tangi L. The dynamics of AI capability and its influence on public value creation of AI within public administration. Government Information Quarterly. 2023a;40(4):14. https://doi.org/10.1016/j.giq.2023.101860.

17.       Bracci E. The loopholes of algorithmic public services: an “intelligent” accountability research agenda. Accounting Auditing & Accountability Journal. 2023;36(2):739-63. https://doi.org/10.1108/aaaj-06-2022-5856

18.       Boban M, Klaric M. Artificial Intelligence in Healthcare Services – Regulation, Implementation and Future Challenges. Piscataway: The Institute of Electrical and Electronics Engineers, Inc. (IEEE); 2023. https://doi.org/10.23919/MIPRO57284.2023.10159797.