The first study to use artificial
intelligence (AI) technology to generate podcasts about research published in
scientific papers has shown the results were so good that half of the papers'
authors thought the podcasters were human.
In research published in the European
Journal of Cardiovascular Nursing (EJCN), researchers led by Professor
Philip Moons from the University of Leuven, Belgium, used Google NotebookLM, a
personalized AI research assistant created by Google Labs, to make podcasts
explaining research published recently in the EJCN.
Prof. Moons, who also presented the
findings at the Association of Cardiovascular Nursing and Allied Professions
(ACNAP) conference in Sophia Antipolis, France, said, "In September 2024,
Google launched a new feature in NotebookLM that enables users to make
AI-generated podcasts. It made me think about how it could be used by
researchers and editors.
"When I did a first test case
with one of my own articles, I was flabbergasted by the high quality and how
natural it sounded. At that moment, I realized that such a system could have
the potential to be used by journals for science communication. Of course, the
quality and accuracy needed to be evaluated. Therefore, we designed this study
to evaluate its potential."
Prof. Moons, who is Professor of
Healthcare and Nursing Science at KU Leuven and editor-in-chief of the EJCN,
and his colleagues selected 10 different types of articles and contacted the
authors to ask if they would agree to take part in the study. The participants
were not informed that the podcast about their study would be generated by AI.
The researchers sent the
AI-generated podcasts to the authors to evaluate for engagement,
trustworthiness and AI detection. They followed up with a questionnaire
(generated with the help of ChatGPT) and a 30-minute interview via Microsoft
Teams.
The authors reported that the
podcasts captured the key points of their papers in "very simple,
easy-to-understand terms," were well-structured, had a good balance in
terms of length and depth, and that the podcast 'hosts' were professional, with
some authors even assuming the speakers had a background in nursing or
medicine. The conversational interaction of the hosts was a significant asset.
Most of the authors said the
podcasts were reliable sources of information. However, some commented on the
American accent and style, including some hyping of research findings with the
use of words such as "amazing," "groundbreaking" and
"totally."
They said there were some
inaccuracies and misrepresentations, there was sometimes lack of context,
incorrect use of medical terminology and mispronunciation of medical words. The
podcasts would need to be checked carefully for accuracy before their release.
AI detection
When they were asked: "Would
you be surprised if I told you this podcast was generated through Artificial
Intelligence?", five authors said "yes," including one author
who had conducted research on AI previously. They said they were "shocked,"
"amazed," or even "having an existential crisis."
Five others said "no."
The participants said that if AI podcasts were released, they should make it
clear that they were AI-generated, acknowledge the authors and reference the
original article.
All the authors said they thought
that patients and the general public would be the most appropriate target for
the podcasts, mainly because of the tone and the ability of the podcasts to
explain the articles in an easy-to-understand manner.
However, some commented that the
podcasts might also be useful for enabling health care workers to stay up to date with the latest research,
access it more easily and make the original research article more visible.
Some of the authors suggested that
podcasts could be tailored for particular audiences according to age, interests
or ethnic background as the technology develops and improves. At present, it is
not possible to change the voices or the language of the AI "hosts,"
but these features will probably be enabled in the future.
Prof. Moons said, "It was
striking how accurate the podcasts were in general. Knowing that we are just at
the beginning of this kind of AI-generated podcasts, the quality will become
better over time, probably within the next few months. Another important aspect
is that these podcasts seem most suitable for a non-technical audience, for
instance, the general public or patients.
"If podcasts could be
generated by AI, that could really be a game-changer. Podcasts could be made
with very little work, just by uploading the article and maybe a bit of
prompting. This could be a sustainable model to get the message out to people who
do not typically read scientific journals.
"I think that this will be a
technology that will allow editors, journals and researchers to communicate
about science to the general public. It will not make human podcasters
redundant.
"There will always be a market
for human-made podcasts, probably because not all topics are capable of being
addressed accurately or appropriately by AI. I can even imagine that there will
be hybrid podcasts, in which human podcasters and AI come together in different
sections of a podcast episode."
Now, the researchers plan to
explore further possibilities of these podcasts for science communication, including seeing what patients and other members of
the public think of them. They also want to investigate if it would be possible
to use AI-generated podcasts for scientific conference sessions.
"For instance, making a
podcast on the content of sessions at the ESC conferences, as a summary for
those who did not participate and want to get a blurb of it," said Prof.
Moons.
The researchers have made an AI-podcast on their research paper about AI-generated podcasts. A link to it is contained in the online published paper.
Source: AI-generated podcasts open new doors to make science accessible
No comments:
Post a Comment