Newswise — When you need accurate information about a serious illness, should you go to Google or ChatGPT?
A research study conducted by computer scientists from the University of California, Riverside, examined internet information gathering services for people seeking information about Alzheimer's disease and other types of dementia. The study involved clinical scientists from the University of Alabama and Florida International University as well. The study assessed the strengths and weaknesses of these information services in providing relevant and reliable information on these medical conditions.
The researchers discovered that Google offers the most up-to-date information, but the search results can be influenced by service and product providers trying to attract customers. On the other hand, ChatGPT provides more unbiased information, but it may be outdated and lacks proper sources for the information presented in its responses.
Vagelis Hristidis, a professor of computer science and engineering in UCR's Bourns College of Engineering, believes that by combining the best aspects of both Google and ChatGPT, a more improved system can be developed. He anticipates that this integration of strengths will likely occur in the coming years.
In their study, Hristidis and his co-authors submitted 60 queries to both Google and ChatGPT that would be typical submissions from people living with dementia and their families.
The researchers chose to focus on dementia because Alzheimer's disease or related conditions affect over 6 million Americans, as stated by study co-author Nicole Ruggiano, who is a professor of social work at the University of Alabama.
According to Ruggiano, research indicates that caregivers of individuals living with dementia are highly involved stakeholders in seeking health information. This is because they are often responsible for making decisions related to the care of their loved ones affected by dementia.
During the research, the queries submitted by the researchers were divided into two categories. Half of the queries focused on obtaining information about the disease processes related to dementia, while the other half aimed to find information about services that could support patients and their families.
The results were mixed.
“Google has more up-to-date information, and covers everything,” Hristidis said. “Whereas ChatGPT is trained every few months. So, it is behind. Let's say there's some new medicine that just came out last week, you will not find it on ChatGPT.”
Although ChatGPT's information may be outdated, it offered more reliable and accurate information compared to Google. This is because the creators of ChatGPT at OpenAI carefully selected trustworthy websites during the computationally intensive machine learning training process. However, the drawback is that users are unaware of the specific sources of information because the responses generated by ChatGPT do not include references or citations.
Google, however, has a reliability problem because it essentially “covers everything from the reliable sources to advertisements,” Hristidis said.
Google's search result pages are influenced by advertisers who pay to have their website links displayed at the top. As a result, users often encounter links to websites of for-profit companies promoting care-related services and products as their first search results. To find reliable information on Google, users need to possess a certain level of skill and experience, as explained by Hristidis. This indicates that the search results may not always prioritize the most reliable and unbiased sources, making it essential for users to be cautious and discerning while conducting their searches.
Co-author Ellen Brown, an associate professor of nursing at the Florida International University, pointed out that families need timely information about Alzheimer's. .
"Although there is no cure for the disease, many clinical trials are underway and recently a promising treatment for early stage Alzheimer's disease was approved by the FDA," Brown said. "Therefore, up-to-date information is important for families looking to learn about recent discoveries and available treatments."
The authors of the study suggest that including both the source and date of health-related information, as well as offering information in other languages, could enhance the usefulness of these platforms for both non-medical and medical professionals. The study titled "ChatGPT vs Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results" was published in the Journal of Medical Internet Research.
Google and ChatGPT both scored low for readability scores, which makes it difficult for people with lower levels of education and low health literacy skills.
According to Hristidis, he predicts that improving the readability of platforms like ChatGPT is more feasible as there are already existing AI methods and tools that can read and rephrase text. However, enhancing reliability, accuracy, and other aspects of such platforms is a more challenging task. Building ChatGPT required many decades of AI research, so any progress in these areas is expected to be gradual and slow.
RSS