User Research for Generative AI: The impact on healthcare charity

• Shaping strategy and future AI services: Fruto conducted research to inform Prostate Cancer UK’s approach to developing new digital services as part of its 10-year strategy by providing leaders with evidence on future AI services through interviews with men aged 50+ regarding real-world AI health searches.

• First insights into audience AI behaviour: We provided the charity with its first detailed insights into how its audience uses Google AI Summaries and ChatGPT by analysing user behaviour, trust, and search triggers related to symptoms and treatments.

• Understanding and reducing website traffic loss: We identified factors contributing to website traffic loss caused by AI tools, noting that users are less likely to visit the charity’s site when AI outputs lack clear, trusted sources or easy click-through options.

Prostate Cancer UK engaged Fruto to address a significant shift in health information searches resulting from Google AI Overviews and conversational tools such as ChatGPT. With sector-wide web traffic declining by 30%, the charity needed to understand how AI summaries affect men’s access to trusted health content and support. There was a risk that exclusive reliance on AI summaries could cause users to miss vital information and connections to the charity.

We partnered with Prostate Cancer UK to investigate how AI-driven health summaries influence users’ search behaviour and access to trusted information, thereby providing valuable insights to inform the charity’s future strategy.

The objectives of the research were:

• Understand how users use Google AI Summaries / ChatGPT to seek and interpret health information about prostate cancer.

• To examine users’ perceptions of the advantages and disadvantages of receiving health information curated by AI compared to information available on the website, including concerns men may have regarding AI overviews.

• Understand the user's "onward journey" after interacting with the AI summaries.

• To compare user experiences across the two platforms.

We conducted one-to-one user research sessions with 14 male participants aged 50 and older, purposely selected to represent diverse ethnicities, education levels, and geographic locations across England, Scotland, and Wales, all of whom had no prior contact with the charity. This ensured the sample reflected the real-world diversity of the target audience impacted by AI-driven shifts in health information seeking.

Participants were split into two groups: those who naturally use Google AI Summaries and those who use ChatGPT, enabling us to compare their experiences and behaviours across the two platforms.

Each participant was given realistic scenarios. The first scenario involved assessing symptoms, such as lower back pain, and their potential link to prostate cancer. The second scenario focused on seeking information about treatment options following a diagnosis of localised prostate cancer.

Throughout the sessions, we observed and recorded search behaviours, the queries used, and how participants interpreted and reacted to the information they discovered.

After each scenario, participants completed a quantitative survey, rating their experience on a scale from 1 to 7 on metrics such as comprehension, accuracy, comfort, and the importance of source credibility.

This structured approach revealed how men interact with AI for health information, the value they derive from it, and the factors that motivate them to seek further validation from trusted sources.

Key research insights:

We found that men’s search behaviour, trust in sources, and likelihood of visiting the charity’s website are influenced by query framing, the visibility of trusted sources, and the platform used. More serious health concerns increase the need to validate AI responses, while AI tools that obscure sources may lead to lower website engagement.

• UK health charities hold a significant trust advantage. Their websites have an excellent reputation and are considered highly reputable sources due to their specialism and scientific research-backed content. Participants grouped them alongside official medical authorities such as the NHS. Charity websites are trusted for their specialism, expertise, impartiality, and non-commercial nature.

• AI was treated as a starting point, but users demanded primary sources for verification. The vast majority of participants (13 out of 14) utilised AI tools as a convenient initial step or "triage" before seeking validation from authoritative sources. The severity of the disease influenced their need to verify; for serious illnesses such as cancer, users indicated they would not rely solely on AI overviews. Instead, their immediate next step was to validate AI claims by accessing trusted primary sources, such as charity websites or the NHS.

• Closed-ecosystem AI applications, such as ChatGPT, may pose a greater threat to website traffic than Google AI summaries. Google AI displays sources upfront, encouraging users to review citations and facilitating click-throughs to external sites, thereby supporting user verification and driving traffic to trusted websites. In contrast, ChatGPT does not display sources by default and retains users within the app, making it more difficult to verify information or navigate directly to trusted sites. Accessing external websites requires an additional step, which increases friction and may reduce click-through rates.

• How the interface changes query framing: User interaction depends heavily on the interface. Google AI users treated the tool like a traditional search engine, using short, keyword-driven fragments without punctuation (e.g., "stiffness lower back prostate cancer"). Conversely, ChatGPT users framed their queries conversationally, writing in full sentences and often providing rich personal context to obtain tailored answers (e.g., "I am 61, Black Caribbean heritage, so what is the risk factor to me having prostate cancer").

• Emotional impact and tone: The manner in which AI structures information can elicit strong emotional responses. In the symptom scenario, Google AI sometimes caused panic by immediately linking lower back pain to advanced prostate cancer at the top of the page, an approach described as "worrying" and alarmist. In contrast, ChatGPT fostered trust by adopting a "ground-up approach," initially suggesting common issues such as muscle strain before mentioning cancer, which conveyed a balanced and measured tone.

Recommendations and strategic opportunities

Based on these insights, Fruto provided Prostate Cancer UK with strategic recommendations to adapt to shifting user behaviours and to future-proof their digital experience.

• Brand recognition and trust are strategic assets: Users trust UK health charities and feel reassured when AI summaries reference credible organisations such as Prostate Cancer UK. To mitigate the risk of AI tools diverting website traffic, the charity should actively promote its reputation as a specialist, research-based authority and raise awareness of the unique value of its content.

• Evolve the website experience to meet changing user expectations: Update the website to provide quick, digestible summaries at the top of each page, accompanied by clear pathways to more detailed content. There is an opportunity to offer personalisation features enabling users to easily find information most relevant to their context, intent, or stage in the journey. This approach reinforces trust and aligns with current user information scanning behaviours.

• Prioritise SEO and GEO (Generative Engine Optimisation): As users increasingly assess reliability based on visible links and cited sources in Google AI, maintaining strong SEO and GEO is essential. This ensures Prostate Cancer UK remains visible and cited in AI summaries, serving as a trust signal and increasing the likelihood that users will click through to reliable, authoritative content as search habits evolve.

• Careful AI product exploration: Should Prostate Cancer UK pursue its own AI initiatives, it must carefully design the tone and information density to avoid alarming users, ensuring responses are measured, empathetic, and consistently reliable.

• Further and ongoing research: We recommend conducting further research with larger sample sizes, potentially through quantitative surveys, to validate behaviours observed in this qualitative study and to include younger users who may be more open and adaptable to changing search behaviours. This should not be considered a standalone study. Given the rapid evolution of technologies such as Generative AI, regular user research is necessary to understand how these changes affect user behaviours, web traffic, and Prostate Cancer UK’s success.

The research insights and strategic recommendations informed Prostate Cancer UK’s digital strategy, content development, and future AI initiatives.

Fruto’s research enabled Prostate Cancer UK to understand and adapt within a rapidly evolving landscape, ensuring that men continue to find trusted support regardless of changes in search behaviour.

• The research report was widely circulated and well received within Prostate Cancer UK, directly informing a new action plan developed with senior leadership.

• The charity obtained its first robust, user-centred evidence on AI’s impact on health information-seeking, enabling decisive action to protect its role as a trusted source and ensuring it remains at the forefront of supporting men’s health in the age of generative AI.