Two major artificial intelligence companies, OpenAI and Anthropic, have recently launched healthcare-specific platforms: ChatGPT Health and Claude for Healthcare. These platforms allow users to upload or connect personal health information, including medical records, pathology and imaging results, and data from wearable devices. Using this information, the AI systems generate summaries, explanations and health related insights. The platforms are being marketed for use in both clinical settings and by individuals seeking to understand their own health information.  

In an ACCHO context: 

For ACCHOs, these developments present both potential opportunities and significant risks that require careful consideration.  

The use of commercial AI platforms understandably raises serious concerns relating to privacy, data security, consent and ongoing control of information, particularly when dealing with sensitive health information, and particularly for Aboriginal patients who are not well represented in the AI learning models that determine its advice and responses. At present, there is limited transparency regarding how data uploaded to these platforms is stored, who may access it, how it may be used over time, or whether it could be shared or repurposed beyond its original purpose. This is not only an ethical question, but one that is especially significant in the context of Aboriginal data governance, sovereignty and community-controlled governance. 

There are also important regulatory and accountability considerations. These AI platforms are not regulated by the Therapeutic Goods Administration, nor supported by any regulatory health body, and have not been assessed for their safety, quality or effectiveness as medical tools in Australia. This creates uncertainty regarding responsibility and liability where AI generated information may contribute to harm, delayed care or inappropriate clinical decision-making. 

Further risks relate to cultural safety and equity. AI systems are not designed for Aboriginal health contexts, knowledges and worldviews, sometimes producing biased, inaccurate or culturally unsafe outputs. Over reliance on automated advice, whether in clinical practice, administration or broader decision-making, may further compound these risks. Without strong safeguards and appropriate governance, the use of such technologies may also contribute to digital exclusion and exacerbate existing health inequities, rather than improving outcomes for Aboriginal communities. 

Considerations for Member Services and communities 

Member Services may wish to take a cautious and staged approach to the use of AI tools. This may include reflecting on governance arrangements, data sovereignty, privacy and consent, as well as cultural safety and accountability. Services may also wish to consider whether the use of AI aligns with local priorities, workforce capacity and digital readiness, and whether it would add value to existing models of care.  

It is also important to recognise that some community members may already be using AI tools to understand their health information. Member Services may therefore wish to consider providing clear, culturally safe messaging to support safer use of these tools. Key messages should reinforce that AI does not replace trusted health workers, may not reflect Aboriginal contexts or needs, and should not be relied upon for diagnosis or urgent care decisions. To support this, AHCSA will develop a wall poster outlining key messages about safe and appropriate use of AI for health information.  

AHCSA support 

AHCSA is committed to supporting Member Services as they navigate these developments. This includes strengthening digital literacy, developing and sharing AI-related resources, and supporting services to work through key considerations relating to governance, policy and risk management. These supports are intended to help services assess readiness and risk before engaging with any AI tools. 

When used appropriately, AI tools may support administrative efficiency and workforce capacity, and assist in producing clearer, more accessible health information for community members. Over time, they may also contribute to service planning, reporting and quality improvements. We recognise that the more exposure AI has to Aboriginal cultural needs and contexts the better its advice will become over time; however, it is essential that in this early time we remain vigilant.  

At this stage, a precautionary approach is recommended, with a focus on governance, sovereignty, and community control.  

For further information or enquiries, please contact AHCSA’s Digital Health Innovation Manager through support requests.