Privacy concerns showcased in Australian ChatGPT investigation similar for NZ organisations
A recent investigation by the Victorian privacy regulator has resulted in a ban on the use of ChatGPT in a government department. The findings highlight that while generative artificial intelligence (GenAI) tools offer significant benefits in the workplace, they also carry risks without robust policies, training and education on their use.
The investigation by the Office of the Victorian Information Commissioner (OVIC) centred on a Protection Application Report (PA Report) prepared by a Child Protection worker, employed by the Department of Families, Fairness and Housing (DFFH). The report was submitted to the Victorian Children’s Court in proceedings against parents charged with sexual offending.
After the report was submitted, a DFFH legal representative reviewed it and noted that its language was overly sophisticated, complex and descriptive. More concerningly, the report included inaccurate information. For example, there were inconsistent references to a child’s doll:
- In the first instance, the PA Report referred to the child’s father using the doll for sexual purposes.
- The PA Report later referred to the same doll as an example of the parents providing the child with “age-appropriate toys,” used to underscore their efforts to nurture the child’s development needs.
As a result, the report inaccurately presented what should be an indicator of risk to the child as an indication of positive caregiving capacity of the parents. This downplayed the severity of the actual or potential harm to the child.
The DFFH representative reported their concerns to its Child Protection division, which investigated. It concluded that the report’s author had used the free version of ChatGPT to draft the report. DFFH withdrew the PA Report from the Court and notified OVIC about a potential privacy breach.
What was the privacy concern?
The concerns were twofold.
The first related to the release of sensitive personal information to ChatGPT. When information is inputted into the free version of ChatGPT, it is disclosed to the tool’s owner, OpenAI (an offshore company). OpenAI can then decide how to use the information, for example by training ChatGPT or sharing the information with third parties. ChatGPT also offers paid versions of the software, which offers greater privacy protections over data inputted by users.
In this case a significant amount of personal and delicate information was entered into ChatGPT without permission. As a result, it was released to OpenAI and it became outside the control of the Department.
The second concern was over the use in the report of content generated by ChatGPT. It contained inaccurate personal information, in this case downplaying the risks to the child in the case.
ChatGPT breach of privacy
OVIC launched an investigation under the Privacy and Data Protection Act 2014 (Vic), which includes similar provisions to New Zealand’s Privacy Act in relation to the collection , use and handling of personal information. The OVIC concluded that, in using ChatGPT to draft the PA Report, DFFH had breached the following Information Privacy Principles (IPPs):
- IPP 3.1 - by failing to take reasonable steps to mitigate risks that ChatGPT use would result in the collection, use and disclosure of inaccurate personal information.
- IPP 4.1 - by failing to take reasonable steps to mitigate risks that ChatGPT use would result in the unauthorised disclosure of personal information.
OVIC was critical of the policies and protections that DFFH had in place to manage the risks of GenAI tools, finding they were not sufficient to ensure compliance with the IPPs. It issued Compliance Notices, requiring, amongst other things, DFFH to block Child Protection staff from accessing various GenAI tools including ChatGPT. It also required DFFH to regularly scan for similar GenAI tools and block access to them by Child Protection staff.
Privacy and New Zealand organisations
Under New Zealand’s Privacy Act 2020, the same or similar outcome would likely have resulted in similar privacy breaches. The IPPs breached by using ChatGPT have the following equivalents under New Zealand’s Privacy Act:
- IPP 8, which provides that agencies must not use or disclose personal information without taking reasonable steps to ensure that the information is accurate, up to date, complete, relevant, and not misleading.
- IPP 5, which provides that agencies must ensure that personal information is protected by reasonable security safeguards against loss, unauthorised access, modification or disclosure and other misuse.
Like OVIC, the New Zealand Office of the Privacy Commissioner has the power to issue Compliance Notices.
The key takeaway for New Zealand organisations is that robust, well thought-out policies, education and training on GenAI use are essential. Lip-service is not enough, as OVIC’s criticisms of the measures in place at the time the PA Report was drafted make clear:
- Education sessions on risks associated with GenAI, directed at managers and leaders only, were not sufficient to educate the general workforce about how GenAI tools work and the privacy risks associated with them.
- There were no specific departmental rules in place about when and how these tools should or should not be used.
- There were no technical controls to restrict access to tools like ChatGPT.
OVIC was similarly underwhelmed by the additional measures adopted by DFFH after it reported the privacy breach:
- The directive style of the GenAI “dos and don’ts” issued to staff was watered-down by the overarching advice that staff must “conduct their own risk assessment which includes balancing any potential benefit and risks of using an GenAI tool.”
- The guidance lacks real-life examples or case studies of appropriate and inappropriate use of GenAI tools.
New Zealand organisations can derive high-level guidance about how to use GenAI in accordance with the Privacy Act from the OPC’s publication Artificial Intelligence and the IPPs and the government’s Generative AI Advice for the Public Sector.
Anita Birkinshaw is a Special Counsel at Simpson Grierson in Auckland, specialising in complex commercial disputes including intellectual property, privacy, and data protection and cyber-security. Jania Baigent is a partner at Simpson Grierson in Auckland, specialising in insurance, product liability, privacy, and media law. Karen Ngan is a partner at Simpson Grierson in Auckland, specialising in technology, telecommunications, and data protection. Michelle Dunlop is a Senior Associate at Simpson Grierson in Auckland, specialising in commercial technology, data protection, and cyber-security.
