The rapid advance of generative AI and its privacy risks calls for a cautious approach, experts say (credit: Michael Dziedzic on Unsplash)
While generative AI is unquestionably a transformative technology being applied virtually everywhere, companies would do well to consider their potential for sharing sensitive personal information when using the tools that could cause them to run afoul of privacy regulations both here and abroad.
As governments and regulators do their best to try to get ahead of the rapid development and use of generative AI, a cautious approach and careful attention to things like the structuring of contract agreements is advised, experts advised.
EU, U.S. Privacy Regulations
There is no federal data privacy law in the U.S., but a number of laws can apply to the use of generative AI. They including the Gramm-Leach-Bliley Act (GLBA), the California Consumer Privacy Act (CCPA), the Health Insurance Portability and Accountability Act (HIPAA), the Federal Trade Commission Act (FTC Act) and the Electronic Communications Privacy Act (ECPA).
In the European Union, the General Data Protection Regulation (GDPR) has stricter standards for what constitutes a violation that can result in a fine. According to a blog post from Chicago law firm Levenfeld Pearlstein, because of the way they’re structured, it’s difficult for generative engines like OpenAI’s GPT-4 to comply with GDPR’s requirement to notify subjects of how their data is being used.
And under CCPA, the firm notes in a related blog, firms “must disclose what personal data is collected, its purpose, and any third-party recipients,” including of course those using GenAI tools to do so. Enacted in 2018, CCPA “gives consumers more control over the personal information that businesses collect about them,” according to the office of California Attorney General Rob Bonta.
This month, California Gov. Gavin Newsom issued an executive order to study how GenAI tools are developed and used, and their risks.
“If AI systems are used to make decisions about or predict the behavior of consumers, companies must explain the underlying logic and the likely outcomes,” Levenfeld Pearlstein notes. “Discrimination against consumers exercising their CCPA rights is prohibited, which pertains to AI systems if users who opt out of data-sharing receive diminished customer service.”
Governments Attempt to Get Their Arms Around It
In June, the European Parliament adopted a draft of the EU AI Act to “ensure that AI developed and used in Europe is fully in line with EU rights and values including human oversight, safety, privacy, transparency, non-discrimination and social and environmental well-being.” The EU AI Act and GDPR both affect any company possessing data on EU citizens.
How all this will be policed is anyone’s guess in these early days, not even a year out from the general release of OpenAI’s ChatGPT. Congress and the White House are trying to sort it all out by holding hearings with tech leaders involved in creating GenAI systems, some of which have produced dire warnings. The most recent hearing was held this month before the Senate Judiciary Committee, including testimony from executives of Nvidia and Microsoft.
There have already been two high-profile lawsuits that could become class-action cases, alleging copyrighted material was used without permission to train OpenAI’s large language model (LLM) as far back as 2018.
Steps Companies Should Take
What then should retail and ecommerce companies be mindful of as they increasingly apply generative AI tools to solve problems for everything from marketing copy to logistics to customer support?
Ken Morris, a managing partner at Cambridge Retail Advisors, said at a minimum, HR departments should develop and disseminate guidelines on acceptable uses of generative AI. “Then they have a chance to stay within the rails until people can figure out,” Morris said. “It’s the wild west right now, and people don’t know where the sheriff lives.”
Morris posited a scenario in which a GenAI chatbot used as a virtual assistant ingests every input from a shopper, who later gets something fed back to her that uses her information. “She wonders, ‘where did they get that? I didn’t expect them to use it against me,’ ” he said.
Beyond that, Morris said retailers should adopt a “crawl/walk/run” approach and not invest too much in GenAI at the beginning. “Their worst fear is, someone will legislate (the technology) out,” he said. Morris also recommended limiting search inputs to internal data.
Enza Iannopollo, a principal analyst at Forrester, said some organizations are taking risk mitigation actions to expand their privacy toolkit to account for specific GenAI risks. This could include capturing new categories such as bias through richer privacy risk assessments, and leveraging data classification that considers business context and automated controls for data protection.
“Creating privacy governance for GenAI that leverages processes, procedures and policies for data access and use that originated with GDPR compliance is also part of the work,” Iannopollo said. “Updating third- party risk management approaches as well as training and awareness for employees are key activities organizations are embracing.”
Liability Is Tricky, But Tends Toward Users
The issue of liability between users and creators of GenAI tools can get complicated, Iannopollo said. It hinges on things like whether a contract includes data controllers only or whether one party acts as a data processor, and whether data sharing and usage is well defined between the parties.
“New regulations might introduce specific requirements for GenAI tool providers on top of contractual agreements,” she said. “It’s critical that companies are laser-focused on the nitty-gritty in their contracts. But, generally speaking, if a company decides to implement the technology and feed personal data into it, that company holds a great deal of liability, not only in front of regulators but also in front of customers, partners and employees.”
Companies are not only liable under GDPR if they possess data on EU citizens, but they’re held to a stricter standard, according to Eric Moody, another analyst with Cambridge Retail Advisors. And the risk is greater with the use of GenAI tools, with their power to access and assimilate data. Moody said this includes the so-called “mosaic effect,” where multiple datasets are overlaid, and previous anonymized data can suddenly identify individuals.
“They don’t realize if their data is housed at all in Europe on a server, they’re exposed to GDPR, including the new AI Act,” Moody said. “If the data is on European soil, it’s within the realm of their jurisdiction. They could be charged, not realizing a server farm switched data to Denmark for a minute.”