The history of data poisoning pre- and post-Chat GPT suggests the potential for potentially dangerous applications.
There are three kinds of data poisoning:
- Accidental misinformation: True or false? 1.) Winters are colder because the earth is farther away from the sun; 2.) A fifty-pound anvil will fall faster than a fifty-pound sack of feathers; 3.) The time, place and date of your birth determine character. Arguments about astrology are hard to resist sometimes, but the real problem here is the potential for deep learning to “misunderestimate” the ubiquity of human error.
- Intentional misinformation: The internet is filled with bad facts and intentional slander. AI-based platforms may be able to tell the difference between facts and opinion–much less fear-mongering or even dog-whistle hate speech.
- Disinformation campaigns: Governments and organizations may have a vested interest in establishing fictitious online narratives. While AI systems can accidentally spread false information, social media channels, Wikipedia and phony news can be deployed to spread deliberate falsehoods.
Data poisoning is difficult to trace back to a specific source. ChatGPT and search engines use massive data sets scraped from public-facing information online. While it is possibly deep-learning can figure out the truth by scoring the number of competing “facts” based on frequency and distribution, an incorrect response to a ChatGPT prompt could very well have its source in a Wikipedia article that was purged after it was scraped–whether it’s there courtesy a whispering campaign spread by a business rival or a hostile nation-state doesn’t really matter.
Another complication arises from the nature of new business development at big tech companies where IP like algorithms and how they work is a tightly guarded secret. OpenAI, Google, Microsoft and others use data from what we share online.
Programming Languages Also Present a Risk
ChatGPT can generate code in many programming languages including Python, PHP and JavaScript. This has the potential for mischief. Any programmer who has spent hours poring over code to find a single project-crashing typo will appreciate what AI will soon have to offer, and what ChatGPT can already do.
From a security standpoint, having the ability to produce effective code, and to explain the structure of existing code presents a double-edged sword. Fewer sloppy mistakes can reduce the potential for major vulnerabilities. On the other hand, a code library that has been data poisoned could generate code that spreads malware or installs backdoors.
ChatGPT relies on massive archives of open source software to produce code. Open source software is built, developed and maintained by volunteers or organizations that release their work to the general public for review. While allowing for greater transparency, it may allow threat actors to introduce rogue code into open source code. Obviously, this would be a bigger threat for projects no longer actively maintained and thus less likely to be under scrutiny by active developers.
Why is this a problem?
AI and deep learning-based systems can be a real time-saver, and make impossibly large tasks possible. The hazard is that bad data, malicious code and outright disinformation could render the work product of these platforms unreliable and ultimately unusable.
Compounding the issue is the appeal for AI-based solutions in the workplace. Human workers require sleep, food, breaks, and are prone to inconsistency based on workload, emotional well-being, and a wide range of other factors. This is a constant challenge for employers, supervisors and HR departments. The same goes for workers; the ability to offload menial tasks onto a computer program saves time and energy in the same way that word processors and office printers eliminated the need for stenographers decades ago.
The common thread is the promise is that we will not have to pay as much attention to details. The danger is that this lack of oversight means an AI-based system may produce results with questionable controls if there are any at all. Even without data poisoning, self-driving cars can be re-routed by traffic cones. AI-based image recognition systems have confused turtles for rifles and misidentified kneecaps as human faces.
A recent study included a curious development in ChatGPT performance. Between March 2023 and June 2023 the platform’s ability to solve math problems dissolved–going from a 98 percent success rate to a concerning 2 percent rate. Whether this is the result of data poisoning or some other ghosts in the machine doesn’t really matter. The takeaway should be clear: AI-generated content is not inherently reliable, and may never be. Until we have a sufficient understanding of the safeguards implemented to protect users from the hazard, we have to assume the worst.
Meanwhile, creatives are assailing AI-generated content as a euphemism for plagiarism and or theft of intellectual property, while ChatGPT is being integrated with new and existing technologies at the speed of commerce. What could go wrong?
What’s the takeaway?
Buckle up because it’s going to be a bumpy ride. Google, Bloomberg and many other companies are investing in the promise of platforms like ChatGPT, but we have no idea how much of their attention is focused on the potential for misuse and/or malicious use.
It seems likely we may see more unforeseen uses for “AI” in multiple industries and accompanying news stories about the unintended consequences.
Remember, just because it pops up on a screen doesn’t make it error-free, and some errors are malicious. Check and double-check all the information you encounter online viewing it with the same level of skepticism you would a stranger encountered offline.
