New anti-AI tool from the team developing Glaze that looks to poison prompts and ruin data sets entirely.
The tool can go so far as to trick AI into generating things it didn’t ask for. 50 “poisoned” images of dogs into a data set yield typical errors companies have been working to get rid of: weird proportions, extra limbs, etc. When they added 300 samples, it was drastic enough to make a data set that was trained to generate “dogs” into “cats” making the tool basically useless.
It’s not out yet, but they’re adding it to Glaze and they intend on making it open source.
A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways.
The tool, called Nightshade, is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission. Using it to “poison” this training data could damage future iterations of image-generating AI models, such as DALL-E, Midjourney, and Stable Diffusion, by rendering some of their outputs useless—dogs become cats, cars become cows, and so forth. MIT Technology Review got an exclusive preview of the research, which has been submitted for peer review at computer security conference Usenix.
AI companies such as OpenAI, Meta, Google, and Stability AI are facing a slew of lawsuits from artists who claim that their copyrighted material and personal information was scraped without consent or compensation. Ben Zhao, a professor at the University of Chicago, who led the team that created Nightshade, says the hope is that it will help tip the power balance back from AI companies towards artists, by creating a powerful deterrent against disrespecting artists’ copyright and intellectual property. Meta, Google, Stability AI, and OpenAI did not respond to MIT Technology Review’s request for comment on how they might respond.
Zhao’s team also developed Glaze, a tool that allows artists to “mask” their own personal style to prevent it from being scraped by AI companies. It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning models to interpret the image as something different from what it actually shows.
You would if you were tired enough. You would if you were hungry enough. You would if your mind and body had been worn down enough, through pain or disease or toil or violent struggle. You might if you were put on the wrong medicine, or you got the wrong kind of head injury, or you were forced to choose between someone else and yourself. You might if your livelihood was staked on it, or all your hopes and dreams. You might if you didn’t know what else to do, if it’s what you were taught or if nobody taught you anything else.
I have not been worn down in most of these ways. I have lived a remarkably privileged life. But I have been worn down in some ways. And they were enough to teach me that in the wrong circumstances, any of us can become someone we don’t want to be. It’s worth keeping that in mind.
every time my bf says “for the time being” i respond with “for the time bean” and then we say “all hail the time bean” and carry on the conversation like nothing happened
Sometimes I talk about corvids, bettas, Pokemon, Cosmere or His Dark Materials. Currently lost in the Tokyo Mew Mew New sauce and waiting for HDM season 3. Sometimes I draw. I am always tired.
