Discussions on Generative AI and originality

[BobbyBrandt]

This story concerns graphic generative AI rather than text, and I don't know if any text equivalent is possible, but it's an interesting development: artists who object to their work being used to train AI can "poison" images with tiny changes that are imperceptible to a human but will sabotage an AI trained on them.

https://www.technologyreview.com/2023/10/23/1082189/data-poisoning-artists-fight-generative-ai/

The concept could work with text in much the same way. A story could be run through a filter when uploaded that alters the metadata within the document so that searches of its content poison an entire AI database.

Imagine if an AI query or prompt on any topic returned sentences where the word "and" gets replaced with "dad" or commas get replaced with question marks.

 

[Bramblethorn]

The concept could work with text in much the same way. A story could be run through a filter when uploaded that alters the metadata within the document so that searches of its content poison an entire AI database.

Imagine if an AI query or prompt on any topic returned sentences where the word "and" gets replaced with "dad" or commas get replaced with question marks.

If you have a way to distinguish between queries from an AI and queries from a human, and serve them different content, you're already pretty much home. Nightshade seems to be intended for situations where you don't have that option, so whatever you're giving the AI still has to be okay for a human viewer.

 

[BobbyBrandt]

If you have a way to distinguish between queries from an AI and queries from a human, and serve them different content, you're already pretty much home. Nightshade seems to be intended for situations where you don't have that option, so whatever you're giving the AI still has to be okay for a human viewer.

The "poison" in both cases is embedded within the metadata which would not affect the way that a human viewed either an image or text. It would only affect the AI database's ability to "view" it correctly.

 

[Bramblethorn]

The "poison" in both cases is embedded within the metadata which would not affect the way that a human viewed either an image or text. It would only affect the AI database's ability to "view" it correctly.

I don't think that's right. Where are you seeing that Nightshade/Glaze change the metadata?

Zhao’s team also developed Glaze, a tool that allows artists to “mask” their own personal style to prevent it from being scraped by AI companies. It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning models to interpret the image as something different from what it actually shows.

"Pixels of images" aren't the metadata.

 

[BobbyBrandt]

I don't think that's right. Where are you seeing that Nightshade/Glaze change the metadata?



"Pixels of images" aren't the metadata.

That has to be what they're changing based upon what they say the process does. The descriptive metadata for an image of a dog, to use one of their examples, gets altered so that it poisons the AI database to present other images when a dog is queried. The original "pixels" of the image are not altered which is why the process is invisible to the human eye.

 

[djrip]

I haven't seen specifics of how this particular method works, but I think the underlying idea is that the things an image-recognition algorithm uses to classify an image aren't necessarily the same as the things a human will focus on.

For instance, a human glancing at this would probably just think "cat". (M.C. Escher's "White Cat I, 1919" to be precise.)
View attachment 2284642

But if I load it into Gimp and turn the brightness way down, and the contrast way up, I get this:

View attachment 2284643

Here, I've overlaid White Cat I with another Escher, "Two Birds", with opacity set to 0.005. If you look closely at the original you might be able to see a hint of it in the black area below the signature, but on the white fur it's pretty much invisible to a human eye without processing.

An image-recognition algorithm processing the first image will "see" these birds. The colour difference between "bird" and "bird outline" is small, too small for us to see in most parts of the image, but it's sharp and consistent and that's an easy thing to pick up for an algorithm that has access to the exact colour values. These bird shapes are very consistent and they're repeated many times in this image.

Meanwhile, if I'm training it on a bunch of other images, the algorithm "knows" that brightness can vary greatly from one image to another, and between different parts of the same image, and that often what matters is local variations. So if I feed it my modified version of White Cat I, tagged as "cat", along with a bunch of similarly-modified images where the cats are different but the birds remain the same, there's a risk that it'll end up learning that "cat" looks like this:

View attachment 2284650

What Glaze/Nightshade is doing won't be anything as simple as just overlaying a faint image of dogs on an image of a cat, but I suspect it's based on a similar idea.



Depending on how it's been altered, you can do things like that. For my particular example, you could probably get rid of the birds without losing much of the cat by compressing colours (anything close to black becomes perfect black, anything close to white becomes perfect white) or by applying Gaussian blur with a radius wide enough to remove the definition on those birds.

But if you don't know whether or how the image has been altered, figuring out exactly what treatment you need to apply to purge that "poison" without destroying the aspects that are important to a human is a hard problem.

Consider this picture. I made this by creating a black background with a white frame (which isn't going to show very well against a white forum screen but I promise it's there) and then overlaying Two Birds with opacity set to 0.005 again, just like I did on White Cat I.

View attachment 2284661
Most humans would label this as "birds" (edit: at least if you turn up the brightness a bit). Just like the first image, there's a very faint bird image overlaid on a strong black-and-white image. But in the first one, the black-and-white image was something we care about, and in the second it's not.

A filter that will purge the subliminal birds from modified White Cat I has an excellent change also purging the non-subliminal birds from the latter image. Part of the method discussed for Glaze/Nightshade involves changing the details of the "poison" from one picture to the next.

Ultimately, the goal for this isn't to make it impossible to train AI on modified images; it's to make it enough of a nuisance that it's easier to go do something else, like restrict the training set to public-domain. The appeal of generative AI isn't that it does stuff humans can't, but that it piggybacks on work humans already did for cheaper than hiring a human. The more effort you have to put into bypassing those protections, the less appealing it is as an option.

(Plus, if you do put work into bypassing such protections, that's pretty strong evidence that you intended to train on material created by somebody who didn't permit that use. I have no idea how the legal side of all this is going to shake out, but it's possible that some court somewhere will decide that generative AI infringes on the copyrights of the originals; if it does, that kind of evidence might be inconvenient.)



Granted, promises can be broken, but this seems to indicate otherwise:

https://glaze.cs.uchicago.edu/what-is-glaze.html

"There is no business model, no subscription, no hidden fees, no startup. We made Glaze free for anyone to use, but not open source, so as to raise the bar for adaptive attacks. Glaze is designed from day 1 to run without a network, so there is no data (or art) sent back to us or anyone else. The only communication Glaze has with our servers is periodic "pings" to ask if there are new software updates."

According to that page, it's not steganographic, but they're a little vague about exactly what it is.

(I note that the first article says they're considering making Nightshade OS, but this one says Glaze isn't OS. IDK how those reconcile - maybe Nightshade specifically is going to be OS but the rest of Glaze won't be?)



Perhaps. From the MIT article, they tested on Stable Diffusion XL, which has over a hundred million images in its training data. Just fifty poisoned images were enough to seriously fuck up its ability to draw "dog" and three hundred were enough to turn "dog" into cats. But I don't know the details of whether they were able to weight the poisoned images more heavily than the original training images.

I knew you'd explain it better than the article!

 

[AlexBailey]

While AI writing is still atrocious for the most part, AI can display some original ideas- probably more like new arrangements of existing ideas.


Alex: Create a story idea and an outline for a romantic three-way relationship that develops into an alpha character, a beta character, and happy muse character.

ChatGPT:

Title: "Hearts Unbound"

Outline:
1. Introduction
- Introduce the three main characters: Alex (Alpha), Jamie (Beta), and Morgan (Muse).
- Set the scene in a small, picturesque town where they all live.

2. Alex (Alpha):
- Alex is a confident, successful, and assertive entrepreneur.
- He's secretly been in love with Jamie, his childhood best friend, for years.
- Alex's journey is about coming to terms with his feelings and taking a risk by revealing his true emotions to Jamie.

3. Jamie (Beta):
- Jamie is a gentle, kind-hearted artist who's been secretly in love with Morgan, their mutual friend and a vibrant free spirit.
- Jamie's character arc involves building the courage to express his feelings to Morgan, who has always seen him as a friend.

4. Morgan (Muse):
- Morgan is a charismatic and adventurous traveler with a lust for life.
- Morgan enjoys spending time with both Alex and Jamie but hasn't realized their affections.
- Morgan's story centers on discovering and navigating the complex emotions that develop when the two people closest to them confess their love.

5. The Love Triangle:
- Alex finally confesses his feelings to Jamie, who reciprocates his affection.
- They decide to pursue a romantic relationship together, keeping it a secret from Morgan.
- Morgan begins to feel an unspoken tension and realizes something has changed.

6. Revelation and Confession:
- After a series of misunderstandings, secrets are revealed, and all three characters confront their feelings and desires.
- Morgan learns about Alex and Jamie's budding romance, leading to an emotional and challenging conversation.

7. Emotional Turmoil:
- The trio navigates through a rollercoaster of emotions, including jealousy, insecurity, and confusion.
- Each character wrestles with their role in the relationship and their own vulnerabilities.

8. Growth and Acceptance:
- Through open communication and support, the three characters come to terms with the unique dynamics of their relationship.
- They explore the idea of polyamory and decide to give it a try.

9. Building a Stronger Bond:
- As they embark on this unconventional journey, the characters learn to embrace their individual strengths and weaknesses.
- Their love deepens as they become more honest and accepting of each other.

10. The Happy Muse:
- The story concludes with Alex, Jamie, and Morgan celebrating their love and the strength of their unique relationship.
- Morgan serves as the muse for both Alex and Jamie's creative endeavors, and together they build a beautiful life filled with love, understanding, and artistic inspiration.

"Hearts Unbound" is a heartwarming story of love, self-discovery, and the triumph of embracing an unconventional romantic relationship. It explores the complexities of human emotions and the beauty of finding happiness in unexpected places.


——————

While it’s nothing great, it could be helpful to an aspiring writer who has only a vague idea or only an idea for a particular scenario they want to expand on.




Alex: Provide examples of similar stories from existing sources.

ChatGPT:

Certainly! While the exact dynamics of the alpha, beta, and muse characters may not be common in traditional romance stories, there are several examples of love triangles and unconventional relationships in literature and film. Here are a few examples that share elements with the outlined story:

1. "Bridges of Madison County" by Robert James Waller:
- This novel explores a romantic relationship between a photographer, Robert Kincaid, and a housewife, Francesca Johnson. It's a poignant tale of a short-lived but intense love affair, which is somewhat similar to the secret romance between Alex and Jamie in the outlined story.

2. "Chasing Amy" (Film):
- Directed by Kevin Smith, this film delves into the complexities of a love triangle. It involves a man, a woman, and another man who realizes he has feelings for his male best friend. While the dynamics differ, the themes of hidden affections and relationship complications are relevant.

3. "Vicky Cristina Barcelona" (Film):
- Directed by Woody Allen, this film explores the dynamics of a love triangle involving two American women, a Spanish artist, and his ex-wife. It showcases the complexities of romantic and sexual relationships among multiple people.

4. "The Great Gatsby" by F. Scott Fitzgerald:
- This classic novel doesn't precisely match the outlined story, but it involves a love triangle between Jay Gatsby, Daisy Buchanan, and Tom Buchanan. It highlights the themes of unrequited love, hidden desires, and social class dynamics.

5. "The Notebook" by Nicholas Sparks (Film adaptation as well):
- While not a love triangle, this story revolves around the enduring love between Noah and Allie. Their love faces societal pressures, misunderstandings, and memory loss, which parallels some of the emotional turmoil in the outlined story.

These examples explore complex relationships, hidden feelings, and the challenges of love in various ways, providing inspiration for stories that feature unconventional romantic dynamics.

 

[Bramblethorn]

That has to be what they're changing based upon what they say the process does. The descriptive metadata for an image of a dog, to use one of their examples, gets altered so that it poisons the AI database to present other images when a dog is queried.

Here's the relevant quote:

Generative AI models are excellent at making connections between words, which helps the poison spread. Nightshade infects not only the word “dog” but all similar concepts, such as “puppy,” “husky,” and “wolf.”

Nothing there says they're changing the metadata. The point is that this contagion is possible because generative AIs already link related words.

Suppose you're building a generative AI like StableDiffusion. You might only have a few thousand training images tagged "husky", which isn't usually a lot for teaching an AI; if you asked it to draw a "husky" based solely on those images, it probably wouldn't do a great job.

But you also have millions of training images tagged "dog", and if your AI knows what "dog" looks like, it already knows almost everything it needs to draw a husky.

If you were trying to instruct a human artist who knew dogs but not huskies, you might tell them: "draw a dog, make it a spitz, long hair, triangular ears, not with a pug face, not curly hair" etc. etc. You can improve the AI in a similar way by incorporating the information that "dog" and "husky" are closely related. Then, instead of needing to learn "husky" as a completely separate concept, it just needs to learn how to modify "dog" to get "husky", which is a much easier task.

But if you then feed that AI poisoned images which corrupt how it draws "dog", that also affects anything else that's piggybacking off its knowledge of "dog".

The original "pixels" of the image are not altered

The article literally says "by changing the pixels of the image".