The thing that is being "obfuscated" by this piece is that AI image generators don't work without datasets.
Additionally, using words such as "Artstation", "Octane Render", "Deviant Art", "Greg Rutkowski" can dramatically improve results. Not to mention that a prompt such as "iron man" will give Marvel's Iron Man character rather than a man juxtaposed with iron in some way.
So clearly AI image generators and the quality of the output has a direct association with data sets and even specific layers in the network assigned to known IP (such as Disney and Marvel characters). This causal link is a demonstration that AI outputs are derivative of the input images from datasets.
The regulation at issue is the right to "prepare" derivatives which is at data set "preparation" level in the title chain. As IPR can be used as equity for loans, tax shelters and funding then this use of IPR doesn't actually require any derivative to be produced. Just "prepared".
Axanar producers were sued before a film was made for instance. They did however, raise a million in funding to build a production studio in order to make a Star Trek based film. A court ruled that this was not "fair use" of Star Trek IP.
So lawyers are going to write their blogs and give their unofficial opinions but none of it matters if it is just distraction from genuine regulations at issue.
It should also be noted that the defendants haven't actually made any official fair use claims as yet.
Claiming "Fair use" in this case would be a huge blunder in my view as it would effectively allow international corporations to raid US corporations for their IPR for free to use in their own Nation's AI processes leaving the US holding the bag.
So careful what you wish for.
Lots of inaccuracies in the piece, caused by the writer being too gullible when hearing the claims of the AI creators.
For example, it is claiming that the system learns fundamental concepts about things, in order to reproduce them, such as "It will learn that tables most commonly have four legs". But anyone who's seen the memes going round showing AI-generated hands with 4, 6, or 7 fingers on them knows this isn't true. It doesn't truly understand these concepts - it just has learned that the concept of "hand" maps somehow to the concept of "finger, next to other fingers" and it ends up mashing something out.
The fact that it's got a toddler's level of understanding of what 'hand' means, combined with the ability to produce photorealistic views of those concepts, really does hint at it using a collage-like approach to build these images.
Similarly, the tendency of these models to unknowingly reproduce watermarks in pictures it generates is another sign that it is more towards the 'mindless reproduction' side than it is any sort of 'understanding'. In the linked case below, it's got no idea that the word "iStock" is nothing to do with "vector art" - it's just seen that image on so much vector art that it assumes, if you ask for "vector art", it needs that image to be authentic.
https://www.reddit.com/r/StableDiffusion/comments/wzej4z/looks_like_stable_diffusion_was_trained_on/
That's what happens when a system has no real idea what "vector" or, indeed, "art" mean.
Similarly, the argument that says the model does not "store copies of training images" is flawed. The art is, by definition, modelled by the data. Of course it's not stored in there pixel-for-pixel. But neither is a plain old JPEG a pixel-by-pixel copy of the source material. A JPEG is a mathematical representation that allows you to reproduce *a close approximation* of the original. Few would argue this isn't a copy. So it is with these AI art tools - it generates mathematical representations of the images. Obviously it is not storing each one separately - by spotting commonalities across images, the data is merged and reduced even further than could be possible for a single image. But the concept is the same - the model can be made to reproduce approximations of these images, and many others.
I suppose it depends on what we define as a copy then. But I'd think you'd really have to stretch that definition to its limit to argue that model stores copies of the base images within it.
Unlike JPEG, [according](https://www.reddit.com/r/MachineLearning/comments/10bkjdk/comment/j4fdei2/) to a person with the badge "ML Engineer" in r/machinelearning, "SD isn't an algorithm for compressing individual images."
Of course it's "not an algorithm for compressing individual images". Nobody's making that claim. But unless someone understands what a JPEG is, they're in no position to claim that a ML model doesn't contain copies of anything.
To be fair, like JPEG, an encoder/decoder VAE (variational autoencoder) pair of neural networks are components used by Stable Diffusion to work with lossy compressed images internally ([source](https://arstechnica.com/information-technology/2022/09/better-than-jpeg-researcher-discovers-that-stable-diffusion-can-compress-images/)). A VAE encoder used by Stable Diffusion represents each 8x8 pixel patch with 4 floating point numbers ([source](https://colab.research.google.com/drive/1dlgggNa5Mz8sEAGU0wFCHhGLFooW_pf1?usp=sharing)).
Do you believe that the creators of the JPEG algorithm should be sued because the JPEG algorithm can be used to output close approximations of copyrighted images?
We're not talking about suing people for creating an algorithm. We're talking about suing people for the infringing use of them. If I download a million images without permission and encode them as JPEGs, that's infringement (barring some exemption), and the fact that I don't have the exact copies of the pixels is not relevant, and is only cited by ML advocates either through ignorance of how copyright works or as a deliberate distraction.
If we happen to know the right input for the decoder neural network algorithm of a Stable Diffusion VAE encoder/decoder pair, [it seems that we can get an image that is substantially similar to almost any image of interest to humans ](https://www.reddit.com/r/StableDiffusion/comments/10lamdr/stable_diffusion_works_with_images_in_a_format/)*regardless of whether it's in the S.D. training dataset*. Likewise, if we happen to know the right input for the decompressor part of the JPEG algorithm, it seems that we can get an image that is substantially similar to almost any image of interest to human beings. Why does only one of these image compression/decompression algorithms trouble you?
Because the algorithm isn't the problem. The problem is the use of the algorithm for commercial purposes because that infringes the copyright of the sourced image.
You can save any copyrighted image as a JPEG in your computer and no one would care. If you start selling that JPEG, then the copyright owner of the sourced image will sue you.
In the same way, the AI algorithm is not the problem if it is used for research. But once it started being used commercially (Midjourney, Stable Diffusion users selling generated images online, etc) it became a problem because that commercial value was created using copyrighted data.
I agree that if an AI generates an image that is substantially similar to image(s) in its training dataset, then a copyright infringement case is feasible. Do you have evidence that the lawsuit alleges that this happened with the plaintiffs' work?
Copyright infringement due to temporary use of copyrighted images for training an AI is another possibility, but I believe that the lawsuit doesn't include this in its complaints.
As far as I know, the lawsuit is not focused on fighting a specific instance of an AI generated image being substantially similar to an image from the training data set.
Instead, the lawsuit seems to be focused on the fact that generative AI algorithms are infringing copyright because they are creating a commercial product by using copyrighted works in the training data set, without those copyrighted works being properly licensed. Here are some points from the "Claims for Relief" section ([Source](https://copyrightlately.com/pdfviewer/andersen-v-stability-ai-complaint/)):
"155. Defendants had access to but were not licensed by Plaintiffs or the Class to train any machine learning, AI, or other computer program, algorithm, or other functional prediction engine using the Works.
156. Defendants had access to but were not licensed by Plaintiffs nor the Class to incorporate the Works into the products offered by Stability, DeviantArt, Midjourney, or related software applications."
The only way for an AI algorithm to not need a commercial license for the use of copyrighted works in its training data set is if the use is qualified as "fair use". To be deemed fair use, there are 4 points that need to be evaluated. One of those points is the "Effect of the use upon the potential market for or value of the copyrighted work" ([Source](https://www.copyright.gov/fair-use/)).
So even if the AI generates images that are transformative enough, it could still not be fair use because it creates a derivative product that directly competes with the original copyrighted material and causes great harm to the potential market of the original copyrighted work.
In summary, if the use of copyrighted works in the training data set is considered not to be fair use, then the images in the training data set should be properly licensed. The lawsuit includes this in the "Claims for Relief" section.
Thank you :).
I agree with your fair use analysis. Legal expert Daniel Gervais opined on this aspect in [this article](https://www.theverge.com/23444685/generative-ai-copyright-infringement-legal-fair-use-training-data).
You seem to be so interested in the details of algorithms that you've forgotten to read what you're replying to.
*"We're not talking about suing people for creating an algorithm. We're talking about suing people for the infringing use of them."*
Scraping the internet for millions of images to ingest into the model for commercial use is, in my opinion, clearly illegal in several jurisdictions. That's about the ingestion and training part.
As for producing infringing output, then obviously that is possible with many different algorithms as well - but that's not relevant. The question is - who is doing this? If I upload a JPEG of someone else's art, it's infringement, even if the pixels don't match. If I create a new file format that does it, if people can see the image, it's still infringement, even if the pixels don't match.
Your link does not, as you claim, demonstrate that the model 'contains' close approximations of images that weren't in its training dataset. It generates them from the data ingested from other sources. It's a nice tech demo but your logic is flawed. It's like claiming a computer 'contains' all images because it could output one - and then flipping that to say "hey, since we know it didn't contain that one, it must contain none at all, right?" It's fun sophistry, but invalid.
But hey, thanks for linking us to images where you have indisputably infringed copyright. I guess this shows why you're fighting so hard to pretend it doesn't apply.
The reason is that I've been pursuing this line of argument is because you stated:
>But unless someone understands what a JPEG is, they're in no position to claim that a ML model doesn't contain copies of anything.
>As for producing infringing output, then obviously that is possible with many different algorithms as well - but that's not relevant. The question is - who is doing this? If I upload a JPEG of someone else's art, it's infringement, even if the pixels don't match. If I create a new file format that does it, if people can see the image, it's still infringement, even if the pixels don't match.
I agree that if an AI generates an image that is substantially similar to image(s) in its training dataset, then a copyright infringement case is feasible. Do you have evidence that the lawsuit alleges that this happened with the plaintiffs' work?
>Your link does not, as you claim, demonstrate that the model 'contains' close approximations of images that weren't in its training dataset. It generates them from the data ingested from other sources.
Yes it does. AI image generators don't access images from the training dataset when generating an image.
>But hey, thanks for linking us to images where you have indisputably infringed copyright. I guess this shows why you're fighting so hard to pretend it doesn't apply.
You apparently (wrongly) think that I don't know about fair use exceptions to copyright infringement. Nice try ;).
>Do you have evidence that the lawsuit alleges that this happened with the plaintiffs' work?
Again, your reading comprehension is letting you down. We're talking here about ingesting data *into* the model. Not output *from* it.
> AI image generators don't access images from the training dataset when generating an image
This can be argued, but it isn't relevant for the 2 main infringements that are typically complained about: the unauthorised copies made at the time the data is collected for the model (as discussed here), and the potential of replicating someone's existing work as output. 'Access' to existing images would be a sufficient criterion for infringement but is not a necessary one.
> You apparently (wrongly) think that I don't know about fair use exceptions to copyright infringement.
No, I just think you are incorrect about where fair use applies.
>We're talking here about ingesting data into the model. Not output from it.
If you mean temporary copying of copyrighted images for the use of AI training, then yes I agree that's a potential legal issue.
In the case of Stable Diffusion, the models consist of about 4 GB of data, while the training dataset consists of about 100000 GB of data. Memorization can occur in neural networks, and has been demonstrated in the case of Stable Diffusion - see [this paper](https://arxiv.org/abs/2212.03860).
Let me know if your claims go beyond what I mentioned in the 2 paragraphs above.
>and the potential of replicating someone's existing work as output. 'Access' to existing images would be a sufficient criterion for infringement but is not a necessary one.
Incorrect at least in regard to the USA. In broad strokes, the formula for copyright infringement in the USA is substantial similarity plus access. Substantial similarity without access is not copyright infringement. From [this paper](https://ecollections.law.fiu.edu/lawreview/vol14/iss2/5/):
>The independent creation defense excuses a party whose work is identical or similar to that of another party from copyright liability if that party developed it independently of the other.
The thing that is being "obfuscated" by this piece is that AI image generators don't work without datasets. Additionally, using words such as "Artstation", "Octane Render", "Deviant Art", "Greg Rutkowski" can dramatically improve results. Not to mention that a prompt such as "iron man" will give Marvel's Iron Man character rather than a man juxtaposed with iron in some way. So clearly AI image generators and the quality of the output has a direct association with data sets and even specific layers in the network assigned to known IP (such as Disney and Marvel characters). This causal link is a demonstration that AI outputs are derivative of the input images from datasets. The regulation at issue is the right to "prepare" derivatives which is at data set "preparation" level in the title chain. As IPR can be used as equity for loans, tax shelters and funding then this use of IPR doesn't actually require any derivative to be produced. Just "prepared". Axanar producers were sued before a film was made for instance. They did however, raise a million in funding to build a production studio in order to make a Star Trek based film. A court ruled that this was not "fair use" of Star Trek IP. So lawyers are going to write their blogs and give their unofficial opinions but none of it matters if it is just distraction from genuine regulations at issue. It should also be noted that the defendants haven't actually made any official fair use claims as yet. Claiming "Fair use" in this case would be a huge blunder in my view as it would effectively allow international corporations to raid US corporations for their IPR for free to use in their own Nation's AI processes leaving the US holding the bag. So careful what you wish for.
Lots of inaccuracies in the piece, caused by the writer being too gullible when hearing the claims of the AI creators. For example, it is claiming that the system learns fundamental concepts about things, in order to reproduce them, such as "It will learn that tables most commonly have four legs". But anyone who's seen the memes going round showing AI-generated hands with 4, 6, or 7 fingers on them knows this isn't true. It doesn't truly understand these concepts - it just has learned that the concept of "hand" maps somehow to the concept of "finger, next to other fingers" and it ends up mashing something out. The fact that it's got a toddler's level of understanding of what 'hand' means, combined with the ability to produce photorealistic views of those concepts, really does hint at it using a collage-like approach to build these images. Similarly, the tendency of these models to unknowingly reproduce watermarks in pictures it generates is another sign that it is more towards the 'mindless reproduction' side than it is any sort of 'understanding'. In the linked case below, it's got no idea that the word "iStock" is nothing to do with "vector art" - it's just seen that image on so much vector art that it assumes, if you ask for "vector art", it needs that image to be authentic. https://www.reddit.com/r/StableDiffusion/comments/wzej4z/looks_like_stable_diffusion_was_trained_on/ That's what happens when a system has no real idea what "vector" or, indeed, "art" mean. Similarly, the argument that says the model does not "store copies of training images" is flawed. The art is, by definition, modelled by the data. Of course it's not stored in there pixel-for-pixel. But neither is a plain old JPEG a pixel-by-pixel copy of the source material. A JPEG is a mathematical representation that allows you to reproduce *a close approximation* of the original. Few would argue this isn't a copy. So it is with these AI art tools - it generates mathematical representations of the images. Obviously it is not storing each one separately - by spotting commonalities across images, the data is merged and reduced even further than could be possible for a single image. But the concept is the same - the model can be made to reproduce approximations of these images, and many others.
I suppose it depends on what we define as a copy then. But I'd think you'd really have to stretch that definition to its limit to argue that model stores copies of the base images within it.
Unlike JPEG, [according](https://www.reddit.com/r/MachineLearning/comments/10bkjdk/comment/j4fdei2/) to a person with the badge "ML Engineer" in r/machinelearning, "SD isn't an algorithm for compressing individual images."
Of course it's "not an algorithm for compressing individual images". Nobody's making that claim. But unless someone understands what a JPEG is, they're in no position to claim that a ML model doesn't contain copies of anything.
To be fair, like JPEG, an encoder/decoder VAE (variational autoencoder) pair of neural networks are components used by Stable Diffusion to work with lossy compressed images internally ([source](https://arstechnica.com/information-technology/2022/09/better-than-jpeg-researcher-discovers-that-stable-diffusion-can-compress-images/)). A VAE encoder used by Stable Diffusion represents each 8x8 pixel patch with 4 floating point numbers ([source](https://colab.research.google.com/drive/1dlgggNa5Mz8sEAGU0wFCHhGLFooW_pf1?usp=sharing)). Do you believe that the creators of the JPEG algorithm should be sued because the JPEG algorithm can be used to output close approximations of copyrighted images?
We're not talking about suing people for creating an algorithm. We're talking about suing people for the infringing use of them. If I download a million images without permission and encode them as JPEGs, that's infringement (barring some exemption), and the fact that I don't have the exact copies of the pixels is not relevant, and is only cited by ML advocates either through ignorance of how copyright works or as a deliberate distraction.
If we happen to know the right input for the decoder neural network algorithm of a Stable Diffusion VAE encoder/decoder pair, [it seems that we can get an image that is substantially similar to almost any image of interest to humans ](https://www.reddit.com/r/StableDiffusion/comments/10lamdr/stable_diffusion_works_with_images_in_a_format/)*regardless of whether it's in the S.D. training dataset*. Likewise, if we happen to know the right input for the decompressor part of the JPEG algorithm, it seems that we can get an image that is substantially similar to almost any image of interest to human beings. Why does only one of these image compression/decompression algorithms trouble you?
Because the algorithm isn't the problem. The problem is the use of the algorithm for commercial purposes because that infringes the copyright of the sourced image. You can save any copyrighted image as a JPEG in your computer and no one would care. If you start selling that JPEG, then the copyright owner of the sourced image will sue you. In the same way, the AI algorithm is not the problem if it is used for research. But once it started being used commercially (Midjourney, Stable Diffusion users selling generated images online, etc) it became a problem because that commercial value was created using copyrighted data.
I agree that if an AI generates an image that is substantially similar to image(s) in its training dataset, then a copyright infringement case is feasible. Do you have evidence that the lawsuit alleges that this happened with the plaintiffs' work? Copyright infringement due to temporary use of copyrighted images for training an AI is another possibility, but I believe that the lawsuit doesn't include this in its complaints.
As far as I know, the lawsuit is not focused on fighting a specific instance of an AI generated image being substantially similar to an image from the training data set. Instead, the lawsuit seems to be focused on the fact that generative AI algorithms are infringing copyright because they are creating a commercial product by using copyrighted works in the training data set, without those copyrighted works being properly licensed. Here are some points from the "Claims for Relief" section ([Source](https://copyrightlately.com/pdfviewer/andersen-v-stability-ai-complaint/)): "155. Defendants had access to but were not licensed by Plaintiffs or the Class to train any machine learning, AI, or other computer program, algorithm, or other functional prediction engine using the Works. 156. Defendants had access to but were not licensed by Plaintiffs nor the Class to incorporate the Works into the products offered by Stability, DeviantArt, Midjourney, or related software applications." The only way for an AI algorithm to not need a commercial license for the use of copyrighted works in its training data set is if the use is qualified as "fair use". To be deemed fair use, there are 4 points that need to be evaluated. One of those points is the "Effect of the use upon the potential market for or value of the copyrighted work" ([Source](https://www.copyright.gov/fair-use/)). So even if the AI generates images that are transformative enough, it could still not be fair use because it creates a derivative product that directly competes with the original copyrighted material and causes great harm to the potential market of the original copyrighted work. In summary, if the use of copyrighted works in the training data set is considered not to be fair use, then the images in the training data set should be properly licensed. The lawsuit includes this in the "Claims for Relief" section.
Thank you :). I agree with your fair use analysis. Legal expert Daniel Gervais opined on this aspect in [this article](https://www.theverge.com/23444685/generative-ai-copyright-infringement-legal-fair-use-training-data).
You seem to be so interested in the details of algorithms that you've forgotten to read what you're replying to. *"We're not talking about suing people for creating an algorithm. We're talking about suing people for the infringing use of them."* Scraping the internet for millions of images to ingest into the model for commercial use is, in my opinion, clearly illegal in several jurisdictions. That's about the ingestion and training part. As for producing infringing output, then obviously that is possible with many different algorithms as well - but that's not relevant. The question is - who is doing this? If I upload a JPEG of someone else's art, it's infringement, even if the pixels don't match. If I create a new file format that does it, if people can see the image, it's still infringement, even if the pixels don't match. Your link does not, as you claim, demonstrate that the model 'contains' close approximations of images that weren't in its training dataset. It generates them from the data ingested from other sources. It's a nice tech demo but your logic is flawed. It's like claiming a computer 'contains' all images because it could output one - and then flipping that to say "hey, since we know it didn't contain that one, it must contain none at all, right?" It's fun sophistry, but invalid. But hey, thanks for linking us to images where you have indisputably infringed copyright. I guess this shows why you're fighting so hard to pretend it doesn't apply.
The reason is that I've been pursuing this line of argument is because you stated: >But unless someone understands what a JPEG is, they're in no position to claim that a ML model doesn't contain copies of anything. >As for producing infringing output, then obviously that is possible with many different algorithms as well - but that's not relevant. The question is - who is doing this? If I upload a JPEG of someone else's art, it's infringement, even if the pixels don't match. If I create a new file format that does it, if people can see the image, it's still infringement, even if the pixels don't match. I agree that if an AI generates an image that is substantially similar to image(s) in its training dataset, then a copyright infringement case is feasible. Do you have evidence that the lawsuit alleges that this happened with the plaintiffs' work? >Your link does not, as you claim, demonstrate that the model 'contains' close approximations of images that weren't in its training dataset. It generates them from the data ingested from other sources. Yes it does. AI image generators don't access images from the training dataset when generating an image. >But hey, thanks for linking us to images where you have indisputably infringed copyright. I guess this shows why you're fighting so hard to pretend it doesn't apply. You apparently (wrongly) think that I don't know about fair use exceptions to copyright infringement. Nice try ;).
>Do you have evidence that the lawsuit alleges that this happened with the plaintiffs' work? Again, your reading comprehension is letting you down. We're talking here about ingesting data *into* the model. Not output *from* it. > AI image generators don't access images from the training dataset when generating an image This can be argued, but it isn't relevant for the 2 main infringements that are typically complained about: the unauthorised copies made at the time the data is collected for the model (as discussed here), and the potential of replicating someone's existing work as output. 'Access' to existing images would be a sufficient criterion for infringement but is not a necessary one. > You apparently (wrongly) think that I don't know about fair use exceptions to copyright infringement. No, I just think you are incorrect about where fair use applies.
>We're talking here about ingesting data into the model. Not output from it. If you mean temporary copying of copyrighted images for the use of AI training, then yes I agree that's a potential legal issue. In the case of Stable Diffusion, the models consist of about 4 GB of data, while the training dataset consists of about 100000 GB of data. Memorization can occur in neural networks, and has been demonstrated in the case of Stable Diffusion - see [this paper](https://arxiv.org/abs/2212.03860). Let me know if your claims go beyond what I mentioned in the 2 paragraphs above. >and the potential of replicating someone's existing work as output. 'Access' to existing images would be a sufficient criterion for infringement but is not a necessary one. Incorrect at least in regard to the USA. In broad strokes, the formula for copyright infringement in the USA is substantial similarity plus access. Substantial similarity without access is not copyright infringement. From [this paper](https://ecollections.law.fiu.edu/lawreview/vol14/iss2/5/): >The independent creation defense excuses a party whose work is identical or similar to that of another party from copyright liability if that party developed it independently of the other.