We're at the beginning of a rapidly evolving situation where out of the blue it is now possible to direct an AI to produce imagery that is interesting using just language. From there to moving animations is a relatively small step and already being worked on. From there to photo realistic scenes directed by just words not that big of a leap either and also something that is being worked on. Are there constraints, glitches, and limitations. Of course. Has that ever stopped artists producing interesting art? Of course not. People have been telling stories via art ever since people figured out that they could scratch things into rock, clay, and whatever.
Perhaps the depressing thing for some artists is that they need to adapt to this and up their game. We've seen the same with photography. Every idiot can now have a decent phone camera and there are plenty of people on Instagram doing not so interesting things with those. Does that invalidate what really good photographers do? No. But it does make their work less special when an AI guided camera in the hands of an absolute amateur can produce photos that are decent. And now with this, you could produce realistic looking photos of things that don't exist by simply asking for them in the right way.
More content means the quality standards are raised. There are not a lot of artists left that make a living scratching things in rocks. But there are more artists than ever. The only real limitation is their imagination.
Speaking of upping their game, I have a friend who worked in photography, and photo and video editing. Over the years, the tools got better and better and more user friendly. The money started to disappear due to more and more people being able to just use phone apps to do it all. He adapted by… leaving the field completely for a tech job.
That’s what we will see: a far smaller percent of people who have the pleasure of paying their bills by working in the arts. It’s all already so commoditized, I don’t know a single artist any more who can do it full time except one woman with a gallery I met the other day on a road trip. Is she going to have to go back to her old career of environmental policy soon?
That is more an economic argument than the philosophical one about what art is. Yes, the vast majority of artists will lose their jobs, if they do art as a vocation, but people who make art, to make it, will continue to create. That is what the parent is implying, I believe.
I wonder how tweakable those generated images are. The most important requirement for 'game art creation' is that it is created in a tight feedback loop between an artists, an art director who needs to enforce an overall artistic vision, and the cold, hard requirements of the game design.
Can I tell the AI "that looks great, keep the trees and house, but I need the door on the right side, and the sky a bit less cloudy". Will it be able to incorporate such instructions without "remixing" a very different image? Can it "understand" the suggestions the same way a human can without having to go into so much details that it essentially becomes procedural generation? Is this even something that's possible with the current approach?
If that's not possible, then I can just as well google and hope to find a matching image.
I’ve been on the beta testing for Stable Diffusion and one thing that they have is give you the seed of the image with which the space is populated. What this enables is enabling you to tweak the image, as you can keep the ‚base‘ image and iterate on the prompt.
It’s not as sophisticated as you described, yet, but close.
In addition I can imagine that it is a matter of the training set. As of now, the AIs seem like jack-of-all-trades, so they know a bit of a lot of different styles and topics. But with Stable Diffusion being able to run locally, you could specialize it by training it with high detail on, let’s say, landscape photography. So then one might be able to direct the AI more precisely.
The quality of the upper echelon of art may be raised, but there's still a discovery problem there. Having to sift through stuff to find the gems is already an issue imo. The OP makes a decent (pessimistic) point
My opinion on the problem of "too much content to watch in several lifetimes" will not be solved on the supply side, but the demand side.
All content is not created equal: we are social animals and what people around us do interests us much more, even if it is of lower quality. So, if anyone can generate professional-looking creative projects with relative little effort, we'll gravitate towards people creating content on niche subjects that interest us; thus creating small communities with high engagement. Even if they have low watch count, they'll matter to those participating in them. Fanfic communities already work that way.
There always be a place for conventional mainstream media outlets creating run-of-the-mill high-production-value works, with themes averaged to appeal to the masses; it's just that they'll have a lot more competition from communities of the first type.
Consider the extreme low quality visuals of SouthPark. They used their low quality imagery as joke enhancement for their presenting ideas far more sophisticated than the majority of animated media.
I might be wrong, but I could imagine that AI will push the absoulute maximum out of human creativity. To beat AI, you'll truly need to make something outstanding and I think there will be people achieving that and truly pushing the boundaries of creativity and art forward, in ways we haven't seen before. And those people will be rewarded. Everyone will have to step up their game.
Probably, artists will use AI not to "beat" it, but as a base tool for exploring the space of possibility and expanding it into new territories. People will see AI as just one more tool in the toolbox.
People using Dall-E or Midjourney naively will be like those unremarkable classicism painters in the late XIX century doing realistic yet conventional paintings which nowadays you can create as studio photographs.
Meanwhile, brilliant artists will train new AI models throwing in data collections that have never been seen before as their training input, to generate completely new styles - just like the -ism movements threw all academic conventions away in pursue of new art styles, bringing us modern and postmodern art.
I think AI will be pretty good at doing recommendations. Show you a bit of random, get your likes and dislikes, exploit what the algorithm learned. TikTok does this well already and, I expect, will continue to do well when the content is AI generated.
I work in the area of recommendations, and this is not a solved problem at all.
You can only recommend what has been shown (without doing coldstart).
One major issue is that other forms of content than 30sec clips can't easily utilize TikToks way of bootstrapping engagement when the item is fresh. Not everyone will understand or appreciate a "new Shakespeare" and it may fall by the wayside.
I too hope it gets better, but it's hard to replace a panel of experts that have sifted through their subject when it comes to quality recommendations in some fields.
Tiktok optimises for what people spend a long time looking at, but I don’t think that anyone would claim the metric it uses is what we would want to define as quality in the broader sense.
I seriously wonder if Tiktok uses eye gaze tracking in their interest assessments. If people's eyes follow the same gaze pattern on a clip repeatedly, that's a damn clear indicator of interest.
My personal hypothesis (based on nothing) is that TikTok just uses the very strong signal of watch time. If you watch a clip all the way, or multiple times, that's good. If you skip early - that's bad.
When I had Netflix I remember being frustrated that Netflix would recommend me shows "based on" content I had watched for a few minutes, decided I didn't like, and backed out of. Why would you recommend me content if you have a strong signal I dislike it?
AI won't be designed to serve the users recommendations, but what is in the best interest of the person designing the AI. There was a pretty good article on HN about this, yesterday I think, that covered this well.
> From there to moving animations is a relatively small step and already being worked on. From there to photo realistic scenes directed by just words not that big of a leap either and also something that is being worked on.
I don’t think DALL-E can even persist a single 2d object between scenes. For example, if you create an animated character you like, I don’t think it’s possible to ask DALL-E to render that same character in a different scene.
And you’d need 3d training data to create 3d rendered scenes, where a camera can pan/zoom realistically through a scene. There’s probably much less of that kind of data available. What we do have is artificially created 3d scenes in movies, but it’s not always very realistic, not to mention proprietary.
If VR kicks off and people can start easily recording 3d moveable scenes then we might start seeing AI creating similar scenes shortly after.
Competitors like Midjourney and Stablediffusion already allow you to re-use an image seed, which makes it much easier to persist style and character across images.
3D can often be inferred by 2D data, 3D training data could also be generated, etc. Think how fast the space has moved just in the last 3 years and extrapolate from that, don't focus too much on today's shortcomings.
But then there are predictions from the 50s that we’d have flying cars by the year 2000.
Sometimes tech hits a plateau.
Animated movies make tons of money, so there’s definitely motivation to make their production faster. I just think the complexities are so intricate that I wouldn’t be surprised if AI-generated animation still seems “not quite right” in the near future.
I think you can. you need to fine tune it and make it learn about a character. I've seen prompts about Darth Vader and the beans from among us produce decent outputs, so Dall E can totally learn how a specific character looks as long as it's in its training set atleast
It's not a small step to move from image generation to animation. In fact, it's quite contrare the complexity grows with length, even creating a short would be exponential in cost. GAN Networks cannot and will never be able to do this.
GAN architecture are no longer used expect for small project, they are unstable to train, they usually not cover the whole distribution and they are hard to condition. Nowadays people use diffusion or transformer models.
The question is, should everyone calling themselves "artist" be able to make a living off their creations? Based on your predictions there will be only a subset which can be professional artists... and I think it's okay this way. Everybody can produce art and call themselves artists, but only a few can also make a living from their art - because by definition art needs a consumer and the (paying) consumers pool is limited.
I don’t want it “whatabout” this, but…Is art really the thing we need to automate right now?
There’s so much labor that we more directly need more of and that we could all benefit a lot more from automating it.
To give a mundane example off the top of my head- street cleaning. I live in a big city (Berlin) which is often filthy with litter. Obviously there are not enough people cleaning it (or it wouldn’t be). Can’t we automate that instead? How about automating construction? Part of the reason housing is expensive are lot prices but surely a lot of it is also labor cost.
It just feels like such a frivolous and relatively useless thing to automate, and a misallocation of resources.
It's not about what we need. Scientific research in this case has been the result of looking around and seen what can be done with the technology at hand without caring if it was what we need or don't. If you feel like "we" should be automating other stuff, you are free to make your own contribution, it's not like OpenAI owns the keys to the field.
But the people at OpenAI aren’t doing it as a hobby, they’ve got a lot of money to conduct this research (from Wikipedia: “The organization was founded in San Francisco in late 2015 by Elon Musk, Sam Altman, and others, who collectively pledged US$1 billion.” [1])
This is the misallocation of resources I’m referring to.
I don't understand where the misallocation of resources is. OpenAI is no longer a non-profit organization. Its goal is not to automate what is most needed now. But to advance the field of AI.
And that's all highly speculative, betting on money to somehow show up, for reasons impossible to predict, if only enough games are changed sufficiently hard. If it can be done it will be done, unless we fundamentally change the way we run the economy.
Conclusions:
(1) Perhaps we should, winning at net zero games is very much a thing in the current way
(2) Didn't know it when I started writing this reply (not at all!), but I guess I agree with you
(3) I really miss Old Google, and how we happily trusted them (deservedly or not)
"This street but cleaner and without the graffiti and mess" would be an awesome prompt for an AI. And possibly an essential step for a cleaning bot to even understand the difference between clean and not clean.
Otherwise these things are not connected in any obvious way and humanity has been known to work on many problems at the same time.
The whole point about Berlin (I live there too) is that it isn't Muenich. Muenich has clean streets, high cost of living, and is frankly a bit bland and boring. People are a bit uptight and conservative there. Not a great place for creatives to express themselves. You find a lot more of those in Berlin. And it's a big part of why Berlin is so awesome. Some Berliners, consider former citizens of Muenich gentrifying their formerly their neighborhoods a problem. Especially, when they start whining about how noisy, unclean and messy things are.
Munich (not "Muenich"), clean streets? Well ok, everything is relative, but even in Munich there is a lot of garbage and graffiti (although I concede that the graffiti in Berlin is better than in Munich, because in Munich most of it seems to be because of the rivalry of the two local football clubs). But, to also say something positive: Munich wouldn't be so expensive if it was as undesirable as you describe it. And yeah, Berlin is desirable too, that's why it now gets gentrified too. That's life!
I beg to differ :) Even with transliterations, it's either München/Muenchen (in German) or Munich (in English). You can even call it "Monaco" if you come from Italy (because, same as the more commonly known Monaco, it was founded by monks), but Münich/Muenich is not a valid name for it in any language I am aware of...
I’m not from Munich (or any place richer/cleaner than Berlin). It was just an example off the top of my head, I don’t think litter in parks and streets is what makes Berlin’s charm/flair.
Automatically cleaning streets is a much harder problem than generating images. There might just be no dedicated work being done on it currently because it isn't a solvable problem given the current state of the art.
The direct application is instant illustration. It's called "art" because it has less requirements and can't be objectively judged. There is a long way to becoming useful, but it can have a very real world impact in everyday jobs.
Lots of people losing their jobs over this also means more available workforce for those more important jobs of yours
I wasn’t seriously expecting investment to change, just kinda funny/sad that in this case the thing getting automated is something humans enjoy doing (illustrating) rather than many things they both don’t enjoy and need more (street cleaning, construction work, menial physical labor in general). So you think in your example illustrators taking up street cleaning work will result in an overall improvement of life for mankind?
Aren’t the machines supposed to make our lives better?
This is what I'm thinking as well. I would go so far to say that it's directly evil to automate away illustration and other forms of creativity. Sure, many people will think it's cool and useful, but so many people will see their reason to live taken away from them. This is not only about making a living, creative arts is something much deeper and meaningful for lots of people.
I wish we could spend our collective brainpower applying AI to fight disease, climate change and poverty instead. That would make life better.
How dare they automate weaving cloth, it's a creative endeavor with a long history! It's deeply meaningful for lots of people in some cultures! Time to smash the looms!
Or recycling - from a conveyor belt full of mixed garbage, pick out the items that are most likely to be a given type of plastic (PET, HDPE, PS, ABS etc. etc.)
Perhaps the depressing thing for some artists is that they need to adapt to this and up their game. We've seen the same with photography. Every idiot can now have a decent phone camera and there are plenty of people on Instagram doing not so interesting things with those. Does that invalidate what really good photographers do? No. But it does make their work less special when an AI guided camera in the hands of an absolute amateur can produce photos that are decent. And now with this, you could produce realistic looking photos of things that don't exist by simply asking for them in the right way.
More content means the quality standards are raised. There are not a lot of artists left that make a living scratching things in rocks. But there are more artists than ever. The only real limitation is their imagination.