Do you think this will be enough context to allow the model to generate novel-length, coherent stories?
I expect you could summarize the preceding, already generated story within that context, and then just prompt for the next chapter, until you reach a desired length. Just speculating here.
The one thing I truly cannot wait for is LLM's reaching the ability to generate (prose) books.
E.g. Kafka's metamorphosis fits entirely in the context window I believe, so short novellas might be possible. But I think you'd still definitely need to guide GPT4 along, I imagine without for example a plan for the plot formulated in advance, the overarching structure might suffer a lot / be incoherent.
They are interactive. What AI is doing with story generation is a text version of the holodeck, not just a plain old book. You can interact with the story, change its direction, explore characters and locations beyond what is provided by just a linear text. And of course you can create stories instantly about absolutely anything you want. You just throw some random ingredients at the AI and it will cook a coherent story out of them. Throw in some image generation and it'll provide you pictures of characters and locations as well. The possibilities are quite endless here. This goes way beyond just generating plain old static books.
I mean, if it is a genuinely good book, I don't care about authorship. Death of the author etc.
"I want <my favorite novel> rewritten in the style of <favorite author> but please focus more on <interesting theme>." I see so many possibilities. Passionate readers could become more like curators, sharing interesting prompts and creations.
Because someone mentioned Kafka:
I'd like to know what Kafka's The Trial written in the style of a PKD novel would be like.
Does it bring them back from the dead? Is writing in the style of Jules Verne, giving us something Jules Verne would create? Ask ChatGPT to make a work of Shakespeare and it does a really bad job of it, it produces puffery but not something like a Shakespeare.
I would be pretty interested already in a work containing typical tropes of Shakespeare, stylistically Shakespearean, but still original enough to be not a rehash of any of his existing works.
I guess I would not be the only one to find that exciting or at least mildy interesting.
But your point is of course valid, it would not be a 'work of Shakespeare'.
Ok, so as I understand it, you're considering having a living human write a new play and then put it through an LLM such as GPT to rewrite it in 'the style of Shakespeare'.
That is possible yes, but only within a limited interpretation of 'the style of Shakespeare'. It could only draw from the lexicon used in the existing body of Shakespeare works, and perhaps some other contemporary Elizabethan playwrights. It wouldn't include any neologisms, as Shakespeare himself invariably included in each new play. It couldn't be a further development of his style, as Shakespeare himself developed his style in each new play. So it would be a shallow mimicry and not something that Shakespeare would have produced himself if he had written a new play (based on a 21st century authors plot).
I personally wouldn't find that interesting. I acknowledge that you wrote only 'mildly interesting' and yes, it could be mildly interesting in the way of what an LLM can produce. But not interesting in the sense of literature, to my mind. Frankly, I'd prefer just to read the original new play written by the living human, if it was good. (I also prefer to not ride on touristic paddle-wheel boats powered by a diesel engine but with fake smokestacks.)
It's frankly stupid to interpret it as anything else.
Sorry for the strong language but this is a ridiculous line to take. A 'work of Shakespeare' is not even remotely open to interpretation as being something produced in the 21st century.
If the book is actually good, then what is interesting about it is that it would still be about something that humans find important and relevant, due to the LLM being trained on human cultural data.
If its context size is >= 25k words, yes. Otherwise it will just discard the start of the prompt. And it’s a sliding window, so the more it generates, the more it forgets.
You could get an 'Illuminatus!' type book out of this, especially if you steered the ending a bit in order to reference earlier stuff. If you're trying to make a sprawling epic that flings a kaleidoscope of ideas, GPT can do that sort of thing, it's just that it won't end up making sense.
GPT is going to be rather poor at priming people for an amazing ending by seeding the ideas and building them into the narrative. Though if you're directing it with enough granularity, you could tell it to do that just like you'd tell yourself to do that when you're doing the writing yourself.
But then you're becoming the executive writer. On a granular enough level, the most ultimate executive control of GPT would be picking individual words, just like you were writing them yourself. Once you want to step away and tell it to do the writing for you, you drift more into the GPT-nature to the point that it becomes obvious.
I expect you could summarize the preceding, already generated story within that context, and then just prompt for the next chapter, until you reach a desired length. Just speculating here.
The one thing I truly cannot wait for is LLM's reaching the ability to generate (prose) books.