I'm not into ML at all, and this was the first idea I had when I learned about the limitations of the context length. Even commented on it here[1] some time later.

Of course, it's one thing to have a good idea, another is to actually implement it well...

[1]: https://news.ycombinator.com/item?id=36531771