Okay I'm still not sure if this is something to record streams for Twitch or something to have a Pi in front of your TV instead of a full PC. Either way, sounds like a configuration issue more than anything. I have processed 1080p 60Hz video with zero input lag on 15 year old 1.5GHz laptops. Higher end stuff should be as easy on modern hardware.
The way a (presumably non-vsynced) game normally outputs to a monitor is it generates new frames as fast as it can, and each time it instantaneously replaces the framebuffer with the latest frame. Meanwhile, the monitor is continuously scanning out the pixels in the framebuffer to the pixels from top to bottom. Therefore, it will output part of one frame at one point in the screen, then part of the next frame under it (above when the end of screen is reached), and so on.
I was going to give a solution but I just realized that if you have no control over scanout you literally will always have input lag and other strange problems. This is not a performance problem. You need to stream each frame from the computer (and each time a new frame is generated, redirect the stream to source from that new frame) to the Pi, compressing on the fly (never buffer a full frame), and the Pi needs to be able to stream this in parallel through the HDMI cable instead of merely writing stuff into a framebuffer. In such a setup, the input lag will merely be how long it takes for a single bit to go through from the computer to the TV. Not sure if there are good compressors for this (other than DSC), but literally the way this stuff is normally supposed to work is through high speed video connections, not a slow ethernet.
If I want to play super duper games I'll hook up the gaming console directly to my non-smart TV through HDMI.
If I want to play retro games or other bullshit that can run on an RPI 4, I'll hook a controller directly up to the RPI 4.
If I want to use my trackpad and keyboard of my laptop in lieu of having a remote control, I use x2x to shuttle my events to the RPI 4, open Chromium on the RPI 4, and then watch netflix or whatever junk from a full-screen browser window. This is the only place I sometimes get lag, but that's just to start a video playing so it doesn't matter.
AFAICT I'm missing the 4k output that apparently requires some model of TV that ships with smart-tv adware. Outside of that I don't see any costs.
The way a (presumably non-vsynced) game normally outputs to a monitor is it generates new frames as fast as it can, and each time it instantaneously replaces the framebuffer with the latest frame. Meanwhile, the monitor is continuously scanning out the pixels in the framebuffer to the pixels from top to bottom. Therefore, it will output part of one frame at one point in the screen, then part of the next frame under it (above when the end of screen is reached), and so on.
I was going to give a solution but I just realized that if you have no control over scanout you literally will always have input lag and other strange problems. This is not a performance problem. You need to stream each frame from the computer (and each time a new frame is generated, redirect the stream to source from that new frame) to the Pi, compressing on the fly (never buffer a full frame), and the Pi needs to be able to stream this in parallel through the HDMI cable instead of merely writing stuff into a framebuffer. In such a setup, the input lag will merely be how long it takes for a single bit to go through from the computer to the TV. Not sure if there are good compressors for this (other than DSC), but literally the way this stuff is normally supposed to work is through high speed video connections, not a slow ethernet.