I'm doing some amateur work on sub pixel matching at the block-level for stereo, currently using cheap APS-C prime lens commercial cameras ( Ricoh GR) and my background is maths not cameras, but what I've observed (all in RAW) is
a) Real world scenes often go below the Nyquist limit ( e.g pattern from a brick wall at sufficient distance)
b)In a camera without a low-pass filter (e.g. the Ricoh GR) in practice you don't see Moire but you can definitely generate it artificially off real world objects
c) The Point Spread function seems quite important (e.g a 1-pixel scene feature for the GR will have half the effect on the sensor as a 2-pixel function because of the point spread)
d) I'm a bit confused as why Fourier/ DCT all work as well as they do, as the Nyquist assumption for a modern digital camera seems incorrect, but on the whole sub-pixel matching in a Fourier space (while Nyquist assumption is not true) seems roughly on par with matching in the spatial domain - I would love to see something that explains why this is so
Thanks for the pointer - I've just skimmed the last 2 chapters and looks like a good book but don't think it solves my specific problem (just to give more context, when I said amateur I'm doing this for fun not money, but have previously built an image processing pipeline from raw sensor output and I get the basics of wavelets, Fourier etc )
My specific question is "If I am matching a RAW stereo block (say 8x8 or 16x16 block), for sub-pixel resolution matching from two similar cameras, why (if at all) is it better to use DCT/Fourier matching e.g. matlab normxcorr2 or the elphel guys https://blog.elphel.com/2018/01/complex-lapped-transform-bay... , rather than try to match in the spatial domain (linear algebra). On writing this I suppose I better look at the H.264 and HEVC algos and see if they work in the frequency or spatial domain when they are looking at sub-pixel matches.
a) Real world scenes often go below the Nyquist limit ( e.g pattern from a brick wall at sufficient distance)
b)In a camera without a low-pass filter (e.g. the Ricoh GR) in practice you don't see Moire but you can definitely generate it artificially off real world objects
c) The Point Spread function seems quite important (e.g a 1-pixel scene feature for the GR will have half the effect on the sensor as a 2-pixel function because of the point spread)
d) I'm a bit confused as why Fourier/ DCT all work as well as they do, as the Nyquist assumption for a modern digital camera seems incorrect, but on the whole sub-pixel matching in a Fourier space (while Nyquist assumption is not true) seems roughly on par with matching in the spatial domain - I would love to see something that explains why this is so
Happy to discuss this more as I am still learning