I do that too, mostly. The only real downside is no transcoding. That's fine as long as you stay on wired Ethernet but a 50G remux via trans-pacific nfs is... suboptimal. Recently I've started running a Jellyfin server on a spare NUC for those times when transcoding is needed, but sticking with direct nfs access when it's not. Seems like the best of both worlds so far.