There is a redundant mode available, but even in default mode when a connection break you should not have any problem on TCP stream that will just use another available connection.

Is MPTCP support required on both source and destination up to the app level?

No, the MPTCP part of the link is between the router and the VPS. The end application doesn't see this.

Correct. It's more like a VPN that bonds multiple commodity internet connections for better performance. OPENMPTCP tries to be as transparent as possible. There are a handful of servers on the internet that actually support end to end MPTCP as well so the above question is legitimate.