I definitely prefer to host my contacts with my email, so they moved to Fastmail when my email did. What email service are you using? Most email services use CardDAV-compatible contact sync of some flavor, nothing Google is doing there is particularly innovative.

I've been a very happy user of https://fruux.com/ for years now.

CalendarServer [1] is good.

[1] https://www.calendarserver.org/