It's not particularly expensive to license music lyrics in terms of doing a start-up. The big problem is you won't be able to get search traffic these days. There was a rush of lyric sites once upon a time, during the content farm wars years (azlyrics.com was one of the few survivors of that, which remained popular). You could easily get a wave of search traffic (and several dozen sites did). Now if you launch a lyric site like that, you're more likely to get tagged as a shallow content farm by Google, and that's that you're done.

The biggest cost is that you have to figure out a substantial, original content way to differentiate from every other lyric source for SEO purposes.