Interestingly, Jukebox from OpenAI was trained on what appears to be copyrighted music and involved styles and renditions that explicitly referenced specific artists. It’s now four years old though. The demo songs don’t seem to be available anymore on Soundcloud.

There is MusicLM from Google (2023) - no lyrics. Also, AudioCraft from Meta (2023) - also no lyrics as far as I can tell.

Llama 3 70b is almost as good. Self-host it and sleep well at night.