Last year around this time, I occasionally saw a kind of cover video that was generated by AI and could change the singer of the song to any singer or even a random celebrity. As soon as I saw it, I couldn't get enough of it and started searching the internet non-stop for cover videos of various AI singers.
The idea is quite simple to understand: the operator collects a large number of dry voices of a particular singer, then trains them with models such as so-vits-svc, RVC, etc., and then obtains a dataset of their voices through the training, so that they can replicate this "timbre" on any song.
This is not a novel idea; many years ago, there was a kind of cover song that was made by cutting out snippets of the sounds of every word the character pronounced, then collaging them, fixing the sound, and finally editing them into a complete music video - this was often used to tease, or make fun of, some politician or celebrity incident. This kind of AI covers I'm hooked on, on the other hand, can basically be seen as serious music works: some of them are full of entertainment, some of them are quirky as hell, and some of them are even unbelievably good. And among these many AI singers, there are a few who have done surprisingly well and have produced some high-quality works - AI Kanye is one of them.
Perhaps due to his controversial nature or his unique voice, AI Kanye has a rich variety of productions. Without Ye himself knowing it, he has performed songs in a range of styles, from old-school rap to country music or Chinese songs; and was even able to help defuse his conflicts by singing Taylor Swift's songs.
Watching such videos is really full of fun. Whereas the people who make these videos are also gradually starting to find that the purpose of creating such videos has changed from making fun of celebrities to wondering what it's like to have their favorite song by some other singer with a unique voice.
But so far, can we call this technique in any way related to artificial creatures or is it just merely a tool to be used to modify sound.
Frankenstein? Human? Robot?
I'd like to answer this with another question (yes, it's tricky):
Do you think Frankenstein is a robot?
Indeed, if the necessity of mechanization is removed, according to Hans Moravec (1988) in his book mind children, robot can be concluded as, something with sensory and behavioral abilities that were once only available to humans and animals. So, in this respect, I think Frankenstein can be regarded as a robot, especially considering several characteristics he shares with robots:
- Can do the movement action by itself
- Responds to data from external sources
- Made from human pieces
I think the idea of making an AI singer can be compared with making a Frankenstein by analogy. The reason is out of curiosity, or a tease or challenge to conventional reality. In addition, the AI singers can also be used as a way of continuing memories, even Paul McCartney has admitted that they used AI technology to isolate the pure Lennon vocals in their newly released single Now And Then. Of course, there are copyright legal issues with unauthorized use of other people's voices. In the long run, the voices copied by AI are also far from having the creativity that human singers have, as well as, emotion (if you agree with that description).
Anyway, the idea of making AI singers is like making your own version of cosplay, it's fun, inspiring - though none of it makes sense without the singers themselves. I don't see AI singers as a gimmick in the huge AI wave, something that will be a flash in for a little while. On the opposite, the root of the idea of using ai to replicate tones actually comes from the desire of humans themselves, and whether that desire will somehow inspire the music production industry or be useful for making more humanoid robots, maybe we'll figure it out in the coming days.
Bonus one.