I tried to tidy up this weekend, but it was so good that I ended up making another one and time just flew by.
These days, you can just ask an AI a little bit about what kind of model it is, so I'll skip the overall introduction and talk about my two favorite features.
https://github.com/multimodal-art-projection/YuE
(I'll give you the address first)
The first is the cover studio function.
If you give it a song that has already been made, it will automatically transcribe it and allow you to remix it using this score.
-I gave it a song from the past and transcribed it, then I gave it the original lyrics and specified the music style as hiphop, rap, 220bpm.
When transcribing, it also places the vocal line, so if the rhythm is roughly right, it seems to fit in that spot.
Personally, I think this is great. When I was making the background music for a magical girl (..) video I made on the weekend, the AI, which already understands prompts with time and divisions, composed music to fit that specific part. It's like commissioning a composition. Not only does it create a complete, consistent song, but it can also compose individual sections as needed and combine them into one song. The strange sound effect (..) at 56 seconds is not from Minimax, but from the Yue model inserted between the songs. This is because it knows what content appears in that part. It's like a musical composition.
The second is also the generation function.
There are many local generators, but the biggest advantage of this generator is that you can use abc musical notation. When transcribing, it also generates scores. You can then remix by genre. The best part, in my opinion, is that unlike other generated works, if you ask the AI to make a 5-minute piece and it decides "I think I'm done," it will cut the length on its own.
This is the worst when you're trying to create background music for a video. But this model uses bpm to create music that fits the video length in one go. For example, if a few bars have a few lines of lyrics, the song length will be exactly right, so you just need to insert the generated score. Plus, remixing is possible, so you can adjust it faster or with a different mood.
And they say that with a little tweaking, it's possible to make 15 minutes according to the 5090 standard (I haven't tried it yet, but I just opened the function limit).
So I'm tinkering with it and making my own app to use. It may not be versatile because it's based on my personal convenience and purpose, but if it becomes usable enough, I'll make it public. There are a lot of other people's apps on GitHub, so feel free to check them out.
Even without this app, if you give the AI the address https://github.com/multimodal-art-projection/YuE and say "do it!", you can install it locally...
P.S. There's also a quantized version available, so you can create it even with low specs.
