[ home / bans / all ] [ amv / jp ] [ maho ] [ f / ec ] [ qa / b / poll ] [ tv / bann ] [ toggle-new ]

/trans/ - Transparency and Moderation

New Reply

Options
Comment
File
Whitelist Token
Spoiler
Password (For file deletion.)
Markup tags exist for bold, itallics, header, spoiler etc. as listed in " [options] > View Formatting "


[Return] [Bottom] [Catalog]

File:Screen Shot 2026-03-02 at ….png (444.29 KB,2003x1640)

 No.32441

So I've been working on a project to re-implement the VOCALOID1 engine.
I'm basing it on the description in Jordi Bonada's PhD thesis "Voice
Processing and Synthesis by Performance Sampling and Spectral Models"
and not the original papers as the former is more detailed, easier to
follow, and also describes the VOCALOID2 engine.

After a lot of trouble with getting TWM f0 estimation to work, I've
finally gotten to implementing MFPA. And amazingly, it seems to have
worked first try.

Compare my results:
https://i.ibb.co/dsvgv0fd/Screen-Shot-2026-03-02-at-3-54-48-PM.png

To the results in the study:
https://i.ibb.co/C3fjdWVd/Screen-Shot-2026-03-02-at-3-55-09-PM.png

Deleted by cool-guy from >>>/maho/7902 Post No. 7906 from thread 7902

 No.32442

>>7914
>Hmm, I see, I see. "Alignment" to be sounds like something would be made more harmonious and pleasant sounding since "noise" (unpleasant sounds) is basically a bunch of spiky sound waves, isn't it?
No it's about phase alignment, not about the frequency envelope.
>Are you planning on doing stuff with this or are you satisfying exploring what it all means? I can't can't understand it so I don't know what this entails, like "I am doing X so I can do Y tomorrow", but I don't understand the X so I don't know what Y you may be working towards.
I would like to release it as an open-source library eventually when I'm done. I would to include support for both the VOCALOID1 and VOCALOID2 engines.

You might think there's an issue with patents, but I think I should be fine after doing quite a bit of research. VOCALOID1 is from 2003 and all the patents have expired. VOCALOID2 still has active patents, but these pertain to additional techniques and uses (cross-synthesis/XSY, growl, chorus, and real-time synthesis) and not the core engine.

Well except for one patent. As far as I can tell, there is exactly one patent that covers the core synthesis engine of VOCALOID2+ - US patent 8,706,496. At first, I thought this spelled big trouble for the project. But I was able to reach out to none other than Dr. Jordi Bonada himself - the author of the paper and the inventor of VOCALOID - and I was able to get a clarification that the patent only pertained to a specific technique for precise estimation of harmonic sinusoidal parameters. Very recently, I realized an important detail, all three independent claims in the patent contain an element specifying a specific interpolation technique for non-power-of-two FFT; and there are no dependent claims mentioning the use of other interpolation methods. Meaning, I believe I can just use a different interpolation method and it shouldn't fall under the scope of the patent. In fact, a specific interpolation method I think might be both more efficient to compute and more accurate. I have not tested any of this yet, but I see no reason why it shouldn't work.

Deleted by cool-guy from >>>/maho/7902 Post No. 7915 from thread 7902

 No.32443

Also, when I'm done with VOCALOID1 engine, I'm thinking of making a post here soliciting samples of your own voices if you would like to here what they sound like when ran through the engine.

Deleted by cool-guy from >>>/maho/7902 Post No. 7916 from thread 7902

 No.32444

>>7920
I plan on releasing as an open-source library for VOCALOID enjoyers and people interested in vocal synthesis to experiment with. It's written in clean C and does not use any external libraries.

Deleted by cool-guy from >>>/maho/7902 Post No. 7921 from thread 7902

 No.32445

>>7922
By "engine" I just mean the core synthesizer, as opposed to the user-facing application called VOCALOID. I'm saying that I'm implementing the the VOCALOID engine and not VOCALOID because I am merely making an API, not a clone or alternative to the VOCALOID GUI and software. I believe the VOCALOID engine was also released as an API called VocaloidAPI or OpenVOCALOID to several other companies. I believe it was used in some computer games.
>Is this any different from a synth plugin you might add to FL Studio?
What is FL studio?

Deleted by cool-guy from >>>/maho/7902 Post No. 7924 from thread 7902

 No.32446

>>7926
>It's a DAW that's popular amoung some eletronic composers who started in the early 2000s. See the image.
Huh, so it's like Pro Tools?
>I'm not sure what the vocaloid engine is.
Let me put it this way. Imagine VOCALOID, but without any of the user interface and just the raw code that does the synthesis - that's what I mean by the engine.

>So my assumption is that you're creating the base layer to API with, then someone would create the interface with the synth, then someone would make a dashboard.
Yes, I am creating an API that you could create other things on top of, including a plugin or other user interface. I should note that I am creating my API for my implementation of VOCALOID, not YAMAHA's API, which is proprietary and is given out only to other companies.

>But you emphasized that it isn't a soundbank, so I thought that you were generating more than a synthetic sound?
VOCALOID is software that uses voicebanks to synthesize a voice. I am creating my own implementation of that. Voicebanks are not software and just contain phoneme sound data and meta data. It's kind of like writing a paint program vs. designing brushes to be used for that program; where in this analogy the brushes are the voicebanks.

Deleted by cool-guy from >>>/maho/7902 Post No. 7927 from thread 7902

 No.32447

Deleted by request >>>/b/15377




[Return] [Top] [Catalog] [Post a Reply]

[ home / bans / all ] [ amv / jp ] [ maho ] [ f / ec ] [ qa / b / poll ] [ tv / bann ] [ toggle-new ]