Vtubers, those real life waifus that have taken the Twitch and Youtube platforms by absolute storm.

I got my first hit when I saw a classic Korone video,

of our best doggo girl with the most contemptuous tirade against a poor turtle and its poor small, small turtle house.

Several memes later and I was interested in the doggo herself, I could feel myself entering dangerous weeb territory. Sure, I've liked anime since I was a kid and Ive basically kept up with it my whole life, but this felt like that one step where you start losing ever more dignity by your mere involvement.

And so I dove head first into the marvelous degeneracy.

None of that matters though, because this fucking blog is about my process for making a Vtuber.

So, to start, there are two primary ways to go about becoming a real animated Vtuber. By far the most popular way is through Live2D models. They're basically images rigged to give the illusion of some basic three dimensionality, and they're super, super dominant from what I can see. I'd imagine this is because of a less technically demanding skillset. Live2D isn't as complicated as the 3D alternative. 3D is a more varied sort of beast, and since I'm a 3D artist obviously I chose that. After trying Live2D for a bit and it not really piquing my interest.

One of my favorite examples of a Live2D Vtuber is Nyanners.

One of the primary examples of a 3D Vtuber is Snuffy.

You can tell in a few ways, the polygon edges along her silhouette are the obvious fuckin' one, but the classier more elegant bigger brained way is the way that the outlines around her model clip in and out. You see, her model is outlined by way of taking her model's polygons, transforming them outwards slightly, and inverting the face of the polygon's... well, their direction basically, it creates that sweet outline effect.

So, first things first, I needed a 3D model. Easy, I know how to do that shit well enough.

Now, I want to be a Vtuber, but I'm a dude, thats strike one, noone likes a cute anime girl that isnt a cute fuckin' anime girl. Obviously.

So first thing I had to do was figure out what I wanted to be. Had to be an animal. Everyones an animal, or a demon, or some shit, so I figured an animal, yeah. Now, I could try and be a sexy anime boy Vtuber, but I've seen some of those, they exist and I dont know of many that are really particularly popular, plus that feels like the kind of obvious choice, yeah?

Nah, I wanted to be big, I wanted to be ugly, I wanted to be what I feel Vtubers have made me into. A big fat fuckin' otaku weeb. Great, so, animal, weeb, narrowing it down. What animal? Obviously gotta be a wet, sweaty looking animal with a bad complexion. First thing that comes to mind is obviously a frog.

[image loading]

So I arrive at this lad, the Frog Prince Frotaku. He has it all, hes fat, hes got wartcne (warts and acne, both, at the same time) hes got some wet sweaty skin.

[image loading]

Sculpted him in Zbrush, retopologized in Maya, UV'ed in Maya, textured in Substance Painter, the rendered views are done in Marmoset for convenience, but the end result will be in Unreal. I initially had him at 12K polys, I figured going low seemed to be what most other Vtubers did, but uh, fuck that I made him 40K polys, so hes closer to a current gen game asset ("closer," a lot of current gen game assets are well into the six digits by now.)

Aight so, this basically through my typical workflow, so thats the easier part done. Next I have to figure out the Vtuber tech thats used for 3D. Snuffy is a good example because her face tracking is really, really good, especially compared to Live2D stuff. Shes mentioned she uses an iPhone, so a little digging found that iPhones are basically the only mother fucking phones that have this lil tech kit for face tracking. Its really good, like I tried it out today for the first time, its really good considering its on a fucking commercial phone.

So, I'm putting this into Unreal, thats my render engine of choice, it has nice Live Link (an app that utilizes the face tracking stuff in an iPhone and directly connects to Unreal) functionality, now, what I have to do is the stuff I typically really don't like to do. Rigging. Skin weight painting. Technical animation setup shit.

That being said, I haven't started that, the first and most important part to work on is BLENDSHAPES. Blendshapes are neat, they're basically taking parts of your model and creating this like, library of deformations. These are primarily used for the facial animations, Apple has a handy list of BlendShapes that they support, its pretty comprehensive, and its not limited, you can add or subtract as necessary, but there are a total of 52 and thats absolutely enough for me.

So I've spent tonight starting this process, I made four BlendShapes. 1. Blinking the left eye, 2. Blinking the right eye, 3. Closing the mouth, 4. Puffing the cheeks.

I set up the blueprints to connect Live Link to Unreal and imported my model with it's BlendShapes and gave it a quick go around as a proof of concept with those four BlendShapes.

The results;

Blinking? Works well. Puffing the cheeks? Hoo boy, that works and is easily the most fun little BlendShape I'll get to make.

The mouth closing doesn't work so much though, it appears to be an issue more with the way Live Link is reading the mouthClose parameters though, the Live Link app has a ton of little numbers on the screen indicating the level of the parameters in action, and closing and opening my mouth barely budges the parameter thats supposed to dictate that, BUT the mouth DOES kind of move, so its nice and easy to isolate the realm that problem lives in and the rest of it FUCKIN' WORKS.

The hardest parts are now staring me in the man titties, so back to making another 40ish BlendShapes and figuring out exactly how I need to go about dealing with the rig for this fat weeby fuck.

Toon in next time for the update where hopefully my model has full BlendShape action, and maybe even a rig capable of moving the neck or shoulder region.

'til then, this has been a truncated journey towards my becoming the fat slimey otaku weeb I've always been at heart, if you have questions address your supplications and Frog Prince Frotaku may answer.

1Lurker1988

3 Posts

April 10 2021 20:03 GMT

--- Nuked ---

WombaT

Northern Ireland15751 Posts

April 11 2021 00:28 GMT

Glorious, but I mean if you want to break into the market it’s got to be some weirdly sexualised anime kid, know your audience!

I mean I’m personally down with B movie Shrek personally, intrigued to see how your project develops

Zambrah

United States5664 Posts

April 11 2021 12:30 GMT

I figured there was probably a small niche of self owning anime fans that might enjoy a visual skewering, but tbh I probably am going to make a Frog Prince anime boi now. I finished learning how to actually do all the shit to make the head and face do things,

and I think Ive come to the conclusion that unless Im going Code Miko hard with real mocap shit, this super inhuman thing isn't going to be within my skill range, at least without spending FUCK TONS of time refining this, and I don't think its worth it. Theres a good reason every Vtuber is a simplified form a la anime, mouth is easy to move, the most difficult part is honestly the eyes, and Im kind of assuming that they basically have a weird unmoving eyeball and the pupil is being moved by the rig, but maybe they just have the two big intersecting eyeballs in their skulls.

Either way, I'm probably going to make an anime, but I like to concept and will continue with the Frog Prince theme, just like, pretty this time.

Or maybe something dark and goth-y like everyone else seems to be doing. If I do that though I'm going to try and draw from Hades and get something a bit more with that incredibly magnificent aesthetic going on about it.

WombaT

Northern Ireland15751 Posts

April 11 2021 21:05 GMT

He’s so sexy that I’ll be damn disappointed if I don’t see him in full animated glory dagnabbit! But no I have enjoyed your chronicling of your project thus far

SeekerIsABitch

1 Post

April 12 2021 06:25 GMT

--- Nuked ---

DoctorHelvetica

United States14929 Posts

April 12 2021 18:28 GMT

I’m considering this because I have too many enemies to make content with my real face

[UoN]Sentinel

United States11279 Posts

April 12 2021 19:03 GMT

I expect him to have a Shrek voice

All of the anime waifus you linked though are straight out of the uncanny valley I don't understand how this is so popular in its current state

Zambrah

United States5664 Posts

April 13 2021 14:08 GMT

I was planning to do a Babycakes voice, since I can do that one and talk at the same time, unfortunately I am not particularly good at voice acting stuff, lol

I think its popular because at least partly because of the streaming nature of it, being a streamer AND a waifu is a big game, the technology also is relatively solid, its definitely not hyper life like but the setup required for that sort of thing is actually just far too intense to be reasonable, I think we're at that level of technology where people can get an okay setup worked out that isnt like, perfect, but does a solid approximation

Hololive is doing some higher end full body stuff though, its pretty cool, makes sense they're doing it since theyre one of the largest Vtuber companies and Japan has the whole Idol thing going on.

Also I've completely moved on from Frotaku at the moment, right now I'm doing a more standard anime-esque character

[image loading]

Ive never actually made an anime character before, one of the weirdest parts is always the eyes. They're SO unnatural and often colossal that they kind of break reality in a 3D context. Like the eyes are so colossal they collide inside of the head which makes rotation of the eyeballs a very odd prospect.

One of the ways I've heard about to deal with it is to basically have no eyeballs and just move the pupils around the plan of the face, and Ive recently learned another very interesting method used by Genshin Impact from this ripped Genshin model,

[image loading]

This is neat, its literally got no eyes, its got two EYE SOCKETS (very shallow eyesockets) instead, and moves the pupils around like I said before, has the little piece of geo for the shine on the bottom of the eye too. Its kind of ingenius, its taken the Built In Eyes thing and just inserted them inside of the skull. Since the game has a simpler flat shading style you don't really notice the kind of fucked up way the socket is, but the slight outward curve helps deliver that eyeball look, and the pupils being inside the socket gets you that natural looking eyeball.

Super clever, been enjoying studying some stuff like that, really cool