2016, The Jiggle Begins

Bio and I had stumbled upon the Shape Matching research paper after weeks of aimless searching, and were racing to realize it in code, skyping back and forth furiously. We couldn't pull off their "least squares rotation matching" algorithm without a mathematically complex "inverse square root" matrix operation, so we were trying to ad hoc it instead. How hard could it be to figure out an object's rotation?

Bio's "torque minimizing" approach in matlab, inspired by their in-progress engineering degree.

Some hairbrained scheme of mine

Back and forth, crazy idea after crazy idea, twitchy messes of cubes... I think Bio was giving up hope and saying we should just do it the right way, when behold! A janky ass method of determining a cube's rotation, using tons of cross products to exploit all the orthogonal lines!    

Of course, it would never work for anything not cube shaped (despite my best efforts), but it worked! And all those silly CS papers just used their jiggly cubes as deformation meshes anyways, so who needs fancy smooth jiggle meshes!

(Spoilers, we needed fancy smooth jiggle meshes. 'Turns out cube based deformation meshes are... Just a pain.)

And so, with my (usually) flawless rotation matching algorithm that could handle any (cube shaped) object you threw at it, I bravely ignored Bio's pleas for a more sensible solution and plowed ahead!

Uh oh... 8 frames per second with only 1000 cubes... Well we're going to need a lot more than that for good T&A...

Not to worry! Some of those papers mentioned that the GPU runs hundreds of times faster than the CPU, so I'll just port it over! Of course, I don't actually know anything about GPU coding... So I'll just use UE4 material blueprints for physics logic!

Oh boy...

While the idea is technically possible, visual scripting isn't meant for anything nearly this complex. Enter months and months of debugging hell.

Oh I did get the basics working... And as hoped, it ran hundreds- maybe thousands- of times faster than the CPU version, as I excitedly declared victory on twitter over and over.

Unfortunately, there's a huge gulf between a set of jiggly cubes and an animated character, and after 3 months of visual scripting hell I'd decided enough was enough.

Goodbye never ending, impossible to find glitches. I'm done.

And since Unreal Engine 4 didn't (and still doesn't, to my knowledge) have dev friendly support for complex shader code, I was also done with UE4. 

The thought of having to start over in a new language and a new game engine (Unity) that I had moved on from years ago was... Demoralizing, to say the least. And I knew deep down that I should really be doing the math the right way. Using the paper sanctioned rotation matching method that could handle arbitrary, rounded meshes. It was all going to be a lot of work...

...So I put it off. That was it for jiggle, for just over 2 years.

8 years of development

Well my game dev twitter got banned for having T-Posing fox girls in the banner. That's about 6 years of game milestone history blocked off. Oof. So here's the gist.

 

It's been 8 years since I posted that first UE4 game demo on deviantart. 

A fun little project I whipped up in a week or two. I'd been dreaming of a lofty open world RPG where you wandered a mountain side town and would get TF'd by all the magitech stuff around, then restart the game the game to be TF'd again... But actually building something in AAA tools showed me just how much modern game engines sucked at the time.

Even 8 years later, game engines still suck. 

The humans just don't look right when they move. They're just empty shells of polygons shifted around by an "armature".

But real life humans are 70% water, big squishy bags of meat that jiggle like jello in slow motion- even muscle. And that lack of movement and subtly ruins it. Bio had shown up to help me mod that demo into a full game (and to get their slime girls into it  =3  ), but as I tried to design better jiggle the limitations of conventional armature animation became all too clear...

So, 2 years of on and off research and experimenting into how to solve the problem lead us to this.

"Shape matching". A complex bit of math to simulate "soft bodies", without computationally expensive masses of FEM springs, that can be even further optimized via "Fast Lattice Shape Matching" (ie by sort of grouping all the soft little pieces into giant, overlapping, stiffer pieces). 

So, it's simple, right? Just write a quick FLSM sim to attach the flesh to physical bones in a more jiggly way, and there's your perfect porn engine! 

I penciled that in for 4 to 8 weeks on my burndown chart.

Well 8 years, over 2000 hours and 2 game engines later we're still working on it. Turns out it's harder than I expected.

You can read more about the early years and motivation in this 2018 summary; why this is the only option in my mind for making the high quality TF content that I want to see, how all the other animation methods I tried first came up short, etc. And how when VR came around it kicked this need into overdrive.

I'll make another retrospective to cover the lost tweets, but for now... Here's what 6 years of working on that bouncy cube looks like:

We're not there yet. But damn it's come a long way!

If you're interested in chatting and watching progress, feel free to drop by the discord. Thankfully that dev history (and its hundreds and hundreds of GIFs) is still intact  =)