Support NeoGAF

[ResurrectedContrarian]

MotionBricks: a large-scale, real-time generative framework

It's a model trained to do realtime animation of character movements, fully environment aware, and dynamically responding to tasks + heavily customizable by injecting prompts along the way.



^ in this video you can see the lines showing where their controller is aiming the character and the character's view, the rest is naturally generated by the model + game engine (sometimes also a prompt like "crouching") without any special tuning or hand-edited animations anywhere

"Notably, MotionBricks applies to new downstream tasks in a zero-shot manner — no fine-tuning or task-specific tagging required — so applications can be assembled in a plug-and-play manner like stacking bricks, without expert animation knowledge."

Basically

• realtime generation of character animation, pluggable into live UE5 or potentially any engine
• responds to environment intelligently, takes freeform prompts (eg "stealthily" or "crawling" or even "hobbling with an injured left leg" and so on, see their full examples)
• easily customizable to your domain

More details and footage: https://nvlabs.github.io/motionbricks/

 

[bender]

I'm suing.

 

[kevboard]

this would work fine for NPCs, but as a character controller this would be absolutely awful. like RDR2's awful controls made even worse kinda awful.

 

[Dacvak]

There's no way this doesn't have a bunch of latency. This looks even more sluggish to control than RDR2.

 

[Diddy X]

Yep, leave that to NPCs only, please.

 

[Magic Carpet]

Only need a GPU with 4TB of memory?

 

[efyu_lemonardo]

Honestly this is cooler than ray tracing.

 

[Kuranghi]

Your actions blend seamlessly, but you don't get to see them because you've gone to bed

 

[Durin]

Seems more useful for NPC or improving automated pathing on them. You do have apps like Cascadeur that allow you to use AI to help generate poses, simulate physics effecting movements, or automate in-between 3d animation work. Then you can still go through to customize anything by hand to make it look better or performant for game responsiveness.

 

[Guardian Monkey]

is this actually generative or just nearest neighbor action mapping?

 

[ResurrectedContrarian]

is this actually generative or just nearest neighbor action mapping?

generating all motion, not just a lookup... see the full videos of even using it to drive robotics

 

[Fahdis]

Prince of Nvidia here looking pretty tight with the assfuck parkour.

 

[kenmasters]

This is the kind of AI solution that will save massive amounts of money in game development once matured and fully integrated into engines and toolchains

 

[TheThreadsThatBindUs]

This is the kind of AI solution that will save massive amounts of money in game development once matured and fully integrated into engines and toolchains

That all hinges on how much it costs performance-wise. Ain't nobody using their entire CPU perf budget on character animation.

 

[ResurrectedContrarian]

That all hinges on how much it costs performance-wise. Ain't nobody using their entire CPU perf budget on character animation.

These kinds of things will eventually, I'm sure, be distilled and tuned for your game (eg a combination of quantization, distillation, and LoRA-like fine tuning) then deployed for relatively modest gpu consumption.

In time, I imagine this to be the pattern, and that would also be a kind of inherent compression of all the hand-tuned systems and animations you'd be storing otherwise.

 

[viveks86]

That all hinges on how much it costs performance-wise. Ain't nobody using their entire CPU perf budget on character animation.

The goal would be to do a lot of this using ML cores as opposed to the CPU. Next gen consoles and PCs will be heavily overprovisioned on that front to be used just for upsampling/denoising.

It's all good stuff. A bit too slow and floaty, but that's more a creative issue than a technical one. Speed it up by 3 to 4x and it will work just fine.

 

[Deerock71]

The goal would be to do a lot of this using ML cores as opposed to the CPU. Next gen consoles and PCs will be heavily overprovisioned on that front to be used just for upsampling/denoising.

It's all good stuff. A bit too slow and floaty, but that's more a creative issue than a technical one. Speed it up by 3 to 4x and it will work just fine.

DAMMIT! I was just posting this.

 

[Parazels]

And this is just the beginning of the AI era in gaming!

 

[pqueue]

yay. the games play themselves.
no need for anyone to buy them. woohoo!

 

[TheThreadsThatBindUs]

These kinds of things will eventually, I'm sure, be distilled and tuned for your game (eg a combination of quantization, distillation, and LoRA-like fine tuning) then deployed for relatively modest gpu consumption.

In time, I imagine this to be the pattern, and that would also be a kind of inherent compression of all the hand-tuned systems and animations you'd be storing otherwise.

The goal would be to do a lot of this using ML cores as opposed to the CPU. Next gen consoles and PCs will be heavily overprovisioned on that front to be used just for upsampling/denoising.

It's all good stuff. A bit too slow and floaty, but that's more a creative issue than a technical one. Speed it up by 3 to 4x and it will work just fine.

Wouldn't doing this on the GPU be inherently problematic, since the game simulation is on the CPU and so if you need any sort of interactivity with collision meshes etc...?

 

[viveks86]

Wouldn't doing this on the GPU be inherently problematic, since the game simulation is on the CPU and so if you need any sort of interactivity with collision meshes etc...?

By the time technology like this is adopted next gen, most collision detection and inverse kinematics will likely move to the GPU as well. GPU work graphs are well suited for that use case. CPU will then deal mostly with game logic, orchestration and triggering animations, while the GPU does everything in between.

 

[The_hunter]

Interesting to think a game could have unique animations that are different every time you play it. On the flipside this lacks any personality, imagine a game like Nier Automata without it's beautiful animation work.

I think we will enter an era of entertainment, where an increasingly amount of games will have less human intent behind their craft.