• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Microsoft's DirectML is the next-generation game-changer that nobody's talking about

CyberPanda

Banned
Microsoft's DirectML is the next-generation game-changer that nobody's talking About

AI has the power to revolutionise gaming, and DirectML is how Microsoft plans to exploit it


DirectML has the potential to be a gamechanger for next-generation hardware, allowing developers to exploit the powers of AI and machine learning to make games more efficient. DirectML has the potential to bring Microsoft's next-generation console to a new level, making the Xbox Series X a lot stronger than its 12 TFLOPS graphics processor.

We've spoken about Microsoft's DirectML before at OC3D, but as a whole, DirectML has received little attention from the media. DirectML isn't a confirmed feature of Microsoft's next-generation console, but the timing of DirectML's development suggests that hardware support is planned for the Xbox Series X.

As many of you know, Microsoft revealed its DXR API in 2018, bringing the world's attention to realtime raytracing. At that time, most people dismissed DirectX Raytracing (DXR) as a feature which won't be ready for the release of Microsoft's next-generation console. Now, DXR support has been confirmed for Microsoft's Xbox Series X, and AMD has confirmed that new Radeon hardware will support hardware-accelerated DXR raytracing in 2020, making DXR a reality for next-generation games.

Microsoft revealed DirectML at the same time as DXR, but DirectML didn't receive much attention. Companies are expected to talk about AI these days, and at the time, it was hard to judge the potential of AI within the gaming market. Raytracing has obvious benefits, but AI does not. Enter Nvidia and its Tensor cores.

Nvidia's DLSS (Deep Learning Super Sampling) technology is the first attempt from a major GPU manufacturer to leverage the power of AI to improve the PC gaming experience. Nvidia uses DLSS to upscale rendered images to higher resolutions, allowing games to run better at higher resolutions by upscaling a lower resolution image. A game could be rendered at 1440p and upscaled to 4K to deliver incredible performance increases to gamers, making higher resolution gaming a lot more accessible (when playing supported titles). While the feature has its downsides (degraded image quality), new implementations of DLSS are difficult to distinguish from native resolution renderings, revealing the potential that AI has within the PC market.



DirectML - The API that could bring DLSS-like features to everyone

While DirectML hasn't been confirmed as a next-generation console feature, you can be sure that Microsoft has been considering the option heavily. Work on DirectML has been happening, at least publically, for as long as DXR has, making it likely that AMD is working on hardware DirectML support for its next-generation graphics cards.

Microsoft has already showcased the potential of machine learning in gaming applications, with the image below showcasing what happens when Machine Learning is used to upscale an image to four times its original resolution (basically from 1080p to 4K) to generate a sharper final image and reduced aliasing. The image below is a comparison between ML Super Sampling and bilinear upsampling.

This technique has also been showcased during one of Microsoft's SIGGRAPH 2018 tech talks. This talk, which is entitled "Deep Learning for Real-Time Rendering: Accelerating GPU Inferencing with DirectML and DirectX 12" showcases Nvidia hardware upscaling Playground Games' Forza Horizon 3 from 1080p to 4K using DirectML in real-time. DirectML has the potential to improve the graphical fidelity of future console and PC games.

" We couldn’t write a graphics blog without calling out how DNNs can help improve the visual quality and performance of games. Take a close look at what happens when NVIDIA uses ML to up-sample this photo of a car by 4x. At first the images will look quite similar, but when you zoom in close, you’ll notice that the car on the right has some jagged edges, or aliasing, and the one using ML on the left is crisper. Models can learn to determine the best color for each pixel to benefit small images that are upscaled, or images that are zoomed in on. You may have had the experience when playing a game where objects look great from afar, but when you move close to a wall or hide behind a crate, things start to look a bit blocky or fuzzy – with ML we may see the end of those types of experiences. "



DirectML will be a big deal for next-generation graphics


Microsoft sees two ways to push the gaming market forward, utilising both Machine Learning and Ray Tracing technologies to enhance computer-generated graphics to levels that have never been seen before. DirectX Raytracing has arrived on Windows 10 as part of Microsoft's October 2018 update, and DirectML support came later with Windows 10's 2019 OS updates.

While DirectML hasn't received as much attention as DirectX raytracing, you can be sure that developers are looking at the new API closely. As screen manufacturers are starting to push beyond 4K, AI upscaling technologies like Nvidia's DLSS will continue to increase in popularity. 4K gaming is already a challenge, and 8K is going to prove to be even more problematic for game makers and hardware vendors. Technologies like DirectML will become vital for future games and consoles; the question now is whether or not this tech will be supported on next-generation consoles.

nv0mVGy.jpg


Ve7ySNg.png

 

psorcerer

Banned
Microsoft's DirectML is the next-generation game-changer that nobody's talking About

AI has the power to revolutionise gaming, and DirectML is how Microsoft plans to exploit it


DirectML has the potential to be a gamechanger for next-generation hardware, allowing developers to exploit the powers of AI and machine learning to make games more efficient. DirectML has the potential to bring Microsoft's next-generation console to a new level, making the Xbox Series X a lot stronger than its 12 TFLOPS graphics processor.

We've spoken about Microsoft's DirectML before at OC3D, but as a whole, DirectML has received little attention from the media. DirectML isn't a confirmed feature of Microsoft's next-generation console, but the timing of DirectML's development suggests that hardware support is planned for the Xbox Series X.

As many of you know, Microsoft revealed its DXR API in 2018, bringing the world's attention to realtime raytracing. At that time, most people dismissed DirectX Raytracing (DXR) as a feature which won't be ready for the release of Microsoft's next-generation console. Now, DXR support has been confirmed for Microsoft's Xbox Series X, and AMD has confirmed that new Radeon hardware will support hardware-accelerated DXR raytracing in 2020, making DXR a reality for next-generation games.

Microsoft revealed DirectML at the same time as DXR, but DirectML didn't receive much attention. Companies are expected to talk about AI these days, and at the time, it was hard to judge the potential of AI within the gaming market. Raytracing has obvious benefits, but AI does not. Enter Nvidia and its Tensor cores.

Nvidia's DLSS (Deep Learning Super Sampling) technology is the first attempt from a major GPU manufacturer to leverage the power of AI to improve the PC gaming experience. Nvidia uses DLSS to upscale rendered images to higher resolutions, allowing games to run better at higher resolutions by upscaling a lower resolution image. A game could be rendered at 1440p and upscaled to 4K to deliver incredible performance increases to gamers, making higher resolution gaming a lot more accessible (when playing supported titles). While the feature has its downsides (degraded image quality), new implementations of DLSS are difficult to distinguish from native resolution renderings, revealing the potential that AI has within the PC market.



DirectML - The API that could bring DLSS-like features to everyone

While DirectML hasn't been confirmed as a next-generation console feature, you can be sure that Microsoft has been considering the option heavily. Work on DirectML has been happening, at least publically, for as long as DXR has, making it likely that AMD is working on hardware DirectML support for its next-generation graphics cards.

Microsoft has already showcased the potential of machine learning in gaming applications, with the image below showcasing what happens when Machine Learning is used to upscale an image to four times its original resolution (basically from 1080p to 4K) to generate a sharper final image and reduced aliasing. The image below is a comparison between ML Super Sampling and bilinear upsampling.

This technique has also been showcased during one of Microsoft's SIGGRAPH 2018 tech talks. This talk, which is entitled "Deep Learning for Real-Time Rendering: Accelerating GPU Inferencing with DirectML and DirectX 12" showcases Nvidia hardware upscaling Playground Games' Forza Horizon 3 from 1080p to 4K using DirectML in real-time. DirectML has the potential to improve the graphical fidelity of future console and PC games.

" We couldn’t write a graphics blog without calling out how DNNs can help improve the visual quality and performance of games. Take a close look at what happens when NVIDIA uses ML to up-sample this photo of a car by 4x. At first the images will look quite similar, but when you zoom in close, you’ll notice that the car on the right has some jagged edges, or aliasing, and the one using ML on the left is crisper. Models can learn to determine the best color for each pixel to benefit small images that are upscaled, or images that are zoomed in on. You may have had the experience when playing a game where objects look great from afar, but when you move close to a wall or hide behind a crate, things start to look a bit blocky or fuzzy – with ML we may see the end of those types of experiences. "



DirectML will be a big deal for next-generation graphics


Microsoft sees two ways to push the gaming market forward, utilising both Machine Learning and Ray Tracing technologies to enhance computer-generated graphics to levels that have never been seen before. DirectX Raytracing has arrived on Windows 10 as part of Microsoft's October 2018 update, and DirectML support came later with Windows 10's 2019 OS updates.

While DirectML hasn't received as much attention as DirectX raytracing, you can be sure that developers are looking at the new API closely. As screen manufacturers are starting to push beyond 4K, AI upscaling technologies like Nvidia's DLSS will continue to increase in popularity. 4K gaming is already a challenge, and 8K is going to prove to be even more problematic for game makers and hardware vendors. Technologies like DirectML will become vital for future games and consoles; the question now is whether or not this tech will be supported on next-generation consoles.

nv0mVGy.jpg


Ve7ySNg.png


Yada yada yada... and no concrete numbers.
How many ms per frame?
 

rofif

Can’t Git Gud
4k fell flat on it's face. Years on and we still cannot reliably easily produce 4k image.
That said - I am ofc rocking 4k 27" freesync ips :D
 

Gavin Stevens

Formerly 'o'dium'
Really hoping we can get this into use for old retro games. The ability to make a game from 20 years ago, play it in 4k/60, with higher resolution textures is amazing.

Couple that with mesh smoothing and boom... you got yourself a better remaster than 90% of the tut released.
 
Last edited:

darkinstinct

...lacks reading comprehension.
Really hoping we can get this into use for old retro games. The ability to make a game from 20 years ago, play it in 4k/60, with higher resolution textures is amazing.

Couple that with mesh smoothing and boom... you got yourself a better remaster than 90% of the tut released.

Doesn't exactly work like that. You have to train the algorithm first. That means have the game run in native 4K for several thousand hours so the algorithm understands what it is meant to do and can then apply it on the fly to a 1080p picture. For retro games you have no training data. There are general machine learning upscaling techniques for textures, but they tend to give everything a very cel shaded look because they simply do not know how to generate additional information out of thin air.
 

IntentionalPun

Ask me about my wife's perfect butthole
It's all based on hardware capabilities.. and there will be a competing open source version that runs on non-MS systems....
 

GHG

Member
as the potential to bring Microsoft's next-generation console to a new level, making the Xbox Series X a lot stronger than its 12 TFLOPS graphics processor.

Why would they need some secret sauce if they have 12TF?

:pie_thinking:
 

Alx

Member
Doesn't exactly work like that. You have to train the algorithm first. That means have the game run in native 4K for several thousand hours so the algorithm understands what it is meant to do and can then apply it on the fly to a 1080p picture. For retro games you have no training data. There are general machine learning upscaling techniques for textures, but they tend to give everything a very cel shaded look because they simply do not know how to generate additional information out of thin air.

I don't think you'd need a specific training per game, you can probably use a generic model to upscale an image based on its own "experience". Like what our brain easily does with the image above : we know what a smooth line is, and can regognize a jagged line, and would be able to trace one over the other based on simple experience.
There's probably a limitation to what you can expect from it obviously, can't upscale from 240p to 8K without introducing made up content, like the Deep Dream dogs for example.
spaghetti-dog.jpg
 

thelastword

Banned
Another game changer I see from Microsoft. Everything from them is a game changer...

DirectML got little traction for good reason, pretty much all these GPU vendors have already stipulated Ai enhancement for picture quality years ago.

Remember AMD announced their highest tier raytracing solution will be cloud based, their Ai based image enhancer will be part of the Fidelity FX software suite.

Vulkan will have its own raytracing pass through lead through the open standard Radeon Rays..... As a matter of fact, you can expect less adoption for Direct X 12, DirectML and Direct X RT next gen since pretty much all base and multiplat development will stem from AMD hardware.
 

Alx

Member
ML is just a new way to write software. Nothing more.
In old terms: procedurally generated.

It's actually a way to have software write itself. In traditional procedural generation, a developer writes down the generation rules. With AI, the rules are automatically learnt from huge datasets. The resulting set of rules is something a human developer couldn't have done on his own.
 
Last edited:
Another game changer I see from Microsoft. Everything from them is a game changer...

DirectML got little traction for good reason, pretty much all these GPU vendors have already stipulated Ai enhancement for picture quality years ago.

Remember AMD announced their highest tier raytracing solution will be cloud based, their Ai based image enhancer will be part of the Fidelity FX software suite.

Vulkan will have its own raytracing pass through lead through the open standard Radeon Rays..... As a matter of fact, you can expect less adoption for Direct X 12, DirectML and Direct X RT next gen since pretty much all base and multiplat development will stem from AMD hardware.
How about we wait and see before shooting it down.
 

Great Hair

Banned
Microsoft's DirectML is the next-generation game-changer that nobody's talking about

While DirectML hasn't been confirmed as a next-generation console feature, you can be sure that Microsoft has been considering the option heavily.

Has anyone played around with gigapixel AI, waifu2x and similar and know how slow and "bad" these so-called-game-changer- features are? Now Microsoft has one, that can do that all in real time?

Does this get better with a neural network of connected Xbox Series X



Watched the demo and while i agree there´s an improvement, i wouldn´t say "it´s much much better like the presenter said". Seems a waste of resources in my book. Racing at 200mp/h ... ya got no time to pay attention if the hud, the levers, the needles, the cockpit is of higher resolution or not.

I can see this work on retro games, something i´ve been asking for years an A.I that improves older games by giving them 50 to 100% more polygons and make them better looking.

Gran Turismo 6 does that since 2013 and no one talks about it "adaptive tessallation". Out of 100K poly cars make 200K, well something like that. Not sure if that´s an Nvidia thing, but wondering why this got not much attention at all.

Depending LOD make the car better looking or reduce details (can´t go below). But with newer 4K assets, this makes no sense at all. Unless we wanna relive that AMD vs Nvidia controversy, where one of the two blamed the other for "cheating" due reducing resolution, textures or using "weaker" FP16 on certain things to win benchmarks.
 
Last edited:

psorcerer

Banned
It's actually a way to have software write itself. In traditional procedural generation, a developer writes down the generation rules. With AI, the rules are automatically learnt from huge datasets. The resulting set of rules is something a human developer couldn't have done on his own.

Dynamic programming, linear programming, etc.
Nothing is fully automatic there.
 

Gavin Stevens

Formerly 'o'dium'
Ahh apologies, I was hoping it would be able to take existing assets and dump them, possibly as an optional download or something, like we can now.
 

Bernkastel

Ask me about my fanboy energy!
Another game changer I see from Microsoft. Everything from them is a game changer...

DirectML got little traction for good reason, pretty much all these GPU vendors have already stipulated Ai enhancement for picture quality years ago.

Remember AMD announced their highest tier raytracing solution will be cloud based, their Ai based image enhancer will be part of the Fidelity FX software suite.

Vulkan will have its own raytracing pass through lead through the open standard Radeon Rays..... As a matter of fact, you can expect less adoption for Direct X 12, DirectML and Direct X RT next gen since pretty much all base and multiplat development will stem from AMD hardware.
For someone with zero credibility you do have a talent to spout bullshit and present it as a fact.
 

01011001

Banned
AI could be something that can help games use 8K TVs more effectively than just reconstruction or upscaling I guess
 

Alx

Member
Dynamic programming, linear programming, etc.
Nothing is fully automatic there.

I don't really see your point. It's like saying "programming is just writing if/else conditions". Of course it does use dynamic/linear programming, and of course it's not fully automatic (you still need knowledge and skill to design your network, your loss functions etc.). Still it is a major field of computer science, with huge benefits in many applications. What's the point of calling it "just another way of programming" when it opens so many possibilities, solved problems that we've been working on for decades ?
 

Alx

Member
Interesting sidebar: Can ML be used for non-visual tasks. I’m thinking enemy AI that learns and adjusts to your gameplay tactics, better than anything we’ve seen before.

It can be used for a lot of stuff. But while enemy AI would be the first thing you think of, it may not be the best idea actually. NPC AI is something that is very deeply linked to the gameplay loop, and it's something the game designer would like to keep under control as much as possible. Making an AI that is at the same time realistic, effective and fun to fight is a lot of work to do through ML when you can get more predictable results through simple rules.
 

psorcerer

Banned
I don't really see your point. It's like saying "programming is just writing if/else conditions". Of course it does use dynamic/linear programming, and of course it's not fully automatic (you still need knowledge and skill to design your network, your loss functions etc.). Still it is a major field of computer science, with huge benefits in many applications. What's the point of calling it "just another way of programming" when it opens so many possibilities, solved problems that we've been working on for decades ?

Because it doesn't. Try to get funding on AI/DNN today.
After all the craze calmed down it became clear that writing DNNs is just another tool for solving software engineering problems.
It's not always the best, and has no magical solutions.
 

Alx

Member
Because it doesn't. Try to get funding on AI/DNN today.
After all the craze calmed down it became clear that writing DNNs is just another tool for solving software engineering problems.
It's not always the best, and has no magical solutions.

Who's talking about magical solution ? Yes it's a tool, sometimes it won't work and sometimes it will. I still don't see your issue with it, especially when discussing a case where it's been proven useful.
Sure there was a craze about it because it's "the latest thing that works". As usual many people overreacted, but it was still popular because it worked. Before that we had the decision forests craze, and those worked too. And before that SVM. And before that adaboost. And before that regular NN, etc. All of those were different steps in the practical use of AI. And all of those steps brought their own practical applications : OCR , deepfakes, speech processing, face detection, instagram filters, biometric identification,...
 
Last edited:

darkinstinct

...lacks reading comprehension.
It can be used for a lot of stuff. But while enemy AI would be the first thing you think of, it may not be the best idea actually. NPC AI is something that is very deeply linked to the gameplay loop, and it's something the game designer would like to keep under control as much as possible. Making an AI that is at the same time realistic, effective and fun to fight is a lot of work to do through ML when you can get more predictable results through simple rules.

I can see it being used in racing games. Sort of what Drivatars already try to do.
 

Alx

Member
I can see it being used in racing games. Sort of what Drivatars already try to do.

Yeah but with drivatars we saw another pitfall of the self-learning approach, the behaviours you learn may not be the ones you want your AI to learn, even if they're realistic. It's like the MS Twitter experiment where users taught the AI racist stuff, it did exactly what it was meant to do, but the end result isn't what you really want.
Also with that kind of scenario you may not need advanced AI anyway, just like drivatars could run on current gen consoles. Even Virtua Fighter 4 had a learning AI on PS2 (and IIRC VF2 on Saturn too).
 

psorcerer

Banned
Who's talking about magical solution ? Yes it's a tool, sometimes it won't work and sometimes it will. I still don't see your issue with it, especially when discussing a case where it's been proven useful.
Sure there was a craze about it because it's "the latest thing that works". As usual many people overreacted, but it was still popular because it worked. Before that we had the decision forests craze, and those worked too. And before that SVM. And before that adaboost. And before that regular NN, etc. All of those were different steps in the practical use of AI. And all of those steps brought their own practical applications : OCR , deepfakes, speech processing, face detection, instagram filters, biometric identification,...

This thread starts from PR talk.
Like literally a wall of text with no numbers, just hype.
 

Alx

Member
This thread starts from PR talk.
Like literally a wall of text with no numbers, just hype.

Come on, it's the same article we discussed in the other thread, with a direct link to the 2018 SIGGRAPH presentation. Are you saying that conference is just PR ?
 

psorcerer

Banned
Come on, it's the same article we discussed in the other thread, with a direct link to the 2018 SIGGRAPH presentation. Are you saying that conference is just PR ?

If it comes from the vendor - yes.
And MSFT and NV are notorious for huge amounts of PR talk.
I've already said what I think: where are the numbers?
For example when AMD presented GAS (sharpening algo) they explicitly said: one RDNA thread. Which is pretty nice.
 

Flintty

Member
It can be used for a lot of stuff. But while enemy AI would be the first thing you think of, it may not be the best idea actually. NPC AI is something that is very deeply linked to the gameplay loop, and it's something the game designer would like to keep under control as much as possible. Making an AI that is at the same time realistic, effective and fun to fight is a lot of work to do through ML when you can get more predictable results through simple rules.

Fair points. I was just thinking out loud (I have no idea how it all works!) 👍🏻
 

Gavon West

Spread's Cheeks for Intrusive Ads
Ok.

Enjoy your secret sauce.

It's good to dream.
Enjoy that PS5. Its good to have logos. They're powerful and stuff.

Holy shit! Talking about dreams. Lol. Kinda like dat durr PS5, huh?
 
Last edited:

cryptoadam

Banned
I don't know if this will be this big deal for MS, but AI/MAchine learning will be a big thing in the PC space, and by the end of this console generation I think PC's will be doing insane things due to AI.

I can imagine a point where almost entire games will be designed by AI. The directing and producing will probably be still humans, but graphics, lighting, frames, res, animations, enemies etc will probably all have sort of AI/Machine learning behind them. Instead of building the Jungle, AI can create the Jungle level, make each grain of dirt individually rendered at 1P and then upscaled to 4K or some crazy shit.

Basically I think AI will blow the doors off graphics and power consumption. where large worlds can be created in fraction of time with 100's of times of the details we have, accompanied by RT level lighting and reflections.
 

Mihos

Gold Member
Yes, machine learning has much to offer games in terms of graphics and gameplay.

None of them are console exclusive.
 
Hardest part of ML in terms of real impact to the industry will be the learning curve.

ML is not a particularly easy field of predictive modelling or AI for your average games developer to grok.

This will prevent all but a very specialist niche of engineers from really contributing to the field.

Once said niche solve for all the low hanging fruit use-cases (e.g. adversarial generative networks for dynamic image upscaling, sophisticated character creators that allow you to take a photo and then turn that into an accurate 3D mesh of your characters head, player voice sampling for dynamic script-to-audio avatar voice synthesis, dynamic weapon generation for loot games to create near-infinite diverse weapon variety, etc) then the more sophisticated class of problems (e.g. dynamic branching role-playing narratives that respond directly to player agency and produce plot-graph structures that adhere to supervised constraints such as ludonarrative coherence) will take the kind of R&D time+budgets most production game teams just dont have in-order to realise, making them the exclusive domain of specialist shops (like Nvidia) to solve for.

This will take a long time to catch on and will require real support from industry aligned academia to produce the next generation of game devs with the right ML chops out the gate to engage and hence innovate.
 
Last edited:
  • Like
Reactions: Alx
Top Bottom