• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Graphics Analysis: The Medium - Too ambitious

VFXVeteran

Banned
All,

Someone on these threads posted a comment about the game The Medium being unoptimized in hopes that it would give credence to the argument that most dev studios need years of learning new hardware in order to extract out the full potential of hardware. I resist this claim and so I decided to download The Medium and take a look at what's going on.

My initial impressions of the Medium is that this renderer is focused on "quality" pixels. What does this mean? It means that most pixels that make up a frame are pretty costly and return an extremely good approximation to the rendering equation. For example, if I pass a vertex through the pipeline and begin to texture it with a typical texture (i.e. 2k) and then evaluate it through a typical directional light source using the simplest diffuse equation (i.e. Lambert) along with adding a basic specular highlight - my pixel is relatively cheap in cost. However, if I was to make that same pixel use a complex Oren Nayar diffuse model, sample an area light, then add a GGX specular model with anisotropy, I've made a very expensive pixel. Here's a vid of the PBR materials and how you can tell how complex they are:



The Medium goes above and beyond this with their graphics engine using RT. But it doesn't stop there. I can only "guess" at how their rendering pipeline would be like with so many individual framebuffers to fill up. Their 2 world camera system with individual rendering isn't exactly half the performance, but it's definitely a big drop. Let's take that system and put RT in the mix and on top of that render RT transparency materials. Too ambitious.

Ray traced transparency is the #2 most computensive calculation in ray-tracing. Here's a pic of a recent talk at the GDC over RT transparency:

4ea62b69d59f4e3d586113bc07571c8f.png

It's great that it's order independent. That means no sorting polygons first before running through the rest of the pipeline. Bad for computing fresnel calculation using an index of refraction term. Downside? Still using texture-space though. Still no worldspace calculations like in film but it's a start.


1ec53c052ec331000e4a0fccb51c164d.png


In the far right upper corner column, you see a ray being cast to an object. It doesn't stop there, it travels through the object to the other side where it has to continue to travel and hit objects behind it. Add a transmission algorithm for the BTDF (transmission function) and possibly many samples to get frosted glass like in the bottom image, and you get a demand that is beyond the capabilities of current hardware (including the 3090). Look at this video of me proving that..




Moving the camera away from the window increases FPS dramactically as the renderer only has to process opaque triangles.

Now to really hit home the point, I did a test whereby I render the entire window door (large object) to take up the entire camera view to force the entire shader to run to simulate what an offline render would do when every single pixel is casting multiple rays in this complex BTDF shader. The results are telling:




FPS drop to the 8s and that's with Balanced DLSS. Realtime RT of this nature is simply not doable on realtime hardware yet. I expect that the next iteration of graphics cards (if they aren't just mild upgrades) could see the likes of nearly the entire rendering engine using worldspace RT with all features enabled. Turning off RT increases the FPS by nearly 6x. A very costly material.

As an aside, I have to nitpick the lack of ambient occlusion. They decided to skip RT ambient occlusion on the dynamically moving character on the floor. She walks across it but her feet aren't planted on the floor. I can't stand this. Nearly every single game still has problems using occlusion properly. We really need to get the GI component of the rendering equation more accurate in these games.



These are my initial impressions of this game. The big takeaway from this post is that some games have cheap cost to each pixel like MIP mapping a small 2k texture/normal map with one directional light source and possibly some screenspace FX. These aren't expensive pixels to render and they show in their appearance if you look at them individually. However, this game and some other ones that have complex rendering algorithms with RT lighting are considered to have "expensive" pixels to render in the pipeline. The Medium does some really radical things with gameplay and forcing a huge bandwidth bottleneck with this game due to it's complex RT decisions and it's dual framebuffer rendering with completely different assets and FX. I think for the Xbox crowd, this should be a game to be proud of because of it's complexity (similar to FS2020 when it releases to XSX).

Lastly, this game is running on UE4. The GPU usage is maxed out at 99%. It's thoroughly rendering the fastest it can and no resources are unused with very little wasted cycles. Unoptimized? Not really. Bad decision to use such an expensive RT feature? Yup. Rendering very expensive pixels and their design for the game forces that. Furthermore, holding 2 different framebuffers where very little data may be cached is asking for trouble. It's not a knock against the studio but really a praise IMO. Very few studios will outright push the hardware to it's knees like Crytek/iDSoftware did back in the day. I like the game and think I'll keep it and skip out on RE8 for now. I'm liking the story.
 
Last edited:
I thought it was weird that the game would hard tank my FPS even with a 3080 at 1440p. Seemed like an optimization issue.

It’s a pretty game but performance on PC seemed very borked in my experience. Not sure if it was ever fixed.
 

VFXVeteran

Banned
I thought it was weird that the game would hard tank my FPS even with a 3080 at 1440p. Seemed like an optimization issue.

It’s a pretty game but performance on PC seemed very borked in my experience. Not sure if it was ever fixed.
It's not a bug. Not optimization per se. The GPU usage is fairly high. That indicates it's quite busy. The RT is just very costly. 1440p or lower won't change this much I think with the way they designed the rendering. UE4 is a very optimized graphics engine *IF* you use it's inherent functions. It's also meant for portablity with other OS. Deviating too far from it's codebase goes into uncharted territory.
 

VFXVeteran

Banned
Very nice analysis and added information from slides, another excellent work VFXVeteran VFXVeteran
I smiled when I indirectly simulated an offline render of that window. The game uses fixed camera angles for a reason. It would probably be a LOT slower running if the camera was free to move about with player control. But they was sloppy letting someone look out a RT transparent window that could take up the entire screenspace! :messenger_tears_of_joy:
 

Guilty_AI

Member
Really interesting analysis, i can't help but compare with your GoT one where the situation seemed to be the exact opposite. One trying to push graphics paradigms the most they could possibly manage, and the other using all sorts of tricks to workaround performance limitations while still getting satisfying visuals.

This really shows how big of a role the developer's ingenuity play at making games look nice, and how its important to separate the notion of good visuals from accurate rendering, the former being something very subjective and the latter possibly a true indication of hardware power and an engine's capabilities.
 
Last edited:

VFXVeteran

Banned
Really interesting analysis, i can't help but compare with your GoT one where the situation seemed to be the exact opposite. One trying to push graphics paradigms the most they could possibly manage, and the other using all sorts of tricks to workaround performance limitations while still getting satisfying visuals.

This really shows how big of a role the developer's ingenuity play at making games look nice, and how its important to separate the notion of good visuals from accurate rendering, the former being something very subjective and the latter possibly a true indication of hardware power and an engine's capabilities.
Very good post and 100% agreed.
 

Kataploom

Gold Member
Good, I love your posts man cause computer graphics is an area I've always wanted to get into. BTW I was doubting to try this on my Vega 8 GPU but I don't want my PC to explode before I can get a GPU lol
 

Jokerevo

Banned
All,

Someone on these threads posted a comment about the game The Medium being unoptimized in hopes that it would give credence to the argument that most dev studios need years of learning new hardware in order to extract out the full potential of hardware. I resist this claim and so I decided to download The Medium and take a look at what's going on.

My initial impressions of the Medium is that this renderer is focused on "quality" pixels. What does this mean? It means that most pixels that make up a frame are pretty costly and return an extremely good approximation to the rendering equation. For example, if I pass a vertex through the pipeline and begin to texture it with a typical texture (i.e. 2k) and then evaluate it through a typical directional light source using the simplest diffuse equation (i.e. Lambert) along with adding a basic specular highlight - my pixel is relatively cheap in cost. However, if I was to make that same pixel use a complex Oren Nayar diffuse model, sample an area light, then add a GGX specular model with anisotropy, I've made a very expensive pixel. Here's a vid of the PBR materials and how you can tell how complex they are:



The Medium goes above and beyond this with their graphics engine using RT. But it doesn't stop there. I can only "guess" at how their rendering pipeline would be like with so many individual framebuffers to fill up. Their 2 world camera system with individual rendering isn't exactly half the performance, but it's definitely a big drop. Let's take that system and put RT in the mix and on top of that render RT transparency materials. Too ambitious.

Ray traced transparency is the #2 most computensive calculation in ray-tracing. Here's a pic of a recent talk at the GDC over RT transparency:

4ea62b69d59f4e3d586113bc07571c8f.png

It's great that it's order independent. That means no sorting polygons first before running through the rest of the pipeline. Bad for computing fresnel calculation using an index of refraction term. Downside? Still using texture-space though. Still no worldspace calculations like in film but it's a start.


1ec53c052ec331000e4a0fccb51c164d.png


In the far right upper corner column, you see a ray being cast to an object. It doesn't stop there, it travels through the object to the other side where it has to continue to travel and hit objects behind it. Add a transmission algorithm for the BTDF (transmission function) and possibly many samples to get frosted glass like in the bottom image, and you get a demand that is beyond the capabilities of current hardware (including the 3090). Look at this video of me proving that..




Moving the camera away from the window increases FPS dramactically as the renderer only has to process opaque triangles.

Now to really hit home the point, I did a test whereby I render the entire window door (large object) to take up the entire camera view to force the entire shader to run to simulate what an offline render would do when every single pixel is casting multiple rays in this complex BTDF shader. The results are telling:




FPS drop to the 8s and that's with Balanced DLSS. Realtime RT of this nature is simply not doable on realtime hardware yet. I expect that the next iteration of graphics cards (if they aren't just mild upgrades) could see the likes of nearly the entire rendering engine using worldspace RT with all features enabled. Turning off RT increases the FPS by nearly 6x. A very costly material.

As an aside, I have to nitpick the lack of ambient occlusion. They decided to skip RT ambient occlusion on the dynamically moving character on the floor. She walks across it but her feet aren't planted on the floor. I can't stand this. Nearly every single game still has problems using occlusion properly. We really need to get the GI component of the rendering equation more accurate in these games.



These are my initial impressions of this game. The big takeaway from this post is that some games have cheap cost to each pixel like MIP mapping a small 2k texture/normal map with one directional light source and possibly some screenspace FX. These aren't expensive pixels to render and they show in their appearance if you look at them individually. However, this game and some other ones that have complex rendering algorithms with RT lighting are considered to have "expensive" pixels to render in the pipeline. The Medium does some really radical things with gameplay and forcing a huge bandwidth bottleneck with this game due to it's complex RT decisions and it's dual framebuffer rendering with completely different assets and FX. I think for the Xbox crowd, this should be a game to be proud of because of it's complexity (similar to FS2020 when it releases to XSX).

Lastly, this game is running on UE4. The GPU usage is maxed out at 99%. It's thoroughly rendering the fastest it can and no resources are unused with very little wasted cycles. Unoptimized? Not really. Bad decision to use such an expensive RT feature? Yup. Rendering very expensive pixels and their design for the game forces that. Furthermore, holding 2 different framebuffers where very little data may be cached is asking for trouble. It's not a knock against the studio but really a praise IMO. Very few studios will outright push the hardware to it's knees like Crytek/iDSoftware did back in the day. I like the game and think I'll keep it and skip out on RE8 for now. I'm liking the story.

The fact that this game runs like ass even on high end PC hardware points to only one thing:team bloober are technically shit at their jobs. They were out of their depth and it shows. Give this project to another more experienced studio and it could have been something really special.
 

Kilau

Gold Member
Nice write up. I never looked into the resolution or effects in the game until now. It has a lot going on but it really doesn’t look all that good imo.

Admittedly, I didn’t get very far into the game before it crashed on my Xbox and I haven’t been back since.
 
S

Shodan09

Unconfirmed Member
I always thought this game looked stunning but that the character didn't look like she was really in the environment. Now I understand why!
 
Thank you for putting this together.

I was also under the impression that this game was unoptimized, But the reality is, it's trying to do some things and the tech isn't there yet. Wow, Not even with a 3090.

Not worth it in my opinion. Mediocre story and gameplay
 
Last edited:

sobaka770

Banned
Thanks for the analysis! However I'm not sure about the praise: the whole point of optimization and generally art-style/graphics is to make decisions for best presentation and performance on a target hardware. While the technicals do matter obviously and need to be pushed, if the game uses the most precise RT effect and didn't do AO properly - that's just not proper decision making because one is much more noticeable than the other. As an average gamer - some effects that works make you salivate as an expert don't do shit for me and probably the majority of the audience - otherwise GoT wouldn't be considered such a graphical masterpiece as it often used low res textures and repeated tilesets. However the choices make it not only look good but also load fast and run at 60 FPS on next gen.

I honestly don't see the point today to drive hardware to it's knees 3 years into the future as you rarely know what new development will happen that will make your game still look outdated. AC unity was optimised for the future - but the pop-in is unacceptable still today when you can run it at 4k60 and the game has no HDR.

The devs should make choices for the game to perform and look best on modern hardware with a little bit of future-proofing in case of pc releases with extra optional bells and whistles but I don't think that's what Medium does.
 

Panajev2001a

GAF's Pleasant Genius
It's not a bug. Not optimization per se. The GPU usage is fairly high. That indicates it's quite busy. The RT is just very costly. 1440p or lower won't change this much I think with the way they designed the rendering. UE4 is a very optimized graphics engine *IF* you use it's inherent functions. It's also meant for portablity with other OS. Deviating too far from it's codebase goes into uncharted territory.

It is an optimisation issue though if you use a costlier setup than needed to get your visuals across. I appreciate things done more correctly is a good bragging point for the dev that implemented it, but as a product for the consumer a slightly cheaper and close enough approximation is where the technical artists and engineering lies IMHO.

UE4 does offer a humongous set of features, its open source, and it is widely used but it is not the game and developers that try to maximise what the HW can do and the output and performance they seek will still have a uphill battle and may have to decide to roll out their tech:

Interesting thread with the responses from a WB Technical Architect:
3rmYPGb.png
W7hhTQ9.jpg
skRTfk7.png
 

Cattlyst

Member
I know nothing about this topic but but thank you OP for the clear and informative post (and examples). This is the kind of thing I come to gaf for. Merci!
 

Clear

CliffyB's Cock Holster
It is an optimisation issue though if you use a costlier setup than needed to get your visuals across. I appreciate things done more correctly is a good bragging point for the dev that implemented it, but as a product for the consumer a slightly cheaper and close enough approximation is where the technical artists and engineering lies IMHO.

UE4 does offer a humongous set of features, its open source, and it is widely used but it is not the game and developers that try to maximise what the HW can do and the output and performance they seek will still have a uphill battle and may have to decide to roll out their tech:

Interesting thread with the responses from a WB Technical Architect:
3rmYPGb.png
W7hhTQ9.jpg
skRTfk7.png


The problem with bespoke engines is staffing, as in its way easier to hire people with UE experience than train them up on your in-house tech. What's more the longer you maintain and augment your own stuff the likelihood is that you are going to run into tech-debt of your own making, which is worse to deal with because its unique, not shared as with a massive dev community like Unreal.

For my money, unless you have massive corporate backing and ready access to the best people, these days the best approach looks something like Housemarque. Where they use a familiar core engine with bespoke tech -built for purpose and product identity- layered on top.

I'd also add that when looking particularly at European developers you need to understand where their roots like. Devs like Housemarque and Remedy came out of the demoscene. Tech is intrinsic to them, whereas I feel like teams like Bloober are more art/design driven.
 

Panajev2001a

GAF's Pleasant Genius
The problem with bespoke engines is staffing, as in its way easier to hire people with UE experience than train them up on your in-house tech.

I can see that happening (same reason that saw a strong push in things such as Cordova/PhoneGap first and React Native second or nodejs + TypeScript in backend services as hiring JS devs is easier and cheaper), but I also think it is a bit overblown too. I see this as a problem for A/AA studios trying to staff up quickly than a small indie or a big AAA studios which are focused on either only creating and supporting the tech that they need for their own game or have a very high bar for what they want to achieve, have the know how, and the support they need to handle it in many cases.
Then again for very small teams the likes of Unity and UE have an appeal as you can get a concept in front of your audience or a publisher sooner. UE was a good choice for Housemarque and allowed them to scale while still customising pieces of it.
In many cases I feel it is a manager fear and need to keep costs down that creates monoculture, focuses more on how you build that what you build to an excess, and tries to make how you build streamlined with the hope that you can afford much cheaper labor and depend on expensive devs less too.

It also depends what kind of engineers you can afford to hire and if the target you are aiming require those people. Are you hiring people with a very narrow focus and experience (and I would say depth in terms of skills and overall experience) or are you hiring people that really know about the fundamentals and know how to learn than just how to be productive with one tool? Are you hiring Software Engineers or UE devs? You may not need the former but the latter… it depends.
It also depend on how well documented and organised your stack is of course.

What's more the longer you maintain and augment your own stuff the likelihood is that you are going to run into tech-debt of your own making, which is worse to deal with because its unique, not shared as with a massive dev community like Unreal.

I feel that it is true if you do not start, out of need, to customise and due to inexperience, time constraints, or a myriad of other reasons you cannot get your changes upstreamed. Now you still deal with your tech debt, UE’s one, as well as changes in UE’s roadmap that might push your closer and closer to a de facto fork of UE that loses a lot of the advantage UE’s community brought. The thread and tweets I linked to and the images I posted here had an interesting discussion from people that have been or are involved in making engines from some very large corporations and not all are sold on the “in house engines are for mega devs and very expensive to setup and maintain” story (see the WB MTL Architect in those tweets).

For my money, unless you have massive corporate backing and ready access to the best people, these days the best approach looks something like Housemarque. Where they use a familiar core engine with bespoke tech -built for purpose and product identity- layered on top.

I'd also add that when looking particularly at European developers you need to understand where their roots like. Devs like Housemarque and Remedy came out of the demoscene. Tech is intrinsic to them, whereas I feel like teams like Bloober are more art/design driven.

Agreed, they made choices that definitely fit well with their studios’ goals and culture.
 
Last edited:
Top Bottom