Search this blog

28 October, 2011

Open questions - my two rules

As I wrote here, there are some fundamental questions in realtime rendering that I wish I knew more about. I do have though two  rules I apply when thinking about rendering techniques

  1. Reduce variance: It's better to be consistent at a lower quality than have glitches/flickering artifacts at a higher quality.
    • Postulate: all graphical effects should be reviewed in motion, crossing quality boundaries
  2. Less is more: It's better to not have a given effect than have it at a too low quality level.
As an example we can analyze shadows. The first rule tells us that it's better to have stable cascades at a lower resolution than having perspective shadowmaps at high resolution. 
The second rule tells us that it's better to have lower filtering and cull shadows from some objects or limit the shadowing maximum distance, than having bad shadows everywhere.

Unrelated, I just saw this as a job opening at Valve... Smart guys!

We believe that the more we know about human behavior, about how and why people do what they do, the better our products will be. All game designers are, in a sense, experimental psychologists. That’s why we’re looking for a experimental psychologist to apply knowledge and methodologies from psychology to game design and all aspects of Valve’s operations. We want to exploit your experience with experimental design, research methods, statistics, and human behavior to help craft even more compelling gameplay experiences for future Valve titles. We’d also expect you to research and weigh in on any and all topics that are relevant to improving the experiences of our customers, partners, and employees.
  • Provide relevant insight into human behavior in order to shape gameplay and customer experience.
  • Perform statistical analyses on all aspects of Valve’s operations: gameplay, financial, and company data.
  • Research compelling new hardware technologies.
  • Design experiments to evaluate various gameplay hypotheses and design choices.
  • Improve existing playtesting methodologies while incorporating novel techniques to improve best practices.
  • Develop innovative ways of acquiring relevant data to answer open questions about all aspects of Valve’s products and business practices.
  • Graduate degree in Psychology (or equivalent) field
  • Advanced knowledge of statistics
  • Familiarity with one or more of the following pieces of data analysis software: SPSS, Systat, Matlab, R, (or equivalent)
  • Four years experience with:
    • Experimental design/research methods
    • Relevant research in cognitive, social, human factors, and related disciplines in psychology
  • Proficiency in one or more of the following programming languages: C++, SQL, PHP, (or equivalent)

26 October, 2011

Open questions

From the series "End of the virtual world" by Robert Overweg

Battlefield, Mass Effect, Modern Warfare, Red Dead, Crysis, Rage, Forza, GT, Alan Wake...

What makes the graphics work? I've played the last three call of duty games on 360, with a projector and a 5.1 system. I found them to be amazing. Then I downloaded MW2 on steam, and the graphics looked mediocre.
Conversely, Mass Effect 2 seemed decent on 360 but way more awesome on PC.
Red Dead pushes less polygons and has more visual defects than Crysis but awed me in a way no other game of this generation did.
Battlefield 3 is a technical jewel but MW seemed to me to have a better atmosphere.

Why? Is my subjective judgment shared by others? Is my memory failing me and have my expectations shifted? Or there is also something technical behind these impressions?

MW textures on PC have the low-res quality of a bad port. Is this, coupled with the increase in resolution, the reason the game looks worse to me on PC, makes my brain "see" the polygons more? Or is it because its atmosphere is better suited for a projector and a couch? Or maybe it's because playing the game I was more immersed into it than when I just looked at the graphics on the PC.

Why was it the opposite for ME2? Is its art style, less reliant on gritty texture detail, making better use of the high resolution and antialiasing of the PC?

Why I enjoyed more ME2 single player graphics than BF3 ones? Are the graphics enhancing the gameplay, or is also the opposite true, game and story do affect the perception of the graphics?

Is it the aliasing that is killing BF? Or is the inability of deferred lighting to express the subtleness of light transport and materials the issue? Is our industry jumping on the deferred lighting approach too fast, without really understanding what it's losing form precomputed lighting?

What about the heavy bloom and flares that BF3 and Crysis2 use? How are they working? How do they alter the perception of the image?

From my experience I observed some patterns, but I don't really know much, I've also found very little research...
Aliasing and other high frequency artifacts quickly tell your brain that it's looking at CG, they are very disturbing. Motionblur at 30fps looks more cinematic and packs more punch than 60fps without blur. We tolerate framerate problems way more if the game looks busy (i.e. A huge explosion) than if they are not connected with game actions. Colour is hard to get right, and ambient lighting and proper occlusion of lighting terms are important to represent volume, rendering the air (haze, fog, scattering, desaturation tricks etc) helps with scale. Crowd variety is achieved more with colour and behavior variation than texture and model. Specular lobes are everywhere, have always fresnel, and we can't recognize errors in the light directions in the specular but we use high-gloss highlights to evaluate shape. Bleeding dark edges (i.e. when dealing with subsampled effects) looks less questionable than having bright halos.

There, I think I didn't miss anything, that's pretty much all I know, it fills a few lines of text. I think that's a big challenge for us, there are more studios that know how to do deferred lighting right and fast than there are studios that know what's important to create an immersive, beautiful game. We don't know what to focus on where, what devices are used for what, which artifacts are tolerable and which are disturbing.

We know the (sometimes) the physics but we have very little math to model the psychology. Yes, KSK fits skin specular well. But what parts of it are important? When does it break and breaks the perception of skin being skin? We need to make hacks, and physically reasonable hacks are fundamental in our line of business, physics are fundamental, but physiologically motivated hacks would be way better!
This also affects all the "tuning" decisions, i.e. more LOD switching but with better detail near the camera, or vice-versa? Geometry or textures? Bigger SSAO radius and more noise or the opposite and so on.
Moreover linking perception (vision) and psychology with rendering would give us more objective tools for art-direction, like what device is the best to convey in the general audience a given intent, what makes a sense of "scale" or of "fear" and so on...

Rendering without knowing about perception is crazy, it's like if musicians knew more about sound waves and instruments than harmony and melody. And yet we often chose rendering techniques based on really faint leads about what is needed to look good. There is little research, and the little there is focuses on issues that seldom are directly applicable to modern videogame rendering techniques.
Even worse, we are just not starting to understand the basics, like color and normals, and not only in the industry but even in most publications you will find little regard towards even basic visual perception metrics.

Videogame rendering today at its best it's a work of iteration, the more you can try and the better feedback you get, the more you inch towards this ill-defined target of visual splendor. But even artists and art-directors with a great eye for light and colour seldom have much experience about realtime CG artifacts and their impact on perception.

Shifting from art to science has to happen in order for our profession to evolve, we can't rely on art direction for technical problems, it's not only too error-prone but also very inefficient. Scientific studies can be shared and described exactly, while art direction remains subjective and does not result in a shared progress.

It of course not something that happens only in rendering (or presentation in general, animation, audio), we even make games with very faint leads about what is needed to make fun. And that's why often you have studios investing big amounts of money, and staffed with great talents, producing results that are impressive but still fail, while only very few games really know how to be fun, and really know how to immerse players in their art...

More to come... Meanwhile you might want to read something (other than the links I've already scattered in the post): Some interesting reads here. Also Holly Rushmeier's work (some is linked in the post - from the website the EG2001 and EG2003 presentations are very neat)
Please comment if you know more resources on the topic. A question has been posted on Quora here...

18 October, 2011

Rendering 102

Follow-up to a simplified and hopefully clearer version of the "how the gpu works" posts. Covers GPU computational model and GPU graphics pipeline.

As usual, let me know in the comments about mistakes and possible improvements!

03 October, 2011

Don't lie.

A task is not done when its code gets checked-in. In production, "done" is:
  • Checked-In
  • Stable (passes automated tests)
  • In-budget (memory and performance)
  • Usable (tools, parameters)
  • Verified (art-directior or designer or lead programmer)
If you ignore the last four, you're living in a dream. A dream in which you will ship the game. Instead, you'll die under the pressure of crunch time and ship a flawed product that does not match the quality standards it should.

That also means that in production we start with a stable, in-budget product. And that we do have means of verifying that this is true for the entire production (tests).

Yes, it will take twice as much time to finish a task. Yes, it will mean that some tasks can't be declared done until you make more space somewhere else, to fit them in.

Of course you can avoid all this. All it takes, is to lie. 

We are a creative industry. We have to deal with change, we don't plan things up ahead and then waterfall until they are all done (not _most_ of the times at least, there are situations in which that applies).

We don't craft a product by following a plan, we make drawings and sketches (prototypes) and then take a canvas and start painting. 
You don't have a person detailing a finger, another one refining an eye, another working on the nose and then hope that everything will stick together just right. Or hope that you will have all the parts done by a given date! And what if it then misses one of the ears? What do you do, take ten artists working on that ear till midnight every day near the deadline?

Only an idiot would do that, and still many game companies work like that, they don't consider that everything has to fit just right, and that change is not local, every change has to fit with the entire context, every brush stroke has to make a sense in the entire painting.
You start with a painting, a rough one, and then refine it, and at all time the painting is a painting, it's not a collection of unrelated pieces. You can stop at any moment and it will be still a painting, maybe not that refined, maybe not as intricate and detailed as you wanted, but it's a painting.

A game is such only if it runs. If it crashes or goes out of memory on the target platform, it's not a game, it's some binary that crashes. If it does not hold its performance, it's not a game, we can't burn it to a disc and call it a game, it won't be shipped, it won't pass certification. Iteration should not break this quality, it should go from a "shippable" game to a "shippable" game. Especially during production.