Home Gaming Enter the GPU – The uncanny valley

Enter the GPU – The uncanny valley

3 min read
37

Digital Ira

After watching the videos in this morning’s next-gen lighting article, I wasn’t nearly as enlightened as I thought I’d be. Many PC gamers would probably feel the same, because it’s something we’ve experienced before. I’m sure that every one will be impressed with what Nvidia’s GPU the GeForce GTX TITAN is capable of though. We’ve seen some really awesome real-time face rendering technology and amazing looking animation over the years, but we’ve never really seen anything like this. This is the human head we hope to see in all of our video games.

NVIDIA Co-founder and CEO Jen-Hsun Huang demonstrated their new Face Works technology running on the GTX TITAN at the annual GPU Technology Conference explaining exactly what the significance of this technology is, the wonder is not only how uncanny the realism is, but the capability of the technology to shrink the size of the data to something more usable in video games.

To create the demo Nvidia partnered with Dr. Paul Debevec of the Institute for Creative Technology (ICT) at the University of Southern California where Debevec and his team have been building next generation systems that can capture facial data to within a tenth of a millimeter without all the extra crap like special makeup or specialized cameras. ICT developed an astonishing ‘light stage’ technology which uses photographic techniques to capture the complete 3D shape of an actor’s face as well as the most important elements to represent the human skin as it really is. From light reflection and the transmission of light through the skin to how light reflects off the oils in the skin, and nearly microscopic lines and bumps on the skin’s surface.

Debevec’s team had an actor model several dozen of basic facial expressions, and after capturing the data they ran it through a compression engine. The results of which is a pared down set of data that can be “mixed and matched to generate a wide range of expressions.” However, all the detail gathered by Debevec’s team is nothing if it can’t be displayed on a screen interactively. This is where Nvidia’s Face Works technology comes in. To be able to process and display every single detail, like light transmission through the ears and nose complex skin shaders have to execute over 8, 000 instructions per pixel to allow realistic rendering of the face in any lighting situation. These are the stats from the Nvidia blog:

“At full HD resolution that’s 82 billion floating point operations per second (FLOPs) per frame – or 4.9 trillion FLOPs per second at 60 frames per second. And all that doesn’t include the 161 filtered texture fetches per pixel.”

However amazing all the detail and the technology allowing the capturing of the data, it’s just not possible to put this into something like a video game, purely because of the size. The data by Debevec’s team containing only a few expressions exceeds 32 gigabytes. Needless to say it’s not something your average GPU would be able to handle. Nvidia’s Face Works technology is critical in making this data usable, as it reduces the size of this data, to a mere 300 megabytes. And how is that done? With sorcery. Or if maybe like this:

“The trick is to take advantage of similarities in the textures for each of the actor’s expressions. Each texture is divided into an eight by eight grid of tiles, and those tiles that are recognized as being nearly identical to the same tile in other expressions are discarded.”

Last Updated: March 20, 2013

37 Comments

  1. Sir Rants-a-Lot Llew

    March 20, 2013 at 14:05

    See nVidia all the way!

    Reply

    • Admiral Chief Erwin

      March 20, 2013 at 14:08

      What GPU is in the new consoles again?

      Reply

      • Sir Rants-a-Lot Llew

        March 20, 2013 at 14:09

        Yeah, in a console. Which is always inferior in quality to a PC. Inferior tech for an inferior machine 😛

        Reply

        • Admiral Chief Erwin

          March 20, 2013 at 14:13

          I categorize enveedee-a in the same area as cod, consoles and feet disease 😛

          Reply

          • Sir Rants-a-Lot Llew

            March 20, 2013 at 14:15

            So you put nvidia in the same area as those things you hate, yet the manufacturer you prefer (AMD) is having their chipsets put in to said hated consoles? While nVidia remains PC?

            Yeah… Ok 😛

          • AndriyP

            March 20, 2013 at 14:17

            Hes crazy doesnt even know what hes talking about….
            nVidia FTW!

          • Sir Captain Rincethis

            March 20, 2013 at 14:20

            Boys! Nuff now, fake Friday, let’s all be bestest of friends!

          • Admiral Chief Erwin

            March 20, 2013 at 14:27

            yes, nVidia is Filthy Tech Waste

            😛

          • Sir Captain Rincethis

            March 20, 2013 at 14:21

            Boys!

          • Admiral Chief Erwin

            March 20, 2013 at 14:24

            *adds LC and Andrey to category*

          • Sir Rants-a-Lot Llew

            March 20, 2013 at 14:24

            I am rubber you are glue! What ever you say bounces off of me and sticks to you!

          • Admiral Chief Erwin

            March 20, 2013 at 14:26

            LC is awesome.

            Hahahahaha

          • Sir Rants-a-Lot Llew

            March 20, 2013 at 14:27

            Now you will forever have the words “LC is awesome” stuck to you. Nanananananaaaaa. Always a reminder of how awesome I am. 😛

          • Sir Captain Rincethis

            March 20, 2013 at 14:20

            Boys!

          • Sir Rants-a-Lot Llew

            March 20, 2013 at 14:22

            Sorry dad 🙁

          • Sir Captain Rincethis

            March 20, 2013 at 14:23

            Don’t sorry me son, GET IN THE GOD DAMN CORNER WITH THAT PRICK OF A RED FIRE ENGINE!

          • Sir Rants-a-Lot Llew

            March 20, 2013 at 14:24

            *grumble grumble poopyhead grumble grumble*

          • Lord Chaos

            March 20, 2013 at 14:50

            All hail ARES II

        • Yolanda Green

          March 20, 2013 at 14:27

          The fact is that they are inferior when it comes to PC as well. The TressFX technology even though it was impressive, isn’t usable by all gamers unless you compromised the over all graphics quality of the game (Tomb Raider). You need a beast of a PC to run the game on high, let alone ultra setting and have TressFX enabled or your frames drop significantly. Nvidia is just better than AMD in some aspects… Having AMD GPU’s in next-gen consoles won’t make it better than Nvidia, it just means we aren’t seeing what Nvidia could have done instead.

          Reply

          • Gustav Willem Diedericks

            March 20, 2013 at 14:53

            Luckily beast of a PC I have 🙂 I’ve been playing it really fine!

          • Yolanda Green

            March 20, 2013 at 15:02

            What GPU are you running?

          • Gustav Willem Diedericks

            March 27, 2013 at 13:26

            I’m running 570 GTX. But I’m also running the version 311.06 driver set

          • Sir Rants-a-Lot Llew

            March 20, 2013 at 15:06

            Very true what you say there

          • Weanerdog

            March 21, 2013 at 15:52

            At $1000 a GPU yeah sure

  2. Sir Rants-a-Lot Llew

    March 20, 2013 at 14:05

    TressFX se moer. Face Works FTW! 🙂

    Reply

    • Admiral Chief Erwin

      March 20, 2013 at 14:08

      Pff, I just look in mirror, now THAT is readl GFX

      Reply

      • Jim Lenoir (Banana Jim)

        March 20, 2013 at 14:34

        But the polycon count is so low… hehehe

        Reply

      • Sir Rants-a-Lot Llew

        March 20, 2013 at 14:39

        You know what I see when I look at real life graphics? I see lag…..

        Reply

        • Admiral Chief Erwin

          March 20, 2013 at 14:45

          I see some ppl out there and think: “This looks shopped”

          XDXDXDXD

          Reply

  3. Sir Rants-a-Lot Llew

    March 20, 2013 at 14:08

    Ok wow. That video is creepy. The facial expressions at the end kinda makes one feel uneasy knowing it’s not real but rather a render

    Reply

  4. Sir Captain Rincethis

    March 20, 2013 at 14:16

    Still in Uncanny valley I think. Damn good, don’t get me wrong, but I would know this sin’t real. Not like that freaky ‘interview’ we saw the other day…

    Reply

    • Yolanda Green

      March 20, 2013 at 14:23

      I think the challenge will always be simulating life 🙂 The Far Cry 3 characters like Vaas were more believable because of their emotion, their voice acting. Imagine Vaas created with this technology.

      Reply

      • Sir Captain Rincethis

        March 20, 2013 at 14:26

        Good point. If you had Mondo doing this I’d probably be shaking under my desk right now…

        Reply

        • Yolanda Green

          March 20, 2013 at 14:35

          I think the point of capturing data as a set of expressions that you are able to mix amd match is that you won’t need a complete performance by an actor, but that you will be able to emulate one. As long as the voice acting matches that, it should be almost flawless. However no predetermined set or pattern of expressions can simulate the randomness that is human life.

          Reply

          • Sir Captain Rincethis

            March 20, 2013 at 14:39

            Agreed. I also don’t want this to completely usurp real actors (imagine no Mondo!) It’s dine to have this for henchmen, but not for VAS!

        • Jean du Plessis

          March 22, 2013 at 08:00

          Vaas on this tech? I’d be having many a sleepless night… o.O

          Reply

  5. Hondsepop

    March 20, 2013 at 14:50

    Hope one day, this will replace the overpriced actors….oops, already happening.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Check Also

Nvidia to finally stop support for Windows 7 and 8

It's hard to believe that the likes of Windows 7 and 8 are still being use by people, but …