If Robert Zemeckis Can’t Cross the Uncanny Valley, What Makes Us Think We Can?

I was getting ready to write a different post about how I think a lot of money could be saved on big production games, and I was going to use last year’s Beowulf movie for a certain example, when I reminded myself of just how HORRIBLE that movie was. For a budget of $150,000,000, Robert Zemeckis, one of my favorite directors of all time with movies like Forest Gump, Roger Rabbit, and Back to the Future to his credit, managed to direct a movie that makes one of the most beautiful women in the world, Angelina Jolie, look terrible (see image below). Through out the movie, I kept getting thrown out of my suspension of disbelief because the motion captured actors continually went so far into the “uncanny valley“.
Beowulf
Zemeckis, now with two of these expensive mo-cap, animated films under his belt, Beowulf and Polar Express, is set to release another one before the end of 2009, A Christmas Carol. Somebody must believe in the process.

If you are unfamiliar with the term uncanny valley, it was coined when computer generated images started to get close to photo realistic, but not quite close enough. People noticed that the closer to reality the images became, the more real they wanted them to be. I learned this lesson in a very painful way back when we created David Wolf: Secret Agent in 1988. For that product, we pioneered using real actors with cameras and digitizers to bring the images and animation into the game. We thought it would be a hit, but the more we put into it, the more we realized it was going to be difficult. It wasn’t a hit.

We continued to experiment with the process for a bunch more titles, but we could never get it right. IMHO, nobody ever got it right. Think about the real actor cut scenes in any game, and they always look like movie “wannabes”. My feeling from then on has been, if you want to make movies, make movies, but don’t try to make games into movies.

But, back to the uncanny valley problem. The absolute best computer animations that I have seen coming out of research labs are close to bridging the valley, but even they have tale tell signs that pull the viewer out of immersion. My prediction is that movies won’t solve the problem any time soon, and they will solve it before games so. If some of the most creative and visual people in the world with huge development budgets for a linear medium can’t get it right, what chance do games have?

-Jeff Tunnell, Game Maker
Make It Big In Games

  • JeremyAlessi

    I had a debate with my friend about CG in 2001 when Star Wars: Episode 2 came out. He hated Yoda and CG in general. He actually preferred the puppets used in the 80's. In my opinion the only way to get better at anything is to actively seek a solution. My answer to him was wait, they're working on it.

    We had the same discussion this year. This time around he agreed with me. We both admit that CG isn't completely there yet and still looks pretty fake most of the time but games these days look like the pre-rendered CG from back then while movies like Iron Man look even better.

    In this world you never know where the epiphany will pop up. I think that games are intrinsically tied to CG where CG for movies is just a bonus. Therefore in my opinion the game development community actually has more of an obligation to get it right. Movies don't need CG but games do. It doesn't matter weather you're doing a Flash game or Crysis you're rendering graphics with a computer.

    Smart people in both industries are aiming for a very similar goal. It's a race and although games are seemingly always behind there's no reason why the idea that fills the uncanny valley up for good can't come from within this industry.

  • JeremyAlessi

    I had a debate with my friend about CG in 2001 when Star Wars: Episode 2 came out. He hated Yoda and CG in general. He actually preferred the puppets used in the 80's. In my opinion the only way to get better at anything is to actively seek a solution. My answer to him was wait, they're working on it.

    We had the same discussion this year. This time around he agreed with me. We both admit that CG isn't completely there yet and still looks pretty fake most of the time but games these days look like the pre-rendered CG from back then while movies like Iron Man look even better.

    In this world you never know where the epiphany will pop up. I think that games are intrinsically tied to CG where CG for movies is just a bonus. Therefore in my opinion the game development community actually has more of an obligation to get it right. Movies don't need CG but games do. It doesn't matter weather you're doing a Flash game or Crysis you're rendering graphics with a computer.

    Smart people in both industries are aiming for a very similar goal. It's a race and although games are seemingly always behind there's no reason why the idea that fills the uncanny valley up for good can't come from within this industry.

    • http://coderhump.com Ben Garney

      I think you have a point. But game developers like to think that they are big budget, hugely talented movie studios. That's where they get into trouble.

  • http://coderhump.com Ben Garney

    I think you have a point. But game developers like to think that they are big budget, hugely talented movie studios. That's where they get into trouble.

  • http://tonemonster.benvesco.com/ Ben Vesco

    Crossing the “uncanny valley” may not be a desirable thing to do based on the hypothesis of http://en.wikipedia.org/wiki/Masahiro_Mori (who is credited with coining the term circa 1970).

    I find highly stylized statements to be much more interesting. “A Scanner Darkly” is a good contrast to “Beowulf” in terms of stylized representations of humanoid forms. Peruse the screen caps at http://www.imdb.com/media/rm1688312064/tt0405296 where you can sample the rotoscoped majesty that is this film. The film is so well done that by half way through you have already forgotten that you are watching an animation even though it doesn't ever pretend to not be animated. Forget about photo-realism and go for a style. Your budget will thank you and so will your future fans.

  • http://tonemonster.benvesco.com/ Ben Vesco

    Crossing the “uncanny valley” may not be a desirable thing to do based on the hypothesis of http://en.wikipedia.org/wiki/Masahiro_Mori (who is credited with coining the term circa 1970).

    I find highly stylized statements to be much more interesting. “A Scanner Darkly” is a good contrast to “Beowulf” in terms of stylized representations of humanoid forms. Peruse the screen caps at http://www.imdb.com/media/rm1688312064/tt0405296 where you can sample the rotoscoped majesty that is this film. The film is so well done that by half way through you have already forgotten that you are watching an animation even though it doesn't ever pretend to not be animated. Forget about photo-realism and go for a style. Your budget will thank you and so will your future fans.

    • http://www.makeitbigingames.com Jeff Tunnell

      I absolutely agree. I like stylized characters much more. That is the reason the guys at Pixar rarely choose real humans as the subject of their animated movies. If they do choose humans, they are highly stylized as in The Incredibles. Mori's theory was actually created in reference to robots, but it applies perfectly to computer generated characters.

      Here is a great article on Slate that says what I was trying to say in a much more elegant way: http://www.slate.com/id/2102086

    • http://www.subreal.net timaste

      I haven't seen that wikipedia article, great read!

    • JeremyAlessi

      I also agree that stylized characters are more interesting but it is through our pursuit to replicate reality in computers that we learn how to create those stylized effects.

      • jgostylo

        I disagree with that. I think stylization is an effort to exaggerate traits and not replicate reality.

        Most entertainment is an exaggeration of traits. Sports features freakishly tall, strong, quick people. Acting exaggerates emotion through overdone tone and body language. I think stylized art is just trying to do the same thing. It is conveying information visually. I think it is so effective because we learn faster visually than through sound. Also, we can learn through both senses simultaniously so it becomes even more effective. The reason for stylization is it reduces ambiguity. Real people are good at holding their cards close to their vest. The more real you get the more subtle the message becomes and the less your audience may get it.

        As for the A Scanner Darkly argument, the real goal when trying for immersion is to give visual consistency. If the whole screen is the same style, our brains give it a bye. For Beowulf and Yoda there were two contrasting styles occupying the same space.

        Another out is when your audience understands and appreciates the inconsistency as in the case of Roger Rabbit. Still it helps to get the physics right. My favorite scene in Roger Rabbit is when the head Weasle splashes Eddie's wash water. Very well done.

        • JeremyAlessi

          Yes obviously stylization isn't used to replicate reality but without the pursuit of more realistic graphics you'd still be stylizing with pixellated blocks. Advances in graphics were created to emulate reality better. Graphics gurus did not have cel shading in mind when they created the technology that allowed it. The goal objectively speaking is to get a firmer grasp on how reality works and then let creative people create abstractions and exaggerations of it.

          • http://coderhump.com Ben Garney

            All the old movements in art are playing out in the digital space, just a lot faster. It's pretty cool to watch.

            Honestly, the best thing about computer graphics is the total control. You can tweak everything – even the laws of physics – to get the look you want. Of course – it helps to know the rules before you break them!

            And even in “realistic” mediums like live action movies – there's so much mediation that goes on. Light, make up, lenses, “film” – not to mention all the post processing. 300 is an extreme case, but even in the nightly news, a lot is going on.

  • http://www.makeitbigingames.com Jeff Tunnell

    I absolutely agree. I like stylized characters much more. That is the reason the guys at Pixar rarely choose real humans as the subject of their animated movies. If they do choose humans, they are highly stylized as in The Incredibles. Mori's theory was actually created in reference to robots, but it applies perfectly to computer generated characters.

    Here is a great article on Slate that says what I was trying to say in a much more elegant way: http://www.slate.com/id/2102086

  • http://www.subreal.net timaste

    I haven't seen that wikipedia article, great read!

  • JeremyAlessi

    I also agree that stylized characters are more interesting but it is through our pursuit to replicate reality in computers that we learn how to create those stylized effects.

  • jgostylo

    I disagree with that. I think stylization is an effort to exaggerate traits and not replicate reality.

    Most entertainment is an exaggeration of traits. Sports features freakishly tall, strong, quick people. Acting exaggerates emotion through overdone tone and body language. I think stylized art is just trying to do the same thing. It is conveying information visually. I think it is so effective because we learn faster visually than through sound. Also, we can learn through both senses simultaniously so it becomes even more effective. The reason for stylization is it reduces ambiguity. Real people are good at holding their cards close to their vest. The more real you get the more subtle the message becomes and the less your audience may get it.

    As for the A Scanner Darkly argument, the real goal when trying for immersion is to give visual consistency. If the whole screen is the same style, our brains give it a bye. For Beowulf and Yoda there were two contrasting styles occupying the same space.

    Another out is when your audience understands and appreciates the inconsistency as in the case of Roger Rabbit. Still it helps to get the physics right. My favorite scene in Roger Rabbit is when the head Weasle splashes Eddie's wash water. Very well done.

  • JeremyAlessi

    Yes obviously stylization isn't used to replicate reality but without the pursuit of more realistic graphics you'd still be stylizing with pixellated blocks. Advances in graphics were created to emulate reality better. Graphics gurus did not have cel shading in mind when they created the technology that allowed it. The goal objectively speaking is to get a firmer grasp on how reality works and then let creative people create abstractions and exaggerations of it.

  • http://coderhump.com Ben Garney

    All the old movements in art are playing out in the digital space, just a lot faster. It's pretty cool to watch.

    Honestly, the best thing about computer graphics is the total control. You can tweak everything – even the laws of physics – to get the look you want. Of course – it helps to know the rules before you break them!

    And even in “realistic” mediums like live action movies – there's so much mediation that goes on. Light, make up, lenses, “film” – not to mention all the post processing. 300 is an extreme case, but even in the nightly news, a lot is going on.

  • Max

    > We continued to experiment with the process for a bunch more titles, but we could never get it right. IMHO, nobody ever got it right. Think about the real actor cut scenes in any game, and they always look like movie “wannabes”.

    While I haven't watched Beowulf and can't comment on that, this point really has another explanation. Nobody could get it right simply because the budgets were always absolutely damn LOW compared to Hollywood. Second-rate actors with cheap backgrounds and crap directing will always look like movie “wannabes”.

    But actually, Wing Commander 3 and 4 were good. There were enough solid actors there to create suspension of disbelief – and WC4 featured cool backgrounds, too.

  • Max

    > We continued to experiment with the process for a bunch more titles, but we could never get it right. IMHO, nobody ever got it right. Think about the real actor cut scenes in any game, and they always look like movie “wannabes”.

    While I haven't watched Beowulf and can't comment on that, this point really has another explanation. Nobody could get it right simply because the budgets were always absolutely damn LOW compared to Hollywood. Second-rate actors with cheap backgrounds and crap directing will always look like movie “wannabes”.

    But actually, Wing Commander 3 and 4 were good. There were enough solid actors there to create suspension of disbelief – and WC4 featured cool backgrounds, too.

  • Ken

    I am glad that people like Zemeckis still has faith in motion capture despite the lashback such as this article. Obviously the process of performance capture is in its infant stages, and it is definately improving, (just watch Polar Express and Beowulf side by side.) Companies like Pixar are shying away from solving the problem of the Uncanny Valley and animation-wise they are completely UN-innovative. People who say “Oh, puppets are better than CGI” are just as bad.

  • Ken

    I am glad that people like Zemeckis still has faith in motion capture despite the lashback such as this article. Obviously the process of performance capture is in its infant stages, and it is definately improving, (just watch Polar Express and Beowulf side by side.) Companies like Pixar are shying away from solving the problem of the Uncanny Valley and animation-wise they are completely UN-innovative. People who say “Oh, puppets are better than CGI” are just as bad.

    • http://www.makeitbigingames.com Jeff Tunnell

      I am glad that somebody is taking the chances to move this forward. I disagree with you in a huge way that Pixar is unoriginal. They are the ones that created the entire computer generated film genre. There is also a huge difference between process originality and content originality. While I don't even agree that Pixar's process is unoriginal, I disagree in a very strong measure that their stories or characters are unoriginal. Their stories, characters, and plots are innovative, fun and fresh. Most of the world agrees with me on this point.

  • http://www.makeitbigingames.com Jeff Tunnell

    I am glad that somebody is taking the chances to move this forward. I disagree with you in a huge way that Pixar is unoriginal. They are the ones that created the entire computer generated film genre. There is also a huge difference between process originality and content originality. While I don't even agree that Pixar's process is unoriginal, I disagree in a very strong measure that their stories or characters are unoriginal. Their stories, characters, and plots are innovative, fun and fresh. Most of the world agrees with me on this point.

  • ldargin

    Did you watch Beowulf in 3D? It definitely worked for many; its rating at Rotten Tomatoes is a good 71%.

    50% of Beowulf's audience was over 25, and 60% was male. That leads me to think that such animation, deep in the Uncanny Valley, can be effective in attracting an older audience. ( http://www.boxofficemojo.com/news/?id=2420&p=.htm )

  • ldargin

    Did you watch Beowulf in 3D? It definitely worked for many; its rating at Rotten Tomatoes is a good 71%.

    50% of Beowulf's audience was over 25, and 60% was male. That leads me to think that such animation, deep in the Uncanny Valley, can be effective in attracting an older audience. ( http://www.boxofficemojo.com/news/?id=2420&p=.htm )

  • http://www.3dcognition.com Brad Strong

    I would disagree – on the point that Angelina looked terrible, and that we necessarily want the cgi characters to be more real. Personally I was impressed with the models, textures, and movement, overall – and it took me on a journey. I was looking forward to what they'd do and accepting it as a type of reality, not necessarily 100% real.

    What did bother me was the way Beowulf moved when he was fighting the Crispin monster (btw I thought Crispin and his character were great and perfect for cgi). It reminded me of the way the elf warrior in The Return of the King moved when he was running up those elephant creatures – too fast, no sense of actual physics involved (gravity, coefficient of friction, general balance issues). That kind of movement drives me crazy.

  • http://www.3dcognition.com Brad Strong

    I would disagree – on the point that Angelina looked terrible, and that we necessarily want the cgi characters to be more real. Personally I was impressed with the models, textures, and movement, overall – and it took me on a journey. I was looking forward to what they'd do and accepting it as a type of reality, not necessarily 100% real.

    What did bother me was the way Beowulf moved when he was fighting the Crispin monster (btw I thought Crispin and his character were great and perfect for cgi). It reminded me of the way the elf warrior in The Return of the King moved when he was running up those elephant creatures – too fast, no sense of actual physics involved (gravity, coefficient of friction, general balance issues). That kind of movement drives me crazy.

    • http://www.makeitbigingames.com Jeff Tunnell

      Well, is sounds like they are crossing the “Valley” for some people, but not for most. Even the most advanced, canned research projects still don't work for me or most of the people that I talk to about it.

      I would take a journey, i.e. a type of reality that I could accept, but, for me, this was not it. On the other hand, I love the CGI stuff that Blizzard does. I always say that I would go watch a movie that they made. Their characters and art direction are not trying to be too real, and I love the look.

      • Justin Boily

        Here`s a thought

        If the uncanny valley is what I really what I think it is, it could only be crossed when there is no more “trying to fool“ but only when your CG character will have everything a human has, from intelligence to curiosity, 24 vertabrae to exocrine glands, heat sensitive skin as well as empathy, memory etc…

        No human can reproduce the exact eye movement, that little twich that happens when you change the focus from the eyes of your interlocutor to the small pimple sitting on his upper eyelid.. or the small blushing of the cheeks when a situation you hear about is reminiscent of one of your tales…

        The book “Do androids dream of electric sheeps“ is all about that last peak to climb towards the other side of the valley… and the ability of the human (even when men have to rely on machines for the task) to track the smallest, most minute detail…

        To sum it all up: When there is no truth, where can the truth be found…

        ————————————

        This applies to the exhaustive search for hyper-realism… my call is to stylise and exploit the smallest details and make them bigger than reality.. that way you can hide stuff and emphasis what you want the spectator to see…

  • http://www.makeitbigingames.com Jeff Tunnell

    Well, is sounds like they are crossing the “Valley” for some people, but not for most. Even the most advanced, canned research projects still don't work for me or most of the people that I talk to about it.

    I would take a journey, i.e. a type of reality that I could accept, but, for me, this was not it. On the other hand, I love the CGI stuff that Blizzard does. I always say that I would go watch a movie that they made. Their characters and art direction are not trying to be too real, and I love the look.

  • Justin Boily

    Here`s a thought

    If the uncanny valley is what I really what I think it is, it could only be crossed when there is no more “trying to fool“ but only when your CG character will have everything a human has, from intelligence to curiosity, 24 vertabrae to exocrine glands, heat sensitive skin as well as empathy, memory etc…

    No human can reproduce the exact eye movement, that little twich that happens when you change the focus from the eyes of your interlocutor to the small pimple sitting on his upper eyelid.. or the small blushing of the cheeks when a situation you hear about is reminiscent of one of your tales…

    The book “Do androids dream of electric sheeps“ is all about that last peak to climb towards the other side of the valley… and the ability of the human (even when men have to rely on machines for the task) to track the smallest, most minute detail…

    To sum it all up: When there is no truth, where can the truth be found…

    ————————————

    This applies to the exhaustive search for hyper-realism… my call is to stylise and exploit the smallest details and make them bigger than reality.. that way you can hide stuff and emphasis what you want the spectator to see…

  • http://nutritionfoods09.blogspot.com/ nutrition foods

    Thanks for the info. May God have mercy on us all.

  • qalice

    Good answer, I am looking for the solution of the same question. Find the movies or mp3 you are looking for at music-and-soundtracks.com the most comprehensive source for free-to-try files downloads on the Web

  • qalice

    Good answer, I am looking for the solution of the same question. Find the movies or mp3 you are looking for at music-and-soundtracks.com the most comprehensive source for free-to-try files downloads on the Web