A global illumination (GI) renderer comparison

wahn · Post by **wahn** » Tue May 29, 2012 1:17 am

Hi,

my name is Jan Walter. I currently work (again) for The Mill in London. I did work in film (MPC, Mill Film, Digital Domain) and commercials (The Mill). I do know a lot about rendering (5 years @ mental images, but I also know the RenderMan world, work on MtoA [Maya to Arnold], and tested many other renderers [for fun]). It started as a hobby and later became more serious. I once worked with NaN (probably 12 years ago) at a time when Blender wasn't open source, but free to use. I still use Blender in my spare time to do render comparisons, but I obviously have access to all the commercial packages (Houdini [my favourite], Maya, XSI, even 3DS Max).

Anyway, I give workshops at the Filmakademie in Ludwigsburg and tech talks at FMX, and recently I showed a lot of different renderers (including Indigo) in action. I have a lot of stuff in repositories on Bitbucket and I'm happy to share scenes, experience etc. with you. Unfortunately I can't grant public access to most of the stuff right now because Arnold isn't out for the public and some content is Arnold related (API and ASS files). But if you contact me directly and ask for Indigo (or other renderer related files - except the Arnold ones) I'm happy to give you access to whatever you'd like to experiment with. Email is jan at janwalter dot com ...

Here from the README file of https://bitbucket.org/wahn/radiance_vs_ ... /wiki/Home :

A global illumination (GI) renderer comparison
----------------------------------------------

The materials collected here was used for a workshop given at the
Filmakademie Ludwigsburg in Germany (on Januray 31st, 2012):

http://www.filmakademie.de/
Filmakademie Baden-Württemberg
Akademiehof 10
71638 Ludwigsburg

In the meantime I updated and added more files to automate the rendering of
test files for a tech talk I gave at FMX 2012 (http://www.fmx.de).

Here is a link to a summary of the tech talk (which was given on May 10th):
http://www.fmx.de/program/event-details ... Event.html

The slides for the presentation are available for download from
http://www.janwalter.com/Download/index.html and as soon as Arnold
will be publicly available I will grant public access to this
repository and my Blender importers/exporters (currently RAD import,
RIB and ASS export) which are on Bitbucket as well:
https://bitbucket.org/wahn/blender-add-ons.

The tech talk covered following renderers:

- Radiance (http://radsite.lbl.gov/radiance)
- Arnold (http://www.solidangle.com)
- V-Ray (http://www.chaosgroup.com)
- Maxwell (http://maxwellrender.com)
- Indigo (http://www.indigorenderer.com)
- iray (http://www.irayrender.com)
- Luxrender (http://www.luxrender.net)
- Octane Render (http://www.refractivesoftware.com)
- and others

I will announce if those other repositories go public on http://janwalter.blogspot.co.uk/ ...

The (work in progress) images are here:

http://www.janwalter.com/RadianceVsYouN ... ameit.html

Slides can be downloaded from http://www.janwalter.com/Download/index.html (as mentioned above).

Let me know if you have other scenes (which could be used e.g. with the creative commons license) you'd like to contribute for more render tests, if you have ideas e.g. to create a forum for these tests where people can share experience independent of any particular renderer ...

As I said, if you want access to the scenes, would like to help with future tests etc. ... my email is mentioned above. Don't get angry if I don't answer immediately. I do all of this in my spare time (therefore NO production scenes from my side) and I'm sometimes quiet busy with some commercial work

Cheers,

Jan

Sonaris · Post by **Sonaris** » Tue May 29, 2012 6:44 pm

Very interesting test. I see Indigo superior compared to other unbiased renderers in many cases. Maxwell for example uses very much post process filters which washes off the details. That's why i like Indigo, because i can turn off every kind of postprocess filter (including super sampling) and this way i get the most physically accurate image. Not to mention the Indigo material system, which in my opinion gives much more realistic results than Maxwell. I just hope that the option to turn off the filters in Indigo will stay there in the future versions too, and that it will not be integrated like in Maxwell or Fryrender where the user don't have the option to turn it off.

CTZn · Post by **CTZn** » Wed May 30, 2012 2:38 am

Hi Jan,

I have perhaps already raised that point but finding matching tonemapping settings (film sensivity) when applicable would enhance the comparison. Some calibration step perhaps...

That would be fair to engines using similar techniques. m~w vs Indigo specifically

edit I am referring to the spheres scene serie... in the other hand it is normal that environments produce a different lighting among applications I assume.

wahn · Post by **wahn** » Wed May 30, 2012 9:35 am

Hi,

I think the topic tonemapping and color manipulation in general can easily spawn a really big discussion on it's own. I think most of the renderers give you several options how to transfer a potentially "overexposed" or "underexposed" HDR image into the "visible" range of colors. And all of those color manipulation tools (in film production you also have 3D LUTs - lookup tables) give you a lot of options to manipulate the final colors. In general what you don't want to do ... is to loose some variety in color range.

I can easily admit that I didn't care too much about making those images look good. In my talk at FMX I wanted to show as many renderers as I could in "action" within a pretty tight 45 minutes talk. That doesn't leave a lot of time for each renderer and the problem with a web page is that I have to put an image online which probably could have looked much better if I would have invested more time in adjusting the colors, but that's where this forums come in handy, because I want to motivate people to do their own tests and come up with good suggestions to make more and more of these image look good (and probably more similar to each other).

What I can't show on the web page is that almost all the renderers allow you to tweak the image (color wise) while it's still rendering. Some of them allow light groups where you can turn light (groups) on and off or weight them differently (while still rendering). Some of them have automatic values etc. whereas for others you pretty much have to decide before you start rendering. But in film or commercials production you almost never use an image as it comes out of the renderer directly. It's very different to do a full CG image (or moving images) vs. integrating real actors and photo/film materials with CG elements, and don't forget that sometimes you want to direct the eye of the observer to a particular object or character (you can do this by lighting but also by a lot of different means). And very often you just want to give someone else different images/layers, which they can tweak and composite in a way until they look really good (which I have no skills in at all).

I'm of course not an artist and rather a more technical programmer, TD, R&D, kind of guy. My interested is rather to help some people doing the first steps with a variety of different (image creating) algorithms, help them understand some kind of artefacts, pitfalls, etc., find workarounds, spawn discussions, and mainly learn from each other.

But, please let me know if you have ideas how I could find matching tonemapping settings etc. for different renderers and how the calibration could be done. I'm happy to update my tests in that regard ... For now I just tried to get "some" images out of all those renderers with basically more or less the same scene (and very roughly the same materials). That's already very time consuming

Sonaris · Post by **Sonaris** » Wed May 30, 2012 9:28 pm

wahn wrote:Hi,

I think the topic tonemapping and color manipulation in general can easily spawn a really big discussion on it's own. I think most of the renderers give you several options how to transfer a potentially "overexposed" or "underexposed" HDR image into the "visible" range of colors. And all of those color manipulation tools (in film production you also have 3D LUTs - lookup tables) give you a lot of options to manipulate the final colors. In general what you don't want to do ... is to loose some variety in color range.

I can easily admit that I didn't care too much about making those images look good. In my talk at FMX I wanted to show as many renderers as I could in "action" within a pretty tight 45 minutes talk. That doesn't leave a lot of time for each renderer and the problem with a web page is that I have to put an image online which probably could have looked much better if I would have invested more time in adjusting the colors, but that's where this forums come in handy, because I want to motivate people to do their own tests and come up with good suggestions to make more and more of these image look good (and probably more similar to each other).

What I can't show on the web page is that almost all the renderers allow you to tweak the image (color wise) while it's still rendering. Some of them allow light groups where you can turn light (groups) on and off or weight them differently (while still rendering). Some of them have automatic values etc. whereas for others you pretty much have to decide before you start rendering. But in film or commercials production you almost never use an image as it comes out of the renderer directly. It's very different to do a full CG image (or moving images) vs. integrating real actors and photo/film materials with CG elements, and don't forget that sometimes you want to direct the eye of the observer to a particular object or character (you can do this by lighting but also by a lot of different means). And very often you just want to give someone else different images/layers, which they can tweak and composite in a way until they look really good (which I have no skills in at all).

I'm of course not an artist and rather a more technical programmer, TD, R&D, kind of guy. My interested is rather to help some people doing the first steps with a variety of different (image creating) algorithms, help them understand some kind of artefacts, pitfalls, etc., find workarounds, spawn discussions, and mainly learn from each other.

But, please let me know if you have ideas how I could find matching tonemapping settings etc. for different renderers and how the calibration could be done. I'm happy to update my tests in that regard ... For now I just tried to get "some" images out of all those renderers with basically more or less the same scene (and very roughly the same materials). That's already very time consuming

But what if the intention of some users is not CG animation with tight deadlines and lot of tricks to get more speed, but rather the best physical correctness? In my opinion those are two very separate categories and I fall in to the second. Tonemaping is mainly a post process, what mathers is the raw light calculation and physically correct shaders without filters and composition. I made some material tests with most of present unbiased renderers and i will post the results soon.

CTZn · Post by **CTZn** » Thu May 31, 2012 12:06 am

Wahn, my concern is not relevant to the purpose of your presentation.

When it comes to unbiased rendering, one prominent criterion for the lambda user is: how fast does noise clear up ? It is a similar concern when we compare different rendering modes with Indigo for one scene: in that case we can focus our attention exclusively on sampling discrepancies because the same tonemapping would be used.

In short, I am used to focus my attention on the final products (images issued from rendering engines), while these images are part of your talk wich addresses broader concerns.

My remark was therefore biased I reckon

lycium · Post by **lycium** » Thu Jun 07, 2012 12:20 pm

The file extension is really ASS?

Zom-B · Post by **Zom-B** » Thu Jun 07, 2012 6:24 pm

For Artists such a comparison is most useful if:
- Scene setup is 98% same regarding material types and lightoutput
- Rendertime is fixed so you can compare who is fastest, and where weak spots of engines are
- Testscenes are (very) complex and work related (nobody cares about how well a cornell-box gets rendered)

Headroom · Post by **Headroom** » Thu Jun 07, 2012 10:47 pm

Aside from the Cornell box, the tea pot is getting a little long in the teeth as well

subpixel · Post by **subpixel** » Fri Jun 08, 2012 6:47 pm

lycium wrote:The file extension is really ASS?

Indeed, and you render scene with application called kick.exe
So you end up with command like:

Code: Select all

kick some.ass

ENSLAVER · Post by **ENSLAVER** » Fri Jun 08, 2012 8:08 pm

Headroom wrote:Aside from the Cornell box, the tea pot is getting a little long in the teeth as well

Blasphemy!