[?] sampling acceleration

[bkircher]

I have some grand ideas concerning accellerating the rendering process
Can the system step up strata widht/oversampling/max. num conseq. rejections over rendertime (determined by a statement similar to halttime):

This way, the general brightness could be established well before the noise is fought in the image...


Sub/Oversampling depending on the primarily visible material, with oversampling at the edges between two materials.

Untextured opaque lamberts could be sampled differently from textured.
Phongs would behave different than transparent objects.


Surface thickness statement for transparent materials, that would insert a virtual geometry behind or before the actual geometry: That would both keep triangle count low and faciliate windows and similar objects.

[CTZn]

I second the edge detection stuff, for all edges. Regarding the rest, if that alterates the unbiased nature of Indigo it's likely you won't see it implemented !

[mrCarnivore]

I think all those improvements would make indigo biased and are therefore not useful additions to indigo...

[CTZn]

I have another project of study about sampling optimization, but it will require the halt_samples upgrade to be relevant, and an API at the end, unless it's integrated by Ono himself. Obviously I won't be the one coding such stuff, and that won't happen if it's not efficient at all. Another approach is possible via batch but my console asks after every command if I want to run the next, so it's useless. Dunno why it prompts after every command... someone knows how to avoid these prompts of cmd.exe ?

And the result is not guaranteed yet, since I can't be a genius in develloping MLT stuff. I can hardly write a loop correctly and maths forced me to stop school

Worth the try anyway, and at a certain point I'll need help (or I'll let the baby to someone).

[suvakas]

What the h*** are you talking about CTZn ?
I don't have a clue what you are trying to say
You start by telling, that you have an optimization idea and you end up telling, that there is something wrong with your cmd.
Was there a nice lady passing by your window while you were typeing the message?
[/joke]

[manitwo]

Was there a nice lady passing by your window while you were typeing the message?
[/joke]

LOL

[zsouthboy]

rofl CTZn - your leet cmd.exe hacking skills are awesome.


how are you entering the commands?

consider downloading and installing msh, .... the WindowsPowerShell, it's much more powerful than cmd alone

or if you know *nix commands, try cygwin and just use a bash shell

[bkircher]

I think all those improvements would make indigo biased and are therefore not useful additions to indigo...

I don't think that has to be neccessarily the case:
Think about starting with a rough unbiased rendering until a rough overall illuminance has been established, then refining later on: That is not really less biased than rendering "low-quality" until you end the simulation, as I understand it.

Also, I think that unbiased means it is getting better and better, not that all samples have to take the same amount of time to sample. So long as the energies are not lost or magnified, is this really impossible?

[CTZn]

What the h*** are you talking about CTZn ?
I don't have a clue what you are trying to say

That's normal, I'm unclear on purpose. I may tell more soon

About the batch: Skills ? Whazzat ? Forget about it, it's not necessary for now. Basically it's a script where I'm launching a scene wich has a halt_time set to some value, the following line is a similar command. The render starts, and when it automatically stops (as expected) instead of running the second command it's asking wether I want to close the cmd window or not. If I say "no" the second line is executed, and the second render starts. But what's the point in doing such a simple script if it is not resuming automatically ?

[zsouthboy]

bkircher:

That still makes indigo biased: at what point do you "turn off" the interpolated data and use the real stuff? until then, it's biased.
Plus, that just takes time away from the render threads.

Useless, IMO. Indigo is fine as it is - you can get a basic idea of how the end result is going to look in 30 seconds. Considering I came from yafray, which meant waiting 15 minutes for damn photon gathering and all the other bullshit - then started rendering, at which time i realize that a setting needs to be tweaked yet again....

CTZn:

Can you either PM me or post somewhere else the contents of your .bat file? I think you may need some help

[CTZn]

CTZn:

Can you either PM me or post somewhere else the contents of your .bat file? I think you may need some help

Won't you prefer if I was to brief you about that idea I have, I'm about to drop computers for a few months/years/forever; I need a desintoxication before attempting anything serious

I've PMed Ono about that a few weeks ago, and he agreed that may have an impact on render times; he asked me to make some tests to see if impact was positive or not, I've some ideas on how but I'm sure you'll be much more efficient than I would. Basically it's just about analyzing some stuff (<- this need a bit of coding, very easy stuff but I'm a noob) then feed the renderer back with that data... er, actually there is more things to define but I'm lacking knowledge about MLT... and there is an even easier method wich needs no analysis at all, only feeding the renderer with some logic. No need to hack Indigo, this is possible since 0.7t3 I think.

I'm still thinking it's worth a try, and I'm sure you'll be seduced by the idea. Do you accept the challenge zsouthboy ?

Estimation of needed worktime to define wether it's working or not is between 6h and 3 days, depending on how accurate you want to be. The forementionned coding is optional, just to avoid some boring step. And again, if you choose the easiest/less optimized method, you'll just have to launch some renders

I'd love to do that myself but unfortunately when it's about working on my computer, I now feel like a frog on a lightbulb hehe and that damn machine blew a capacitor but it's still working, no luck I keep being a nerd :/

So what ?

[zsouthboy]

I'm still not sure what you mean, CTZn. You're analyzing some aspect of the sampling method?

I wanted to help you with your batch file - what you do with it is your business

I'm writing a program to help with network rendering across many computers right now.

Basically the first thing I suggest RE: your batch file is to add this line first:
@ECHO OFF

That will prevent commands from being "echoed" to the console before they run.

[bkircher]

bkircher:

That still makes indigo biased: at what point do you "turn off" the interpolated data and use the real stuff? until then, it's biased.
Plus, that just takes time away from the render threads.

Useless, IMO. Indigo is fine as it is -

. You're right as far as different sampling strategies are concerned, there is a (certain) bias to the rendering, but only to the interpolation on pixel level, not to light distribution.
On the other hand, assuming that a lambertian material needs less supersampling than a phong makes sense to me: Ono states somewhere that the idea to supersample the whole image is to improve quality around bright spots: Bad for caustics if supersampling is skipped on them, but otherwise I don't see the harm for a lambert material. And you still get closer to a reality on a pixel level, only grained finer in certain areas and averaged later on (perhaps subsampling is really a bad Idea, except in an initial phase maybe). After a sufficiently long enough rendering, both solutions should be nearly identical.

. A different, 2nd means to change the rendering charackter is stepping e.g. strata width up after certain intervalls, so that you fight noise once there is enough information in the image to make this worthwhile.

. concerning "Indigo is fine as it is"... It sure is, that's what the appeal of this software is. And sure, you can get a fairly good first impression, if there is enough light in your image in the first place. As far as I understand it, less light = more noise.
I'm interested in rendering indirectly lit interiors, with good contrasts between "suede" diffuse shaders and phong shaders, and an as small amount of noise as possible. I love the way how indigo handels huge models, how straigth-forward instancing is, etc.

.Also:

Plus, that just takes time away from the render threads.

How so, in any meaningful way?


p.s.: I'm writing this stuff mainly to make waiting for bidir in 0.7.6 more entertaining, dont you?

[CTZn]

I'm still not sure what you mean, CTZn. You're analyzing some aspect of the sampling method

Yes, the rate of samples/seconds. Didn't you notice it wasn't linear ? That it's correlated with some settings you can actually feed Indigo with ? The most ambitious project is to maintain this ratio the highest possible (this need analysis and advanced scripting, or coding as soon as an API allows it. Looks like the settings have to be constantly evolving in order to get max ratio, that's why the API would be much more effcient), while the most easy (easy when you know what these settings do exactly, wich is not my case but empirically speaking) consists of resuming an image with successively different, but related settings (optimization would require halt_samples).

It's about harmonizing noise levels to get rid of it, the best you can in a predetermined amount of time. It's still theorical and unverified, but you'll agree that could bring Indigo to a place it wasn't before

Makes sense now ? I know it's a bit confuse but I'm not a math guy, I'm more on a conceptual side.

----

Basically the first thing I suggest RE: your batch file is to add this line first:
@ECHO OFF

Code: Select all

@echo off
echo starting stuff...
C:\Progs\3d\indigo\indigo_07_test5\indigo.exe -r C:\Progs\3d\indigo\indigo_07_test5\renders\im1172610055.igi C:\work\3d\test\renderData\3dnoiz00_Indigo.xml

echo starting more stuff...
C:\Progs\3d\indigo\indigo_07_test5\indigo.exe -r C:\Progs\3d\indigo\indigo_07_test5\renders\im1172610055.igi C:\work\3d\test\renderData\3dnoiz00_Indigo.xml

That was not an issue, I scanned the system for *.bat and learned basics that way. Surprisingly it's working now, maybe because a capacitor from my mobo melted last week Before, when the first render was over the command window would hang but now it's working as expected, cool !

And now that it's working I need even more advanced scripting, like:

1 start a render (easy)
2 get the new image created when the render is over, hopefully because halt_samples limit is reached (hu...)
3 use it to resume another scene (the very same than the former but with different settings)
4 reapeat until there is no more scene with the same prefix to render
5 open the last, neat igi in violet

3 and 4 are not necessary since the same scene can be used if it is edited by hand after Indigo parsed it. Obviously the most difficult part is to get the name of the last created image and insert it into the next command... but at testing stage everything can be done by hand anyway

----

bkircher, sorry I'm squatting your post, maybe I should start a new one ? Thx for creating this one

[bkircher]

It seems you're trying to implement something I'm theoretically speculating about...

btw., in batch files you can use "%1". Drop a file onto a .bat, and it will substitute %1 with the filename. Basics I know, but if you want to start 10 files one after the other, use <halttime> and write your lines like:

indigo %1 [options]
copy initfile.txt initfile_ori.txt
indigo %1 [options]
copy initfile1.txt initfile.txt
indigo %1 [options]
...