Shader tests

General questions about Indigo, the scene format, rendering etc...
User avatar
OnoSendai
Developer
Posts: 6243
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Post by OnoSendai » Wed Jun 25, 2008 8:16 am

ZomB wrote:
OnoSendai wrote:
Whaat wrote:Yup, basically you embed a short program in the scene file, that describes the albedo, exponent, blend factor, displacement, etc.. of any material.
The program is executed whenever a ray hits the material, and can therefore use such information as the normal at the hit position, hit position, surface uv coordinates at the hit position etc.. to calculate the final colour or whatever
Hmmm... how about processing speed, running the script for each ray hitting the material could maybe slow down the whole rendering Process if the shaders getting more complicated.

How about GPU support for this part of Indigo, actually this should be a classic GPU homeground...

Check this place for advice: http://www.gpgpu.org/
One step ahead of you ZomB.
If the shader language is sufficiently general, then hopefully the code generation can be retargetted to CUDA etc..

User avatar
Zom-B
1st Place 100
Posts: 4701
Joined: Tue Jul 04, 2006 4:18 pm
Location: ´'`\_(ò_Ó)_/´'`
Contact:

Post by Zom-B » Wed Jun 25, 2008 9:07 am

That's my boy :D
polygonmanufaktur.de

User avatar
CTZn
Posts: 7240
Joined: Thu Nov 16, 2006 4:34 pm
Location: Paris, France

Post by CTZn » Wed Jun 25, 2008 6:17 pm

Wonderfull but I hope we'll have a format specification of this before further fixing bugs, adding long-requested features, and replying to emails from you ?

Another example maybe :)
obsolete asset

User avatar
vmuriel
Posts: 17
Joined: Thu Apr 03, 2008 12:45 pm
Contact:

Post by vmuriel » Thu Jun 26, 2008 6:43 pm

Custom shader language, really really nice.

Maybe, using this new ability, we can make ours output layers for production ready compositions: diffuse layer, specular layer, etc, etc, while the master Ono makes a layer system.

mgix
Posts: 12
Joined: Thu Jun 26, 2008 10:35 am

Re: Shader tests

Post by mgix » Sat Jun 28, 2008 6:45 am

OnoSendai wrote:
I looked around at a few existing languages / VMs, but none of them provided quite what i wanted, or had too much overhead, so I decided to roll my own. Perhaps I'll switch to an existing language later, we'll see.
Have you looked at LUA (I know the pull of designing ones
own language is a very strong one, but ... :) ) ?

LUA's VM is super lightweight, completely portable, and the
language -albeit very weakly typed - is very clean functional
language (a la python, etc) and has the advantage of having
a syntax that is super easy for non-programmer types to grok.

Also: now that Indigo is going the programmable shaders
route, it has been my experience that in tricky production
renders, most of the computing time is spent in evaluating
shaders.

Render architectures that more or less properly tackle this
problem (i.e. renderman) do this by massively vectorizing
the evaluation of said shaders (i.e they amortize the cost
of interpreting each instruction of the VM bytecode by having
said instruction act upon a large number of pixels).

Architecture-wise, this is relatively easy to do in a scanline render,
but a tad harder in a raytracer.

Will Indigo be able to do this ?

User avatar
Borgleader
Posts: 2149
Joined: Mon Jun 16, 2008 10:48 am

Post by Borgleader » Sat Jun 28, 2008 7:33 am

Wan't LUA the language used by WoW-addon coders?

User avatar
zsouthboy
Posts: 1395
Joined: Fri Oct 13, 2006 5:12 am

Post by zsouthboy » Sat Jun 28, 2008 8:19 am

LUA is used by Blizzard themselves to do all interface work.
Take a look, the actual code is sitting right there in your WoW directory, ready to be modified :)

User avatar
Borgleader
Posts: 2149
Joined: Mon Jun 16, 2008 10:48 am

Post by Borgleader » Sat Jun 28, 2008 8:55 am

zsouthboy wrote:LUA is used by Blizzard themselves to do all interface work.
Take a look, the actual code is sitting right there in your WoW directory, ready to be modified :)
I uninstalled wow a while ago. Moved on to AoC

mgix
Posts: 12
Joined: Thu Jun 26, 2008 10:35 am

Post by mgix » Sat Jun 28, 2008 11:21 am

Borgleader wrote:Wan't LUA the language used by WoW-addon coders?
LUA is used in a lot of games, both as an add-on
language and as an internal scripting glue language.

One of the reasons for that is that is very easy
to interface it with native C routines.

Sukrim
Posts: 66
Joined: Wed Jan 09, 2008 6:47 am

Post by Sukrim » Sat Jun 28, 2008 9:28 pm

The AI scripts in Crysis for example are LUA too...

User avatar
Kram1032
Posts: 6649
Joined: Tue Jan 23, 2007 3:55 am
Location: Austria near Vienna

Post by Kram1032 » Sun Jun 29, 2008 4:42 am

According to Wikipedia (German), LUA is used in the following games:

Heroes of Might and Magic V, Monkey Island 4, Empire at War , Dawn of War, Spellforce, GMod 9 & 10, Far Cry, Crysis, World of Warcraft, Ragnarok Online, Microsoft Freelancer, The Settlers V, Parkan 2, Star Wars Battlefront II, Company of Heroes, Stalker, Total Annihilation Spring, Angband 3, ZAngband and ToME.

User avatar
OnoSendai
Developer
Posts: 6243
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Re: Shader tests

Post by OnoSendai » Sun Jun 29, 2008 3:04 pm

mgix wrote:
OnoSendai wrote:
I looked around at a few existing languages / VMs, but none of them provided quite what i wanted, or had too much overhead, so I decided to roll my own. Perhaps I'll switch to an existing language later, we'll see.
Have you looked at LUA (I know the pull of designing ones
own language is a very strong one, but ... :) ) ?

LUA's VM is super lightweight, completely portable, and the
language -albeit very weakly typed - is very clean functional
language (a la python, etc) and has the advantage of having
a syntax that is super easy for non-programmer types to grok.

Also: now that Indigo is going the programmable shaders
route, it has been my experience that in tricky production
renders, most of the computing time is spent in evaluating
shaders.

Render architectures that more or less properly tackle this
problem (i.e. renderman) do this by massively vectorizing
the evaluation of said shaders (i.e they amortize the cost
of interpreting each instruction of the VM bytecode by having
said instruction act upon a large number of pixels).

Architecture-wise, this is relatively easy to do in a scanline render,
but a tad harder in a raytracer.

Will Indigo be able to do this ?
Yup, I tried out Lua, in particular LuaJIT, which extends the Lua VM with a JIT compiler. Lua does indeed have a nice, lightweight, and low overhead VM. The JIT does work pretty well, at least for simple arithmetical code.
However I found the vector/composite structure handling of Lua to be quite slow. The JIT'er doesn't seem to work well here, due to calls into the VM and memory allocations etc..
Also, while Lua is threadsafe in the sense that you can have 1 VM per thread, it's a bit wasteful to duplicate so much shared data over mulitple threads.

SIMD shader evaluation is something I'll think about. Right now the architecture of Indigo doesn't allow for SIMD execution, but perhaps in the future I'll change it.

mgix
Posts: 12
Joined: Thu Jun 26, 2008 10:35 am

Re: Shader tests

Post by mgix » Sun Jun 29, 2008 8:57 pm

OnoSendai wrote: Yup, I tried out Lua, in particular LuaJIT, which extends the Lua VM with a JIT compiler. Lua does indeed have a nice, lightweight, and low overhead VM. The JIT does work pretty well, at least for simple arithmetical code.
However I found the vector/composite structure handling of Lua to be quite slow. The JIT'er doesn't seem to work well here, due to calls into the VM and memory allocations etc..
Also, while Lua is threadsafe in the sense that you can have 1 VM per thread, it's a bit wasteful to duplicate so much shared data over mulitple threads.
I see you have done your homework :D
OnoSendai wrote: SIMD shader evaluation is something I'll think about. Right now the architecture of Indigo doesn't allow for SIMD execution, but perhaps in the future I'll change it.
I wasn't very clear in my post, so just checking that we're talking about
the same thing. Your looking at LuaJIT seems to imply you're worried
about the speed of the VM when in fact it may not matter much.

I wasn't simply talking about SIMD the way it's generally understood
these days on (i.e. vectors of size 4 or 8 using SSE or Altivec), but
rather about _very_ long vectors (~16K long).

If you do implement this, then the speed of your VM becomes kind of
irrelevant because the VM overhead for decoding and executing an
instruction is amortized on 16K pixels. Renderman, for example,
implements this, and they never really had to make their VM fast.

There's of course the problem of dealing with diverging code path
for different pixels in the same vector, so it's not all fun and games :)

Another thing that's worth looking at if your brain is focused on
the programmable shader thing is LLVM (http://llvm.org/).

Although a tad bulky, the project allows you to generate native
machine code on the fly whatever your shading language is.

Just my $0.02.

Keep up the good work with Indigo !

Cheers.

User avatar
OnoSendai
Developer
Posts: 6243
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Post by OnoSendai » Mon Jun 30, 2008 12:36 am

yeah, I know what you mean by large SIMD widths.
Since i don't have an appropriate design for doing such SIMD computations in Indigo currently, VM speed and overhead is an issue though.

I did have a look at LLVM, and it looks nice. Sadly it won't compile in Visual Studio however, and I don't want to have to build it in MinGW.

mgix
Posts: 12
Joined: Thu Jun 26, 2008 10:35 am

Post by mgix » Mon Jun 30, 2008 1:01 am

OnoSendai wrote:yeah, I know what you mean by large SIMD widths.
Since i don't have an appropriate design for doing such SIMD computations in Indigo currently, VM speed and overhead is an issue though.
Okay.
OnoSendai wrote: I did have a look at LLVM, and it looks nice. Sadly it won't compile in Visual Studio however, and I don't want to have to build it in MinGW.
Awwww :(

As nice an IDE as Visual Studio maybe, you shouldn't let mere tools
get in the way of doing what's architecturally right for your code.

Post Reply
223 posts

Who is online

Users browsing this forum: No registered users and 1 guest