Page 1 of 2
memory errors of death
Posted: Fri Feb 15, 2008 3:37 am
by drBouvierLeduc
Hi all,
I was wondering, when using latest "official" blendigo version, how do you deal with memory errors ?
From what I understand, memory errors aren't related to the entire polycount of the scene, but rather on the polycount of each mesh.
I believe the limit is around ~300k or something ? So if no mesh is higher then this in the scene, there should be no problem.
In the project I'm working on, lots of meshes are sculpted and quite detailled. So I have split each of them, so that no one is higher than 200k polys. And still I get those memory errors...
So I'm a little lost there...
Smartden, I know you're working on that problem, is it possible to test that version of blendigo ?
Posted: Fri Feb 15, 2008 4:44 am
by zsouthboy
Use an x64 build of blender, and x64 python, and add tons of ram.
The magic limit disappears.
(The limit is actually a python problem - our blender exporters essentially try to work on an entire mesh at once..)
Posted: Fri Feb 15, 2008 4:57 am
by drBouvierLeduc
unfortunally I'm on windows, so no x64blender... Anyway I heard that the limit is blowned away in the upcoming version of blendigo.
Posted: Fri Feb 15, 2008 5:43 am
by dougal2
can you get blender to dump .obj files instead ?
Posted: Fri Feb 15, 2008 5:59 am
by drBouvierLeduc
You mean export the scene as obj. files then import them in indigo ? Too much work for my lazyness !
Posted: Fri Feb 15, 2008 6:13 am
by dougal2
Well, I didn't mean manually, of course that'd be loads of painful work. It was more of an open question/suggestion to the exporter writer(s).
Posted: Fri Feb 15, 2008 8:56 am
by PhilBo
You could setup a new partition with Linux 64 bit and then do the stuff there. Indigo runs great under wine or you could bring the files back over to the Windows partition and run it from there.
Posted: Fri Feb 15, 2008 10:12 am
by Wedge
All I can advise is to use the exporter I maintain until Blendigo is fixed.
Or do something about the big meshes.
The limit is not directly your fault. The old Blender exporters (And I'm assuming without looking at Blendigo code, that it exports like the old exporters) try to put all of the mesh data into one string variable. This is bad, and this is the problem.
32 bit system can handle this fine. I export 1 million vert objects for fun on my old computer. (it has 2gb ram)
Posted: Sat Feb 16, 2008 9:47 am
by zsouthboy
Oh, I forgot to link it - use the LAA compiled Blender.
http://www.graphicall.org/builds/builds ... how&id=493
i was using it for displacement export on win64 - it'll happily take 3g+ of ram before crashing, rather than ~1.2 G
Posted: Sat Feb 16, 2008 1:07 pm
by OnoSendai
Wedge is correct, I had a quick look at Blendigo, and it keeps the mesh data as one (or more?) temporary variables.
This is probably causing the out of memory errors.
To handle large meshes efficiently, it should write them out to disk as soon as possible.
There may be some issues relating to collapsing shared vertices, but this probably isn't done in Blendigo anyway.
I'm not sure if Smartden is working on this, but I'd be happy personally if someone else fixed it, as it seems a problem for a lot of people

Posted: Sun Feb 17, 2008 12:35 am
by dougal2
I'm not a python programmer, nor have I ever seen blendigo's source, but shouldn't it be fairly simple to change
var += something
to
file.write( something )
?
I've done precisely this change within MtI for writing large numbers of instances. Much quicker and uses less memory.
Posted: Sun Feb 17, 2008 1:44 am
by OnoSendai
That's more or less it Dougal2, but it will require a little bit of refactoring.
Posted: Sun Feb 17, 2008 5:01 am
by SmartDen
i'm working on it already. and i can say, it's make me pretty "paint in the ass". Ono has right, it needs some refactoring. But i think drBouvierLeduc and me will soon present you new version, that can handle big meshes

I plea you, be a little bit longer patient.
Posted: Sun Feb 17, 2008 6:07 am
by BbB
Thanks SmartDen. That'd be great. Also suffering from this a bit.
Posted: Sun Feb 17, 2008 8:46 am
by Wedge
The exporters are written in modules. Just take out the exportMesh module from mine and throw it in Blendigo. Do you need my help?
Shouldn't take too long. Let it write directly out to file one obj per file like I did. Best way to do it. And object file is better than Indigo format. (no offense) This is because the less characters are used to identify what is what. I'd make obj file even more efficient by using only one letter to represent textures and normals rather than two. For every vert, normal, or texture location all you should need is one letter allowing the program to figure out what it is. Makes file size much smaller when working with large mesh.
Should remove any calling to NMESH as well, it has been deprecated for quite some time now.
I don't mind helping, just have to ask me. I'd like to think that the exporter I work on could be the 'core' of Blendigo and Blendigo can go on making its connection with the user while I fix the problems behind the scene and do not care so much for the ease of use.
I just get disappointed to see errors keep showing up months after I fixed them. And even worse to see fixes being brought up that aren't really fixes.
