Page 2 of 5

Posted: Sat Sep 06, 2008 12:13 pm
by Deus
Suvakas my friend. a binary char takes 1 byte the corresponding char takes 3.333 bytes in a "text" file. So in practice XML is about 5:1 loss in data size.

Posted: Sat Sep 06, 2008 8:19 pm
by suvakas
Gotcha.
Thanks. Yes, I get the point now.
Sounds like a good deal 8)

Posted: Sun Sep 07, 2008 3:27 pm
by Deus
Damn I cant do simple multiplication. It should be 2.4 bytes. The approximate formula to calculate this for any number of bits is nbits*0.3 ex 8*0.3 = 2.4 bytes for a byte 4.8 bytes for a 16 bit number etc.

XML tags, line feeds, whitespace etc also adds overhead obviously.

Its dependant on the XML structure and other things but its rougly 5:1 overhead in the general case. The normal way of "fixing" this architecturally in a modern file format while still retaining size and the strength and flexibility of XML is to use zlib or similar to compress the XML stream. Good examples of this in practice is JARs, XPS (Vistas pdf "replacement") and Indigo files. It requires CAREFUL implementation and reading speed and size will still suffer a bit but not as bad as 5:1 size wise.

It's important to realize is that XML is really strong because

A) It can be manually typed in an editor
B) Its easily extended
C) Underlying architecture is easily understood by just reading documents written in the specification and understanding = engineering trust and engineering trust = something that gets used i.e. xml makes people want to use the product or platform

However. For "trivial" things that is well known prior art and that is highly unlikely to change over time XML is NOT the best choice. Examples of this is bitmaps, masses of raw data (audio, geometry)

Onos choice to go for a binary format here is a good choice and I hope I gave some of you a bit more insight into software architecture and data formats.

Im starting to babble here ;) It's important stuff though.

/ D

Posted: Sun Sep 07, 2008 9:41 pm
by fused
Deus wrote:Onos choice to go for a binary format here is a good choice and I hope I gave some of you a bit more insight into software architecture and data formats.
right. my words. :)

i implemented it into cindigo and the results are quite impressive:

Code: Select all

Exported mesh: Buddah, 1085617 tris, 548494 vertices, 543717 uvs. Processing time: 3.125s. Total export time: 3.502s.
igmesh size: 45.6MB
heres another one:

Code: Select all

Exported mesh: Landscape, 3992000 tris, 1996002 vertices, 2000997 uvs. Processing time: 7.333s. Total export time: 8.698s.
igmesh size: 167MB
it writes 4m tris in 1.365 seconds. thats impressive.

will export those to embedded_2 and post the results, too. for coparison.

Posted: Sun Sep 07, 2008 9:53 pm
by fused
results embedded_2:

Code: Select all

Exported mesh: Buddah, 1085617 tris, 548494 vertices, 543717 uvs. Total export time: 21.78s.
igs size(mesh element only): 153MB

Code: Select all

Exported mesh: Landscape, 3992000 tris, 1996002 vertices, 2000997 uvs. Total export time: 60.27s.
igs size(mesh element only): 429MB

Posted: Sun Sep 07, 2008 10:12 pm
by OnoSendai
Thanks for posting some results, fused.

Posted: Mon Sep 08, 2008 1:04 am
by pixie
Fused is the man! :D

Posted: Mon Sep 08, 2008 1:06 am
by pixie
@ Deus:
I think that models are one of those things that binary makes sense, the scene however makes sense to be in xml, still as there's includes there's already a match made in heaven for those two :)

Posted: Mon Sep 08, 2008 4:54 am
by Deus
Ono can you please supply Vec2,Vec3 code too so I can integrate directly into indigoworld? :)

Posted: Mon Sep 08, 2008 3:30 pm
by OnoSendai
Deus wrote:Ono can you please supply Vec2,Vec3 code too so I can integrate directly into indigoworld? :)
Nope, just write some conversion functions.

Posted: Mon Sep 08, 2008 4:41 pm
by Whaat
This may be a stupid question, but I'm tired. :) Can this binary format be created using Ruby code or do I have to use C++? Is there a way to create the mesh files using Ruby? Forgive me...I only briefly looked at the C++ code.

Posted: Mon Sep 08, 2008 7:34 pm
by OnoSendai
Yes, you should be able to write the format from ruby just fine.

Posted: Tue Sep 09, 2008 2:49 pm
by Whaat
OnoSendai wrote:Yes, you should be able to write the format from ruby just fine.
good to know....too bad I don't have a clue how to do it. :(

Posted: Tue Sep 09, 2008 5:35 pm
by OnoSendai
Whaat wrote:
OnoSendai wrote:Yes, you should be able to write the format from ruby just fine.
good to know....too bad I don't have a clue how to do it. :(
Try array.pack

Posted: Wed Sep 10, 2008 3:54 am
by Whaat
ok. That helps a bit. Do I need to create an IndigoMesh class in Ruby that is identical to the C++ class? And then, do I have to figure out how to write the information contained in the IndigoMesh object to a file using the same template as the one that you have provided? :?