We’ve covered quite a lot of ground when it comes to the indie 3D Engine I’m working on. We’ve decided on what sort of things it should be capable of, how we’ll deal with material and lightings, and how we’re managing the definition of a model and instances of one. But one thing we haven’t touched on yet, is how we get the 3D models into the game in the first place. So today, we’ll be talking about 3D file formats.
If you start looking around on the web, you’ll find a lot of 3D file formats out there, so it’s easy to go about this all the wrong way, by asking "What file format should I use?". The answer to that question is, always and unavoidable, you should use your own custom file format.
There’s simply no way around this one. It’s better to have your own format to avoid having it bloated by information that your engine doesn’t use (and believe me, loading unnecessary information will have an adverse effect on load times). But more importantly, it’s essential to have your own format so you can store any extra information that your engine needs, and which won’t be supported by an off-the-shelf formats.
Creating your own file format can be a big, complex task, or reasonably fast and easy, depending on how you go about it. Now, what most commercial engines do, is create their own custom format, and then they write exporters and material plugins for all the modelling packages they support (usually the big, commercial one). That’s a lot of work, and though not impossible, it would be impractical to do it for the engine I’m working on (for starters, I’d have to get a license for each of the big modelling packages, and I couldn’t afford that).
Instead, I’m going to take the easier route. I’m going to create my own file format, but rather than writing export plugins for all the different modelling packages, I will write stand-alone conversion programs which can convert from a fileformat already supported by most modelling packages into my own, custom format.
When I worked in the games industry, I worked on many games where loading times was just insane. And there wasn’t even a good reason for it, there was just a lot of processing of the data taking place at load time. For some of the game, we did speed it up some towards the end of development, but it was still kind of slow.
So for my Pixie engine, I’ve made a point of making loading fast, by making sure there’s no procesing of data going on at load time, using custom binary file format where a majority of stuff can be loaded in large chunks straight into the memory area it will occupy during runtime. I’m going to do so for the 3D engine too, but there’s one disadvantage to binary formats: they break if you change anything. So, rather than having my converters output straight to the binary format, they will generate a custom, intermediary XML-based format.
XML is nice, because it’s possible to read by humans, and structured enough to allow for expressing all kinds of data. Due to its nature though (text based hierarchical), it’s way too slow for loading large amounts of data (even with the fastest parsers).
Anyway, here’s a simple example of how a 3D file in the Pixie XML format would look. Its a very simple model, just a plane with a texture on it, but it still shows the various elements of a model: Nodes, transforms, meshes, materials, vertices and indices.
I’ve started to write some converters for the format too, and I’m done with a converter from the .obj format, and almost done with one for the .fbx format (which is built-in to both 3D Studio and Maya, and is probably the most reliable way I’ve used to get model data exported from those packages). Additionally, I’ve written a custom exporter for Milkshape (which is the only modelling package I have myself). And I think I will leave it at that for now, but I’m sure I’ll be adding a few more later (maybe one for the .x format).
Having a structured and stable pipeline for getting content into your game is really important for productivity, and I think this way will prove quite good for our needs.