When making games, you find yourself dealing with lots of data of different sorts, everything from the movement speed and jump height of the player to the level layouts or game art, and it can sometimes be of great importance how you deal with it. Some things (especially the smaller, simpler things), you might as well just store in the code itself, while other things might be better to store in external files. This entry will be about storing game data, and more specifically about using XML files for storage.

In the various online forums and messageboards about game development, the question about how to store game data pops up from time to time. Usually, some people will reply to this by recommending their favourite data format (be it scripts, like LUA, or data formats like INI-files or XML) and then keep defending their choice, touting it as the best in any given situation. These are people with enough experience to feel confident, but not nearly enough experience to be worth listening to. Give them a few years and a couple of large projects, and they’ll see the error of their ways

The fact is, that how you store data depends on your game, the data in question and (yes) your personal preferences.

Don’t think that you’ll be using the same format for all the data in a game, or that you will use the same formats for every game you make. Attempting to do so will result in a lot of wasted time, that could have been better spent.

If the data is large (like bitmaps, 3d models etc), you must use custom, binary formats. There’s no other way to get maximum load speed, and there’s no reason to make your players wait for your game to complete a bunch of text-parsing during loading (and yes, even for small games, text-based formats make a significant impact on load times, even on fast computers). If you need the data to control flow or check conditions, you need a scripting language of some sort. But it’s usually not a good idea to use an out-of-the box scripting language – you need a simpler, custom solution which will let you express your game scripts in the most simple way possible, to make them easier to write/

Sometimes you need to store small amounts of data which you need to be fairly easy to edit by hand, or you need an intermediate format for larger amounts of data (which will be converted to a custom binary format before release, like my 3d model file format), and XML is a good choice for this (in many cases). Note though, that it’s not the only one – always evaluate your options and make the choice that you feel comfortable with.

When making games using my own free, public domain game engine, I use XML for some things from time to time, and if used right, it’s a really nice way of storing data. I won’t go into too much detail about the XML format itself – there’s plenty of online resources describing XML basics. There’s one thing I think I should mention though:

There’s two different ways of reading and working with XML files. two different classes of parsers: DOM (Document Object Model) and SAX (Simple API for XML).

With DOM, the game programmer tells the XML parser to read and parse an XML file. The parser will go through the file, and create a hierarchical datastructure in memory, mirroring the content of the XML file. The game programmer will then add code to parse this datastructure, pulling out strings from it and use this to create his own in-game structures.

With SAX, the game programmer also tells the XML parser to read and parse an XML file. The parser will go through the file, and for each element or attribute it encounters, it will call a function implemented by the game programmer, passing along the data. The game programmer can do pretty much what he wants in this function, and will use this to create his own in-game structures.

Though they appear quite similar, there’s a couple of significant differences between the SAX and DOM parsing philosophies. With DOM, there’s a lot of small memory allocations going on, as it will have to allocate string buffers and hierarchy nodes as it goes along. Memory allocations is very slow, and you definitely don’t want a lot of them going on during load time, if you can avoid it. With SAX, on the other hand, there won’t be much allocations going on, except whatever ones the game programmer makes in the callback functions, to store the in-game data. Also, using DOM, you will have to keep the entire contents of the XML file in memory while extracting the data – with SAX, you only hold bits of it at a time.

DOM parsers have their uses, especially if you need to load a hierarchy, change it, and save it back to file, but for games where you just want to read the data and store it in your own datasets, the obvious choice is a SAX parser.

So what do most games programmers use? Well, being games programmers, they use DOM parsers. Which doesn’t make much sense at all. I think the reason is that there’s a popular DOM parser called TinyXML, which a lot of programmers are recommending on forums and such. Not because it is better than other parsers, but because it is the only one they have used (or in some cases, heard that others have used). I think the "Tiny" part of the name appeals to a lot of programmers, and it is true that the source code is quite small, and the feature set is limited. But it’s still a DOM parser, so its memory footprint, number of allocations or general speed won’t be tiny.

A better choice is a little known SAX parser called expat, which is also the one I’m using in my engine. It’s small, easy to use and free to use and distribute in any way.

Using a SAX-based parser is not difficult, and just requires you to register some callback functions with the parser, and keep track of where the data should go. However, as I’m using object-oriented C++ for most things, I’ve built a simple system on top of expat, which allows for even easier loading of XML data, and I will be writing more about this system in a later blog entry.

The last couple of weeks (in between studying for my exam ) I’ve been working on a new project, a quite hardcore (p&p style heavy number crunching) fantasy RPG. I won’t go into much details about it yet, but it’s progressing rather well, and I’ll keep posting little bits about it here as I go along.

Here’s a little preview, one of the characters from the game:

The game will (as usual) be free, and I’ll make the source code public domain (again, as usual) so that it may (potentially) be of some use to others

So far, I’ve mostly been writing about games I like, and unfortunately, Duell is not one of them.

The game is basically a card game, though you have a simple game board as well. The objective of the game is to be the first player to win five rounds, and you win a round by landing your piece on top of your opponents.

When playing the game, you hold five cards in your hand, and on your go, you play a card and move your piece that number of moves backwards or forward. You may not move into your opponents half of the board, and you can’t move backwards and forwards in the same go. If you are able to move on top of your opponent, you win the round, unless your opponent can counter your move by playing a card with the same number.

There’s a few special rules as well, but they don’t do much to make this game more fun. It’s not a really bad game, it’s just that it gets quite boring quite quickly. It feels like it’s a bit too random, and I think that if there had been a stronger element of strategy, it would have been much better.

When it comes to 3D games, you often find yourself in a position where you need to have a big area where the action takes place – the terrain on which to place your roads, buildings and other things. There’s many ways to go about this (ranging from having the artists model everything by hand to having it all procedurally generated), and today I’ll be talking about a very simple, straightforward way to generate terrain, which also gives a great result.

This approach requires you to defined a few base texture maps, which we will use to generate and texture our terrain. First out is the height map, which in many ways is the most important one, as it will define the shape of the generated terrain. The heightmap is just a grayscale image, where black means lowest and white means highest point. There’s various ways to go about acquiring a heightmap, including procedurally generating one, but the simplest way is simply to paint it in a paint program.

In addition to the heightmap, we’ll need a few (four in this example) texture maps for the ground. It tends to work best if they are quite distinct, I’ve chosen sand, grass, and two different kinds of rocks. The idea is to divide the terrain into four different height spans, and use Texture map A for the first span, Texture map B for the second, and so on.

Once we’ve calculated the pixel color, we perform simple light calculations (N dot L lighting works fine) and do a simple ray-test to see if the light source can be seen from this point on the terrain (this is as simple as stepping along a line in three dimensions, checking the lines y-position against the heightmap value at each point). This gives us static lighting and shadows, which we can bake into the large terrain texture.

To make use of this in our game, we just create a grid of triangles corresponding to our height map, and textured with the large texture. The value for the y-coordinate of each vertex is taken from the heightmap, giving us a three-dimensional, fully textured model of the terrain. Since the texture map is stretched across such a large area, it can look way too blurred and stretched out, so to make it appear more detailed, we use the detail map together with the large texture, but tiled instead of stretched. As you can see on the picture above, this gives a very nice result, for very little effort – once you have a decent set of base texture maps, you can keep making new landscapes just by painting heightmaps!

If you add a plane with a semi-transparent water-texture on it, you can get rivers and lakes pretty much automatically, by adjusting the height of the water-plane, making your landscape much more alive.

A drawback of this technique is that it’s using the same detail map for the whole terrain, so you can’t have, for example a grass-pattern detail map in some areas, and a rock-patterned one elsewhere, but instead you have to settle for a pretty non-descript map which sort-of-works everywhere. There are other techniques which can address this issue, and I might write about them some other time…

I’ve been into RPGs, of the pen & paper variety, since before I got into video games, so I’ve always had a soft spot for computer RPGs too. The problem is, so many computer RPGs have a tendency to simplify things too much, making them more like traditional video games rather than the numbers-heavy simulations of most RPG (at least the early ones).

I enjoyed Phantasie 3 – The Wrath of Nikademus at the time I played it, on the Atari ST. It sure didn’t make any efforts to hide the complexity of the numbers by trying to show it by graphical visualizations. Like the character sheet above, it’s all numbers, and I quite like that

It got a bit more visual when you where travelling on the roads. You could move your group of adventurers around, and sometimes you would encounter enemies, and have to fight it out. Though the enemies wasn’t displayed on the map, you just got a message box telling you about the attack, before taking you to the battle screen.

When you encountered a city, there was various things you could do, like train or learn spells, or recruit more members for your party. This was done by simple menus, which worked quite well.

Battles was turn-based, and at the start of your turn you would set up which action each of your characters would perform during the battle.

Battle would then be resolved, letting you know who had been damaged and if any of the enemies had been killed. Your own characters could be injured in various ways, including having limbs chopped off!

There was also dungeons to explore, and that worked pretty much the same way as when travelling along the roads (but without the cities, obviously).

It’s rare to see RPGs with this level of complexity these days. Focus seems to have shifted to fast-paced action and pre-defined storylines, with less interest in detailed simulations and oldschool RPG mechanics. Which is a shame, because I’m sure that there’s quite a few of us out there who really likes them, and would love to play some newer ones…

If you have followed my entries on the RetroBox, you’re going to like this: the first version is officially released now, and you can download it here. Included is full source code and a simple assembler program which displays a couple of sprites and some colors on the screen, like this:

To use it, just drag and drop any RetroBox asm file onto it. When developing, you might want to run the _Debug version, and specify -window on the commandline as well as the name of your asm file, to make things a bit nicer.

There’s still a lot of work left to do before the RetroBox is "finished", but if you feel like having a go at some RB assembler programming, it’s definitely good enough to start using. The next big step is making the high-level language, which will probably be a bit like BASIC, but more modern and with support for objects/classes etc…

So, download this:

RetroBox_1_0.zip

to give it a go, or just to have a look at the source code and see how it’s all put together. And please let me know what you think

Note: If you want to try your hand at some ASM programming for the RB, you’ll need thislist of assembler instructions.

To give you an idea of what assembler for the RetroBox looks like, here’s the full source code for the sample asm-program shown above:

Through the years, my wife and I have made a habit of having a look for boardgames whenever we happen across a charity shop, flea market or toy store. Sometimes you can find interesting games that you’ve never heard about, and at bargain prices, making it worth the risk that the game will suck or not be complete. This game here, we picked up at a toy store when we were living in Dundee (Scotland).

The objective of the game is simple: put down all your cannibals on the board before your opponent have put down all of his. You play the game by taking turns, and on each turn you roll three dice (which are special dice, numbered 1 to 3 instead of the usual 1 to 6). You add up the sum of the dice, and the number you get is the column on the board on which you must place your next piece (the columns are labelled 3 to 9).

If you successfully place your cannibals on either side of an opponents piece (even diagonally or wrapping from top to bottom of the board), it will be "eaten", and returned to his hand, taking him further from his goal of placing all his pieces down.

As with a lot of fun games, the basic mechanic is dead easy, but after a couple of rounds, you’ll find yourself thinking through your moves with great care, thinking ahead and setting traps.

It’s a very nice game which doesn’t take long to set up and play a round or two, and if you come across it, I’d recommend you to give it a try.

I’ve talked a bit about racing games before, and as I’ve said, I prefer 3D racing games to 2D ones, but this old 2D racing classic from the Commodore 64 is still worth having a look at: yes, it’s Pitstop!

This is probably one of the first games I ever played. At the core, it’s a quite basic racing game; you push the joystick around to control your car, and you must avoid bumping into the other cars or the edge of the road.

If you do bump into anything, you will both lose speed and your tires will take damage (and will change color to indicate this). When they’ve taken enough damage, the will explode, and you’ll have lost the race. You also have a fuel meter, and if it ever reach empty, you will also lose.

So what you need to do, is drive into the pit when you pass it (once per lap), to change tires and fuel up. You control the guys using the joystick, and it is quite fiddly until you get used to the controls, but once you’re getting good at it, you can start the fuelling up process first, and then change all four tires before the tank is full, thus saving valuable seconds. But be careful: you need to stop filling the tank before it gets too full, or it will explode and you’ll lose the game!

The addition of the pit is what takes this game from just a standard racing game to a game which stood out at the time, and which still is quite good fun!

We’ve covered quite a lot of ground when it comes to the indie 3D Engine I’m working on. We’ve decided on what sort of things it should be capable of, how we’ll deal with material and lightings, and how we’re managing the definition of a model and instances of one. But one thing we haven’t touched on yet, is how we get the 3D models into the game in the first place. So today, we’ll be talking about 3D file formats.

If you start looking around on the web, you’ll find a lot of 3D file formats out there, so it’s easy to go about this all the wrong way, by asking "What file format should I use?". The answer to that question is, always and unavoidable, you should use your own custom file format.

There’s simply no way around this one. It’s better to have your own format to avoid having it bloated by information that your engine doesn’t use (and believe me, loading unnecessary information will have an adverse effect on load times). But more importantly, it’s essential to have your own format so you can store any extra information that your engine needs, and which won’t be supported by an off-the-shelf formats.

Creating your own file format can be a big, complex task, or reasonably fast and easy, depending on how you go about it. Now, what most commercial engines do, is create their own custom format, and then they write exporters and material plugins for all the modelling packages they support (usually the big, commercial one). That’s a lot of work, and though not impossible, it would be impractical to do it for the engine I’m working on (for starters, I’d have to get a license for each of the big modelling packages, and I couldn’t afford that).

Instead, I’m going to take the easier route. I’m going to create my own file format, but rather than writing export plugins for all the different modelling packages, I will write stand-alone conversion programs which can convert from a fileformat already supported by most modelling packages into my own, custom format.

When I worked in the games industry, I worked on many games where loading times was just insane. And there wasn’t even a good reason for it, there was just a lot of processing of the data taking place at load time. For some of the game, we did speed it up some towards the end of development, but it was still kind of slow.

So for my Pixie engine, I’ve made a point of making loading fast, by making sure there’s no procesing of data going on at load time, using custom binary file format where a majority of stuff can be loaded in large chunks straight into the memory area it will occupy during runtime. I’m going to do so for the 3D engine too, but there’s one disadvantage to binary formats: they break if you change anything. So, rather than having my converters output straight to the binary format, they will generate a custom, intermediary XML-based format.

XML is nice, because it’s possible to read by humans, and structured enough to allow for expressing all kinds of data. Due to its nature though (text based hierarchical), it’s way too slow for loading large amounts of data (even with the fastest parsers).

Anyway, here’s a simple example of how a 3D file in the Pixie XML format would look. Its a very simple model, just a plane with a texture on it, but it still shows the various elements of a model: Nodes, transforms, meshes, materials, vertices and indices.

I’ve started to write some converters for the format too, and I’m done with a converter from the .obj format, and almost done with one for the .fbx format (which is built-in to both 3D Studio and Maya, and is probably the most reliable way I’ve used to get model data exported from those packages). Additionally, I’ve written a custom exporter for Milkshape (which is the only modelling package I have myself). And I think I will leave it at that for now, but I’m sure I’ll be adding a few more later (maybe one for the .x format).

Having a structured and stable pipeline for getting content into your game is really important for productivity, and I think this way will prove quite good for our needs.

I like boardgames that are flexible enough to last a long time, which is a big reason for me liking games like Magic and HeroClix, where you can build your deck/team yourself, and the way you play and what strategies you use depends on how what you’ve constructed. So here’s a game I think is real neat: the Pirates game from WizKidsGames.

Basically, it’s a game where you move your pirate ships around, fighting each other, trying to gather as much treasure (the little gold coin markers) as possible on your home island (the game ends when all the available treasure is on either players home island, and the player with the most gold is the winner). There’s many different types of ships, capable of moving att different rates, and having varying attack capabilities and cargo capacity. As you construct your fleet yourself, there’s lots of opportunity to make any trade-offs you find reasonable.

When you buy the game, it’s in the form of a pack of cards, and each pack contains everything needed for one player (so if you get two packs, you’re good to start playing) – there’s no booster pack. All the game pieces (ships, crew members, gold pieces etc) are done as cut-out pieces on the cards (which are thick plastic cards). I think this is a real nice way of doing it, as it gives you nice three-dimensional playing pieces at a quite low cost (compared to other collectible miniature games). You even get three miniature dice, and a map (with rules printed on the back) with every pack.

It’s definitely worth giving this game a go, especially considering the low cost to get started.You can get it here.