Archive for June, 2011

Simple and efficient management of shared resources

June 22, 2011

Recently, Cliff Harris of Positech Games made a blog post called Optimising my ‘dumb’ textures, where he describes the optimisations he is doing to his latest game, to increase the load times when dealing with large number of textures that needs to be managed automatically, but also be shared between different game systems.

This made me think of resource managers in general, and how I’ve been planning for a while to replace the one in my Pixie Game Engine with something more robust, efficient and generic. It also struck me that this is a topic that often comes up, and I think all developers eventually reach a point where they need to solve this problem, so I thought I might as well give it a shot and share the resulting code as public domain, as I usually do.

So, my requirements for this system is that it should be really easy to use, but at the same time very efficient both in terms of memory use and performance.

In the solution I came up with, I use efficient hash tables to store the resources, but I wrap the access to them in a convenient resource class, which can be used as simply as this:

Resource<Texture> myTexture("myBitmap.tga");

In this example, Texture is the class that we want our resource manager to manage, and Resource is the templated wrapper class that we use to have it managed automatically.

There’s no problem having multiple instances referencing the same physical resource, like this:

Resource<Texture> myTexture("myBitmap.tga");
Resource<Texture> myOtherTexture("myBitmap.tga");

Even though we have two different resources, the fact that they’re created from the same filename means that they will both be pointing to the same Texture instance. The first of these two lines will cause the Texture instance to be created, while the second line will merely add an additional reference to the same instance.

Resources are automatically reference counted, so as resources are destroyed, the reference count goes down, and when it reaches zero (meaning there are no longer any Resource referencing the instance in question), the instance will be deleted automatically.

A good thing about this resource implementation is that Resource instances take very little memory – only 4 bytes – as they only store a pointer to the instance they reference. Also, they can be freely copied and assign, and the reference count will be adjusted accordingly, like this:

Resource<Texture> myTexture("myBitmap.tga"); // New instance created
Resource<Texture> myTexture2("myBitmap2.tga"); // New instance created
Resource<Texture> myTexture3(myTexture); // Ups ref count on "myBitmap.tga"
myTexture2 = myTexture3; // Ups ref count on "myBitmap.tga”
// and also causes "myBitmap2.tga" instance to be destroyed

This makes it possible to work freely with Resources, assigning them and copying them around, knowing that it’s all very efficient under the hood, and that the bulk of the data (as would be stored in, say, the Texture class) will be shared among all of them.

Additionally there’s a couple of helper functions to get the filename that a resource was created from, and to get a list of all managed resources of a specific type – which can come in handy if you need to loop through and perform an operation on each of them, such as Cliffski’s case where he needed to restore lost surfaces after Alt+Tab.

The code is public domain, it’s stand-alone and compiles as is – but it does include several of the data containers I originally developed for Pixie, so there’s quite a few files in this package, even though the resource system itself is only four fairly small files.

Download Resource Manager Source Code

I hope someone will find this useful, if nothing else as a starting point for their own system. I’ve done some quick performance tests, and on my computer I can create and destroy one million resources in less than half a second, so the resource system adds very little overhead to the process of loading and managing your assets :-)