Why I Picked XCF Instead of OpenRaster for Vara

My digital painting application Vara could export the final work, but was still lacking a Save feature. That was okay so far because Vara wasn't originally meant for multi-session drawings. That is, the original use case was that one would launch Vara, scribble something casually, and export the final image. Done.

Things had changed. While the early releases had only two layers, recently it started supporting any number of layers. Linear RGB workflow and gamma correction had landed. The in-memory pixel storage and processing was of high dynamic range from the beginning. Enough to carry someone away from casual scribbling. Then one would want to save the session so that they could resume later with Vara or some other application.

So I decided to start working on the feature finally, and got it implemented within a week. Vara v0.1.240823 can save the document in XCF format, which is the native format of GIMP.

I think reusing an existing format instead of inventing anything new is understandable: it has the advantage of making your program instantly interoperable as well as benefiting from the lessons that others have already learned (at the cost of conforming to the specs laid out by someone else). But when it came to the selection of an open format that supported high dynamic range and multiple layers, there seems to have been a better choice: OpenRaster. Then why did I pick XCF for Vara?

Why OpenRaster?

First let's see why OpenRaster makes a better choice. While distribution (export) formats like PNG and JPEG are not tied to any particular application, project (saving) formats like PSD, XCF, and kra are tied to their original applications. Their only goal is to let the original application store and restore an editing session, without any promises for third-party users. This means they can be underdocumented, unstable, and tightly coupled with the internals of the original application. This is why the official spec of XCF itself warns against its use in other applications.

Why XCF?

To my best knowledge, OpenRaster is the only raster project format whose primary goal is interoperability. So I naturally wanted it to be the default project format for Vara. Then why did I change my mind?

The primary issue for me was that OpenRaster was a ZIP-based format, encoding and decoding which required more work, that too on top of depending on additional libraries. Basic file I/O routines provided by the C standard library are enough to write XCF, unless you need advanced compression. The reduced complexity was already enough to make my mind, but more reasons enforced the decision:

1. OpenRaster is still a draft, despite having started in 2006 or so
2. Applications seem have their own variants of it
3. XCF has better documentation and updates compared to OpenRaster
4. XCF isn't as GIMP-specific as it is said to be; you don't even have to look into the data structures used by GIMP

Of course, all of this could change, but for the time being, it felt like supporting OpenRaster wasn't worth the trouble.

Please note that this is not to downplay the efforts of OpenRaster. It's an important initiative with an important goal, and you should support it if you have time.

Debugging

Getting XCF to work wasn't an easy task though. The format has its own quirks. For example, the number of bytes used to store is four in older versions while it is eight in newer ones. Numbers are stored in big-endian. Presence of certain fields and their accepted values are version-dependent. But that's the whole point of versioning. The spec is relatively clear about all of this (despite leaving some questions here and there).

Hex editors and the built-in logging in GIMP (GIMP_DEBUG=xcf gimp FILENAME) were helpful to check the XCF files I wrote. I didn't refer to the source code of any XCF writers, but interestingly, hacking into the code of an XCF reader was helpful during the early tests. What I did was modify the source code of the XCF reading code in ImageMagick to make it print debug messages while it read my XCF outputs. This helped me identify what the reader was expecting in each step and whether it was lacking in my output.