17.2: Binary Data Files

Normally, when writing notes like these, I progress from the easy to the hard, or the boring to the interesting, or the deficient to the recommended. This chapter is the reverse; I heartily recommend that you use the text data files of the previous section whenever possible. This section on binary data files is included for completeness, and you're welcome to skip it if you're not interested in using binary data files or if it doesn't make sense.

We've already seen two examples of writing and reading binary data files, in section 16.7 of the previous chapter. To write out an array of integers, we called

	fwrite(array, sizeof(int), na, fp);

To read them back in, we called

	na = fread(array, sizeof(int), 10, fp);

To write out a structure, we called

	fwrite(&x, sizeof(struct s), 1, fp);

To read it back in, we called

	fread(&x, sizeof(struct s), 1, fp);

(which returns 1 if it succeeds).

These examples certainly seem attractive: they will result in compact data files, they will probably be quite efficient, and they are certainly simple for the programmer to write. However, data files created in this way fare quite badly when evaluated against our other criteria. They will not be human-readable; they will contain sets of inscrutable byte values which are exact copies of the memory regions used to contain the data structures. They will not be at all portable; they cannot be correctly read (at least, not with the simple calls to fread) on machines where basic types such as int have different sizes, or where the basic types are laid out differently in memory (e.g. ``big endian'' vs. ``little endian'', or different floating-point representations). They may not even be able to be read by the same code compiled under a different compiler on the same machine, since different compilers may use different sizes for integers, or lay out the fields of structures differently in memory. (The fields will always be in the order you expect, but different compilers may, for various reasons, leave different amounts of empty space or ``padding'' between certain fields.) These binary files will have no provision whatsoever for backwards or forwards compatibility; any change to the structure definition will completely change the implied format of the data file, with no hope of reading older (or newer) files. The only other benefit these files have is that if the data is for any reason sensitive, it will certainly be a bit better concealed from prying eyes.

We can get around these disadvantages of binary data files, but in so doing we'll lose many of the advantages, such as blinding efficiency or programmer convenience. If we care about data file portability or backwards or forwards compatibility, we will have to write structures one field at a time, not in one fell swoop. Furthermore, if we have an int to write, we may choose not to write it using fwrite:

	fwrite(&i, sizeof(int), 1, fp);

but rather a byte at a time, using putc:

	putc(i / 256, fp);
	putc(i % 256, fp);

In this way, we'd have precise control over the order in which the two halves of the int are being written. (We're assuming here that there's no more than two bytes' worth of data in the int, which is a safe assumption if we're portably assuming that ints can only hold up to +-32767.) When it came time to read the int back in, we might do that a byte at a time, too:

	i = getc(fp);
	i = 256 * i + getc(fp);

(We could not collapse this to i = 256 * getc(fp) + getc(fp), because we wouldn't know which order the two calls to getc would occur in.)

We might also choose to use tags to mark the various ``fields'' within our binary data file; the fields would be more likely to be byte codes such as 0x00, 0x01, and 0x02 than the character or string codes we used in the tagged text data file of the previous section.

If you do choose to use binary data files, you must open them for writing with fopen mode "wb" and for reading with "rb" (or perhaps one of the + modes; the point is that you do need the b). Remember that, in the default mode, the standard I/O functions all assume text files, and translate between \n and the operating system's end-of-line representation. If you try to read or write a binary data file in text mode, whenever your internal data happens to contain a byte which matches the code for \n, or your external data happens to contain bytes which match the operating system's end-of-line representation, they may be translated out from under you, screwing up your data.

Read sequentially: prev next up top