Stuff ► Code Snippets ► Data Packing (C++)

You are viewing an old version of the ionflux.org website! Please visit the new version here.

Data Packing (C++)

For writing data to files or transmitting it over a network, it may be necessary to write functions which pack the data from its native representation in memory into a sequence of bytes. Fortunately, it is not necessary to write a function for each type separately, thanks to C++ templates. The following code snippets illustrate how data can be packed into a string and unpacked from a string without writing specialized functions.

The only requirement in order for this to work is that the length of the data can be obtained using the sizeof() operator. But even if this is not the case, it is fairly easy to implement a specialized override of the function which uses the function template to do most of the actual work. See below for an example which packs standard C++ strings so they can be restored later from the byte sequence.

Pack Data

The following function template packs data into a string. This works for any type the size of which can be determined using the sizeof() operator. A parameter is provided which allows the caller of the function to specify whether data should be appended to the target buffer, or whether the target buffer should just be overwritten.

#include <string> template<class T> void pack(const T& source, std::string& target, bool append = true) { static const unsigned int dataSize = sizeof(T); std::string buffer; buffer.assign(dataSize, '0'); const unsigned char* rawData = reinterpret_cast<const unsigned char*>(&source); for (unsigned int i = 0; i < dataSize; i++) buffer[i] = rawData[dataSize - i - 1]; if (append) target.append(buffer); else target = buffer; }

Unpack Data

The following code can be used to unpack data from a string which has been created by the pack() function. A parameter is provided to specify an offset from which to start unpacking. This is useful if multiple data records are packed into the same buffer. The function also returns the new offset, which can then be used to unpack data starting from the end of the sequence of bytes which has already been unpacked.

#include <string> template<class T> int unpack(const std::string& source, T& target, int offset = 0) { static const unsigned int dataSize = sizeof(T); if ((offset + dataSize) > source.size()) return -1; unsigned char* rawData = reinterpret_cast<unsigned char*>(&target); for (unsigned int i = 0; i < dataSize; i++) rawData[dataSize - i - 1] = source[i + offset]; return offset + dataSize; }

Pack String

This code uses the pack() function template to pack the length of a string and the string itself into a buffer. This data can then be used to restore the string without any additional information. It is an example of how to use the function template defined above to implement specialized packing functions for arbitrary types.

#include <string> void pack(const std::string& source, std::string& target, bool append = true) { std::string buffer; pack(static_cast<unsigned int>(source.size()), buffer, false); buffer.append(source); if (append) target.append(buffer); else target = buffer; }

Unpack String

This is the function for unpacking strings from a sequence of bytes. It relies on the unpack() function template specified above to obtain the length of the string before the actual data is extracted from the source buffer.

#include <string> int unpack(const std::string& source, std::string& target, int offset = 0) { unsigned int dataSize = 0; offset = unpack(source, dataSize, offset); if ((offset == -1) || ((offset + dataSize) > source.size())) return -1; target = source.substr(offset, dataSize); offset += dataSize; return offset; }