Binary iostreams in C++
In the past, I’ve reached for the C stdio library when I need to do binary file I/O in C++. Why? Well, much of the C++ iostreams interface is designed for character- rather than byte-based I/O:
- I/O is generally done through the << and >> operators, which format types to a character representation.
- Stream objects can be “imbued” with C++ locale objects, affecting various aspects of I/O, including how numbers are formatted and how bytes are translated into characters.
- The ios::binary flag only affects line-ending conversion, not character conversion.
- Stream objects’ locales are inherited from the global locale.
I figured this would make binary I/O difficult, but I never researched it.
The opportunity to figure this out has just come up. I wrote a class which builds a tar file on the fly by writing it to a generic iterator. Its AddFile() also reads the contents of the file to store from an iterator.
I wrote this under test without hitting the disk, but the last step is to supply binary file input and output iterators. After successfully implementing a binary file input iterator, I thought, “It’s hard to believe that Boost doesn’t already have these.” I posted on the list and quickly received an answer:
Use the standard streambuf iterators, istreambuf_iterator and ostreambuf_iterator.
Can we do that? What about locales and character set translation?
So finally I looked into this locale thing. It seems the ISO C++ specification guarantees that codecvt<>, which is the facet of the locale that translates characters, performs no translation in the case of converting from a narrow char buffer to a narrow char buffer. The specification considers this a “degenerate case.”
Conclusion: The C++ language specification guarantees that narrow iostreams will pass through untranslated bytes regardless of the current locale.
Interesting insight. One thing that is worth noting too is that the standard stream interface set up correctly can even allow you to do generic transformations using the iterator interfaces. What doesn’t feel right is the current interface provided by the standard. Maybe when the next standard comes up and we have better standard library implementations from vendors we might not have to worry about these things.
Also, FWIW, there is no standard way to deal with C file handles through the C++ Stream interfaces — which might be a good thing, but really hard for those who are comfortable with the C interface already.
Well, you can apparently write your own streambuf that uses a FILE*, though I’ve never done it. I’m curious whether this isn’t done in general because most people find it scary, or because it’s actually hard. I should figure this out one of these days.
After reviewing the specification for the above, it doesn’t seem as scary or hard as I once thought, though it is non-obvious.