Cristian Adam

C++ I/O Benchmark

In this post I will talk about copying files. I will read one file in chunks of 1MB and write it to another file.

C++ provides three cross platform APIs for I/O (input/output):

  1. C FILE API (fopen, fread, fwrite)
  2. C++ API (std::ifstream, std::ofstream)
  3. POSIX API (open, read, write)

The POSIX API requires a bit of #ifdef-ing to get it working cross platform, but it’s not that scary.

Reading and writing 1 MB of data should work more or less as fast for all APIs, right?

I have run the benchmark on my SSD powered Lenovo Core i7 laptop running Windows 10 and Kubuntu 15.10, and on a SSD powered Raspberry PI2 running the latest Raspbian.

The code for the benchmark is below:

Code

#include <stdio.h>
#include <fcntl.h>

#if defined(__unix__) || defined (__CYGWIN__)
    #include <unistd.h>
#else
    #include <io.h>
#endif

#ifndef O_BINARY
    #define O_BINARY 0
#endif

#include <chrono>
#include <iostream>
#include <functional>
#include <fstream>
#include <map>
#include <string>
#include <vector>
 
using namespace std::chrono;

struct measure
{
    template<typename F, typename ...Args>
    static std::chrono::milliseconds::rep ms(F func, Args&&... args)
    {
        auto start = system_clock::now();
        func(std::forward<Args>(args)...);
        auto stop = system_clock::now();
        
        return duration_cast<milliseconds>(stop - start).count();
    }
};
 
void testCFileIO(const char* inFile, const char* outFile, std::vector<char>& inBuffer)
{
    FILE* in = ::fopen(inFile, "rb");
    if (!in)
    {
        std::cout << "Can't open input file: " << inFile << std::endl;
        return;
    }
 
    FILE* out = ::fopen(outFile, "wb"); 
    if (!out)
    {
        std::cout << "Can't open output file: " << outFile << std::endl;
        return;
    }
 
    fseek(in, 0, SEEK_END);
    size_t inFileSize = ::ftell(in);
    fseek(in, 0, SEEK_SET);
   
    for (size_t bytesLeft = inFileSize, chunk = inBuffer.size(); bytesLeft > 0; bytesLeft -= chunk)
    {
        if (bytesLeft < chunk)
        {
            chunk = bytesLeft;
        }
        
        ::fread(&inBuffer[0], 1, chunk, in);
        ::fwrite(&inBuffer[0], 1, chunk, out);
    }
  
    ::fclose(out);
    ::fclose(in);
}

void testCppIO(const char* inFile, const char* outFile, std::vector<char>& inBuffer)
{
    std::ifstream in(inFile, std::ifstream::binary);
    if (!in.is_open())
    {
        std::cout << "Can't open input file: " << inFile << std::endl;
        return;
    }
 
    std::ofstream out(outFile, std::ofstream::binary);
    if (!out.is_open())
    {
        std::cout << "Can't open output file: " << outFile << std::endl;
        return;
    }
 
    in.seekg(0, std::ifstream::end);
    size_t inFileSize = in.tellg();
    in.seekg(0, std::ifstream::beg);
   
    for (size_t bytesLeft = inFileSize, chunk = inBuffer.size(); bytesLeft > 0; bytesLeft -= chunk)
    {
        if (bytesLeft < chunk)
        {
            chunk = bytesLeft;
        }
        
        in.read(&inBuffer[0], chunk);
        out.write(&inBuffer[0], chunk);
    }
}
  
void testPosixIO(const char* inFile, const char* outFile, std::vector<char>& inBuffer)
{
    int in = ::open(inFile, O_RDONLY | O_BINARY);
    if (in < 0)
    {
        std::cout << "Can't open input file: " << inFile << std::endl;
        return;
    }

    int out = ::open(outFile, O_CREAT | O_WRONLY | O_BINARY, 0666);
    if (out < 0)
    {
        std::cout << "Can't open output file: " << outFile << std::endl;
        return;
    }
 
    size_t inFileSize = ::lseek(in, 0, SEEK_END);
    ::lseek(in, 0, SEEK_SET);
   
    for (size_t bytesLeft = inFileSize, chunk = inBuffer.size(); bytesLeft > 0; bytesLeft -= chunk)
    {
        if (bytesLeft < chunk)
        {
            chunk = bytesLeft;
        }

        ::read(in, &inBuffer[0], chunk);
        ::write(out, &inBuffer[0], chunk);
    }

    ::close(out);
    ::close(in);
}
 
int main(int argc, char* argv[])
{
    std::vector<std::string> args(argv, argv + argc);
    if (args.size() != 4)
    {
        std::cout << "Usage: " << args[0] << " copy_method (c, posix, c++) in_file number_of_times" << std::endl;
        return 1;
    }

    typedef std::map<std::string, std::function<void (const char*, const char*, std::vector<char>&)>> FuncMap;
    FuncMap funcMap { {"c", testCFileIO}, {"posix", testPosixIO}, {"c++", testCppIO}};

    auto it = funcMap.find(args[1]);
    if (it != funcMap.end())
    {       
        std::vector<char> inBuffer(1024 * 1024);
        
        auto dest = args[2] + ".copy";
        const auto times = std::stoul(args[3]);
        
        milliseconds::rep total = 0;
        for (unsigned int i = 0; i < times; ++i)
        {
            total += measure::ms(it->second, args[2].c_str(), dest.c_str(), inBuffer);
            ::unlink(dest.c_str());
        }
        std::cout << "Average " << args[1] << " I/O took: " << total / double(times) << "ms" << std::endl;
    }
    else
    {
        std::cout << "Not supported copy method: " << args[1] << std::endl;
    }    
}

I have used Boost 1.60 zip package file (125 MB) as the file to copy around.

My test script looks like this:

@echo off
test_io.exe c boost_1_60_0.zip 10 > nul
test_io.exe c boost_1_60_0.zip 100
test_io.exe posix boost_1_60_0.zip 10 > nul
test_io.exe posix boost_1_60_0.zip 100
test_io.exe c++ boost_1_60_0.zip 10 > nul
test_io.exe c++ boost_1_60_0.zip 100

For the Linux variant just replace @echo off with /bin/bash, > nul with /dev/null and the line endings smile

Windows 10

I have tested Visual C++ 2013 32 and 64 bit, Clang 3.7.1 with Visual C++ 2013 32 and 64 bit, MinGW 4.9.2 32 bit from Qt 5.6 distribution, MinGW 5.3.0 64 bit from Nuwen, Cygwin GCC 5.3.0 64 bit, and Cygwin Clang 3.7.1 64 bit.

Visual C++ and Clang compilation line was cl /O2 /EHsc test_io.cpp, for MinGW I had g++ -O2 test_io.cpp -o test_io -std=c++11, and for Cygwin Clang clang -O2 test_io.cpp -o test_io -std=c++11 -lstdc++.

I have also disabled the real time protection from Windows Defender.

The results are below:

Compiler C FILE POSIX C++
Visual C++ 2013 32 111.8 ms 111.8 ms 320.91 ms
Visual C++ 2013 64 111.44 ms 109.74 ms 309.27 ms
Visual C++ 2015 32 107.22 ms 107.47 ms 315.7 ms
Visual C++ 2015 64 109.57 ms 106.87 ms 305.6 ms
Clang 3.7.1 32 101.43 ms 101.38 ms 446.26 ms
Clang 3.7.1 64 101.71 ms 99.5 ms 460.8 ms
MinGW 4.9.2 32 104.7 ms 108.78 ms 110.67 ms
MinGW 5.3.0 Nuwen 110.34 ms 107.48 ms 110.83 ms
Cygwin GCC 5.3.0 64 124.91 ms 108.36 ms 181.32 ms
Cygwin Clang 3.7.1 64 121.74 ms 105.91 ms 181.65 ms

Surprisingly only MinGW GCC provides the same performance for all three APIs.

Visual C++ and Clang using Visual C++’s CRT library has a 2.87x, respectively a 4.39x slower C++ API than C or POSIX API !!!

On Cygwin the C and C++ APIs are slower than the POSIX API.

It is very interesting to know why GCC’s libstdc++ behaves on Cygwin slower than on MinGW!

Kubuntu 15.10

I have booted my Linux distribution and ran the same test there, results below:

Compiler C FILE POSIX C++
GCC 5.2.1 64 109.17 ms 105.85 ms 107.23ms
Clang 3.6.2 64 110.26 ms 105.72 ms 107.71 ms

Nothing to see here but consistency! smile

Raspberry PI2

Thanks to this test I have finally managed set up my Raspberry PI2 smile

I had a bit of fun making the USB SSD hard drive to work with Raspberry PI2, increasing partition size, and so on.

The results of the test a below:

Compiler C FILE POSIX C++
GCC 4.9.2 1277.07 ms 1239.34 ms 1238.49 ms
Clang 3.5.0 1282.46 ms 1262.77 ms 1284.25 ms

The C++ API for GCC was the fastest! sunglasses

Interesting to see that Raspberry PI2 was ~12 times slower than my Core i7 laptop.

Conclusion

The POSIX API provides the best results on all platforms tested!

Comments