*How* to customise I/O

[milasudril]

From the API documentation, it appears that there is no official way for I/O customization (custom implementations of read, write and flush). I found #1370 from 2018 which is sort of related but none of it is documented.

[nlohmann]

There is no customization for output (only ideas...), but there is the concept of an input adapter. I don't know though if it is usable via a public API. I will check.

[milasudril]

I/O customization have many use-cases. Sockets, data embedded in a streams etc.

[nlohmann]

So basically you can parse anything as long as you can expose it as input iterator pairs.

Here is a simple example needlessly wrapping a string in a user-defined class allowing access via iterators. This should allow you to work with any input as long as you can express a "start here" (begin), "end here" (end), "step" (operator++), and "give me a char" (operator*()) function.

#include <iostream>
#include <iterator>
#include <string>

#include <nlohmann/json.hpp>

using json = nlohmann::json;

class MyInputWrapper {
public:
    explicit MyInputWrapper(const std::string& input)
        : data_(input) {}

    class iterator {
    public:
        using iterator_category = std::input_iterator_tag;
        using value_type = char;
        using difference_type = std::ptrdiff_t;
        using pointer = const char*;
        using reference = const char&;

        iterator() = default;
        explicit iterator(const char* ptr)
            : ptr_(ptr) {}

        reference operator*() const {
            return *ptr_;
        }

        pointer operator->() const {
            return ptr_;
        }

        iterator& operator++() {
            ++ptr_;
            return *this;
        }

        iterator operator++(int) {
            iterator tmp(*this);
            ++(*this);
            return tmp;
        }

        friend bool operator==(const iterator& lhs, const iterator& rhs) {
            return lhs.ptr_ == rhs.ptr_;
        }

        friend bool operator!=(const iterator& lhs, const iterator& rhs) {
            return lhs.ptr_ != rhs.ptr_;
        }

    private:
        const char* ptr_ = nullptr;
    };

    iterator begin() const {
        return iterator(data_.data());
    }

    iterator end() const {
        return iterator(data_.data() + data_.size());
    }

private:
    std::string data_;
};

int main()
{
    MyInputWrapper input(R"({"status":"ok"})");

    json result = json::parse(input.begin(), input.end());
    std::cout << result.dump(2) << '\n';
}

[milasudril]

What if I have no end?

[nlohmann]

Internally, it will go through this interface:

template<typename IteratorType>
class iterator_input_adapter
{
  public:
    using char_type = typename std::iterator_traits<IteratorType>::value_type;

    iterator_input_adapter(IteratorType first, IteratorType last)
        : current(std::move(first)), end(std::move(last))
    {}

    typename char_traits<char_type>::int_type get_character()
    {
        if (JSON_HEDLEY_LIKELY(current != end))
        {
            auto result = char_traits<char_type>::to_int_type(*current);
            std::advance(current, 1);
            return result;
        }

        return char_traits<char_type>::eof();
    }

    // for general iterators, we cannot really do something better than falling back to processing the range one-by-one
    template<class T>
    std::size_t get_elements(T* dest, std::size_t count = 1)
    {
        auto* ptr = reinterpret_cast<char*>(dest);
        for (std::size_t read_index = 0; read_index < count * sizeof(T); ++read_index)
        {
            if (JSON_HEDLEY_LIKELY(current != end))
            {
                ptr[read_index] = static_cast<char>(*current);
                std::advance(current, 1);
            }
            else
            {
                return read_index;
            }
        }
        return count * sizeof(T);
    }

  private:
    IteratorType current;
    IteratorType end;

    template<typename BaseInputAdapter, size_t T>
    friend struct wide_string_input_helper;

    bool empty() const
    {
        return current == end;
    }
};

As you can see, the end iterator just needs to be value different from all other iterators - it is never dereferenced, and we also do not do any arithmetic on it.

[mutluit]

I tried to adapt the above posted MyInputWrapper class to read blockwise from a gzip compressed file (file.gz),
but changing/resetting the pointers after processing the initial block crashes the library (exception).
Does the library need all of the data be available in memory before starting the parsing?
A basic example that reads blockwise (even bytewise) via the gzread(), or any other file read function would be useful.

Update:
Solved! By using the gzstream library --> see the latest postings in the other thread: