Chunked image encoding

[cklein05]

As you may know, we are implementing a fully featured WMS server based on Node Mapnik. It turns out, that most of Mapnik's incredible rendering speed, compared to many other WMS servers, has only little effect with slow network connections. For example, many typical map requests are served about 1.5 to 2.5 times faster than by GeoServer on a local 1 GB network (not to mention localhost requests). However, this factor tends to fall to 1.0 or even 0.8 on typical Internet links.

One reason may be the missing ability to implement chunked HTTP Transfer-Encoding as GeoServer does. Here, the final map image is encoded to, for example, PNG format in chunks. Each chunk is sent down the wire as soon as it is available. With Node Mapnik, the whole rendered image is encoded into one single chunk by one of the save_to_string methods. So, with GeoServer, there is a level of parallelism by overlapping image encoding and sending, which actually brings some extra speed compared to our implementation.

So, I'm asking for a new encodeChunked method for the Node Mapnik Image object. Likely the simplest implementation works with a callback function that gets called for each chunk available. Typically, one would specify the desired chunk size as an argument when the function is called. The callback function gets invoked once for each chunk, which is passed as a Node.js Buffer object. Additionally, a boolean flag last_chunk is required to signal which one is the last chunk.

Since I'm no C++ expert, I currently have no idea how to implement this on the C++ side. This seems to require a stream/buffer, that triggers an event/callback after N bytes have been written to the stream. Maybe there is some Boost template for that? Also, I have no idea whether it's better to implement this as a sync or async method (from a performance point of view), since for each chunk some (expensive?) inter-thread access seems to be required. Actually, this new method could as well be encodeChunkedSync.

On the Javascript side this could simply look like (heavily shortened):

// encode and send image in 512 byte chunks
myImage.encodeChunked(512, function(err, buffer, last_chunk) {
    response.write(buffer);
    if (last_chunk) {
        response.end();
    }
}();

[cklein05]

Any ideas on that?

[talaj]

This does not necessarily need to be a part of Node Mapnik. There might be some Node module that can encode PNG this way giving it raw data from img.data().

[flippmoke]

@cklein05 I believe that you need not do anything on the node side to do this, but rather if you use apache or nginx they will be able to control this completely (but this is just my hunch). Overall, I do not feel that node-mapnik should have any responsibilities in doing this and so I do not think we should add this to the library.

With Node Mapnik, the whole rendered image is encoded into one single chunk by one of the save_to_string methods. So, with GeoServer, there is a level of parallelism by overlapping image encoding and sending, which actually brings some extra speed compared to our implementation.

In node-mapnik, the slowest part of the processing should be simply rendering the image. save_to_string simply takes an uncompressed image and compresses it into a buffer for saving. We can not split up processing of the image into partial sections (which would be very complex) and frankly not worth it.

My best suggestion is this:

If speed is a requirement, I would suggest you investigate another solution besides WMS. WMS is not a fast format for many reasons, one of them being caching.

[cklein05]

@flippmoke OK, I have to accept this. However,

I believe that you need not do anything on the node side to do this, but rather if you use apache or nginx they will be able to control this completely (but this is just my hunch).

Of course, Node.js as well as both Apache and NGINX do support chunked encoding perfectly. However, this is not of much performance benefit, if we must wait until Mapnik has encoded the whole image. The idea was to overlap/parallelize image encoding and streaming. Imagine to replace the destination string in save_to_string with the Node's response stream. That is real streaming and is actually another way to implement this (but may be complicated, since it requires to work with Node's response stream from C++).

So, an approximation of the above may have been these buffers sent back by a callback function. The idea was not to split up image processing/encoding, but to split up or buffer the output stream.

[talaj]

On the other hand mapnik::save_to_stream() exists in the public interface so it should be simple to expose it in Node.

[flippmoke]

mapnik::save_to_string() already utilizes this underneath. So its not a stretch at all to do this, and perhaps we could use the NaN::Buffer() object here. However, this will probably be a good amount of wrapper code that we might need to add and maintain.

template <typename T>
MAPNIK_DECL std::string save_to_string(T const& image,
                                       std::string const& type)
{
    std::ostringstream ss(std::ios::out|std::ios::binary);
    save_to_stream(image, ss, type);
    return ss.str();
}

I am not sold however, that this would actually lower memory footprint much or at all, if we preserve the life of the std::ostringstream() we simply hold that memory a bit longer, at least with the current method this will be dropped once the std::string is created and passed to NaN for ownership. It could possibly be a bit faster, but it might be far easier to simply provide a buffer around the str output from this method in node. I am somewhat concerned about how this Buffer would be used as well, because we would have to figure out how to maintain the std::ostringstream object for the lifetime of the buffer in node.

In short, I don't think its impossible, but could be hard to implement properly.

[talaj]

I am not sold however, that this would actually lower memory footprint much or at all

The goal is not to save memory, rather this:

level of parallelism by overlapping image encoding and sending, which actually brings some extra speed compared to our implementation.

For this purpose, I would create a new std::ostream based class which would call a javascript callback on every write or when some intermediate buffer is full.

[talaj]

Something like this:

#include <array>
#include <iostream>

template<typename Callback, std::size_t Size, class Char = char>
class callback_streambuf : public std::basic_streambuf<Char>
{
public:
    using base_type = std::streambuf;
    using char_type = typename base_type::char_type;
    using int_type = typename base_type::int_type;

    callback_streambuf(Callback callback)
        : buffer_{},
          callback_(callback)
    {
        base_type::setp(buffer_.begin(), buffer_.end());
    }

protected:
    int sync()
    {
        bool ok = callback_(base_type::pbase(),
                            base_type::pptr() - base_type::pbase());
        base_type::setp(buffer_.begin(), buffer_.end());
        return ok ? 0 : -1;
    }

    int_type overflow(int_type ch)
    {
        int ret = sync();
        if (ch == base_type::traits_type::eof())
        {
            return ch;
        }
        base_type::sputc(ch);
        return ret == 0 ? 0 : base_type::traits_type::eof();
    }

private:
    std::array<char_type, Size> buffer_;
    Callback callback_;
};

struct cout_callback
{
    template<class Char, class Size>
    bool operator()(const Char* buffer, Size size) const
    {
        std::cout << "CALLBACK: '";
        std::cout.write(buffer, size);
        std::cout << "' (" << size << ")" << std::endl;
        return true;
    }
};

main()
{
    cout_callback callback;
    callback_streambuf<cout_callback, 10> streambuf(callback);
    std::ostream stream(&streambuf);
    stream << "Lorem ipsum dolor sit amet, "
        "consectetur adipiscing elit, "
        "sed do eiusmod tempor incididunt ut "
        "labore et dolore magna aliqua.";
    stream.flush();
}

 $ g++ -std=c++14 callback_stream.cpp -o callback_stream &&  ./callback_stream
CALLBACK: 'Lorem ipsu' (10)
CALLBACK: 'm dolor si' (10)
CALLBACK: 't amet, co' (10)
CALLBACK: 'nsectetur ' (10)
CALLBACK: 'adipiscing' (10)
CALLBACK: ' elit, sed' (10)
CALLBACK: ' do eiusmo' (10)
CALLBACK: 'd tempor i' (10)
CALLBACK: 'ncididunt ' (10)
CALLBACK: 'ut labore ' (10)
CALLBACK: 'et dolore ' (10)
CALLBACK: 'magna aliq' (10)
CALLBACK: 'ua.' (3)

[cklein05]

@talaj That's it. That's what I was asking for... 👍 Thank you for investigating and creating that descriptive sample application as well as the std::ostream based class.

It's still to check whether this actually works with an asynchronous NaN method or if this must be implemented synchronously.

[cklein05]

Some remarks:

Template parameter std::size_t Size should be an argument provided by the corresponding NaN method. I believe, you can't use variables inside <...> but have to pass the desired buffer size to the constructor instead:

int chunk_size = 10;      /* from NaN method */
callback_streambuf<cout_callback> streambuf(callback, chunk_size );

A boolean parameter indicating the last chunk should be passed to the callback function, since the last chunk is not necessarily smaller than the desired regular chunk size (requires to detect this stream's eof state). If this new encoding method actually works asynchronously, this seems required in order to give the caller a chance to properly end the request.

[talaj]

I'm sorry for not responding, I have vacation. I will try to take a look on it later this week.

Template parameter std::size_t Size should be an argument provided by the corresponding NaN method.

It depends on whether you want to set size of the intermediate buffer in runtime. Probably yes, it was just an example.

A boolean parameter indicating the last chunk should be passed to the callback function, since the last chunk is not necessarily smaller than the desired regular chunk size

I'm also thinking about distinct callback function or indication by zero chunk_size.

[cklein05]

Never mind. That's all voluntary and there's no rush. Enjoy your vacation :)

[talaj]

@cklein05 Take a look on #799.