Halide（三）Buffer与cv::Mat的相互转换

目前OpenCV的NN模块中已经有Halide一部分代码，可以通过 net.setPreferableBackend(DNN_BACKEND_HALIDE); 告诉OpenCV NN引擎尽可能使用Halide进行计算。除非以后OpenCV完全实现Halide的运行机制，否则在图像处理任务中，Halide与OpenCV协同工作是不可避免的。这篇博客主要介绍Halide的图像类型和OpenCV图像类型的相互转化。

一、代码示例

代码胜千言：

#include <stdio.h>
#include <Halide.h>
#include <halide_image_io.h>
#include <opencv2/opencv.hpp>


int main(int argc, char* argv[]) {
#ifdef HAVE_OPENCV
    cv::Mat image = cv::imread("images/rgb.png");
    cv::cvtColor(image, image, cv::COLOR_BGR2RGB);
    cv::imshow("in", image);
    cv::waitKey(10);
    Halide::Buffer<uint8_t> input(Halide::Buffer<uint8_t>::make_interleaved(image.data, image.cols, image.rows, image.channels()));

    // If the OpenCV matrix has padding between the rows, the longer form is
    // a halide_dimension_t is the min coordinate, the extent, and then the stride/step in that dimension.
    //halide_dimension_t shape[3] = { {0, image.cols, image.step1(1)},
    //                           {0, image.rows, image.step1(0)},
    //                           {0, image.channels(), 1}
    //};
    //Halide::Buffer<uint8_t> buffer(image.data, 3, shape);
#else
    Halide::Buffer<uint8_t> input = Halide::Tools::load_image("images/rgb.png");
#endif
    Halide::Func brighter;
    Halide::Var x, y, c;
    Halide::Expr value = input(x, y, c);
    value = Halide::cast<float>(value);
    value = value * 1.5f;
    value = Halide::min(value, 255.0f);
    value = Halide::cast<uint8_t>(value);
    brighter(x, y, c) = value;

    Halide::Buffer<uint8_t> output =
        brighter.realize(input.width(), input.height(), input.channels());

#ifdef HAVE_OPENCV
    cv::Mat image_out = cv::Mat::zeros(output.height(), output.width(), CV_8UC3);
    for (int i = 0; i < output.height(); ++i) {
        for (int j = 0; j < output.width(); ++j) {
            for (int n = 0; n < output.channels(); ++n) {
                image_out.at<uchar>(i, j*output.channels()+n) = (uchar)output(j, i, n);
            }
        }
    }
    cv::imshow("out", image_out);
    cv::waitKey(10);
#else
    Halide::Tools::save_image(output, "brighter.png");
#endif

    return 0;
}

代码上传到了这个仓库：

二、OpenCV的cv::Mat转化为Halide Buffer

这个直接用Halide Buffer的构造就可以了

Halide::Buffer<uint8_t> input(Halide::Buffer<uint8_t>::make_interleaved(image.data, image.cols, image.rows, image.channels()));

如果数据buffer本身有ROI或者stride之类的存储规则，那么也有初始化api可以使用：

// If the OpenCV matrix has padding between the rows, the longer form is 
// a halide_dimension_t is the min coordinate, the extent, and then the stride/step in that dimension. 
halide_dimension_t shape[3] = { {0, image.cols, image.step1(1)}, 
 {0, image.rows, image.step1(0)}, 
 {0, image.channels(), 1} 
}; 
Halide::Buffer<uint8_t> buffer(image.data, 3, shape);

博客最后会对Halide::Buffer初始化api进行一下总结

三、Halide Buffer转化为OpenCV的cv::Mat

如果时单通道图可以memcpy，但是多通道图没找到写起来简单的方法，因为Halide Buffer里面数据存储顺序和cv::Mat的数据存取规则相距甚远，只有逐个pixel去取了

cv::Mat image_out = cv::Mat::zeros(output.height(), output.width(), CV_8UC3); 
for (int i = 0; i < output.height(); ++i) {
 for (int j = 0; j < output.width(); ++j) {
  for (int n = 0; n < output.channels(); ++n) {
    image_out.at<uchar>(i, j*output.channels()+n) = (uchar)output(j, i, n); 
  }
 }
}

四、Halide Buffer初始化方式总结

参考链接：HalideにおけるバッファHalide::Buffer<T>の操作（HalideとOpenCVの相互変換）

首先简单看下Buffer有哪些属性：

Func grad;
grad(x, y, c) = cast<uint8_t>((x + y) % 256);
Halide::Buffer<uint8_t> output = realize(256,256);

output.width();//宽
output.height();//高
output.channels()//通道
output.dimensions()//维度
output2.size_in_bytes()//字节（长ｘ宽ｘ通道ｘsizeof(T)）．

uchar* data = output2.begin();//RAW数据的开头
uchar* data = output2.data();//RAW数据的开头，和上面是一样的
uchar* data = output2.end();//RAW数据的结尾

下面开始看Halide::Buffer的初始化方法：

1. 使用Halide::Buffer结构初始化

构造函数：

Halide::Buffer<uint8_t> input(Halide::Buffer<uint8_t>::make_interleaved(image.data, image.cols, image.rows, image.channels()));

2. 初始化时只指定buffer size，而不给出数据

void halideInitTest2(Mat& dest)
{
    Var x, y, c;
    Func grad;
    grad(x, y, c) = cast<uint8_t>((x + y) % 256);
    Halide::Buffer<uint8_t> output(dest.cols, dest.rows, dest.channels());

    grad.realize(output);

    uchar* data = output.data();
    memcpy(dest.data, data, output.size_in_bytes());
}

 

3. 根据数据大小和数据头部指针初始化

void halideInitTest3(Mat& dest)
{
    Var x, y, c;
    Func grad;
    grad(x, y, c) = cast<uint8_t>((x + y) % 256);
    //dest.ptr<uchar>(0)はOpenCVの画像データの先頭ポインタ
    Halide::Buffer<uint8_t> output(dest.ptr<uchar>(0), dest.cols, dest.rows, dest.channels());

    grad.realize(output);
}

 

4. 先申请内存，再逐个pixel初始化

void halideInitTest6(Mat& src, Mat& dest)
{
    Var x, y, c;
    Halide::Buffer<uint8_t> input(src.cols, src.rows, src.channels());
    for (int j = 0; j < src.rows; j++)
    {
        for (int i = 0; i < src.cols; i++)
        {
            for (int n = 0; n < src.channels(); n++)
            {
                input(i, j, n) = src.at<uchar>(j, i*src.channels() + n);
            }
        }
    }
}

 

OK，See You Next Chapter!