OpenCV bindings for luajit and lua 5.1/5.2/5.3/5.4.
The aim is to make it as easy to use as opencv-python.
Therefore the OpenCV documentation should be the reference.
- Installation
- Examples
- Running examples
- Hosting you own binary rocks
- Keyword arguments
- How to translate python/c++ code
- Lua Gotchas
- Switching from sol2 to plain-c
Prebuilt binaries are available for LuaJIT 2.1 and Lua 5.1/5.2/5.3/5.4, and only on Windows and Linux.
- Install CMake >= 3.25
- Install Git
- Install LuaRocks
- Install NodeJS
- Install Python
- Install Visual Studio 2022 >= 17.7.2 with .NET Desktop and C++ Desktop
- In your windows search, search and open the
x64 Native Tools Command Prompt for VS 2022
- Install CMake >= 3.25
- Install LuaRocks
- Install Ninja
- Install NodeJS
- Install needed packages (see below for you corresponding distruction).
- Tell luarocks to use Ninja as cmake generator
luarocks config --scope project cmake_generator Ninja
sudo apt install -y build-essential curl git libavcodec-dev libavformat-dev libdc1394-dev \
libjpeg-dev libpng-dev libreadline-dev libswscale-dev libtbb-dev \
pkg-config python3-pip python3-venv qtbase5-dev unzip wget zip
sudo apt install -y libtbbmalloc2 || apt install -y libtbb2
sudo dnf install -y curl gcc gcc-c++ git \
libjpeg-devel libpng-devel readline-devel make patch tbb-devel \
libavcodec-free-devel libavformat-free-devel libdc1394-devel libswscale-free-devel \
pkg-config python3-pip qt5-qtbase-devel unzip wget zip
sudo dnf install -y curl gcc-toolset-12-gcc gcc-toolset-12-gcc-c++ git \
libjpeg-devel libpng-devel readline-devel make patch tbb-devel \
pkg-config python3.12-pip qt5-qtbase-devel unzip wget zip && \
sudo config-manager --set-enabled powertools && \
sudo dnf install -y epel-release && \
sudo dnf install -y https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-8.noarch.rpm
sudo dnf install -y https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-8.noarch.rpm && \
sudo dnf update -y && \
sudo dnf install -y ffmpeg-devel && \
source /opt/rh/gcc-toolset-12/enable
sudo dnf install -y curl gcc gcc-c++ git \
libjpeg-devel libpng-devel readline-devel make patch tbb-devel \
pkg-config python3-pip qt5-qtbase-devel unzip wget zip && \
sudo config-manager --set-enabled crb && \
sudo dnf install -y epel-release && \
sudo dnf install -y https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-9.noarch.rpm
sudo dnf install -y https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-9.noarch.rpm && \
sudo dnf update -y && \
sudo dnf install -y libavcodec-free-devel libavformat-free-devel libdc1394-devel libswscale-free-devel
I recommend you to try installing the prebuilt binary with
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua
Or to specify the target lua version with one of the following commands
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0luajit2.1
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0lua5.4
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0lua5.3
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0lua5.2
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0lua5.1
Those prebuilt binaries should work on Windows and many linux distributions and have been tested on:
- Windows 11
- Ubuntu 20.04
- Ubuntu 22.04
- Ubuntu 24.04
- Debian 10
- Debian 11
- Debian 12
- Fedora 38
- Fedora 39
- Fedora 40
- Almalinux 8
- Almalinux 9
If none of the above works for you, then install the source rock with
luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0
On Windows, the lua_modules modules should be added to PATH as show with luarocks path
set "PATH=%LUA_MODULES%\bin;%APPDATA%\luarocks\bin;%PATH%"
LUA_MODULES
is a variable pointing to your lua_modules folder.
For example, in your lua project, initialized with luarocks init
, modify the file lua.bat
and after the line with set "LUAROCKS_SYSCONFDIR=
, add
set LUA_MODULES=%~dp0lua_modules
set "PATH=%LUA_MODULES%\bin;%APPDATA%\luarocks\bin;%PATH%"
local opencv_lua = require("opencv_lua")
local cv = opencv_lua.cv
local img = cv.imread("lena.jpg")
cv.imshow("Image", img)
cv.waitKey()
cv.destroyAllWindows()
local opencv_lua = require("opencv_lua")
local cv = opencv_lua.cv
local int = opencv_lua.math.int
local camId = 0
local cap = cv.VideoCapture(camId)
if not cap:isOpened() then error("!>Error: cannot open the camera " .. camId) end
local CAP_FPS = 60
local CAP_SPF = int(1000 / CAP_FPS)
cap:set(cv.CAP_PROP_FRAME_WIDTH, 1280)
cap:set(cv.CAP_PROP_FRAME_HEIGHT, 720)
cap:set(cv.CAP_PROP_FPS, CAP_FPS)
while true do
-- Without this, memory grows indefinitely
collectgarbage()
local start = cv.getTickCount()
local read, frame = cap:read()
if not read then
io.stderr:write("!>Error: cannot read the camera.\n")
end
local fps = cv.getTickFrequency() / (cv.getTickCount() - start)
-- Flip the image horizontally to give the mirror impression
local frame = cv.flip(frame, 1)
cv.putText(frame, string.format("FPS : %.2f", fps), { 10, 30 }, cv.FONT_HERSHEY_PLAIN, 2, {255, 0, 255}, 3)
cv.imshow("capture camera", frame)
local key = cv.waitKey(CAP_SPF)
if key == 0x1b or key == string.byte("q") or key == string.byte("Q") then break end
end
cv.destroyAllWindows()
local opencv_lua = require("opencv_lua")
local cv = opencv_lua.cv
local img = cv.imread("lena.jpg")
local angle = 20
local scale = 1
local size = { img.width, img.height }
local center = { img.width / 2, img.height / 2 }
local M = cv.getRotationMatrix2D(center, -angle, scale)
local rotated = cv.warpAffine(img, M, size)
cv.imshow("Rotation", rotated)
cv.waitKey()
cv.destroyAllWindows()
local opencv_lua = require("opencv_lua")
local cv = opencv_lua.cv
local img = cv.imread("pic1.png")
local img_grey = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
local ret, thresh = cv.threshold(img_grey, 100, 255, cv.THRESH_BINARY)
local contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
print("Found " .. #contours .. " contours")
cv.drawContours(img, contours, -1, { 0, 0, 255 }, 2)
cv.imshow("Image", img)
cv.waitKey()
cv.destroyAllWindows()
local opencv_lua = require("opencv_lua")
local cv = opencv_lua.cv
--[[
Sources:
https://docs.opencv.org/4.x/d4/dc6/tutorial_py_template_matching.html
--]]
local img_rgb = cv.imread("mario.png")
assert(not img_rgb:empty(), "file could not be read, check with os.path.exists()")
local img_gray = cv.cvtColor(img_rgb, cv.COLOR_BGR2GRAY)
local template = cv.imread("mario_coin.png", cv.IMREAD_GRAYSCALE)
assert(not template:empty(), "file could not be read, check with os.path.exists()")
local h, w = template.height, template.width
local res = cv.matchTemplate(img_gray, template, cv.TM_CCOEFF_NORMED)
local threshold = 0.8
-- transform into an lua table for faster processing in lua
local rows, cols = res.rows, res.cols
res = res:table()
for j = 1, rows do
local y = j - 1
for i = 1, cols do
local x = i - 1
if res[j][i] >= threshold then
cv.rectangle(img_rgb, { x, y }, { x + w, y + h }, { 0, 0, 255 }, 2)
end
end
end
cv.imshow("res.png", img_rgb)
cv.waitKey()
cv.destroyAllWindows()
More expamples can be found in the samples directory.
All the examples in the samples directory can be run by folling theses instructions.
- Install Git
- Install NodeJS
- Install Visual Studio 2022 >= 17.7.2 with '.NET desktop development' and 'Desktop development with C++'
- In your windows search, search and open the
x64 Native Tools Command Prompt for VS 2022
- Install NodeJS
- Install CMake >= 3.25
- Install Ninja
- Install needed packages:
- Debian, Ubuntu:
sudo apt install -y curl g++ gcc git libgl1 libglib2.0-0 libreadline-dev libsm6 libxext6 make python3-pip python3-venv unzip wget
- Fedora, Almalinux 9:
sudo dnf install -y curl gcc gcc-c++ git glib2 readline-devel libglvnd-glx libSM libXext make patch python3-pip unzip wget
- Almalinux 8:
sudo dnf install -y curl gcc gcc-c++ git glib2 readline-devel libglvnd-glx libSM libXext make patch python3.12-pip unzip wget
- Debian, Ubuntu:
git clone --depth 1 --branch 4.10.0 https://github.com/opencv/opencv.git
git clone --depth 1 --branch v0.0.5 https://github.com/smbape/lua-opencv.git
cd lua-opencv
@REM build.bat "-DLua_VERSION=luajit-2.1" --target luajit --install
@REM available versions are 5.1, 5.2, 5.3, 5.4
build.bat "-DLua_VERSION=5.4" --target lua --install
build.bat "-DLua_VERSION=5.4" --target luarocks
@REM luarocks\luarocks.bat install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0luajit2.1
luarocks\luarocks.bat install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua
luarocks\luarocks.bat install --deps-only samples\samples-scm-1.rockspec
npm ci
node scripts\test.js --Release
git clone --depth 1 --branch 4.10.0 https://github.com/opencv/opencv.git
git clone --depth 1 --branch v0.0.5 https://github.com/smbape/lua-opencv.git
cd lua-opencv
# ./build.sh "-DLua_VERSION=luajit-2.1" --target luajit --install
# available versions are 5.1, 5.2, 5.3, 5.4
./build.sh "-DLua_VERSION=5.4" --target lua --install
./build.sh "-DLua_VERSION=5.4" --target luarocks
# ./luarocks/luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua 4.10.0luajit2.1
./luarocks/luarocks install --server=https://github.com/smbape/lua-opencv/releases/download/v0.0.5 opencv_lua
./luarocks/luarocks install --deps-only samples/samples-scm-1.rockspec
npm ci
node scripts/test.js --Release
If the provided binray rocks are not suitable for your environnment, you can install the source rock.
However, installing the source rock takes a long time (01h25mn on my old computer, 30mn on the new one).
Therefore, it is not practical to repeat that process again.
To avoid that long install time, you can host your own prebuilt binary rocks on a private server.
Windows: Hosting you own binary rocks on Windows
Linux: Hosting you own binary rocks on Linux
Similar to python keyword arguments or c# Named parameters, keyword arguments free you from matching the order of arguments to the order of parameters in the parameter lists of called methods.
When a function has multiple default parameters, keyword arguments allow you to override only the needed parameters without specifying the previous default parameters.
As an example, given the documentation of cv.normalize
void cv::normalize ( InputArray src,
InputOutputArray dst,
double alpha = 1,
double beta = 0,
int norm_type = NORM_L2,
int dtype = -1,
InputArray mask = noArray()
)
Python:
cv.normalize( src, dst[, alpha[, beta[, norm_type[, dtype[, mask]]]]] ) -> dst
the following expressions are equivalent:
cv.normalize(src, dst, alpha, beta, norm_type, dtype, mask)
cv.normalize(src, dst, alpha, beta, norm_type, dtype, opencv_lua.kwargs({ mask = mask }))
cv.normalize(src, dst, alpha, beta, norm_type, opencv_lua.kwargs({ dtype = dtype, mask = mask }))
cv.normalize(src, dst, alpha, beta, opencv_lua.kwargs({ norm_type = norm_type, dtype = dtype, mask = mask }))
cv.normalize(src, dst, alpha, opencv_lua.kwargs({ beta = beta, norm_type = norm_type, dtype = dtype, mask = mask }))
cv.normalize(src, dst, opencv_lua.kwargs({ alpha = alpha, beta = beta, norm_type = norm_type, dtype = dtype, mask = mask }))
cv.normalize(src, opencv_lua.kwargs({ dst = dst, alpha = alpha, beta = beta, norm_type = norm_type, dtype = dtype, mask = mask }))
cv.normalize(opencv_lua.kwargs({ src = src, dst = dst, alpha = alpha, beta = beta, norm_type = norm_type, dtype = dtype, mask = mask }))
Of course, optional parameters are not mandatory. In other words, if you only want to change the norm_type
parameter, you can do
cv.normalize(src, dst, opencv_lua.kwargs({ norm_type = cv.NORM_L1 }))
The transformation will usually be straight from python.
tuples
and arrays
become tables
.
numpy
calls and arrays
manipulations have their cv.Mat
counter parts.
keyword arguments
will be wrapped within opencv_lua.kwargs
.
import cv2 as cv
img = cv.imread(cv.samples.findFile("pic1.png"))
img_grey = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
ret, thresh = cv.threshold(img_grey, 100, 255, cv.THRESH_BINARY)
contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
print(f"Found { len(contours) } contours")
cv.drawContours(img, contours, -1, ( 0, 0, 255 ) , 2)
cv.imshow("Image", img)
cv.waitKey()
cv.destroyAllWindows()
local opencv_lua = require("opencv_lua")
local cv = opencv_lua.cv
local img = cv.imread(cv.samples.findFile("pic1.png"))
local img_grey = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
local ret, thresh = cv.threshold(img_grey, 100, 255, cv.THRESH_BINARY)
local contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
print("Found " .. #contours .. " contours")
cv.drawContours(img, contours, -1, { 0, 0, 255 }, 2)
cv.imshow("Image", img)
cv.waitKey()
cv.destroyAllWindows()
Arrays are 1-indexed in lua
, and 0-indexed in python. Therefore, if you see in python p[0]
, you should write in lua p[1]
More over, opencv functions still expect 0-indexed parameters. For exemple
for i = 1, #contours do
-- Notice the i - 1
cv.drawContours(src, contours, i - 1, {0, 0, 255}, 2)
end
Instance methods are called with :
not .
.
If you see in python
cap = cv.VideoCapture(camId)
read, frame = cap.read()
You should write in lua
cap = cv.VideoCapture(camId)
read, frame = cap:read()
For backwards compatibility, python still allows an old syntax for enums and call constructors. This library does not.
If you see in python
cv.ml.SVM_create(...)
cv.ml.SVM_C_SVC
cv.TERM_CRITERIA_MAX_ITER
You should write in lua
cv.ml.SVM.create(...)
cv.ml.SVM.C_SVC
cv.TermCriteria.MAX_ITER
Image processing can use up a lot of memory. Relying solely on the behaviour of the garbage collector can cause your program to consume more memory than necessary.
Therefore, think about calling collectgarbage
in those cases.
local cap = cv.VideoCapture(0)
while true do
-- Without this, memory grows indefinitely
collectgarbage()
local read, frame = cap:read()
cv.imshow("capture camera", frame)
if cv.waitKey(0) == 0x1b then break end
end
For maximum speed performance, when you need to manipulate matrices in lua, convert them to table
s, do your manipulation and convert them back to matrices
-- transform into an lua table for faster processing in lua
local rows, cols, type_ = res.rows, res.cols, res:type()
res = res:table()
for j = 1, rows do
local y = j - 1
for i = 1, cols do
local x = i - 1
if res[j][i] >= threshold then
cv.rectangle(img_rgb, { x, y }, { x + w, y + h }, { 0, 0, 255 }, 2)
end
end
end
res = cv.Mat.createFromArray(res, type_)
If you are using LuaJIT, there is even a way to get close to the c performance with the usage of the FFI Library;
local ffi = require("ffi")
local rows = 512
local cols = 512
local channels = 3
local mat = cv.Mat({ rows, cols, channels }, cv.CV_8U, 255.0)
local bytes = ffi.cast("unsigned char*", mat.data)
local sum = 0
for i = 0, rows - 1 do
for j = 0, cols - 1 do
for k = 0, channels - 1 do
sum = sum + bytes[i * cols * channels + j * channels + k]
end
end
end
My aim was to create a binding that was at least as fast as the python alternative.
The first implementation using sol2 was functional but slower than python in some cases.
Those cases included class instance recognition, functions with multiple overloads (cv::Mat has 20 overloads), returning maps and vectors as userdata.
A small test with plain c/c++ bindings of those cases was much faster (5x).
The key performance points were as follows:
- comparing the hash of the instance's metatable to determine whether it is an instance of a specific class. Inspired by luaarray
- checking argument type directly in stack without the sol::variadic_args intermediary
- returning map/vector as plain tables, which are handled more quickly in lua
In addition, some side effects were a smaller binary (30% less) and a faster compilation time (an average gain of 5mn for 30mn).
I may have used sol2 in an inefficient way. Nevertheless, I now have a more efficient implementation using plain c/c++.
It is possible to outperform python, however, I haven't found a way that comes close to python's sweet syntaxic sugars.