Minimal Swift for TensorFlow OpenCV4 bindings, partially ported from gocv.
OpenCV Functions exposed:
- resize
- getRotationMatrix2D
- warpAffine
- copyMakeBorder
- GaussianBlur
- remap
- imdecode
- imread
- cvtColor
- flip
- transpose
- VideoCapture (Not supported on Colab)
- ImShow (Not supported on Colab)
- WaitKey
OpenCV's Mat can be converted to S4TF's Tensor and ShapedArray types (and back).
See Extra/Tests.ipynb and Tests as an example of usage.
Try demo notebook in Colab.
Include as SwiftPM package:
.package(url: "https://github.com/vvmnnnkv/SwiftCV.git", .branch("master"))In a Jupyter Notebook:
%system curl -sL https://github.com/vvmnnnkv/opencv-colab/raw/master/opencv4.tar.gz | tar zxf - -C / && ldconfig /opt/opencv-4.1.0/lib/ && ln -s /opt/opencv-4.1.0/lib/pkgconfig/opencv4.pc /usr/lib/pkgconfig/opencv4.pc
%install-location $cwd/swift-packages
%install '.package(url: "https://github.com/vvmnnnkv/SwiftCV.git", .branch("master"))' SwiftCVNOTE: OpenCV4 must installed in order for package to compile.
Run the install/install_cv4.sh script (written by Jeremy Howard) to install opencv4.
// load image in memory
let url = "https://live.staticflickr.com/2842/11335865374_0b202e2dc6_o_d.jpg"
let imgContent = Data(contentsOf: URL(string: url)!)
// make opencv image
var cvImg = imdecode(imgContent)
// convert color scheme to RGB
cvImg = cvtColor(cvImg, nil, ColorConversionCode.COLOR_BGR2RGB)
show_img(cvImg)resize(cvImg, nil, Size(100, 50), 0, 0, InterpolationFlag.INTER_AREA)let zoomMat = getRotationMatrix2D(Size(cvImg.cols, cvImg.rows / 2), 0, 2)
warpAffine(cvImg, nil, zoomMat, Size(600, 600))let rotMat = getRotationMatrix2D(Size(cvImg.cols / 2, cvImg.rows / 2), 20, 1)
warpAffine(cvImg, nil, rotMat, Size(cvImg.cols, cvImg.rows))copyMakeBorder(cvImg, nil, 40, 40, 40, 40, BorderType.BORDER_CONSTANT, RGBA(0, 127, 0, 0))GaussianBlur(cvImg, nil, Size(25, 25))flip(cvImg, nil, FlipMode.HORIZONTAL)transpose(cvImg, nil)let cap = VideoCapture(0)
// Optional, reduces latency a bit
cap.set(VideoCaptureProperties.CAP_PROP_BUFFERSIZE, 1)
let frame = Mat()
while true {
cap.read(into: frame)
ImShow(image: frame)
WaitKey(delay: 1)
}This requires Swift for TensorFlow.
// convert image to floats Tensor
var imgTens = Tensor<Float>(Tensor<UInt8>(cvMat: cvImg)!) / 255
let contr:Float = 1.8
let lightn:Float = 0.2
let mean = imgTens.mean()
imgTens = (imgTens - mean) * contr + mean + lightnThis requires Swift for TensorFlow.
// convert image to Tensor
var imgTens = Tensor<Float>(Tensor<UInt8>(cvMat: cvImg)!) / 255
let randTens = Tensor<Float>(randomNormal: imgTens.shape) * 0.1
imgTens += randTensCurrently this package is just an example of OpenCV/S4TF integration with no safety checks and guarantees to work properly :)
OpenCV C API, (c) Copyright gocv authors.