Add native I/O support for numpy arrays as image types

[myselfhimself]

Basically Gmic's run() method should accept 1 or several nparray for its images= parameter.
Numpy type official documentation page

We want as low-level optimization as possible, grabbing the numpy buffers arrays, and without any extra binary dependency.

Workflows yielding a nparray for an image are typicall from scikit-image and PIL/Pillow.

De-interlacing and re-interlacing at the gmic.run()'s input and output are necessary.

TODO list

• if numpy is not importable, all of the following will fail with an ImportError or ModuleNotFound exception; no ugly segfault.

- [ ] GmicImage(data=a_numpy_array """no dimensions info, they are inferred from the object's shape""") -> GmicImage
- [ ] GmicImage(ndarray) unit testing

• GmicImage.to_numpy_array(self, astype=numpy.float32: numpy.dtype, interleave=True: bool) -> numpy.ndarray

• GmicImage().to_numpy_array() unit testing

• GmicImage.from_numpy_array(cls, obj: numpy.ndarray, deinterleave=True: bool) -> GmicImage()

• GmicImage().from_numpy_array() unit testing

- [ ] gmic.run(images=List<numpy.ndarray>|List<GmicImage>) -> in place change of images=List ; same behaviour for gmic.Gmic(...) or gmic.Gmic(...).run(...)
- [ ] gmic.run() (& gmic instance alternatives) unit testing

• add numpy support C ifdefs with setup.py togglable flags

- [ ] add complex numbers numpy array dtype support
- [ ] unit test complex numbers support

- [ ] potential memory leaks care

• add numpy as a build flag stored in gmic.__build__ string
• have numpy examples running without any issue

[myselfhimself]

OpenCV inspiration code for any type of pixel size to CImg: https://github.com/dtschump/CImg/blob/master/plugins/cvMat.h

[myselfhimself]

Per-pixel size is in ndarray.dtype

>>> im.dtype
dtype('uint8')

[myselfhimself]

Work is in progress for uint8 pixel-dtype 2D RGB nparrays, the binding still crashes.. I will post back when this works. After then, we shall be lenient with other dtypes (uint16 etc..) and be tolerant with other shapes: 1D, 2D, 3D, 1-255 channels by point.

[myselfhimself]

Design notes:

INPUT

For inputting of numpy.ndarrays, the path is pretty clear: on the fly conversion in GmicImage(data:numpy.ndarray) constructor or gmic.run(images:List<numpy.ndarray>|numpy.ndarray)
The input numpy.ndarray dtype will be read, and data will be cast to GmicImage's T type (float).
By itself, the code will tolerant to mixed-types images list ie. having GmicImage and numpy.ndarray elements, without extra coding needed.
Envisioned acceptable channel value dtypes are : 1bit, 8bit signed/unsigned/int/float, 16bit signed/unsigned/int/float, 32bit signed/unsigned/int/float.
(The list of numpy dtypes is here.)

OUTPUT

• For outputting, if any numpy.ndarray was given as input, despite any possible mixed existence of GmicImage/numpy.ndarray as mentioned above, all images will be converted to numpy.ndarray will be handed for all outputs.
• A boolean flag will allow for setting the output dtype, but by default it will be Gmic's, that is some float32 (using the numpy.array.astype() method as a convenience for the user).
• The images list will be rewritten in-place, but its contents are first removed (dereferenced) then filled out-of-place; that is, the incoming numpy arrays are not edited at all, and new numpy arrays are provided instead.
• Example .run() signature:

from numpy import array
from copy import deepcopy
my_numpy_arrays =[array(...), array()...]
numpy_arrays_copy = deepcopy(my_numpy_arrays)
gmic.run(images=my_numpy_arrays, command="blur", numpy_output_dtype='uint16')
assert my_numpy_arrays != numpy_arrays_copy # True

[myselfhimself]

Single item images run(images=aNumpyNdarray, command="...") objects will be edited in place though

[myselfhimself]

as far as 2D RGB images are concerned, these POCs are ready but uncomitted yet:

• uint8 numpy.ndarray input with deinterlacing
• float32 numpy.ndarray output with reinterlacing, pipeable using numpy.ndarray.astype(dtype=numpy.uint8|numpy.uint16 etc..)

The design is changing slightly:
(checkboxes moved to the top message)

Those interlace and deinterlace flags could be logical-bit-&'ed maybe... Let us be imperfect for now.

[myselfhimself]

Work on the class method GmicImage.from_numpy_array(obj: numpy.ndarray, deinterlace=True: bool) -> GmicImage() has started... it will be hopefully stable this week

[myselfhimself]

Working full time on this now

[myselfhimself]

libgmic 2.9.0 which was just released is targetted

[myselfhimself]

interleaving and dtype checking+conversion should not happen in any of the gmic.run() input/output images processing loop which is quite long already, but in the from_numpy() to_numpy() methods.

[myselfhimself]

There are a few unit tests already regarding numpy support, but I am lost in them.

Here is a simple handmade test that shows OK support for the numpy ndarray output: (image.png is a lena sample picture)

import gmic
import PIL
import PIL.Image
from numpy import asarray, array_equal
from matplotlib import pyplot

i = PIL.Image.open('image.png')
ii = asarray(i)

l = []
gmic.run("sp lena", l)
ll = l[0].to_numpy_array(interleave=True, astype=int)
array_equal(ll,ii)

pyplot.imshow(ll)
pyplot.show()
pyplot.imshow(ii)
pyplot.show()

let me see for simple numpy ndarray Gmic input hand testing...

[myselfhimself]

the testing scenario now works OK a bit further (except that dtype variety is not yet covered):

# TEST 1: numpy(interleaved)->pyplot_display(interleaved)->gmic_from_nparray(deinterleaved)->gmic_display(deinterleaved)
import gmic
import PIL
import PIL.Image
from numpy import asarray, array_equal
from matplotlib import pyplot

gmic.run("sp lena output image.png")
i = PIL.Image.open('image.png') # Lena sample
ii = asarray(i)

l = []
gmic.run("sp lena", l)
ll = l[0].to_numpy_array(interleave=True, astype=int)
array_equal(ll,ii) # True

pyplot.imshow(ll)
pyplot.show() # Lena in good colors, dimensions and orientation

pyplot.imshow(ii)
pyplot.show() # Lena in good colors, dimensions and orientation

j = gmic.GmicImage.from_numpy_array(ll, deinterleave=True)
gmic.run("display", j) # Lena in good colors, dimensions and orientation

jj = gmic.GmicImage.from_numpy_array(ii, deinterleave=True)
gmic.run("display", jj) # TODO STRANGE IMAGE

[myselfhimself]

Progress on above empirical test by extending it thanks to to_numpy_array and from_numpy_array simplification and solidification. See below. Sold unit tests will be written. Will check boxes in the above message when those upcoming tests pass for each method.
Oustanding work is also for GmicImage(numpy.ndarray) initialization (code removed for now) and gmic.run(images=List<numpy.ndarray>) (started) calls which are comfort-related and could maybe be ifdef-ed for later toggling and maintainability... Those calls will also need proper unit testing...
Depending on time, complex numbers could be in for this numpy-support release, for now only boolean-int-float scalars are to be supported.

# TEST 1: numpy(interleaved)->pyplot_display(interleaved)->gmic_from_nparray(deinterleaved)->gmic_display(deinterleaved)
import gmic
import PIL
import PIL.Image
import numpy
from numpy import asarray, array_equal
from matplotlib import pyplot

gmic.run("sp lena output image.png")
i = PIL.Image.open('image.png') # Lena sample
ii = asarray(i)
print(ii.shape) #(512, 512, 3)
print(ii.dtype) #uint8

l = []
gmic.run("sp lena", l)
ll = l[0].to_numpy_array(interleave=True, astype=numpy.uint8, squeeze_shape=True)
array_equal(ll,ii) # True
print(ll.shape) #(512, 512, 3)
print(ll.dtype) #uint8

pyplot.imshow(ll)
pyplot.show() # Lena in good colors, dimensions and orientation

pyplot.imshow(ii)
pyplot.show() # Lena in good colors, dimensions and orientation

j = gmic.GmicImage.from_numpy_array(ll, deinterleave=True)
gmic.run("display", j) # Lena in good colors, dimensions and orientation
"""
[gmic]-1./ Display image [0] = '[unnamed]', from point (256,256,0).
[0] = '[unnamed]':
  size = (512,512,1,3) [3072 Kio of floats].
  data = (225,225,223,223,225,225,225,223,225,223,223,223,(...),78,78,78,77,91,80,79,89,77,79,79,82).
  min = 8, max = 251, mean = 128.241, std = 58.9512, coords_min = (457,60,0,1), coords_max = (425,20,0,0).
"""

jj = gmic.GmicImage.from_numpy_array(ii, deinterleave=True)
gmic.run("display", jj) # Lena in good colors, dimensions and orientation
"""
[gmic]-1./ Display image [0] = '[unnamed]', from point (256,256,0).
[0] = '[unnamed]':
  size = (512,512,1,3) [3072 Kio of floats].
  data = (225,225,223,223,225,225,225,223,225,223,223,223,(...),78,78,78,77,91,80,79,89,77,79,79,82).
  min = 8, max = 251, mean = 128.241, std = 58.9512, coords_min = (457,60,0,1), coords_max = (425,20,0,0).
"""

# TEST 2: numpy(deinterleaved)->gmic_from_nparray(deinterleaved)->gmic_display(deinterleaved)
import gmic
import PIL
import PIL.Image
from numpy import asarray, array_equal

gmic.run("sp lena output image.png")
i = PIL.Image.open('image.png')
ii = asarray(i)
print(ii.shape) # (512, 512, 3)
print(ii.dtype) # uint8

l = []
gmic.run("sp lena", l)
ll = l[0].to_numpy_array(interleave=False, astype=int, squeeze_shape=True) # default astype=float32, default squeeze_shape=True
print(ll.shape) # (512, 512, 3)
print(ll.dtype) # int64
array_equal(ll,ii) # False; uint8 vs. int64 match but deinterleave vs interleave unmatch

j = gmic.GmicImage.from_numpy_array(ll, deinterleave=False)
gmic.run("display", j) # Lena in good orientation, color, size
"""
[gmic]-1./ Display image [0] = '[unnamed]', from point (256,256,0).
[0] = '[unnamed]':
  size = (512,512,1,3) [3072 Kio of floats].
  data = (225,225,223,223,225,225,225,223,225,223,223,223,(...),78,78,78,77,91,80,79,89,77,79,79,82).
  min = 8, max = 251, mean = 128.241, std = 58.9512, coords_min = (457,60,0,1), coords_max = (425,20,0,0).
"""

jj = gmic.GmicImage.from_numpy_array(ii, deinterleave=True)
gmic.run("display", jj)  # Lena in good orientation, color, size
"""
[gmic]-1./ Display image [0] = '[unnamed]', from point (256,256,0).
[0] = '[unnamed]':
  size = (512,512,1,3) [3072 Kio of floats].
  data = (225,225,223,223,225,225,225,223,225,223,223,223,(...),78,78,78,77,91,80,79,89,77,79,79,82).
  min = 8, max = 251, mean = 128.241, std = 58.9512, coords_min = (457,60,0,1), coords_max = (425,20,0,0).
"""

[myselfhimself]

Note that GmicImage.to_numpy_array() has a new default parameter squeeze_shape=True, where the default shape that G'MIC likes(w,h,d,c) is clear from its dimensions =1, eg. 2d RGB images in shape (512,512,1,3) will have a (512,512,3) shape instead, which matplot will not refuse for displaying.

[myselfhimself]

Updated TODO list in topmost message above

[myselfhimself]

• a GmicImage() can now be initialized with an empty data buffer parameter, it will be filled with zeros according to the provided image dimensions (w,h,d,c)

[myselfhimself]

will be finished in #59 // numpy support subpart