Missing help on multiple Outputs using applier

[gillins]

Original report by cedric lardeux (Bitbucket: clardeux, GitHub: clardeux).

---

Hi,

not sure it's the best way to post my request but this is the only way I found. Sorry for disturbance if I am wrong.

In the help of RIOS about multiple input/outputs using applier it's well explained how to manage multiple inputs using list of file path but in case of multiple outputs it's not explained.

Indeed I tried to use the next command for multiple output (inspired from input) but I have the following error.

Command

#!python

outfiles = applier.FilenameAssociations()
outfiles.imgs = aOutputRasterListPath

Error

#!python

AttributeError: 'BlockAssociations' object has no attribute 'imgs'

Probably I have do a mistakes but I think that it'should be very nice to ad an example with multiple input and multiple output.

Best regards

[gillins]

Original comment by Sam Gillingham (Bitbucket: gillins, GitHub: gillins).

---

Yes this the best way to contact us at present.

We do support writing a list of rasters. Looking at the error it seems you are not setting imgs in your function (you should set this to a list of numpy arrays). If you are still have a problem after checking this please post your script here.

Sam.

[gillins]

Original comment by cedric lardeux (Bitbucket: clardeux, GitHub: clardeux).

---

Thanks for answer and the first mistake was to try to iterate over outfiles.imgs .

Now I write a 3d numpy array of 14 size in first dimension.
Now I have another error that I don't understand because when I print the shape of the numpy array I have 14,266,266 where the number of output file is well 14.

Here after the 2 functions that I call to run the process.
Have you an idea of the problem, may be I need to give some settings in controls ?

Best regards and thanks again for help.

#!python

    applier.apply(QueganTemporalSpeckleFilteringRIOS, infiles, outfiles,otherargs, controls=controls)
  File "/home/cedric/App/RIOS/RIOSInstall/lib/python2.7/site-packages/rios/applier.py", line 670, in apply
    controls, info)
  File "/home/cedric/App/RIOS/RIOSInstall/lib/python2.7/site-packages/rios/applier.py", line 770, in writeOutputBlocks
    creationoptions=controls.getOptionForImagename('creationoptions', name))
  File "/home/cedric/App/RIOS/RIOSInstall/lib/python2.7/site-packages/rios/imagewriter.py", line 215, in __init__
    "Shape of array to write must be 3-d. Shape is actually %s"%repr(firstblock.shape))
rios.rioserrors.ArrayShapeError: Shape of array to write must be 3-d. Shape is actually (266, 266)

#!python


'''
This function apply teporal filtering over a 3d numpy array
'''
def QueganTemporalSpeckleFilteringRIOS(info, inputs, outputs, otherargs):
    eps = 1e-16
    NumTime = len(inputs.imgs) - 1
    
    # Read first image as sumpart just to create a 2D array due to RIOS
    aSumPart = inputs.imgs[0].astype(np.float32)
    
    # We remove the first band dimension equal to one
    aSumPart = aSumPart[0,:,:]
    
    # Replace all value to 0.
    aSumPart.fill(0.)
    
    # Get row, cols
    Rows, Cols = aSumPart.shape
    
    # Test if previous filtering if yes we read the data
    if otherargs.PrevFilt:
        print 'In quegan filtering func : YES previous filtering'
        aSumPart = inputs.imgs[NumTime].astype(np.float32)
        # We remove the first band dimension equal to one
        aSumPart = aSumPart[0,:,:]
    else:
        print 'In quegan filtering func : NO previous filtering'

    FiltTempArray = np.zeros(shape=(NumTime, Rows, Cols),dtype=np.float)
    
    for img in inputs.imgs[:-1]:
        # We remove the 1d band dimension due to RIOS
        img = img[0,:,:]
        aSumPart += img / ( uniform_filter(img, size=otherargs.WinSize) + eps )
        
    for i, img in enumerate(inputs.imgs[:-1]):
        # We remove the 1d band dimension due to RIOS
        img = img[0,:,:]
        FiltTempArray[i,:,:] += uniform_filter(img, size=otherargs.WinSize) \
        * aSumPart / (float(NumTime) + float(otherargs.NPrevDates))

    OutNP = np.concatenate((FiltTempArray, np.expand_dims(aSumPart, axis=0)))
    
    print 'OutNP.shape', OutNP.shape
    outputs.imgs = OutNP

        
'''
Apply block processing temporal filtering
'''
def TileTemporalFilteringRIOS(aInput_Data_Folder, aInputRasterListPath, aOutputRasterListPath,
                            aInputPrevFiltPath, aBlockSize,aTempWindowSize):
    NumBands = len(aInputRasterListPath)
    # Boolean of previous filtering
    BoolPrevFilt = False

    # We get InputRaster dates
    aOuputPrevFiltPath = ''
    NumPrevDates = 0
    aListDates = [getDayFromS1FileOrFolder(aRastPath) for aRastPath in aInputRasterListPath ]
    UniqueDates = list(set(aListDates))
    UniqueDates.sort()
    
    # Test if it's first processing time or if previous filtering files is existing
    if aInputPrevFiltPath != '':
        # Get first and last dates and number of date from this format QueganVV_20150221_20161118_15.tif
        BaseName = os.path.basename(os.path.splitext(aInputPrevFiltPath)[0])
        NumPrevDates = float(BaseName.split('_')[3])

        InputPrevFiltNameSplit = BaseName.split('_')
        InputPrevFiltNameSplit[2] = UniqueDates[-1]
        InputPrevFiltNameSplit[3] = str(int(NumPrevDates + NumBands))

        aOuputPrevFiltPath = os.path.join(os.path.dirname(aInputPrevFiltPath) ,
                                          '_'.join(InputPrevFiltNameSplit) + '.tif')
        BoolPrevFilt = True
    else:
        # Put existing file in order that rios find it in input
        aInputPrevFiltPath = aInputRasterListPath[0]
    
        if '_VV_' in aInputRasterListPath[0]:
            aOuputPrevFiltPath = os.path.join(aInput_Data_Folder,
            'TempProcStackVV_' + UniqueDates[0] + '_' + UniqueDates[-1] + '_' + str(NumBands) + '.tif')
        elif '_HH_' in aInputRasterListPath[0]:
            aOuputPrevFiltPath = os.path.join(aInput_Data_Folder,
            'TempProcStackHH_' + UniqueDates[0] + '_' + UniqueDates[-1] + '_' + str(NumBands) + '.tif')
        elif '_VH_' in aInputRasterListPath[0]:
            aOuputPrevFiltPath = os.path.join(aInput_Data_Folder,
            'TempProcStackVH_' + UniqueDates[0] + '_' + UniqueDates[-1] + '_' + str(NumBands) + '.tif')
        elif '_HV_' in aInputRasterListPath[0]:
            aOuputPrevFiltPath = os.path.join(aInput_Data_Folder,
            'TempProcStackHV_' + UniqueDates[0] + '_' + UniqueDates[-1] + '_' + str(NumBands) + '.tif')
            
        BoolPrevFilt = False

    # Append the Input Prev field path to put in the quegan filtering
    aInputRasterListPath.append(aInputPrevFiltPath)
    
    # Append the Output Prev field path to put in the quegan filtering
    aOutputRasterListPath.append(aOuputPrevFiltPath)
                                
    # Settings of RIOS lib
    # Beacause we need overlap between blocks
    controls = applier.ApplierControls()
    # for a 3x3 the overlap is 1, 5x5 overlap is 2 etc
    Radius = int(aTempWindowSize / 2.)
    controls.setOverlap(Radius)
    
    # Define other args to use
    otherargs = applier.OtherInputs()
    # Window size
    otherargs.WinSize = aTempWindowSize
    
    # If previous filtering exist
    otherargs.PrevFilt = BoolPrevFilt
    otherargs.NPrevDates = NumPrevDates
    
    # Because we use an arbitrary number of input output
    infiles = applier.FilenameAssociations()
    outfiles = applier.FilenameAssociations()
    
    infiles.imgs = aInputRasterListPath
    outfiles.imgs = aOutputRasterListPath
    
#    print infiles.imgs, '\n' 
    print len(outfiles.imgs)
#    print outfiles.imgs, '\n' 
    
    applier.apply(QueganTemporalSpeckleFilteringRIOS, infiles, outfiles,otherargs, controls=controls)

[gillins]

Original comment by Sam Gillingham (Bitbucket: gillins, GitHub: gillins).

---

outputs.imgs must be a list of 3d numpy arrays, not a single array. This is because you pass outfiles.img as a list - they must match in type.

[gillins]

Original comment by cedric lardeux (Bitbucket: clardeux, GitHub: clardeux).

---

Thanks a lot for your help and sorry for my dev newbie question.

Now it's working well. Thanks a lot for your library that should be better known for everybody.

I think that it should be good to add an example of multiple output in your on line documentation.

Best regards

[gillins]

Original comment by Sam Gillingham (Bitbucket: gillins, GitHub: gillins).

---

Thanks for you feedback. Feel free to make a pull request with improved documentation (the sphinx docs are in the rios/doc/source subdirectory of the source tree). Alternatively post a small example here and I'll put it in.