Shape matching 2d

module demonstrating how to match shapes on a 2d grey level images More...

module demonstrating how to match shapes on a 2d grey level images

Overview

This application matches the objects in an input image with input templates (both input and template images are loaded from TIFF image files).

To proceed, the application extracts the shapes from each image (see Label shape extraction 2d) and computes for each shape in the image its distance with all the template's shapes. The template yielding the minimum distance is assigned as the best match.

The user can specify a minimum distance. If the matched template has a higher distance than the given threshold, no template is close enough to match the shape.

The application also allows to blur the image to smooth the edges in the input image. It can be useful to deblur noise on the input images, or in the case of default images which has only two intensities (one for the background and one for the foreground), it generates a shaded contour. If the user specifies a value 0, the smoothing step is skipped.

Finally, the user can specify the binarization threshold used for the shape extraction (see Label shape extraction 2d for more details about the shape extraction).

The application's ouput are a label image, where each shape is labelled with the matched template's index, and a CSV file summarizing the matches for each shape.

Usage

The application can be called through a command line as follows:

   <application_exe_filename> [--inputImgFilePath <input_image_file_path>]  [--inTemplatePathList <input_template_file_path_list>] 
                              [--inThresholdBinarization <input_binarization_threshold>] [--inThresholdDistance <input_distance_threshold>] 
                              [--inStdDev <input_blur_standard_deviation>]
                              [--outputImgFilePath <output_image_file_path>] [--outputCsvResultPath <output_csv_file_path>] 
     
   Arguments:
      --inputImgFilePath            optional; specifies the name of the TIFF file, from
                                    which the 2d input image will be loaded; if not 
                                    specified by the user, the input image is loaded from
                                    file <DEV_ROOT>/data/Sample/images/shapes.tif

      --inTemplatePathList          optional; specifies the names of the TIFF files, from
                                    which the 2d input template images will be loaded; if not 
                                    specified by the user, 4 images are loaded from
                                    file <DEV_ROOT>/data/Sample/images/Templates.
                                    To specify a list, the syntax is:
                                    --inTemplatePathList path_template1 path_template2 ...

      --inThresholdBinarization     optional; binarization threshold used by the 
                                    shape extraction;
                                    if not specified by the user, this value equals 15

      --inThresholdDistance         optional; distance threshold used by the 
                                    shape extraction;
                                    if not specified by the user, this value equals 0.2

      --inStdDev                    optional; deviation used to blur the image;
                                    if the user specifies a value <= 0, this step is skipped by 
                                    the application (i.e. the input image is not blurred)
                                    if not specified by the user, this value equals 1
                          
      --outputImgFilePath           optional; specifies the name of the TIFF file, in
                                    which the 2d output image resulting from the 
                                    shape matching will be saved;
                                    if not specified by the user, the output image is 
                                    saved in file <TEMPORARY_IPSDK_DIR>/Sample/shapeMatch.tif
                          
      --outputCsvResultPath         optional; specifies the name of the CSV file containing the
                                    summary of the shape matching;
                                    if not specified by the user, the output file is 
                                    saved in the same directory than outputImgFilePath
                                    and is named shapeMatch.csv

Here is a snapshot of default input image used by the application with the desired templates (on the left). On the right of the image, the output match image and the CSV result are shown. All the parameters are set to their default values:

Source code documentation

We start by importing all necessary libraries:

import os
import sys, getopt
import PyIPSDK
import PyIPSDK.IPSDKIPLAdvancedMorphology as advmorpho
import PyIPSDK.IPSDKIPLBinarization as bin
import PyIPSDK.IPSDKIPLFiltering as filter
import PyIPSDK.IPSDKIPLIntensityTransform as itrans
import PyIPSDK.IPSDKIPLShapeSegmentation as shapesegmentation

Then we define the input parameters.

    # Retrieve program parameters
    inputImgPath, inputTemplatePathList, outputImgPath, outputCsvResultPath, inputThresholdBinarization, inputThresholdDistance, stdDev = readCmdArguments(argv)

We load our input image from the associated Tiff file, by calling the function ipsdk::image::file::loadTiffImageFile.

    # Opening of input image
    inImg = PyIPSDK.loadTiffImageFile(inputImgPath)

In the same way, we load the template images. The paths are stored in a list and we declare another list containing the ipsdk images.

    # Loading all the template images and store them in a list
    templateImgList = []
    nbTemplates = len(inputTemplatePathList)
    for i in range(0, nbTemplates):
        templateImgList.append(PyIPSDK.loadTiffImageFile(inputTemplatePathList[i]))

Since the default images only have two different intensities (the background intensity is 0 and the geometric shapes pixels have an intensity of 255), the application allows to blur the input image and the template images to smooth the edges. This step can also be used to smooth the noise. To skip this step, the user can set the Gaussian standard deviation to a value lower or equal than 0.

    # Blur the images if the standard deviation is strictly positive
    if stdDev > 0:
        filter.gaussianSmoothing2dImg(inImg, stdDev, stdDev, PyIPSDK.createGaussianCoverage(0.99, 2), inImg)
        for i in range(0, nbTemplates):
            filter.gaussianSmoothing2dImg(templateImgList[i], stdDev, stdDev, PyIPSDK.createGaussianCoverage(0.99, 2), templateImgList[i])

Now the images are ready to be processed. We extract the shapes from the input image and from the template images. The shape templates are stored in a list juste like for the ipsdk template images.

    # Label contour extraction computation for the input image
    binImg = bin.thresholdImg(inImg, inputThresholdBinarization, 255)
    inLabelImg = advmorpho.connectedComponent2dImg(binImg)
    imShape2dColl = shapesegmentation.labelShapeExtraction2d(inLabelImg)
    
    # Label contour extraction computation for each template
    templateShapesList = []
    for i in range(0, nbTemplates):
        binImg = bin.thresholdImg(templateImgList[i], inputThresholdBinarization, 255)
        labelImg = advmorpho.connectedComponent2dImg(binImg)
        templateShape2dColl = shapesegmentation.labelShapeExtraction2d(labelImg)
        templateShapesList.append(templateShape2dColl)

We can now use the extracted shapes to compute the matching by calling the function ipsdk::geom::matchShapes. This function takes a reference (i.e. template) shape and a shape collection (for instance, the one extracted from the input image) and computes for each of these shapes its distance to the template shape. Hence, the application have to loop over all the templates to know the closest template for each shape.

Note

In the case of default template images, there is only one shape to extract : a square, a triangle, a hexagon or a circle. For other images, more shapes can be extracted because the template image may contain non connected objects. In this case, the application chooses the first shape.

Warning

IPSDK also extracts the background as a shape with index 0 in the shape collection. This shape is skipped during the process.

    matchesList = []
    for i in range(0, nbTemplates):
        # Match shape computation
        matches = PyIPSDK.matchShapes(templateShapesList[i].getColl()[1], imShape2dColl, PyIPSDK.eSDT_Type3, False)
        matchesList.append(matches)

Once this step done, we know the distance for each shape with every template. The next step involves determining the closest distance to be able to match a template. The result is stored in the bestMatches variable, which contains two collections with the same size as the number of shapes extracted from the input image (excluding the background). The first collection stores the closest shape indices whereas the second one contains their distances.

The user can define a threshold, whose default value is 0.2. If the minimum distance is higher than this threshold, the application considers that the shape could not be matched with any template and the index is set to sys.maxsize.

    # Declare and allocate the collection
    # bestMatches is the collection of (_shapeHuDistance, _shapeIdx) structures
    # where _shapeHuDistance is the minimum distance between a shape in the 
    # input image and the most similar template of index _shapeIdx
    bestMatches = ([], [])
    nbShapes = len(matchesList[0][0])
    # Initialize the best matches for each templates
    for shapeIdx in range(0, nbShapes) :
        bestMatch = (-1, sys.float_info.max)

        # For each template
        for templateIdx in range(0, nbTemplates) :
            # Check if we find a new minimum distance
            matches = matchesList[templateIdx]
            if (bestMatch[1] > matches[1][shapeIdx]) :
                bestMatch = (templateIdx, matches[1][shapeIdx])
       
        bestMatches[0].append(bestMatch[0])
        bestMatches[1].append(bestMatch[1])

        # Validate the match : if the distance is lower than the input threshold,
        # then the algorithm considers that the match is correct, otherwise, it
        # notes the match is unsatisfying by assigning the shape index to its 
        # maximum possible value
        if (bestMatches[1][shapeIdx] >= inputThresholdDistance):
            bestMatches[0][shapeIdx] = sys.maxsize

Now, we know for each shape of the image its closest template. We use this information to generate the ouput image, by transforming the LUT of the previously calculated label image.

    # Initialize the LUT
    lut = PyIPSDK.createIntensityLUT(0, 1, [])

    # Add the first value : the background is 0
    lut.lookupTable.append(0)

    # Add the other values
    nbLabels = imShape2dColl.getSize()
    for i in range(1, nbLabels) :
        matchIndex = bestMatches[0][i - 1] + 1
        if matchIndex < sys.maxsize:
            lut.lookupTable.append(matchIndex)
        else:
            lut.lookupTable.append(0)

    # Apply the LUT
    matchImg = itrans.lutTransform2dImg(inLabelImg, lut);

If all the previous operations were successfully completed, we save the image resulting from the matching operation to the TIFF file specified by the user (or to a default TIFF file, if the user did not specify anything):

    PyIPSDK.saveTiffImageFile(outputImgPath, matchImg)

We also save the result of the matching in a CSV file. This file contains for each shape of the input image its index, the index of the correspponding template if its distances is lower than the distance threshold criterion (an error message is written otherwise) and its distance in question.

    # save the matching results to output csv file
    writeToCsvFile(outputCsvResultPath, bestMatches, inputThresholdDistance)

The content of the writeToCsvFile function is :

def writeToCsvFile(outputCsvResultPath, bestMatches, threshold):
    file = open(outputCsvResultPath, "w") 
 
    file.write("Shape Index;Matched Template Index; Distance\n") 
    
    # For each shape (do not process the background)
    nbTemplates = len(bestMatches[0])
    for i in range(0, nbTemplates) :
        file.write(str(i + 1) + ";")

        # Test for a bad match
        if (bestMatches[1][i] < threshold):
            file.write(str(bestMatches[0][i]))
        else:
            file.write("Bad match according to threshold ( = " + str(threshold) + ")")
        file.write(";" + str(bestMatches[1][i]) + "\n")
     
    file.close() 

See the full source listing