Shape Matching 2d

Application used to match shapes on a 2d grey level images. More...

Application used to match shapes on a 2d grey level images.

Overview

This application matches the objects in an input image with input templates (both input and template images are loaded from TIFF image files).

To proceed, the application extracts the shapes from each image (see Label shape extraction 2d) and computes for each shape in the image its distance with all the template's shapes. The template yielding the minimum distance is assigned as the best match.

The user can specify a minimum distance. If the matched template has a higher distance than the given threshold, no template is close enough to match the shape.

The application also allows to blur the image to smooth the edges in the input image. It can be useful to deblur noise on the input images, or in the case of default images which only two intensities (one for the background and one for the foreground), it generates a shaded contour. If the user specifies a value 0, the smoothing step is skipped.

Finally, the user can specify the binarization threshold used for the shape extraction (see Label shape extraction 2d for more details about the shape extraction).

The application's ouput are a label image, where each shape is labelled with the matched template's index, and a CSV file summarizing the matches for each shape.

Usage

The application can be called through a command line as follows:

   <application_exe_filename> [--inputImgFilePath <input_image_file_path>]  [--inTemplatePathList <input_template_file_path_list>] 
                              [--inThresholdBinarization <input_binarization_threshold>] [--inThresholdDistance <input_distance_threshold>] 
                              [--inStdDev <input_blur_standard_deviation>]
                              [--outputImgFilePath <output_image_file_path>] [--outputCsvResultPath <output_csv_file_path>] 
     
   Arguments:
      --inputImgFilePath            optional; specifies the name of the TIFF file, from
                                    which the 2d input image will be loaded; if not 
                                    specified by the user, the input image is loaded from
                                    file <DEV_ROOT>/data/Sample/images/shapes.tif

      --inTemplatePathList          optional; specifies the names of the TIFF files, from
                                    which the 2d input template images will be loaded; if not 
                                    specified by the user, 4 images are loaded from
                                    file <DEV_ROOT>/data/Sample/images/Templates.
                                    To specify a list, the syntax is:
                                    --inTemplatePathList path_template1 path_template2 ...

      --inThresholdBinarization     optional; binarization threshold used by the 
                                    shape extraction;
                                    if not specified by the user, this value equals 15

      --inThresholdDistance         optional; distance threshold used by the 
                                    shape extraction;
                                    if not specified by the user, this value equals 0.2

      --inStdDev                    optional; deviation used to blur the image;
                                    if the user specifies a value <= 0, this step is skipped by 
                                    the application (i.e. the input image is not blurred)
                                    if not specified by the user, this value equals 1
                          
      --outputImgFilePath           optional; specifies the name of the TIFF file, in
                                    which the 2d output image resulting from the 
                                    shape matching will be saved;
                                    if not specified by the user, the output image is 
                                    saved in file <TEMPORARY_IPSDK_DIR>/Sample/shapeMatch.tif
                          
      --outputCsvResultPath         optional; specifies the name of the CSV file containing the
                                    summary of the shape matching;
                                    if not specified by the user, the output file is 
                                    saved in the same directory than outputImgFilePath
                                    and is named shapeMatch.csv

Here is a snapshot of default input image used by the application with the desired templates (on the left). On the right of the image, the output match image and the CSV result are shown. All the parameters are set to their default values:

Source code documentation

We start by including all the necessary header files:

// --- IPSDK includes
// ------------------

// used to initialize IPSDK environment
#include <IPSDKCore/Config/LibraryInitializer.h>

// used to retrieve usual folders (IPSDK temporary folder, root development folder, etc.)
#include <IPSDKUtil/Tools/LibraryPath.h>

// used to catch exceptions potentially thrown  by functions loadRawImageFile and saveRawImageFile
#include <IPSDKImageFile/Logger/IPSDKImageFileException.h>

// used to read/write an image from/to a TIFF file:
#include <IPSDKImageFile/Tiff/TiffImageFileUtils.h>

// used to smooth images with a Gaussian kernel
#include <IPSDKIPL/IPSDKIPLFiltering/Processor/GaussianSmoothing2dImg/GaussianSmoothing2dImg.h>

// used to threshold images
#include <IPSDKIPL/IPSDKIPLBinarization/Processor/ThresholdImg/ThresholdImg.h>

// used to compute a label image
#include <IPSDKIPL/IPSDKIPLAdvancedMorphology/Processor/ConnectedComponent2dImg/ConnectedComponent2dImg.h>

// used to exstract shapes from label images
#include <IPSDKIPL/IPSDKIPLShapeSegmentation/Processor/LabelShapeExtraction2d/LabelShapeExtraction2d.h>

// used for Shape2dColl
#include <IPSDKBaseShapeSegmentation/Entity/2d/Shape2dColl.h>

// used for Hu moments and its distance
#include <IPSDKGeometry/Measure/2d/Moments/HuMomentsUtils.h>

// used for numeric limits (minimum, maximum and epsilon)
#include <IPSDKUtil/Tools/NumericLimits.h>

// used for LUT transform
#include <IPSDKIPL/IPSDKIPLIntensityTransform/Processor/LUTTransform2dImg/LUTTransform2dImg.h>

// used to display log messages
#include <Sample/Sample_ShapeMatching2d/Logger/Sample_ShapeMatching2dLog.h>

// --- third-party boost includes
// ------------------------------

// boost/filesystem/*: contains functions and classes providing facilities to
// manipulate files and directories, and associated paths
#include <boost/filesystem/path.hpp>
#include <boost/filesystem/convenience.hpp>

// boost/program_options/*: contains functions and classes used to manage and
// interpret arguments of command line
#include <boost/program_options/cmdline.hpp>
#include <boost/program_options/options_description.hpp>
#include <boost/program_options/parsers.hpp>
#include <boost/program_options/variables_map.hpp>

// --- third-party log4cplus include
// ---------------------------------

// used to add console as output support of logs
#include <log4cplus/consoleappender.h>

// --- STL include
// ---------------

#include <iostream>
#include <fstream>

In the main function body, we start by asking to display all the log messages generated by IPSDK libraries and by our application itself to the application console:

int
main(int argc, char* argv[])
{
    // add console appender for application logs
    log4cplus::SharedAppenderPtr pConsole(new log4cplus::ConsoleAppender);
    log4cplus::Logger::getRoot().addAppender(pConsole);
    log4cplus::Logger::getRoot().setLogLevel(log4cplus::INFO_LOG_LEVEL);

Next, we initialize the IPSDK environment by invoking "ipsdk::core::LibraryInitializer::getInstance().init()". This method must be called before using any entity or function of IPSDK libraries. It returns an object of type ipsdk::core::LibInitResult, that tells us whether the initialization was OK or not. If the initialization failed (because the IPSDK license file was not found, for instance), we notify the user with an appropriate log message, and we close the application.

    // initialize IPSDK environment (first call to be done before calling any
    // function or using any entity of IPSDK environment)
    const ipsdk::core::LibInitResult initRes =
        ipsdk::core::LibraryInitializer::getInstance().init();

    switch (initRes.getResult().value()) {
    case ipsdk::core::eLibInitStatus::eLIS_Warn:
        // IPSDK library is initialized but there were warnings;
        // notify the user by displaying a message
        SAMPLE_SHAPEMATCHING2D_LOG_WARN(
            [eSample_ShapeMatching2dMessage::eIPSDKLibInitWarn] % initRes.getMsg());
        break;
    case ipsdk::core::eLibInitStatus::eLIS_Failed:
        // IPSDK library initialization; notify the user and exit
        SAMPLE_SHAPEMATCHING2D_LOG_ERROR(
            [eSample_ShapeMatching2dMessage::eIPSDKLibInitFailed] % initRes.getMsg());
        return -1;
        break;
    default:
        break;
    }

We then declare the variables where the input arguments will be stored. First, we declare the paths of the input and output images, the path of the output CSV file and also the collection of the paths for the templates.

    // boost objects, used to store input and output images files paths
    boost::filesystem::path inImgFilePath, outImgFilePath, outputCsvResultPath;
    std::vector<boost::filesystem::path> vInTemplateImgFilePath;

We declare variables for the scalar parameters as well.

    // variables storing dimensions of input image
    ipReal64 thresholdDistance, thresholdBinarization;
    ipReal32 stdDev;

All these variables are initialized through the call of the "readCmdArguments" function. As its name suggests, it parses the command line to initialize the input and output images files paths, depending on the options specified by the user. The definition of this function is not explained here, because it mainly uses boost functions, and no IPSDK code.

    // read program options from command line, and, if appropriate,
    // initialize input and output images files paths
    if (!readCmdArguments(argc, argv, inImgFilePath, vInTemplateImgFilePath, outImgFilePath, outputCsvResultPath, thresholdBinarization, thresholdDistance, stdDev))
        return -1;

Once the IPSDK environment correctly initialized, we load our input image from the associated TIFF file, by calling the function ipsdk::image::file::loadTiffImageFile. This function takes our object 'inImgFilePath" (of type boost::filesystem::path) as argument and returns an object of type ipsdk::image::ImagePtr, that contains our input image. We store it in "pInImg" variable.

The call of "loadTiffImageFile is enclosed in a try/catch block, to handle the case where an exception is thrown by loadTiffImageFile (if the file is not found or corrupted, for instance). If an error occurs, a message is displayed to the user, IPSDK environment is cleaned by calling "ipsdk::core::LibraryInitializer::getInstance().clear()" and the application terminates.

    // declare the variable that will contain the input image, loaded from TIFF file
    ipsdk::image::ImagePtr pInImg;

    try {
        // read input image from specified path
        pInImg = image::file::loadTiffImageFile(inImgFilePath);
    }
    catch (const image::file::IPSDKImageFileException& e) {

        // loadRawImageFile function threw an exception; display error log
        // message
        SAMPLE_SHAPEMATCHING2D_LOG_ERROR(
            [eSample_ShapeMatching2dMessage::eErrorLoadingInputImgFromTiffFile]
        % inImgFilePath.string() % e.getMsg());

        // clear IPSDK environment features; should be called before exiting
        // program
        ipsdk::core::LibraryInitializer::getInstance().clear();

        // quit the application with an exit code indicating an error
        return -1;
    }

In the same way, we load the template images. The paths are stored in a std::vector and we declare another std::vector containing the ipsdk ImagePtr objects.

    const ipUInt32 nbTemplates = static_cast<ipUInt32>(vInTemplateImgFilePath.size());

    // declare the collection of template images, loaded from TIFF file
    std::vector<ipsdk::image::ImagePtr> vTemplateImg;
    vTemplateImg.resize(nbTemplates);

    // For each template image path
    for (ipUInt32 i = 0; i < nbTemplates; ++i) {
        try {
            vTemplateImg[i] = image::file::loadTiffImageFile(vInTemplateImgFilePath[i]);
        }
        catch (const image::file::IPSDKImageFileException& e) {

            // loadRawImageFile function threw an exception; display error log
            // message
            SAMPLE_SHAPEMATCHING2D_LOG_ERROR(
                [eSample_ShapeMatching2dMessage::eErrorLoadingTemplateImgFromTiffFile]
            % vInTemplateImgFilePath[i].string() % e.getMsg());

            // clear IPSDK environment features; should be called before exiting
            // program
            ipsdk::core::LibraryInitializer::getInstance().clear();

            // quit the application with an exit code indicating an error
            return -1;
        }
    }

Since the default images only have two different intensities (the background intensity is 0 and the geometric shapes pixels have an intensity of 255), the application allows to blur the input image and the template images to smooth the edges. This step can also be used to smooth the noise. To skip this step, the user can set the Gaussian standard deviation to a value lower or equal than 0.

    if (stdDev > 0.f) {
        // Blur the images to degrade them and avoid smooth the edges
        try {
            const ipReal32 gaussRatio = 0.99f;
            ipsdk::imaproc::filter::gaussianSmoothing2dImg(pInImg, stdDev, stdDev, ipsdk::imaproc::attr::createGaussianCoverage(gaussRatio, 2), pInImg);
            for(ipUInt32 i = 0; i<nbTemplates; ++i)
                ipsdk::imaproc::filter::gaussianSmoothing2dImg(vTemplateImg[i], stdDev, stdDev, ipsdk::imaproc::attr::createGaussianCoverage(gaussRatio, 2), vTemplateImg[i]);
        }
        catch (const image::file::IPSDKImageFileException& e) {

            // loadRawImageFile function threw an exception; display error log
            // message
            SAMPLE_SHAPEMATCHING2D_LOG_ERROR(
                [eSample_ShapeMatching2dMessage::eErrorBlurringImages] % e.getMsg());

            // clear IPSDK environment features; should be called before exiting
            // program
            ipsdk::core::LibraryInitializer::getInstance().clear();

            // quit the application with an exit code indicating an error
            return -1;
        }
    }

Now the images are ready to be processed. We extract the shapes from the input image and from the template images. The shape templates are stored in a std::vector juste like for the ipsdk::image::ImagePtr template images.

    Shape2dCollPtr pShape2dColl;
    std::vector<Shape2dCollPtr> vTemplateShape2dCollPtr;

    // Label image pointer is declared out of the try-catch block because it is used later
    ImagePtr pLabelImg;
    try {
        // Binarize the input image
        ImagePtr pBinImg = ipsdk::imaproc::bin::thresholdImg(pInImg, thresholdBinarization, 255);

        // Compute the corresponding label image
        pLabelImg = ipsdk::imaproc::advmorpho::connectedComponent2dImg(pBinImg);

        // Extract the shapes from the label image
        pShape2dColl = ipsdk::imaproc::shape::segmentation::labelShapeExtraction2d(pLabelImg);

        // For each template input image
        for (ipUInt32 i = 0; i < nbTemplates; ++i) {
            // Binarize the current template image
            ImagePtr pBinImg_template = ipsdk::imaproc::bin::thresholdImg(vTemplateImg[i], thresholdBinarization, 255);
            // Compute the label image
            ImagePtr pLabelImg_template = ipsdk::imaproc::advmorpho::connectedComponent2dImg(pBinImg_template);

            // Extrac tthe current template shapes
            Shape2dCollPtr pShape2dColl_template = ipsdk::imaproc::shape::segmentation::labelShapeExtraction2d(pLabelImg_template);
            vTemplateShape2dCollPtr.push_back(pShape2dColl_template);
        }
    }
    catch (const image::file::IPSDKImageFileException& e) {

        // loadRawImageFile function threw an exception; display error log
        // message
        SAMPLE_SHAPEMATCHING2D_LOG_ERROR(
            [eSample_ShapeMatching2dMessage::eErrorExtractingShapes] % e.getMsg());

        // clear IPSDK environment features; should be called before exiting
        // program
        ipsdk::core::LibraryInitializer::getInstance().clear();

        // quit the application with an exit code indicating an error
        return -1;
    }

We can now use the extracted shapes to compute the matching by calling the function ipsdk::geom::matchShapes. This function takes a reference (i.e. template) shape and a shape collection (for instance, the one extracted from the input image) and computes for each of these shapes its distance to the template shape. Hence, the application have to loop over all the templates to know the closest template for each shape.

Note

In the case of default template images, there is only one shape to extract : a square, a triangle, a hexagon or a circle. For other images, more shape can be extracted because the template image may contain non connected objects. In this case, the application chooses the first shape.

Warning

IPSDK also extracts the background as a shape with index 0 in the shape collection. This shape is skipped during the process.

    std::vector<ipsdk::geom::MatchShapesInfo> vMatches;
    vMatches.resize(nbTemplates);

    // For each template
    for (ipUInt32 templateIdx = 0; templateIdx < nbTemplates; ++templateIdx) {
        // Compute the distance between the template and the shapes in the input image
        ipsdk::geom::MatchShapesInfo matches = ipsdk::geom::matchShapes(vTemplateShape2dCollPtr[templateIdx]->getColl()[1], pShape2dColl->getColl(), ipsdk::geom::eShapeHuDistanceType::eSDT_Type3, false);

        // Store the result
        vMatches[templateIdx] = matches;
    }

Once this step done, we know the distance for each shape with every template. The next step involves determining the closest distance to be able to match a template. The result is stored in the bestMatches variable, which contains two collections with the same size as the number of shapes extracted from the input image (excluding the background). The first collection stores the closest shape indices whereas the second one contains their distances.

The user can define a threshold, whose default value is 0.2. If the minimum distance is higher than this threshold, the application considers that the shape could not be matched with any template and the index is set to the numeric limit of the ipsdk::ipUInt32 type.

    // Declare and allocate the collection
    // bestMatches is the collection of (_shapeHuDistance, _shapeIdx) structures
    // where _shapeHuDistance is the minimum distance between a shape in the 
    // input image and the most similar template of index _shapeIdx
    ipsdk::geom::MatchShapesInfo bestMatches;
    const ipUInt32 nbShapes = static_cast<ipUInt32>(vMatches[0]._shapesIdx.size());
    bestMatches._shapesHuDistance.resize(nbShapes);
    bestMatches._shapesIdx.resize(nbShapes);

    // Initialize the best matches for each templates
    for (ipUInt32 shapeIdx = 0; shapeIdx < nbShapes; ++shapeIdx) {
        bestMatches._shapesHuDistance[shapeIdx] = NumericLimits<ipReal64>::max();

        // For each template
        for (ipUInt32 templateIdx = 0; templateIdx < nbTemplates; ++templateIdx) {
            // Check if we find a new minimum distance
            const ipsdk::geom::MatchShapesInfo matches = vMatches[templateIdx];
            if (bestMatches._shapesHuDistance[shapeIdx] > matches._shapesHuDistance[shapeIdx]) {
                bestMatches._shapesHuDistance[shapeIdx] = matches._shapesHuDistance[shapeIdx];
                bestMatches._shapesIdx[shapeIdx] = templateIdx;
            }
        }

        // Validate the match : if the distance is lower than the input threshold,
        // then the algorithm considers that the match is correct, otherwise, it
        // notes the match is unsatisfying by assigning the shape index to its 
        // maximum possible value
        if (bestMatches._shapesHuDistance[shapeIdx] >= thresholdDistance)
            bestMatches._shapesIdx[shapeIdx] = NumericLimits<ipUInt32>::max();
    }

Now, we know for each shape of the image its closest template. We use this information to generate the ouput image, by transforming the LUT of the previously calculated label image.

    // Initialize the LUT
    boost::shared_ptr<ipsdk::imaproc::attr::IntensityLUT> pLut = ipsdk::imaproc::attr::IntensityLUT::createNode();
    pLut->setValue<ipsdk::imaproc::attr::IntensityLUT::BinWidth>(1.0);
    pLut->setValue<ipsdk::imaproc::attr::IntensityLUT::InMin>(.0);

    // Add the first value : the background is 0
    pLut->push_back<ipsdk::imaproc::attr::IntensityLUT::LookupTable>(0);

    // Add the other values
    const ipUInt32 nbLabels = pShape2dColl->getSize();
    for (ipUInt32 i = 1; i < nbLabels; ++i) {
        const ipUInt32 matchIndex = static_cast<ipUInt32>(bestMatches._shapesIdx[i - 1] + 1);
        if(matchIndex < NumericLimits<ipUInt32>::max())
            pLut->push_back<ipsdk::imaproc::attr::IntensityLUT::LookupTable>(matchIndex);
        else
            pLut->push_back<ipsdk::imaproc::attr::IntensityLUT::LookupTable>(0);
    }
    // Apply the LUT
    ImagePtr pMatchImg = ipsdk::imaproc::itrans::lutTransform2dImg(pLabelImg, pLut);

If all the previous operations were successfully completed, we save the image resulting from the matching operation to the TIFF file specified by the user (or to a default TIFF file, if the user did not specify anything):

    try {
        ipsdk::image::file::saveTiffImageFile(outImgFilePath, pMatchImg);
    }
    catch (const image::file::IPSDKImageFileException& e) {

        SAMPLE_SHAPEMATCHING2D_LOG_INFO(
            [eSample_ShapeMatching2dMessage::eErrorSavingImgTiffFile] % outImgFilePath % e.getMsg());

        // clear IPSDK environment features; should be called before exiting
        // program
        ipsdk::core::LibraryInitializer::getInstance().clear();

        // quit the application with an exit code indicating an error
        return -1;
    }

We also save the result of the matching in a CSV file. This file contains for each shape of the input image its index, the index of the correspponding template if its distances is lower than the distance threshold criterion (an error message is written otherwise) and its distance in question.

    // save matching results to output csv file
    ipBool bWritten = writeToCsvFile(outputCsvResultPath, bestMatches, thresholdDistance);
    if (bWritten == false) {

        SAMPLE_SHAPEMATCHING2D_LOG_INFO(
            [eSample_ShapeMatching2dMessage::eFailedToSaveCsvResultsFile] % outputCsvResultPath.string());

        // clear IPSDK environment features; should be called before exiting
        // program
        ipsdk::core::LibraryInitializer::getInstance().clear();

        return -1;
    }

The content of the writeToCsvFile function is :

ipBool writeToCsvFile(const boost::filesystem::path& fileName, const ipsdk::geom::MatchShapesInfo& bestMatches, const ipReal64 threshold)
{
    // Open the tile
    std::ofstream file(fileName.c_str(), std::ios::out | std::ios::trunc);
    if (!file)
        // Return false if the file does not exist
        return false;

    // Write the titles
    file << "Shape Index;Matched Template Index; Distance\n";

    // For each shape (do not process the background)
    const ipUInt32 nbTemplates = static_cast<ipUInt32>(bestMatches._shapesIdx.size());
    for (ipUInt32 i = 0; i < nbTemplates; i++) {
        file << (i + 1) << ";";

        // Test for a bad match
        if (bestMatches._shapesHuDistance[i] < threshold)
            file << bestMatches._shapesIdx[i];
        else
            file << "Bad match according to threshold ( = " << threshold << ")";
        file << ";" << bestMatches._shapesHuDistance[i] << "\n";
    }

    // Close the file
    file.close();

    return true;
}

The final action consists in cleaning the IPSDK environment before exiting. This cleaning operation (call to "ipsdk::core::LibraryInitializer::getInstance().clear()") must be the last call to IPSDK environment. It guarantees in particular that all threads created by IPSDK libraries are complete.

    // clearing IPSDK environment features; should be called before exiting
    // program
    ipsdk::core::LibraryInitializer::getInstance().clear();
    return 0;
}

See the full source listing