.. _sec-pipelines:

Pipelines
=========

BIP pipelines have two purposes. First, they simplify and accelerate the design and the application of image processing sequences composed of several steps. Second, they also facilitate tracing the operations that have been applied to images.

Writing and using pipeline files
--------------------------------

Consider the following 3-step sequence: Gaussian filtering, Otsu’s thresholding, and object labelling. Without pipelines, the corresponding commands to enter into the terminal would be:

::

   bip gaussian-filter 1.0 image.tif
   bip otsu-thresholding image-gaussian-filter.tif
   bip labelling image-gaussian-filter-otsu-thresholding.tif

This is long, both to type and to execute because of writing/reading operations of intermediate files. This is also error-prone when applying this sequence to different files and at different times. In addition, though the intermediate files generated by such a sequence may be of interest at the design stage, they are generally of no interest in routine application. Using a pipeline can address all these issues at once.

To use a pipeline with BIP, you first create a text file containing the sequence of operations to apply. The file can also contain comments (pieces of text starting with the ’\ ``#``\ ’ character, which are ignored by BIP). For the above sequence, one would write the following code and save it to a file named, for example, ``process.bip``:

::

   # contents of process.bip
   gaussian-filter 1.0
   otsu-thresholding
   labelling

.. note::

   The syntax to invoke an operator from a pipeline is identical to the one used from the command line. This feature enables users to rapidly write pipelines as soon as they have learn to use BIP in simple command-line mode.

To apply a pipeline, one invokes the ``pipeline`` operator with the corresponding file:

::

   bip pipeline process.bip image.tif

When running a pipeline, the name of the output file for each input image is generated by concatenating the input image filename with the basename of the pipeline file. In our example, the result would be stored in ``image-process.tif``.

Of course, pipelines can be applied in batch as any other operator:

::

   bip pipeline process.bip ../input/*.tif

Pipelines without pipeline files
--------------------------------

In some applications, it may happen that short pipelines have to be applied and/or that keeping trace of the applied pipelines into text files is not required. In these cases, writing pipeline files may be cumbersome. BIP allows defining pipelines upon invocation of the ``pipeline`` operator.

*Pipeline expressions* are used to define pipelines upon invocation of the ``pipeline`` operator in the command-line. As in pipeline files, the syntax in pipeline expressions is identical to the syntax used to invoke operators individually. The only specificity in expression syntax is that the successive operators are separated by the ’\ ``|``\ ’ symbol (a.k.a. pipe operator under Unix/Linux). The whole expression should be enclosed within a pair of quotes.

The ``-e`` option of the ``pipeline`` operator is used to specify a pipeline expression instead of a pipeline file. Using a pipeline expression, our sample pipeline above would thus be applied using the following call:

::

   bip pipeline -e "gaussian-filter 1.0 | otsu-thresholding | labelling" image.tif

Pipeline variables
------------------

It is frequent that a pipeline does not consist in a purely sequential set of operations but instead requires that different images undergo different operations before being combined. As a prototypical example, consider the morphological tophat, defined for an image :math:`I` as:

.. math:: \mathrm{tophat}(I)=I - \mathrm{opening}(I)

(note this is a purely pedgogical example, as BIP provides a native ``tophat-white`` operator that implements this operation). In a pipeline, the opening step (here, using a circular structuring element of radius 2) can be computed by writing

::

   opening-filter ball 2

The problem is that after the execution of this pipeline line, the initial image is lost. This shows that two images are needed to compute the tophat: one to store the original image and another to store the opening of this image. Hence, we need to create a copy of the original image that can be retrieved after having computed the opening.

This is where variables come into play. Remember that the image to which the operators are applied in a pipeline is implicit. A variable is a name that designates this current image at a given step, the step corresponding to the point where the variable is declared. A variable declaration consists of the keyword ``store`` followed by the variable name, preceded by the special character ’\ ``$``\ ’:

::

   store $<variable-name>

The current image in a pipeline can be set to a previously stored image by invoking the ``recall`` operator followed by the corresponding variable name:

::

   recall $<variable-name>

In the tophat example, we need to create a variable to store the input (= current image at the beginning of the pipeline) image before computing its opening. We then recall the original image and subtract the opening. This yields the following pipeline:

::

   # save a copy of the input image somewhere in memory
   store $image

   # replace current image by its morphological opening
   # and store the result somewhere in memory
   opening ball 2
   store $opening

   # set the current image to the previously stored input image
   recall $image

   # subtract the morphological opening from the current image
   substract $opening