semantic_segmentation_augmentations
Data augmentation is a regularisation technique that generates new training samples from the original dataset by applying colour or geometric transformations. Although this technique has been applied in other computer vision fields, such as image classification or object detection, the application of this technique in semantic segmentation is not yet widespread.
This library groups some data augmentation techniques for semantic augmentation, such as CutOut
or CutMix
.
Install
To install the library, run:
pip install semantic_segmentation_augmentations
How it works
This data augmentation techniques are defined as fastai
callbacks
. Given this fact, you need to train your models using fastai
’s API to use them.
As these techniques modifies the input image changing some pixels regions for something else, those callbacks have been defined as the union of two subcomponents: the HoleMakerTechnique
, that defines how to make the hole (how to select the region to replace) and the HolesFilling
, that defines how to fill the region defined below (the way to fill the region gives the name to the technique used).
The HoleMakerTechnique
can be replaced in order to change the behavior of the selection of the region to replace. Doing so, you can use a CutOut technique that select the region randomly or based on the information bounded in that region.
You can also define your custom techniques defining how to fill a hole. You just need to extend the HolesFilling
class and define the before_bacth
(remember: we are using callbacks
from fastai
) abstract method. You will have two methods to simplify the process: the make_hole
function, that uses the selected HoleMakerTechnique
to make the hole and returns two slices (the region boundaries) and the fill_hole
function, that fills the hole with something.
How to use it
In order to use these techniques, you just need to define the Learner
from fastai
with the Callbacks
that represent the techniques that you want to use.
For example, if you want to create an U-net
Leaner
with resnet18
backbone and use the CutOut technique with HoleMakerRandom
as region selection techinique (which is the default one), you just need to import it:
from semantic_segmentation_augmentations.holesfilling import CutOutRandom
And add it to your learner:
learner = unet_learner(dls, resnet18, cbs = CutOutRandom(hole_maker = HoleMakerRandom()))
If you want to change the default behaviour of the CutOutRandom technique (i. e. to select the region with a HoleMakerAttention
technique), you just need to define the technique as follows:
learner = unet_learner(dls, resnet18, cbs = CutOutRandom(hole_maker = HoleMakerAttention(attention_threshold = 0.25, hole_size = (150, 150))))
Finally, you can apply geometrical augments to the selected regions (using the Albumentations
augments). Maybe this augmnets are not useful with the CutOut
techniques, but with the CutMix
ones. In order to do so, you just need to implement a RegionModifier
with a list with the augments to apply:
learner = unet_learner( dls,
resnet18,
cbs = CutMixRandom( holes_num = 1,
modifier = RegionModifier(A.Compose([A.Blur(), A.GaussNoise()])),
hole_maker = HoleMakerRandom((250, 250)),
p = 1
)
)