A strong bilateral filter smooths flat regions while keeping edges sharp, and therefore is great as an automatic cartoonifier or painting filter, except that it is extremely slow (that is, measured in seconds or even minutes, rather than milliseconds!). Therefore, we will use some tricks to obtain a nice cartoonifier, while still running at an acceptable speed. The most important trick we can use is that we can perform bilateral filtering at a lower resolution and it will still have a similar effect as a full resolution, but run much faster. Let's reduce the total number of pixels by four (for example, half width and half height):
Size size = srcColor.size();
Size smallSize;
smallSize.width = size.width/2;
smallSize.height = size.height/2;
Mat smallImg = Mat(smallSize, CV_8UC3);
resize(srcColor, smallImg, smallSize, 0,0, INTER_LINEAR);
Rather than applying a large bilateral filter, we will apply many small bilateral filters, to produce a strong cartoon effect in less time. We will truncate the filter (refer to the following diagram) so that instead of performing a whole filter (for example, a filter size of 21 x 21, when the bell curve is 21 pixels wide), it just uses the minimum filter size needed for a convincing result (for example, with a filter size of just 9 x 9 even if the bell curve is 21 pixels wide). This truncated filter will apply the major part of the filter (gray area) without wasting time on the minor part of the filter (white area under the curve), so it will run several times faster:
Therefore, we have four parameters that control the bilateral filter: color strength, positional strength, size, and repetition count. We need a temp Mat since the bilateralFilter() function can't overwrite its input (referred to as in-place processing), but we can apply one filter storing a temp Mat and another filter storing back the input:
Mat tmp = Mat(smallSize, CV_8UC3);
auto repetitions = 7; // Repetitions for strong cartoon effect.
for (auto i=0; i<repetitions; i++) {
auto ksize = 9; // Filter size. Has large effect on speed.
double sigmaColor = 9; // Filter color strength.
double sigmaSpace = 7; // Spatial strength. Affects speed.
bilateralFilter(smallImg, tmp, ksize, sigmaColor, sigmaSpace);
bilateralFilter(tmp, smallImg, ksize, sigmaColor, sigmaSpace);
}
Remember that this was applied to the shrunken image, so we need to expand the image back to the original size. Then, we can overlay the edge mask that we found earlier. To overlay the edge mask sketch onto the bilateral filter painting (left-hand side of the following image), we can start with a black background and copy the painting pixels that aren't edges in the sketch mask:
Mat bigImg;
resize(smallImg, bigImg, size, 0,0, INTER_LINEAR);
dst.setTo(0);
bigImg.copyTo(dst, mask);
The result is a cartoon version of the original photo, as shown on the right-hand side of the following image, where the sketch mask is overlaid on the painting:
