Benchmarking : Smil vs scikit-image

Morphological Image Libraries

nestor - Resources : CPU and memory usage


This page shows how both libraries make use of CPU and memory when handling a gray level image of size 16384x16384.

The test image was created as a mosaic of 64x64 images of the traditional image lena (which original size is 256x256).

In order to ease visual comparison, vertical axes (memory size or CPU) are presented in the same scale while horizontal axes (time) not because time complexity aren't the same in both libraries.

See the discussion page for comments.

open

Image lena.png - mosaic 16384x16384
CPU
Smil skImage
Memory usage
Smil skImage

Image handling time


Image size Smil (ms) skImage (ms) SpeedUp
16384x16384 70.724 8598.485 121.578

hMinima

Image lena.png - mosaic 16384x16384 Image lena.png - mosaic 8192x8192
CPU
Smil skImage Smil skImage
Memory usage
Smil skImage Smil skImage

As explained in the methodology page, we weren't able to run this test on the nestor computer with skimage library because the available memory (16 GiB RAM) limit was reached and the process begun to swap. So we decided to run this test also with an image size four times smaller (8192x8192).

Image handling time


Image size Smil (ms) skImage (ms) SpeedUp
8192x8192 23713 56290 2.374
16384x16384 98995 4705695 47.534
32768x32768 406021 N.A. N.A.

The screenshot below confirms that the process is making use of the swap space mainly by the high percentual of CPU time in waiting state and the strong presence of process kswapd0.


top - 14:18:49 up 5 days, 19:43,  5 users,  load average: 5.31, 2.93, 1.24
Tasks: 242 total,   2 running, 240 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  4.0 sy,  0.0 ni, 33.0 id, 62.6 wa,  0.0 hi,  0.1 si,  0.0 st
MiB Mem :  15899.8 total,    130.5 free,  15664.6 used,    104.7 buff/cache
MiB Swap:  15625.0 total,   6828.4 free,   8796.6 used.     19.6 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
    129 root      20   0       0      0      0 D  14.2   0.0   1:35.49 kswapd0
  70097 martins   20   0   23.7g  14.6g    108 D  13.2  93.8   3:25.95 python3
    ...

A fast test had shown that, in this particular computer, Smil was still able to run this function with images of size 32768x32768 (~1 GiPixels), while the limit for scikit-image is something around 12288x12288 (~144 MiPixels), so around 7 times bigger in the pixel count, without the help of swap memory. Obviously, this assumption is valid only for this function and this platform.

watershed

Image lena.png - mosaic 16384x16384
CPU
Smil skImage
Memory usage
Smil skImage

Image handling time


Image size Smil (ms) skImage (ms) SpeedUp
16384x16384 24116.282 352619.320 14.622