Benchmarking : Smil vs scikit-image

Morphological Image Libraries
1. Home
Introduction
Site overview
About
2. Benchmarking
Comparing Libraries
Benchmark : what and how to
Functions equivalence
Test images
About Segmentation
Test platforms
3. Summary
Image size - taurus
Image size - nestor
StrElt size - taurus
StrElt size - nestor
Elapsed times
4a. Results - taurus
Binary images
Gray level images
Resources usage
4b. Results - nestor
Binary images
Gray level images
Resources usage
5. Conclusion
Discussion
Conclusion
Bibliography and links

Summary of elapsed times


Knowing the order of magnitude of elapsed time of each function is interesting as this allows us to identify where to concentrate efforts to optimize an application.

In this page we show, pour chaque function, the minimum and maximum of elapsed times for the set of submitted images, for two image sizes (2048x2048 and 8192x8192) and for the two test platforms.

An interesting indicator is the ratio between maximum and minimum values. This indicator is also shwon in the tables below. This ratio is always greater than 1.

Independently of the order of magnitude of elapsed times, surely very different, these ratio gives some interesting information.

What can be observed in these tables …

  • ratio is near 1. - this means that the algorithm doesn't depends on the image content (or complexity);

  • ratio is much bigger than 1. - this means that the algorithm depends on the the image content (or complexity). Usually an iterative algorithm. The bigger this ratio, the stronger is the dependency;

  • Smil and skImage ratios for the same function - probably different algorithms or programming paradigm. The library with the bigger ratio presents stronger dependency on the image content (complexity);

  • Equal ratios for binary and gray images - this is the trivial case for erode() and open() functions.

  • label() and areaThreshold() functions - clearly Smil is the looser here. The reason is that is that both the Smil implementation of label()= function is more general than in skImage and areaThreshold() makes a call to label() inside it. The situation can be reverted when replacing label() by fastLabel() which is a semi parallelized version of label()+. More about it in the discussion page.


Image size 2048x2048

Elapsed times for functions on binary images (ms)


Function nestor taurus
Smil skImage Smil skImage
Min Max Ratio Min Max Ratio Min Max Ratio Min Max Ratio
erode 0.37 0.39 1.05 53.10 53.70 1.01 0.10 0.11 1.13 52.20 55.30 1.06
open 0.69 0.71 1.03 111.00 112.00 1.01 0.19 0.22 1.19 109.00 111.00 1.02
distance 110.00 125.00 1.14 444.00 767.00 1.73 141.00 153.00 1.08 550.00 915.00 1.66
label 13.50 120.00 8.87 28.90 55.20 1.91 18.10 150.00 8.27 38.60 63.70 1.65
fastLabel 50.40 58.30 1.16 29.00 55.30 1.91 66.70 86.90 1.30 38.50 68.30 1.77
segmentation 727.00 888.00 1.22 655.00 7310.00 11.20 938.00 1130.00 1.20 812.00 6220.00 7.66
areaThreshold 28.00 165.00 5.89 54.50 109.00 2.00 48.70 229.00 4.71 71.30 145.00 2.03
thinning 1330.00 9280.00 6.98 7570.00 53200.00 7.03 772.00 5400.00 6.99 7780.00 54500.00 7.00

Elapsed times for functions on gray images (ms)


Function nestor taurus
Smil skImage Smil skImage
Min Max Ratio Min Max Ratio Min Max Ratio Min Max Ratio
erode 0.37 0.38 1.05 53.10 53.40 1.01 0.10 0.11 1.08 52.40 53.30 1.02
open 0.69 0.71 1.03 111.00 111.00 1.00 0.19 0.22 1.16 109.00 110.00 1.01
tophat 1.35 1.59 1.17 123.00 124.00 1.01 0.33 0.35 1.08 125.00 126.00 1.01
gradient 2.76 2.80 1.02 741.00 764.00 1.03 1.50 1.56 1.04 809.00 901.00 1.11
hMinima 786.00 1080.00 1.38 2470.00 3720.00 1.51 984.00 1160.00 1.17 2250.00 4140.00 1.84
watershed 217.00 270.00 1.24 1140.00 2160.00 1.89 335.00 389.00 1.16 1270.00 2320.00 1.82
segmentation 1080.00 1240.00 1.15 3140.00 4290.00 1.36 1420.00 1670.00 1.18 3690.00 4890.00 1.33
areaOpen 277.00 369.00 1.33 7390.00 30000.00 4.06 322.00 391.00 1.21 7430.00 29400.00 3.96


Image size 8192x8192

Elapsed times for functions on binary images (ms)


Function nestor taurus
Smil skImage Smil skImage
Min Max Ratio Min Max Ratio Min Max Ratio Min Max Ratio
erode 10.00 10.20 1.01 838.00 840.00 1.00 1.45 1.61 1.11 874.00 883.00 1.01
open 17.20 17.50 1.01 1740.00 1760.00 1.01 2.41 2.74 1.14 1760.00 1910.00 1.08
distance 3020.00 3080.00 1.02 9040.00 10800.00 1.19 5310.00 6080.00 1.15 17500.00 18700.00 1.07
label 214.00 2080.00 9.71 438.00 855.00 1.95 266.00 2200.00 8.27 620.00 1020.00 1.65
fastLabel 718.00 775.00 1.08 438.00 857.00 1.96 759.00 830.00 1.09 614.00 1030.00 1.68
segmentation 14100.00 20200.00 1.43 13100.00 39800.00 3.04 18100.00 27700.00 1.53 22200.00 44500.00 2.01
areaThreshold 419.00 2680.00 6.40 838.00 1720.00 2.05 717.00 3630.00 5.06 1170.00 2000.00 1.71
thinning 84200.00 595000.00 7.07 504000.00 3590000.00 7.13 47800.00 341000.00 7.13 514000.00 3670000.00 7.14

Elapsed times for functions on gray images (ms)


Function nestor taurus
Smil skImage Smil skImage
Min Max Ratio Min Max Ratio Min Max Ratio Min Max Ratio
erode 10.00 10.10 1.01 838.00 839.00 1.00 1.42 1.64 1.16 876.00 885.00 1.01
open 17.30 17.40 1.01 1750.00 1750.00 1.00 2.42 2.76 1.14 1770.00 1770.00 1.00
tophat 35.40 36.20 1.02 1940.00 1940.00 1.00 11.60 12.30 1.06 2000.00 2050.00 1.03
gradient 42.10 42.60 1.01 12000.00 12700.00 1.06 18.50 19.30 1.04 13100.00 14300.00 1.09
hMinima 15900.00 27300.00 1.71 53100.00 82600.00 1.56 17800.00 30800.00 1.73 48500.00 75800.00 1.56
watershed 3430.00 3800.00 1.11 14600.00 25500.00 1.74 5030.00 5220.00 1.04 17200.00 29900.00 1.74
segmentation 16800.00 18200.00 1.08 47600.00 59700.00 1.26 20100.00 22300.00 1.11 55800.00 69800.00 1.25
areaOpen 4920.00 6780.00 1.38 290000.00 2280000.00 7.86 5530.00 7910.00 1.43 269000.00 2500000.00 9.28