- Email: [email protected]
Reduction of bubble-like frames using a RSS filter in wireless capsule endoscopy video
Optics and Laser Technology 110 (2019) 152–157
Contents lists available at ScienceDirect
Optics and Laser Technology journal homepage: www.elsevier.com/locate/optlastec
Full length article
Reduction of bubble-like frames using a RSS filter in wireless capsule endoscopy video
T
⁎
Qian Wanga,d, Ning Panb,d, , Wei Xiongc, Heng Lue, Nan Lif, Xijing Zoua a
School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan 430073, China College of Biomedical Engineering, South Central University for Nationalities, Wuhan 430074, China c College of Computer Science, South Central University for Nationalities, Wuhan 430074, China d Hubei Key Laboratory of Medical Information Analysis and Tumor Diagnosis & Treatment, Wuhan 430074, China e Department of Gastroenterology and Hepatology, Jinling Hospital, Nanjing University School of Medicine, Nanjing 210002, China f Department of Gastroenterology, Zhongda Hospital Affiliated to Southeast University, Nanjing 210009, China b
H I GH L IG H T S
novel automatic approach based on morphology properties to detect the bubble-like frames in capsule video endoscopy. • AHessian eigen values are used to construct the ring selective filters to enhance the boundary areas of the bubbles. • Both of hematrix local edge and the global round shape properties have been considered in the ring selective filter. • The bubble-like frames under the no uniform illumination conditions can also be detected effectively. •
A R T I C LE I N FO
A B S T R A C T
Keywords: Wireless capsule endoscopy Image sensor Bubble-like frames Hessian matrix eigenvalues RSS filter
Wireless capsule endoscopy (WCE) is an advanced technique for inner visualization of the digestive tract. However, its large amount of video data collected continuously for several hours by wireless image sensors usually results in tiresome and tedious reading for physicians. To develop this technique in practice, the automatic reduction of large amounts of video data arises as a helpful alternative, which is partly implemented by detecting the non-useful frames, such as those filled with intestinal juices accompanying large bubble-like regions. Based on the bubbles’ morphology properties, a novel automatic approach for detecting the bubble-like frames in capsule video endoscopy that is based on a new ring shape selective (RSS) filter is presented. According to the shape of the bubbles’ bright ring boundary, a group of the shape probability functions and the response intensity functions produced by Hessian matrix eigenvalues are used to construct the RSS filters to enhance the boundary areas of the bubbles. Then, the bubble-like frames can be distinguished by in the bright areas after binary and morphological processing in the filtered images. The experimental results show that the uninformative bubble-like frames can be detected effectively by this approach even under non-uniform illumination conditions.
1. Introduction WCE is a new technology for detecting gastrointestinal diseases that has flourished quickly in recent years [1,2]. The typical WCE system is usually composed of the four parts shown in Fig. 1, which include a capsule endoscope device, a wireless sensor array, a data logger and an image data browsing workstation. When this system works to check the digestive tract, the wireless sensor array and data logger are worn on the patient's abdomen first, and then the capsule endoscope is swallowed by the patient, which automatically moves by gastrointestinal
⁎
peristalsis and is finally discharged through the anus. During this process, the real-time video data from the digestive tract are collected from CMOS image sensors in the capsule endoscope device. At the same time, these data are transmitted to the data logger through the outside wireless sensor array. The physician then reads the video data that is downloaded to the workstation from the data logger. Compared with the traditional examination, this method is noninvasive, more convenient and has a wider visual field. Thus, many hospitals worldwide have chosen the WCE as a first device for examining diseases and abnormalities in the digestive tract, especially in
Corresponding author. E-mail addresses: [email protected] (Q. Wang), [email protected] (N. Pan), [email protected] (W. Xiong).
https://doi.org/10.1016/j.optlastec.2018.08.051 Received 29 June 2018; Received in revised form 27 August 2018; Accepted 29 August 2018 Available online 05 September 2018 0030-3992/ © 2018 Elsevier Ltd. All rights reserved.
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
Fig. 1. The four parts of the wireless capsule endoscope system. (a) Capsule endoscope device, (b) wireless sensor array, (c) data logger, and (d) image data browsing workstation.
2. Principles of the RSS filter
the small intestine [3,4]. However, there is a major drawback of WCE that might restrict its development and popularization in the clinic, which is the large amount of photographic data that needs to be read [4–6]. Generally, this whole examination takes approximately 6–8 h, and if the real-time photographs were recorded by WCE every 2 s, approximately 40–60 thousand frames are produced during the examination. Such a large amount of frame data exhausts the physicians that read the data, which easily causes misdiagnosis [5,7]. To resolve this problem, some researchers have tried to reduce the images without useful diagnostic information by automatically detecting them with the computer to improve the diagnosis efficiency [8]. One category of these non-useful information frames contains the regions of the small intestinal wall hindered by intestinal juices mixed with the remains of food, which is always accompanied by the presence of plentiful bubbles [9,10]. Sometimes, the bubble-like frames may account for approximately 20% of all videos. At present, two main methods have been reported for analyzing the WCE image semantics [11–13] used to detect bubble-like frames. One method is color property-based. Bejakovic [14] used an MPEG-7 dominant color descriptor to detect bubbles; the detection accuracy was 87.3%. Bashar [15,16] used local color moments and HSV color histogram to determine highly contaminated nonbubbled frames as the first step to detecting bubbles-like frames. Due to the unpredictability of the imaging environment in the digestive tract, the single-color-based detection method might be invalid under non-uniform illumination conditions. At this time, the other method based on morphology properties is more compatible. Bejakovic [14] also tried to compute the edge histogram descriptor to represent the bubbles’ light boundary, and the detection accuracy was 88.9%. Vilarino [17] tried to detect the bubble-like frames using texture information extracted by a group of Gabor filters in multiple directions and scales; in their experiments, approximately 95% of the bubble-like frames were detected exactly, and 5% mistaken bubble-like frames were detected. Bashar [15,16] implemented a Gauss Laguerre transform-based multi-resolution texture feature to characterize the bubble structures in capsule video frames; they obtained 85.4% of the average detection accuracy by two groups of frames from 6 videos. In our research, we found that the ring shape is an essential morphological property of the bubbles. Thus, a new ring-shape selective filter constructed by eigenvalues of the Hessian matrix was used to enhance the bubble-like regions in frames. Due to the combination of the local edge shape and the global round shape in this ring filter, relatively high accuracy has been found for our morphology propertybased method to detect and reduce the bubble-like frames in capsule video endoscopy. The details of our approach are as follows.
2.1. Shape selective filters based on Hessian matrix Eigenvalues of the Hessian matrix have significant geometric implications that are widely applied to filter and enhance medical images [18–20]. In a two-dimensional image, a Hessian matrix, denoted by H, can be constructed for each pixel (x , y ) by computing the second-order partial differential of the image density function f. It can be formulated as follows:
⎡ fxx fxy ⎤ H=⎢ f f ⎥ ⎣ yx yy ⎦
(1)
In contrast, fxx , fxy , f yx and f yy are the second-order partial differential at pixel (x , y ) of the image density function f in the different directions. In the X direction (horizontal direction), the second-order partial differential is fxx ,
fxx =
∂ 2f = f (x −1, y ) + f (x + 1, y )−2f (x , y ) ∂x 2
(2)
In the Y direction (vertical direction), the second-order partial differential is f yy ,
f yy =
∂ 2f = f (x , y−1) + f (x , y + 1)−2f (x , y ) ∂y 2
(3)
On the diagonal line of the X and Y directions, the mixture secondorder partial differential fxy and f yx can be calculated as follows,
fxy = f yx
∂ 2f = f (x + 1, y + 1) + f (x , y )−f (x + 1, y )−f (x , y + 1) ∂x ∂y (4)
They describe the varieties of the image density in these four directions. H is a real symmetric matrix; it must have two real eigenvalues. We denote them as λ1 and λ2 , which can be obtained by the following formulations.
λ1 = K +
K 2−Q 2
λ2 = K − K 2−Q 2
(5) (6)
where
K = (fxx + f yy )/2
(7)
Q=
(8)
fxx f yy −fxy f yx
The mode values of these two eigenvalues represent special geometric implications. If an ellipse is used to approximate a local curve of 153
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
gcircle = |λ2|
(a) a bubble-like frame
In this group function, the absolute value of the nonzero eigenvalues is used to depict a similar degree between the current shape and the ideal shape. In detail, for a point on a line-like shape, the larger absolute value of the nonzero eigenvalues (that is, |λ1|) is greater, which indicates that the length of the long axis of the approximation ellipse to the local curve at this point is longer. Thus, this local curve looks more like a line. In another case, while the point is on a round-like shape, the smaller absolute value of the nonzero eigenvalues (that is, |λ2|) is greater, which implies that the length of the short axis of the approximation ellipse to the local curve at this point is long and may be closer to the length of the long axis. Thus, this local shape looks more like a circle. In other words, the greater the value of the response intensity function is, the more the degree looks like the corresponding shape. Based on the above, the output functions of the line shape and circle shape selective filters can be defined as follows. They are denoted by z line and z circle , respectively.
(b) a normal tissue frame
Fig. 2. Frames in video sensing by wireless capsule endoscopy.
the image density function on its gradient direction at point(x , y ) , the lengths of the long axis and short axis for this ellipse correspond to the mode values of λ1 and λ2 (|λ1| ⩾ |λ2|). Varying the length of the two axes can shift the shape of the ellipse, which can exactly describe the properties of the curve shape. Thus, designing the corresponding functions of the eigenvalues can distinguish different shapes [21]. Ideally, we analyze the features of the eigenvalues corresponding to the shape of the line and the circle. For a line, the value of |λ1| is a positive real number, and the value of |λ2| equals zero. However, for a point, the value of |λ1| is a positive real number and equals the value of|λ2|. Thus, a function η can be used to distinguish these two shapes, where
η=
|λ2 | |λ1 |
(13)
z line = pline gline = |λ1−λ2| z circle = pcircle gcircle =
(14)
|2
|λ2 |λ1 |
(15)
Furthermore, a point on the bright rings of the bubble edge can be considered as belonging to a line boundary and attributed to a circle structure at the same time. Thus, the output function of the bubble-like (ring) shape selective filter can be designed as follows.
(9)
As the value of the function η approaches 1, the local curve becomes more similar to the circle shape. Otherwise, as the value of the function η approaches 0, the local curve becomes more similar to the line shape.
z ring = z line z circle =
|λ1−λ2|·|λ2 |2 |λ1 |
(16)
2.2. The construction of the RSS filter 3. Automatic detection of bubble-like frames As shown in Fig. 2(a), the bubbles in the frames of the capsule endoscopy video appear as bright rings, while the tissues, such as the walls of the small intestines, take the shape of thick lines or planes, as shown in Fig. 2(b). Thus, the shape is an effective characteristic for identifying the bubble-like areas. It can be enhanced by a RSS filter constructed by a Hessian matrix. A ring can be considered the combination of a line and a circle, where the line property can be regarded as its local edge shape and the circle property can be regarded as its global round shape. A group of eigenvalue functions is introduced by the Hessian matrix to describe a line and circle shape and can be denoted by pline and pcircle , respectively, which are the shape probability function and are defined as follows,
pline =
pcircle =
|λ1|−|λ2 | |λ1 |
|λ2 | |λ1 |
At the first stage, the original frames of the capsule video endoscopy should be processed by the RSS filter defined in Formula (16). We denote a filtered image as M , and p is a pixel in M with the intensityM (p) . The boundaries of the bubbles correspond to the relatively light pixels in M enhanced by RRS in a frame. Thus, a binary image MB can be obtained by Formula (17) with a thresholdt .
1, M (p) ⩾ t MB (p) = ⎧ ⎨ ⎩ 0, M (p) < t t=
t0 ∑ M (p) |M|
(18)
where t0 is a given threshold (in our experiments, it is set to 100/
(10)
255) and the denominator term ∑ M (p) represents the mean value of |M| the enhanced image M , which can improve the adaptability of threshold t between the images with small and large bubble areas. After binarization of the enhanced images, some morphology methods are used to improve the results of the bubble-like areas. Some physicians have suggested that a frame containing more than 20–50% (in our experiments, the area ratio is prescribed as 30%) is a non-useful region, such as bubbles and juices, and can be considered uninformative for the diagnosis. Thus, the detection decision is made: if the light regions in MB overlap more than 30% of the whole effective area, the corresponding frame can be regarded as bubble-like uninformative and should be removed from the original video. The detailed steps to automatically detect the bubble-like frames are as follows.
(11)
The possibility of a point belonging to a line or a circle shape is given by the value of pline and pcircle , ranging from 0 to 1. In detail, if the point is on an ideal line, its response value of pline equals 1; meanwhile, the response value of pcircle equals 0. Otherwise, if the point is on an ideal circle, its response value of pcircle equals 1, and the response value of pline equals 0. In the most practical situations, if the point is on the other shape, the response values of both pline and pcircle are between 0 and 1. Thus, this group of shape probability functions mainly focuses on distinguishing different shapes. Additionally, to improve the performance of the shape-selective enhancement, the output value of the filter can not only distinguish different shapes effectively but also describe the various degrees of the similar shapes accurately. Another group of eigenvalue functions named the response intensity functions for the line and the circle are defined as follows and are denoted by gline and gcircle , respectively.
gline = |λ1|
(17)
Step 1 For each frame, the gray imageF should first be transformed from the color image; Step 2 The RRS filter defined in Formula (16) is used to processF , and the filtered image M is obtained to enhance the bubble-like areas with ring shapes;
(12) 154
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
Step 3 Binarize the image M by the threshold t defined in Formula (18); the binarization image MB is generated. Step 4 Optimize MB by the morphological close operation to fill the areas with small cavities. Step 5 Remove the small areas (less than 50 pixels) in MB ; Step 6 Make a detect decision to reduce the bubble-like frames based on the condition as follows,
MB (light areas ) > 30% F (effective areas )
(19)
4. Experiments and results We test our method with various WCE images from different patients provided by Nanjing Jinling Hospital of Nanjing University School of Medicine. The data in our experiments are divided into 4 groups. Each group contains 500 frames in the video from WCE with a size of 240 × 240. The effective area in the center of each frame contains 41,778 pixels. The possible range of the radius of bubbles is 2–15 pixels. The useless bubble-like frames selected by physicians are taken as the gold standard. We use the RSS filter to process each frame, and the filtered results are shown in Fig. 3. The bubble-like areas with ring shapes have been remarkably enhanced by the RSS filter, which appears as light-point regions in the second column in Fig. 3(b). At the same time, the regions of the small intestine walls with a thick line or plane shapes in the normal frames are converted into dark regions, which are shown in the second column in Fig. 3(a). In general, the enhanced light regions by the RSS filter correspond to the areas with numerous bubbles. Sometimes, the small intestinal lumen looks like a dark circular hole, and its boundary is always enhanced to form a light region by the RSS filter, which is shown in the second normal frame in Fig. 3(a). However, the bright spots caused by uneven illumination were not enhanced after filtering, as shown in the first and last normal frames in Fig. 3(a). The corresponding binary images processed by morphological methods on filtered images are shown in the third columns in both Fig. 3(a) and (b); the white areas are located mostly in the bubble areas. To further verify our method, a comparison has been performed between the RRS-based method and the Gabor-based method (proposed by Ref [14]) to detect the bubble-like frame from the same WCE video data. The partial results of these two methods are shown in Fig. 4. Because the Gabor-based method is a linear filter for edge extraction, some small bubbles with weak edges cannot be detected. The RSS considers both the edge-line and ring-shape features; thus, bubbles with weak edges and gastric juice with bright textures can also be detected well. Some statistical experiments were used to analyze the four groups of data. The detection sensitivity and specificity for the useless bubble-like frames in each group are shown in Table 1. When a frame with more than 30% bubble-like areas in the whole effective area is detected, the non-useless frame is removed. The average values of detection sensitivity are 92.7% and 82% of the RSS-based method and the Gabor-based method, respectively. The corresponding specificity was equal to 100% for both methods. Due to the random motion of the capsule endoscopy, the intensity of the frames has large diversity. Our approach can adapt to this variety in intensity owing to the shape selection. The data and results in Table 1 indicates that the RSS-based method can detect the redundant bubblelike frame more effectively, including the nonuniform illumination frames. Moreover, to give the unbiased statistic of the detection results, we have drawn ROC (receiver operating characteristic) curves [22] of our detection approach with the compared methods, as shown in Figs. 5 and 6.
(a) the normal frames with non- or small bubbles
(b) the non-useful frames with plentiful bubbles Fig. 3. The results processed by the RSS filter. (From the left to the right, the first column is the original frame, the second column is the enhanced image by the filter, and the third column is the binary image with bubble areas detected by this filter.)
155
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
Fig. 4. Detected results of bubble regions using RSS-based and Gabor-based methods. (a) Original frame images; (b) binary images using Gabor-based method; (c) binary images using RSS-based method; (d) overlapped images using Gabor-based method; (e) overlapped images using RSS-based method. Table 1 The statistical results of the detection sensitivity and specificity for the bubblelike frames. Group sequence
1 2 3 4
Proportions of useless bubblelike frames
Proportions of nonuniform illumination frames
Sensitivity
Specificity
RRS
Gabor
RRS
Gabor
19% 23% 28% 16%
20% 16% 24% 8%
82.8% 97% 100% 91.1%
75.9% 84.6% 80% 87.5%
100% 100% 100% 100%
100% 100% 100% 100%
Fig. 5. ROC curves of RRS-based and Gabor-based methods to detect useless bubble-like frames.
In Fig. 5, comparing the two ROC curves of the RSS-based and Gabor-based methods, most of the curve of the former method is above the latter one. Specifically, as the specificity equals 100%, there are no normal frames erroneously removed, which is better in practical clinical applications to avoid missing crucial information in each useful frame. Among the points on the ROC curves in this condition, the RRS-based
method clearly has better sensitivity. At the same time, we also gave the corresponding ROC curves of the LSS filter (line shape-selective filter, with the definition in Formula (14)) and the CRR (circle shape-selective filter, with the definition in Formula (15)) to detect the useless bubble-like frames in Fig. 6. When these three filters were compared, more accurate detection results were 156
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
References [1] O. Pietri, G. Rezgui, A. Histace, et al., Development and validation of an automated algorithm to evaluate the abundance of bubbles in small bowel capsule endoscopy, Endosc. Int. Open 6 (4) (2018) E462–E469. [2] T. Ghosh, S.A. Fattah, K.A. Wahid, et al., Cluster based statistical feature extraction method for automatic bleeding detection in wireless capsule endoscopy video, Comput. Biol. Med. 94 (2018). [3] I.N. Figueiredo, C. Leal, L. Pinto, et al., Dissimilarity measure of consecutive frames in wireless capsule endoscopy videos: a way of searching for abnormalities, IEEE, International Symposium on Computer-Based Medical Systems (2017) 702–707. [4] S. Charfi, M.E. Ansari, Computer-aided diagnosis system for ulcer detection in wireless capsule endoscopy videos, International Conference on Advanced Technologies for Signal and Image Processing (2017) 1–5. [5] S. Suman, F.A. Hussin, N. Walter, et al., Automatic Detection and Removal of Bubble Frames from Wireless Capsule Endoscopy Video, IEEE, 2017. [6] M. Nakamura, A. Murino, A. O’Rourke, et al., A critical analysis of the effect of view mode and frame rate on reading time and lesion detection during capsule endoscopy, Digest. Dis. Sci. (2015) 1–5. [7] S. Segui, M. Drozdzal, F. Vilarino, et al. Categorization and segmentation of intestinal content frames for wireless capsule endoscopy, in: IEEE Transactions on Information Technology in Biomedicine, vol. 16, no. 6, pp. 1341–1352. [8] H. Liu, N. Pan, H. Lu, et al., Wireless capsule endoscopy video reduction based on camera motion estimation, J. Digital Imaging 26 (2) (2013) 287–301. [9] O. Haji Maghsoudi, A. Talebpour, H. Soltanian-Zadeh, et al. Automatic informative tissue's discriminators in WCE, in: 2012 IEEE International Conference on Proceedings of the Imaging Systems and Techniques (IST), IEEE, pp. 18–23. [10] O.H. Maghsoudi, A. Talebpour, H. Soltanian-Zadeh, et al., Informative and uninformative regions detection in WCE frames, J. Adv. Comput. 3 (1) (2014) 12–34. [11] Q. Zhou, W. Yang, G. Gao, et al., Multi-scale deep context convolutional neural networks for semantic segmentation, World Wide Web-internet Web Inform. Syst. 7 (2018) 1–16. [12] Q. Zhou, B. Zheng, W. Zhu, et al., Multi-scale context for scene labeling via flexible segmentation graph, Pattern Recogn. 59 (C) (2016) 312–324. [13] W. Yang, Q. Zhou, Y. Fan, et al., Deep context convolutional neural networks for semantic segmentation, CCF Chinese Conference on Computer Vision (2017) 696–704. [14] S. Bejakovic, R. Kumar, T. Dassopoulos, et al., Analysis of Crohn's disease lesions in capsule endoscopy images, in: 2009 ICRA'09 IEEE International Conference on Proceedings of the Robotics and Automation, IEEE, 2009, pp. 2793–2798. [15] M.K. Bashar, T. Kitasaka, Y. Suenaga, et al., Automatic detection of informative frames from wireless capsule endoscopy images, Med. Image Anal. 14 (3) (2010) 449–470. [16] M.K. Bashar, K. Mori, Y. Suenaga, Detecting informative frames from wireless capsule endoscopic video using color and texture features, Medical Image Computing and Computer-Assisted Intervention-miccai ICCAI (2008) 603–610. [17] F. Vilarino, P. Spyridonos, O. Pujol, et al., 2006 ICPR 2006 18th International Conference on Proceedings of the Pattern Recognition, IEEE, 2006, pp. 719–722. [18] J. You, B. Chen, New approach to retinal image enhancement based on hessian matrix, Jisuanji Yingyong/J. Comput. Appl. 31 (6) (2011) 1560–1562. [19] R. Su, C. Sun, C. Zhang, et al. A new method for linear feature and junction enhancement in 2D images based on morphological operation, Oriented anisotropic gaussian function and hessian information. Pattern Recognition. [20] X. Jiang, Y. Qiu, An extraction method of cerebral vessels based on multi-threshold Otsu classification and hessian matrix enhancement filtering, Frontier and Future Development of Information Technology in Medicine and Education (2014) 2675–2682. [21] Q. Li, F. Li, K. Doi, Computerized detection of lung nodules in thin-section CT images by use of selective enhancement filters and an automated rule-based classifier, Acad. Radiol. 15 (2) (2008) 165-75. [22] J.A. Hanley, B.J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology 143 (1) (1982) 29–36. [23] S. Serikawa, H. Lu, Underwater Image Dehazing Using Joint Trilateral Filter, Pergamon Press, Inc., 2014. [24] H. Lu, Y. Li, S. Mu, et al., Motor anomaly detection for unmanned aerial vehicles using reinforcement learning, IEEE Int. Things J. (2017) (99):1-1. [25] H. Lu, Y. Li, M. Chen, et al., Brain intelligence: go beyond artificial intelligence, Mob. Netw. Appl. 23 (2) (2018) 368–375. [26] H. Lu, B. Li, J. Zhu, et al., Wound intensity correction and segmentation with convolutional neural networks, Concurr. Comput. Pract. Exper. (2017) 29(6).
Fig. 6. ROC curves of various shapes selective filter to detect useless bubble-like frames.
Fig. 7. A few poorly enhanced results by RSS.
obtained by the RSS filter because it has the uppermost ROC curve [22]. Thus, this newly constructed filter combining the line and the circle property is more effective in detecting the bubble-like frames than the filter based on only one of these two properties. In the bubble-like regions missed by our approach (e.g., Fig. 7), severe motion blurs can be found, which resulted in fuzzy lines or deformations of the ring structures for the bubbles. The preprocessing of motion blur reductions may help to improve these results. 5. Conclusions In our research, a new RSS filter constructed by eigenvalues of the Hessian matrix was used to process each frame in a video from WCE. The bubble-like regions in the frames were well enhanced in the filtered images even under nonuniform illumination conditions. With the combination of both the local edge and the global round shape properties in this RSS filter, this approach obtained higher detection accuracy than some existing filters, such as Gabor-based, LSS-based and CRR-based methods. Our future work will develop this method in the direction of a multiscale ring filter that can be self-adapting to various bubble sizes to be more robust. Additionally, more capsule endoscopy video data should be collected to validate our method through experiments, which can also provide the necessary data to develop intelligence methods [23–26] with machine learning in future work. Acknowledgments This work was supported by the National Natural Science Foundation of China (Grant No. 61602519); the Natural Science Foundation of Hubei Province, China (Grant No. 2013CFC090 and 2017CFB552); and the Applied Basic Research Programs of Wuhan, China (Grant No. 2017060201010160).
157
Contents lists available at ScienceDirect
Optics and Laser Technology journal homepage: www.elsevier.com/locate/optlastec
Full length article
Reduction of bubble-like frames using a RSS filter in wireless capsule endoscopy video
T
⁎
Qian Wanga,d, Ning Panb,d, , Wei Xiongc, Heng Lue, Nan Lif, Xijing Zoua a
School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan 430073, China College of Biomedical Engineering, South Central University for Nationalities, Wuhan 430074, China c College of Computer Science, South Central University for Nationalities, Wuhan 430074, China d Hubei Key Laboratory of Medical Information Analysis and Tumor Diagnosis & Treatment, Wuhan 430074, China e Department of Gastroenterology and Hepatology, Jinling Hospital, Nanjing University School of Medicine, Nanjing 210002, China f Department of Gastroenterology, Zhongda Hospital Affiliated to Southeast University, Nanjing 210009, China b
H I GH L IG H T S
novel automatic approach based on morphology properties to detect the bubble-like frames in capsule video endoscopy. • AHessian eigen values are used to construct the ring selective filters to enhance the boundary areas of the bubbles. • Both of hematrix local edge and the global round shape properties have been considered in the ring selective filter. • The bubble-like frames under the no uniform illumination conditions can also be detected effectively. •
A R T I C LE I N FO
A B S T R A C T
Keywords: Wireless capsule endoscopy Image sensor Bubble-like frames Hessian matrix eigenvalues RSS filter
Wireless capsule endoscopy (WCE) is an advanced technique for inner visualization of the digestive tract. However, its large amount of video data collected continuously for several hours by wireless image sensors usually results in tiresome and tedious reading for physicians. To develop this technique in practice, the automatic reduction of large amounts of video data arises as a helpful alternative, which is partly implemented by detecting the non-useful frames, such as those filled with intestinal juices accompanying large bubble-like regions. Based on the bubbles’ morphology properties, a novel automatic approach for detecting the bubble-like frames in capsule video endoscopy that is based on a new ring shape selective (RSS) filter is presented. According to the shape of the bubbles’ bright ring boundary, a group of the shape probability functions and the response intensity functions produced by Hessian matrix eigenvalues are used to construct the RSS filters to enhance the boundary areas of the bubbles. Then, the bubble-like frames can be distinguished by in the bright areas after binary and morphological processing in the filtered images. The experimental results show that the uninformative bubble-like frames can be detected effectively by this approach even under non-uniform illumination conditions.
1. Introduction WCE is a new technology for detecting gastrointestinal diseases that has flourished quickly in recent years [1,2]. The typical WCE system is usually composed of the four parts shown in Fig. 1, which include a capsule endoscope device, a wireless sensor array, a data logger and an image data browsing workstation. When this system works to check the digestive tract, the wireless sensor array and data logger are worn on the patient's abdomen first, and then the capsule endoscope is swallowed by the patient, which automatically moves by gastrointestinal
⁎
peristalsis and is finally discharged through the anus. During this process, the real-time video data from the digestive tract are collected from CMOS image sensors in the capsule endoscope device. At the same time, these data are transmitted to the data logger through the outside wireless sensor array. The physician then reads the video data that is downloaded to the workstation from the data logger. Compared with the traditional examination, this method is noninvasive, more convenient and has a wider visual field. Thus, many hospitals worldwide have chosen the WCE as a first device for examining diseases and abnormalities in the digestive tract, especially in
Corresponding author. E-mail addresses: [email protected] (Q. Wang), [email protected] (N. Pan), [email protected] (W. Xiong).
https://doi.org/10.1016/j.optlastec.2018.08.051 Received 29 June 2018; Received in revised form 27 August 2018; Accepted 29 August 2018 Available online 05 September 2018 0030-3992/ © 2018 Elsevier Ltd. All rights reserved.
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
Fig. 1. The four parts of the wireless capsule endoscope system. (a) Capsule endoscope device, (b) wireless sensor array, (c) data logger, and (d) image data browsing workstation.
2. Principles of the RSS filter
the small intestine [3,4]. However, there is a major drawback of WCE that might restrict its development and popularization in the clinic, which is the large amount of photographic data that needs to be read [4–6]. Generally, this whole examination takes approximately 6–8 h, and if the real-time photographs were recorded by WCE every 2 s, approximately 40–60 thousand frames are produced during the examination. Such a large amount of frame data exhausts the physicians that read the data, which easily causes misdiagnosis [5,7]. To resolve this problem, some researchers have tried to reduce the images without useful diagnostic information by automatically detecting them with the computer to improve the diagnosis efficiency [8]. One category of these non-useful information frames contains the regions of the small intestinal wall hindered by intestinal juices mixed with the remains of food, which is always accompanied by the presence of plentiful bubbles [9,10]. Sometimes, the bubble-like frames may account for approximately 20% of all videos. At present, two main methods have been reported for analyzing the WCE image semantics [11–13] used to detect bubble-like frames. One method is color property-based. Bejakovic [14] used an MPEG-7 dominant color descriptor to detect bubbles; the detection accuracy was 87.3%. Bashar [15,16] used local color moments and HSV color histogram to determine highly contaminated nonbubbled frames as the first step to detecting bubbles-like frames. Due to the unpredictability of the imaging environment in the digestive tract, the single-color-based detection method might be invalid under non-uniform illumination conditions. At this time, the other method based on morphology properties is more compatible. Bejakovic [14] also tried to compute the edge histogram descriptor to represent the bubbles’ light boundary, and the detection accuracy was 88.9%. Vilarino [17] tried to detect the bubble-like frames using texture information extracted by a group of Gabor filters in multiple directions and scales; in their experiments, approximately 95% of the bubble-like frames were detected exactly, and 5% mistaken bubble-like frames were detected. Bashar [15,16] implemented a Gauss Laguerre transform-based multi-resolution texture feature to characterize the bubble structures in capsule video frames; they obtained 85.4% of the average detection accuracy by two groups of frames from 6 videos. In our research, we found that the ring shape is an essential morphological property of the bubbles. Thus, a new ring-shape selective filter constructed by eigenvalues of the Hessian matrix was used to enhance the bubble-like regions in frames. Due to the combination of the local edge shape and the global round shape in this ring filter, relatively high accuracy has been found for our morphology propertybased method to detect and reduce the bubble-like frames in capsule video endoscopy. The details of our approach are as follows.
2.1. Shape selective filters based on Hessian matrix Eigenvalues of the Hessian matrix have significant geometric implications that are widely applied to filter and enhance medical images [18–20]. In a two-dimensional image, a Hessian matrix, denoted by H, can be constructed for each pixel (x , y ) by computing the second-order partial differential of the image density function f. It can be formulated as follows:
⎡ fxx fxy ⎤ H=⎢ f f ⎥ ⎣ yx yy ⎦
(1)
In contrast, fxx , fxy , f yx and f yy are the second-order partial differential at pixel (x , y ) of the image density function f in the different directions. In the X direction (horizontal direction), the second-order partial differential is fxx ,
fxx =
∂ 2f = f (x −1, y ) + f (x + 1, y )−2f (x , y ) ∂x 2
(2)
In the Y direction (vertical direction), the second-order partial differential is f yy ,
f yy =
∂ 2f = f (x , y−1) + f (x , y + 1)−2f (x , y ) ∂y 2
(3)
On the diagonal line of the X and Y directions, the mixture secondorder partial differential fxy and f yx can be calculated as follows,
fxy = f yx
∂ 2f = f (x + 1, y + 1) + f (x , y )−f (x + 1, y )−f (x , y + 1) ∂x ∂y (4)
They describe the varieties of the image density in these four directions. H is a real symmetric matrix; it must have two real eigenvalues. We denote them as λ1 and λ2 , which can be obtained by the following formulations.
λ1 = K +
K 2−Q 2
λ2 = K − K 2−Q 2
(5) (6)
where
K = (fxx + f yy )/2
(7)
Q=
(8)
fxx f yy −fxy f yx
The mode values of these two eigenvalues represent special geometric implications. If an ellipse is used to approximate a local curve of 153
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
gcircle = |λ2|
(a) a bubble-like frame
In this group function, the absolute value of the nonzero eigenvalues is used to depict a similar degree between the current shape and the ideal shape. In detail, for a point on a line-like shape, the larger absolute value of the nonzero eigenvalues (that is, |λ1|) is greater, which indicates that the length of the long axis of the approximation ellipse to the local curve at this point is longer. Thus, this local curve looks more like a line. In another case, while the point is on a round-like shape, the smaller absolute value of the nonzero eigenvalues (that is, |λ2|) is greater, which implies that the length of the short axis of the approximation ellipse to the local curve at this point is long and may be closer to the length of the long axis. Thus, this local shape looks more like a circle. In other words, the greater the value of the response intensity function is, the more the degree looks like the corresponding shape. Based on the above, the output functions of the line shape and circle shape selective filters can be defined as follows. They are denoted by z line and z circle , respectively.
(b) a normal tissue frame
Fig. 2. Frames in video sensing by wireless capsule endoscopy.
the image density function on its gradient direction at point(x , y ) , the lengths of the long axis and short axis for this ellipse correspond to the mode values of λ1 and λ2 (|λ1| ⩾ |λ2|). Varying the length of the two axes can shift the shape of the ellipse, which can exactly describe the properties of the curve shape. Thus, designing the corresponding functions of the eigenvalues can distinguish different shapes [21]. Ideally, we analyze the features of the eigenvalues corresponding to the shape of the line and the circle. For a line, the value of |λ1| is a positive real number, and the value of |λ2| equals zero. However, for a point, the value of |λ1| is a positive real number and equals the value of|λ2|. Thus, a function η can be used to distinguish these two shapes, where
η=
|λ2 | |λ1 |
(13)
z line = pline gline = |λ1−λ2| z circle = pcircle gcircle =
(14)
|2
|λ2 |λ1 |
(15)
Furthermore, a point on the bright rings of the bubble edge can be considered as belonging to a line boundary and attributed to a circle structure at the same time. Thus, the output function of the bubble-like (ring) shape selective filter can be designed as follows.
(9)
As the value of the function η approaches 1, the local curve becomes more similar to the circle shape. Otherwise, as the value of the function η approaches 0, the local curve becomes more similar to the line shape.
z ring = z line z circle =
|λ1−λ2|·|λ2 |2 |λ1 |
(16)
2.2. The construction of the RSS filter 3. Automatic detection of bubble-like frames As shown in Fig. 2(a), the bubbles in the frames of the capsule endoscopy video appear as bright rings, while the tissues, such as the walls of the small intestines, take the shape of thick lines or planes, as shown in Fig. 2(b). Thus, the shape is an effective characteristic for identifying the bubble-like areas. It can be enhanced by a RSS filter constructed by a Hessian matrix. A ring can be considered the combination of a line and a circle, where the line property can be regarded as its local edge shape and the circle property can be regarded as its global round shape. A group of eigenvalue functions is introduced by the Hessian matrix to describe a line and circle shape and can be denoted by pline and pcircle , respectively, which are the shape probability function and are defined as follows,
pline =
pcircle =
|λ1|−|λ2 | |λ1 |
|λ2 | |λ1 |
At the first stage, the original frames of the capsule video endoscopy should be processed by the RSS filter defined in Formula (16). We denote a filtered image as M , and p is a pixel in M with the intensityM (p) . The boundaries of the bubbles correspond to the relatively light pixels in M enhanced by RRS in a frame. Thus, a binary image MB can be obtained by Formula (17) with a thresholdt .
1, M (p) ⩾ t MB (p) = ⎧ ⎨ ⎩ 0, M (p) < t t=
t0 ∑ M (p) |M|
(18)
where t0 is a given threshold (in our experiments, it is set to 100/
(10)
255) and the denominator term ∑ M (p) represents the mean value of |M| the enhanced image M , which can improve the adaptability of threshold t between the images with small and large bubble areas. After binarization of the enhanced images, some morphology methods are used to improve the results of the bubble-like areas. Some physicians have suggested that a frame containing more than 20–50% (in our experiments, the area ratio is prescribed as 30%) is a non-useful region, such as bubbles and juices, and can be considered uninformative for the diagnosis. Thus, the detection decision is made: if the light regions in MB overlap more than 30% of the whole effective area, the corresponding frame can be regarded as bubble-like uninformative and should be removed from the original video. The detailed steps to automatically detect the bubble-like frames are as follows.
(11)
The possibility of a point belonging to a line or a circle shape is given by the value of pline and pcircle , ranging from 0 to 1. In detail, if the point is on an ideal line, its response value of pline equals 1; meanwhile, the response value of pcircle equals 0. Otherwise, if the point is on an ideal circle, its response value of pcircle equals 1, and the response value of pline equals 0. In the most practical situations, if the point is on the other shape, the response values of both pline and pcircle are between 0 and 1. Thus, this group of shape probability functions mainly focuses on distinguishing different shapes. Additionally, to improve the performance of the shape-selective enhancement, the output value of the filter can not only distinguish different shapes effectively but also describe the various degrees of the similar shapes accurately. Another group of eigenvalue functions named the response intensity functions for the line and the circle are defined as follows and are denoted by gline and gcircle , respectively.
gline = |λ1|
(17)
Step 1 For each frame, the gray imageF should first be transformed from the color image; Step 2 The RRS filter defined in Formula (16) is used to processF , and the filtered image M is obtained to enhance the bubble-like areas with ring shapes;
(12) 154
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
Step 3 Binarize the image M by the threshold t defined in Formula (18); the binarization image MB is generated. Step 4 Optimize MB by the morphological close operation to fill the areas with small cavities. Step 5 Remove the small areas (less than 50 pixels) in MB ; Step 6 Make a detect decision to reduce the bubble-like frames based on the condition as follows,
MB (light areas ) > 30% F (effective areas )
(19)
4. Experiments and results We test our method with various WCE images from different patients provided by Nanjing Jinling Hospital of Nanjing University School of Medicine. The data in our experiments are divided into 4 groups. Each group contains 500 frames in the video from WCE with a size of 240 × 240. The effective area in the center of each frame contains 41,778 pixels. The possible range of the radius of bubbles is 2–15 pixels. The useless bubble-like frames selected by physicians are taken as the gold standard. We use the RSS filter to process each frame, and the filtered results are shown in Fig. 3. The bubble-like areas with ring shapes have been remarkably enhanced by the RSS filter, which appears as light-point regions in the second column in Fig. 3(b). At the same time, the regions of the small intestine walls with a thick line or plane shapes in the normal frames are converted into dark regions, which are shown in the second column in Fig. 3(a). In general, the enhanced light regions by the RSS filter correspond to the areas with numerous bubbles. Sometimes, the small intestinal lumen looks like a dark circular hole, and its boundary is always enhanced to form a light region by the RSS filter, which is shown in the second normal frame in Fig. 3(a). However, the bright spots caused by uneven illumination were not enhanced after filtering, as shown in the first and last normal frames in Fig. 3(a). The corresponding binary images processed by morphological methods on filtered images are shown in the third columns in both Fig. 3(a) and (b); the white areas are located mostly in the bubble areas. To further verify our method, a comparison has been performed between the RRS-based method and the Gabor-based method (proposed by Ref [14]) to detect the bubble-like frame from the same WCE video data. The partial results of these two methods are shown in Fig. 4. Because the Gabor-based method is a linear filter for edge extraction, some small bubbles with weak edges cannot be detected. The RSS considers both the edge-line and ring-shape features; thus, bubbles with weak edges and gastric juice with bright textures can also be detected well. Some statistical experiments were used to analyze the four groups of data. The detection sensitivity and specificity for the useless bubble-like frames in each group are shown in Table 1. When a frame with more than 30% bubble-like areas in the whole effective area is detected, the non-useless frame is removed. The average values of detection sensitivity are 92.7% and 82% of the RSS-based method and the Gabor-based method, respectively. The corresponding specificity was equal to 100% for both methods. Due to the random motion of the capsule endoscopy, the intensity of the frames has large diversity. Our approach can adapt to this variety in intensity owing to the shape selection. The data and results in Table 1 indicates that the RSS-based method can detect the redundant bubblelike frame more effectively, including the nonuniform illumination frames. Moreover, to give the unbiased statistic of the detection results, we have drawn ROC (receiver operating characteristic) curves [22] of our detection approach with the compared methods, as shown in Figs. 5 and 6.
(a) the normal frames with non- or small bubbles
(b) the non-useful frames with plentiful bubbles Fig. 3. The results processed by the RSS filter. (From the left to the right, the first column is the original frame, the second column is the enhanced image by the filter, and the third column is the binary image with bubble areas detected by this filter.)
155
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
Fig. 4. Detected results of bubble regions using RSS-based and Gabor-based methods. (a) Original frame images; (b) binary images using Gabor-based method; (c) binary images using RSS-based method; (d) overlapped images using Gabor-based method; (e) overlapped images using RSS-based method. Table 1 The statistical results of the detection sensitivity and specificity for the bubblelike frames. Group sequence
1 2 3 4
Proportions of useless bubblelike frames
Proportions of nonuniform illumination frames
Sensitivity
Specificity
RRS
Gabor
RRS
Gabor
19% 23% 28% 16%
20% 16% 24% 8%
82.8% 97% 100% 91.1%
75.9% 84.6% 80% 87.5%
100% 100% 100% 100%
100% 100% 100% 100%
Fig. 5. ROC curves of RRS-based and Gabor-based methods to detect useless bubble-like frames.
In Fig. 5, comparing the two ROC curves of the RSS-based and Gabor-based methods, most of the curve of the former method is above the latter one. Specifically, as the specificity equals 100%, there are no normal frames erroneously removed, which is better in practical clinical applications to avoid missing crucial information in each useful frame. Among the points on the ROC curves in this condition, the RRS-based
method clearly has better sensitivity. At the same time, we also gave the corresponding ROC curves of the LSS filter (line shape-selective filter, with the definition in Formula (14)) and the CRR (circle shape-selective filter, with the definition in Formula (15)) to detect the useless bubble-like frames in Fig. 6. When these three filters were compared, more accurate detection results were 156
Optics and Laser Technology 110 (2019) 152–157
Q. Wang et al.
References [1] O. Pietri, G. Rezgui, A. Histace, et al., Development and validation of an automated algorithm to evaluate the abundance of bubbles in small bowel capsule endoscopy, Endosc. Int. Open 6 (4) (2018) E462–E469. [2] T. Ghosh, S.A. Fattah, K.A. Wahid, et al., Cluster based statistical feature extraction method for automatic bleeding detection in wireless capsule endoscopy video, Comput. Biol. Med. 94 (2018). [3] I.N. Figueiredo, C. Leal, L. Pinto, et al., Dissimilarity measure of consecutive frames in wireless capsule endoscopy videos: a way of searching for abnormalities, IEEE, International Symposium on Computer-Based Medical Systems (2017) 702–707. [4] S. Charfi, M.E. Ansari, Computer-aided diagnosis system for ulcer detection in wireless capsule endoscopy videos, International Conference on Advanced Technologies for Signal and Image Processing (2017) 1–5. [5] S. Suman, F.A. Hussin, N. Walter, et al., Automatic Detection and Removal of Bubble Frames from Wireless Capsule Endoscopy Video, IEEE, 2017. [6] M. Nakamura, A. Murino, A. O’Rourke, et al., A critical analysis of the effect of view mode and frame rate on reading time and lesion detection during capsule endoscopy, Digest. Dis. Sci. (2015) 1–5. [7] S. Segui, M. Drozdzal, F. Vilarino, et al. Categorization and segmentation of intestinal content frames for wireless capsule endoscopy, in: IEEE Transactions on Information Technology in Biomedicine, vol. 16, no. 6, pp. 1341–1352. [8] H. Liu, N. Pan, H. Lu, et al., Wireless capsule endoscopy video reduction based on camera motion estimation, J. Digital Imaging 26 (2) (2013) 287–301. [9] O. Haji Maghsoudi, A. Talebpour, H. Soltanian-Zadeh, et al. Automatic informative tissue's discriminators in WCE, in: 2012 IEEE International Conference on Proceedings of the Imaging Systems and Techniques (IST), IEEE, pp. 18–23. [10] O.H. Maghsoudi, A. Talebpour, H. Soltanian-Zadeh, et al., Informative and uninformative regions detection in WCE frames, J. Adv. Comput. 3 (1) (2014) 12–34. [11] Q. Zhou, W. Yang, G. Gao, et al., Multi-scale deep context convolutional neural networks for semantic segmentation, World Wide Web-internet Web Inform. Syst. 7 (2018) 1–16. [12] Q. Zhou, B. Zheng, W. Zhu, et al., Multi-scale context for scene labeling via flexible segmentation graph, Pattern Recogn. 59 (C) (2016) 312–324. [13] W. Yang, Q. Zhou, Y. Fan, et al., Deep context convolutional neural networks for semantic segmentation, CCF Chinese Conference on Computer Vision (2017) 696–704. [14] S. Bejakovic, R. Kumar, T. Dassopoulos, et al., Analysis of Crohn's disease lesions in capsule endoscopy images, in: 2009 ICRA'09 IEEE International Conference on Proceedings of the Robotics and Automation, IEEE, 2009, pp. 2793–2798. [15] M.K. Bashar, T. Kitasaka, Y. Suenaga, et al., Automatic detection of informative frames from wireless capsule endoscopy images, Med. Image Anal. 14 (3) (2010) 449–470. [16] M.K. Bashar, K. Mori, Y. Suenaga, Detecting informative frames from wireless capsule endoscopic video using color and texture features, Medical Image Computing and Computer-Assisted Intervention-miccai ICCAI (2008) 603–610. [17] F. Vilarino, P. Spyridonos, O. Pujol, et al., 2006 ICPR 2006 18th International Conference on Proceedings of the Pattern Recognition, IEEE, 2006, pp. 719–722. [18] J. You, B. Chen, New approach to retinal image enhancement based on hessian matrix, Jisuanji Yingyong/J. Comput. Appl. 31 (6) (2011) 1560–1562. [19] R. Su, C. Sun, C. Zhang, et al. A new method for linear feature and junction enhancement in 2D images based on morphological operation, Oriented anisotropic gaussian function and hessian information. Pattern Recognition. [20] X. Jiang, Y. Qiu, An extraction method of cerebral vessels based on multi-threshold Otsu classification and hessian matrix enhancement filtering, Frontier and Future Development of Information Technology in Medicine and Education (2014) 2675–2682. [21] Q. Li, F. Li, K. Doi, Computerized detection of lung nodules in thin-section CT images by use of selective enhancement filters and an automated rule-based classifier, Acad. Radiol. 15 (2) (2008) 165-75. [22] J.A. Hanley, B.J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology 143 (1) (1982) 29–36. [23] S. Serikawa, H. Lu, Underwater Image Dehazing Using Joint Trilateral Filter, Pergamon Press, Inc., 2014. [24] H. Lu, Y. Li, S. Mu, et al., Motor anomaly detection for unmanned aerial vehicles using reinforcement learning, IEEE Int. Things J. (2017) (99):1-1. [25] H. Lu, Y. Li, M. Chen, et al., Brain intelligence: go beyond artificial intelligence, Mob. Netw. Appl. 23 (2) (2018) 368–375. [26] H. Lu, B. Li, J. Zhu, et al., Wound intensity correction and segmentation with convolutional neural networks, Concurr. Comput. Pract. Exper. (2017) 29(6).
Fig. 6. ROC curves of various shapes selective filter to detect useless bubble-like frames.
Fig. 7. A few poorly enhanced results by RSS.
obtained by the RSS filter because it has the uppermost ROC curve [22]. Thus, this newly constructed filter combining the line and the circle property is more effective in detecting the bubble-like frames than the filter based on only one of these two properties. In the bubble-like regions missed by our approach (e.g., Fig. 7), severe motion blurs can be found, which resulted in fuzzy lines or deformations of the ring structures for the bubbles. The preprocessing of motion blur reductions may help to improve these results. 5. Conclusions In our research, a new RSS filter constructed by eigenvalues of the Hessian matrix was used to process each frame in a video from WCE. The bubble-like regions in the frames were well enhanced in the filtered images even under nonuniform illumination conditions. With the combination of both the local edge and the global round shape properties in this RSS filter, this approach obtained higher detection accuracy than some existing filters, such as Gabor-based, LSS-based and CRR-based methods. Our future work will develop this method in the direction of a multiscale ring filter that can be self-adapting to various bubble sizes to be more robust. Additionally, more capsule endoscopy video data should be collected to validate our method through experiments, which can also provide the necessary data to develop intelligence methods [23–26] with machine learning in future work. Acknowledgments This work was supported by the National Natural Science Foundation of China (Grant No. 61602519); the Natural Science Foundation of Hubei Province, China (Grant No. 2013CFC090 and 2017CFB552); and the Applied Basic Research Programs of Wuhan, China (Grant No. 2017060201010160).
157