- Email: [email protected]
In-pit coal mine personnel uniqueness detection technology based on personnel positioning and face recognition
International Journal of Mining Science and Technology 23 (2013) 357–361
Contents lists available at SciVerse ScienceDirect
International Journal of Mining Science and Technology journal homepage: www.elsevier.com/locate/ijmst
In-pit coal mine personnel uniqueness detection technology based on personnel positioning and face recognition Sun Jiping, Li Chenxin ⇑ Research Institute of Information Engineering, China University of Mining & Technology, Beijing, Beijing 100083, China
a r t i c l e
i n f o
Article history: Received 29 August 2012 Received in revised form 16 September 2012 Accepted 21 December 2012 Available online 5 June 2013 Keywords: Coal mine Uniqueness detection Recognition of personnel positioning cards Face recognition Generalized symmetry transformation
a b s t r a c t Since the coal mine in-pit personnel positioning system neither can effectively achieve the function to detect the uniqueness of in-pit coal-mine personnel nor can identify and eliminate violations in attendance management such as multiple cards for one person, and swiping one’s cards by others in China at present. Therefore, the research introduces a uniqueness detection system and method for in-pit coal-mine personnel integrated into the in-pit coal mine personnel positioning system, establishing a system mode based on face recognition + recognition of personnel positioning card + release by automatic detection. Aiming at the facts that the in-pit personnel are wearing helmets and faces are prone to be stained during the face recognition, the study proposes the ideas that pre-process face images using the 2D-wavelet-transformation-based Mallat algorithm and extracts three face features: miner light, eyes and mouths, using the generalized symmetry transformation-based algorithm. This research carried out test with 40 clean face images with no helmets and 40 lightly-stained face images, and then compared with results with the one using the face feature extraction method based on grey-scale transformation and edge detection. The results show that the method described in the paper can detect accurately face features in the above-mentioned two cases, and the accuracy to detect face features is 97.5% in the case of wearing helmets and lightly-stained faces. Ó 2013 Published by Elsevier B.V. on behalf of China University of Mining & Technology.
1. Introduction The in-pit coal-mine personnel positioning system requires a uniqueness detection function to detect the personnel identity of coal mines inside the pit and a device to be located at all pit entrances and exits to detect the recognition cards work normally and to perform the uniqueness detection [1,2]. The potential risks are present in management because in the current actual application the in-pit coal-mine personnel positioning system cannot achieve effectively detection of the uniqueness of the personnel inside the pit, nor eliminate the violations such as attendance cheat and card swiping by others. Thus, it is of great importance in developing a good and complete in-pit coal-mine personnel positioning system and promoting safety production management to study the technology on uniqueness detection over personnel of coal mines inside the pit. The biologic feature recognition technology can fulfill the uniqueness detection by comparing the unique biologic features on human bodies. Recognition technologies on palm print, fingerprint, iris and face can be used to detect the identity uniqueness of human biological features, with respect to palm print, ⇑ Corresponding author. Tel.: +86 15210198505. E-mail address: [email protected] (C. Li).
fingerprint, iris and face, and application of biologic feature recognition technologies have been discussed in some literatures in the fields such as security, attendance, ticket management, medical care and banking [3–6]. However, in comparison with other fields, coal mines are different in the facts that biologic features of in-pit coal mine personnel are easy to be stained with coal dust affecting the detection precision, and in some literature, discussions are made to the application of the image monitoring and face detection technology in the coal mine [7–9]. The research compares the main technical features, as shown in Table 1. As shown in Table 1, the biologic features in palm print and fingerprint recognitions are not suitable for the uniqueness detection over in-pit coal mine personnel because of reproducibility and need of active cooperation of subjects for feature acquisition, while iris recognition is not appropriate for wide application to a large extent because of its high cost in acquisition devices though it is hardly affected by factors present in coal mines. Face recognition may be subject to the acquisition environment and factors related to the recognized persons to some degrees, and with some modifications, it can be applied in the uniqueness detection over in-pit personnel. In this paper, in-pit personnel uniqueness detection system on the basis of a face feature detection is introduced to be integrated into the in-pit coal mine personnel positioning system.
2095-2686/$ - see front matter Ó 2013 Published by Elsevier B.V. on behalf of China University of Mining & Technology. http://dx.doi.org/10.1016/j.ijmst.2013.05.014
358
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
Table 1 Comparison of four types of biometric recognition technology. Type
Palm print recognition
Fingerprint recognition
Iris recognition
Face recognition
Detection method Human’s behavior needed or not Cost of the device Acquisition speed Affecting factor Stability Likelihood of reproduction
Contact Yes Low Fast Injury, age, tiredness High Reproducible under static condition
Contact Yes Low Fast Dryness, stain, injury High Reproducible under static condition
Non-contact Yes High Faster Adjustment of camera lens Highest Very low reproducibility
Non-contact No Low Fastest Light, age, shielding High Low reproducibility
2.1. System design The uniqueness detection, as a supplement to the in-pit coalmine personnel positioning system, can be realized with the existing functions of the latter and therefore it is proper to integrate the former into the latter. In this paper, discussion is given to the in-pit personnel uniqueness detection based on face recognition + personnel positioning card recognition + automatic detection release mode. The system is comprised of 3 modules, as shown in Fig. 1. The personnel entry module includes the smart channel controller 1, the gravity sensor with location mark. The automatic detection module includes the low-power card reader for in-pit coal-mine personnel positioning (the card recognition technology is compliant with that in the current personnel positioning system, e.g., RFID-based, ZigBee based and WiFi-based technologies), CCD cameras, the image processing host and the judgment host. The movement-limiting alarm module consists of the smart speaker and the smart channel controller 3.
Entry into the mark of detection point
Reopen the channel
Sense Personnel s entry
Close the channel
Detect the positioning card
Acquire the realtime face image
Only one card or not
Image pre processing
Yes No
Read information saved on the card
Extract face features
Output pre-stored face features Alarm against for multiple cards for one person and close the channel
Output acquired face feature
Match or not
No
Detection is over
2. Uniqueness detection system
Alarm against for card-swiping by others and close the channel
Yes Manual check of identity
Normally entry into / or exit from pit
Manual check of identity
2.2. System’s working principle Fig. 2. Coal mine personnel uniqueness detection process.
The ‘‘face recognition + personnel positioning card recognition + automatic detection release mode’’ system’s working principle is shown in Fig. 2. When the coal mine personnel arrive at the detection point and the marked location according to the marking module, the gravity sensor at the marked location detects personnel’s entry, then the sensor sends the command ‘‘start detection’’ to the smart channel controller 1 and the camera, and the smart channel controller 1 closes the channel entrance after receiving the command ‘‘start detection’’ and waits for the command ‘‘end detection’’. Two persons of coal mines and more are not permitted to arrive at the detection points at the same time.
2 3
Automatic detection module
Movement-limiting alarm module 2 3
3
2 1
1
1 Personnel entry module
Fig. 1. Design of in-pit coal mine personnel uniqueness detection system.
The low-power card reader for in-pit coal mine personnel positioning of the automatic detection module detects the information of the quantity of the personnel positioning recognition card in the area in the radius half of the length of the detection point. If the detection shows that there is one card only, then it continues to read entity and face information saved in the recognition card and sends the face information to the judgment host for feature judgment; if the detection shows that there is more than one card, then it is judged as violation ‘‘multiple cards for one person’’ and it sends the command ‘‘alarm for banned movement’’ to the movement-limiting alarm module, and as a result, the smart speaker buzzes for alarm, and the smart channel controller 3 keeps the channel closed to forbid the person with multiple cards to enter or leave the pits for manual check. While the automatic detection module is detecting the recognition cards, the CCD camera takes photos of the mark location, and sends to the image processing host for image pre-processing and face feature detection and then to the feature judgment host. The feature judgment computer compares the face information saved in the recognition card and the extracted instant face information to judge the face information in tally with each other or not; if not, it is judged as violation ‘‘card swiping by others’’ and the command ‘‘alarm for banned movement’’ is sent to the movement-limiting alarm module, and as a result, the smart speaker buzzes for alarm, and the smart channel controller 3 keeps the channel closed to forbid the person with multiple cards to enter or leave the pits for manual check; If yes, the command ‘‘release’’ is sent to the movement-limiting alarm module, and the smart channel controller 3 opens the exit of the channel to permit the
359
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
person to leave/enter the pit normally and the detection is over, and the command ‘‘detection over’’ is sent to the personnel entry module to open the entrance of the detection channel for the next detection. With integration of the personnel entry module and the automatic detection module, the system allows effectively recognizing and forbidding the violation ‘‘multiple cards for one person’’, and with comparison between the face feature information saved in the recognition card and the acquired one, it enables to effectively recognize and forbid the violation ‘‘card-swiping by others’’, shortening the time duration of detection due to retrieving information on face features. The detection accuracy is increased using the detection location marking function to provide the basic front face image for face feature detection. Overall, the system can be used for in-pit coal mine personnel uniqueness detection.
coordinates, can be expressed as: qn ¼ ðrn ; hn Þ, where @ rn ¼ logð1 þ krzn kÞ and hn ¼ arctan j @@x zn = @y zn j are expressed as the gradient strength and direction at point zn . cij is the counterclockwise angle of the link of zi and zj and the horizontal line, and the range of cij is expressed as ½0; p [13–14]. Define the set of all dotted pairs using z as links’ central point as:
kzi zj k 1 Lr ði; jÞ ¼ pffiffiffi e 2r 2pr
ð4Þ
3. In-pit coal mine personnel face feature detection
Zði; jÞ ¼ ð1 cosðhi þ hj 2cij ÞÞð1 cosðhi hj ÞÞ
ð5Þ
The factors present in the coal mines are mainly that the coaldust is prone to adhere on the face skin and miners are required to wear helmets, so the study propose a face feature detection method based on modified generalized symmetry transformation: first, apply the 2D wavelet Mallat algorithm to pre-process the face image; second, apply the generalized symmetry transformation algorithm to detect the face feature. The face features to be detected include the eyes and mouths that are not easy to be stained and the miner lights fixed on the helmet and located in the front of faces. All are of geometrically symmetric structure.
z þ z i j ¼ z 2
CðzÞ ¼ ði; jÞ
ð3Þ
Define the distance weight function and the phase weight function as:
where r and subsequently-mentioned r0 ; r1 ; r2 are the ratio of the detected image width and the image width. Define the strength of generalized symmetry transformation at point z as:
Sr ðzÞ ¼
X
Cði; jÞ
ð6Þ
ði;j2CðzÞÞ
where
Cði; jÞ ¼ Lr ði; jÞZði; jÞr i r j
ð7Þ
Define the direction of the generalized symmetry transformation as:
3.1. Mallat algorithm for 2D wavelet transformation
hi þ hj ; ði; j 2 CðzÞÞ 2
The acquired image is resolved and reconstructed using the Mallat algorithm for 2D wavelet transformation to pre-process the image. The resolution algorithm’s formula is:
uðzÞ ¼ Max/ðzÞ/ðzÞ ¼
X 8 > C ¼ hl2n hj2m ckþl;l;j > > k;n;m > l;j > > > X > 1 > > d ¼ hl2n g j2m ckþl;l;j > > < k;n;m l;j X 2 > dk;n;m ¼ g l2n hj2m ckþl;l;j > > > > l;j > > > X > 3 > > g l2n g j2m ckþl;l;j > : dk;n;m ¼
4. Test of face feature detection
ð1Þ
l;j
where g is the high-pass filter o (HPF); h the low-pass filter (LPF); and 1 2 3 k; n; m; l; j 2 Z; fck ; dk ; dk ; dk the 2D wavelet transformation ckþ1 (le1 2 3 vel 1), of which dk ; dk ; dk respectively corresponds to the horizontal high frequency (vertical edge), the vertical high frequency (horizontal edge) and the high frequencies in both directions (angle point) in the frequency domain characteristics [10–12]. The equation for the reconstruction algorithm is as follows:
ckþ1;n;m
Using the software MATALB, face features are acquired from 80 miners’ face images, where 40 are clean faces with no helmets and the remaining 40 are lightly-stained faces with helmets. Comparison was made to the result from the method proposed in the paper and the one from the method based on grey-scale transformation and edge detection. The process for the wavelet resolution method of 2D signals is shown in Fig. 3. The process for wavelet reconstruction method of 2D signals is shown in Fig. 4. The face feature detection is based on generalized symmetry transformation algorithm.
X X 1 ¼ hn2l hm2j ck;n;m þ hn2l g m2j dk;n;m l;j
Column convolution with LPF h
l;j
X X 2 3 þ g n2l hm2j dk;n;m þ g n2l g m2j dk;n;m l;j
ð8Þ
ð2Þ
Line sampling , reserve even rows
l;j
Line convolution with LPF h
Column sampling, reserve even rows
ck
Line convolution with HPF g
Column sampling, reserve even rows
d k1
Line convolution with LPF h
Column sampling, reserve even rows
d k2
Line convolution with HPF g
Column sampling, reserve even rows
d k3
ck +1
3.2. Generalized symmetry transformation algorithm As a local operator of the gradient graphics, the symmetry transformation is used to process the point symmetry of points on an image. Assume zn ¼ ðxn ; yn Þ is a random point on an image, @ where n ¼ 1; 2; 3; . . . ; N: Define: rzn ¼ j @@x zn ; @y zn j is the gradient operator at point zn and rzn , when transformed into the polar
Column convolution with HPF g
Line sampling , reserve even rows
Fig. 3. Mallat algorithm for 2D wavelet resolution.
360
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
Column interpolation column 0
Line convolution with LPF h
d k1
Column interpolation column 0
Line convolution with HPF g
d k2
Column interpolation column 0
Line convolution with LPF h
Column interpolation column 0
Line convolution with HPF g
ck
d k3
⊕
Line interpolation line 0
Column convolution with LPF h
⊕ ⊕
Line interpolation line 0
ck +1
Column convolution with HPF g
Fig. 4. Mallat algorithm for 2D wavelet reconstruction. Fig. 5. In-pit personnel face feature detection.
The miner’s light is considered as a strong symmetric round structure in detection. Based on the image width ratio, assume r0 ¼ 0:25 and the distance weight function is:
" # ðxj xi Þ2 þ ðyj yi Þ2 1 Lr ði; jÞ ¼ pffiffiffi exp 2r20 2pr
ð9Þ
The directional weight function is:
p Zði; jÞ ¼ ð1 þ cosðhi cij Þ cosðhj cij pÞÞcij 2
ð10Þ
Eye detection: the eye is divided into the round symmetric area abstracted from the pupil area and the ellipse symmetric area of the whole eyeball. Based on the image width ratio, assuming r0 is 0.05 of image width, r1 ¼ 4r0 and r2 ¼ r0 , correspondingly, the distance weight function of symmetric area is divided into two sections for expression:
h i 8 ðx x Þ2 þðy y Þ2 > < pffiffi21pr exp j i 2r2 j i ; r < r0 0 Lr ði; jÞ ¼ h i h i ðx x Þ2 ðy y Þ2 > 1 : pffiffiffiffiffiffiffiffiffiffiffiffi exp j2r2i exp j2r2i 2pr1 r2
2
1
ð11Þ
2
2
where r ¼ ðxj xi Þ þ ðyj yi Þ . The directional weight function at the pupil area is:
p Zði; jÞ ¼ 1 þ cosðhi cij Þ cos hj cij p cij 2
ð12Þ
Mouth detection: in analysis, it is considered as an ellipse area. Based on the image width ratio, assume r2 ¼ 0:3 image width, and since the mouth can open or close any time, assume r1 ¼ mr2 ; m 2 ð1; 3Þ. The distance function is:
" # " # ðyj yi Þ2 1 ðxj xi Þ2 exp Lr ði; jÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi exp 2r21 2r22 2pr1 r2
ð13Þ
The directional weight function is revised as:
p Zði; jÞ ¼ ð1 cosðhi þ hj 2cij ÞÞð1 cosðhi hj ÞÞcij 2
ð14Þ
Fig. 5 is the face feature detection result from in-pit personnel face images. It must be noted that, by applying the systemic structure and the detection method proposed in the paper, the image from the acquired features is basically the front face image with fixed sizes because the system structure in the paper limits the location of subjects receiving detection using the detection marks. Fig. 6 shows the comparison between the effect of the greyscale-and edge-detection-based face detection method and the one based on Mallat algorithm for 2D wavelet transformation and generalized symmetry transformation algorithm. Fig. 6a is an original image, and Fig. 6b is the detection result with MATLAB test from the method based on grey-scale transformation and edge detection on face. Fig. 6c is the result with MATLAB test from the method mentioned in the above.
Fig. 6. Effect comparison between the face detection results.
Table 2 Accuracy of face feature detections. Accuracy (quantity of images correctly detected) (%)
Method based on grey-scale transformation and the edge detection
Method proposed in the paper
Faces with no stains and no helmets Faces with light stains and helmets Total
100/40
100/40
20/8
97.5/39
60/48
98.75/79
Fig. 6 shows that there is great difference in positioning and detection of faces with coal dust stains and wearing helmets using the grey-scale-and-edge-detection-based method while the method proposed in the paper acquires precise face features about the miner light, eye and mouth of in-pit personnel. Table 2 shows the accuracy of face detection with the two methods. It can be seen from Table 2, the accuracy of the grey-scaletransformation and edge detection-based method is high for faces with no stain and helmets, but is very low for faces with stains and helmets, while the accuracy of the method proposed, affected little by these factors, is 97.5% with overall accuracy of 98.75%. Therefore, the method proposed in the paper is effective in face feature detection. 5. Conclusions (1) The mode ‘‘face recognition + personnel positioning card recognition + automatic detection release’’ can effectively recognize and eliminate violations like ‘‘multiple cards for one person’’ and ‘‘card swiping by others’’, rooting out any violations on pit entering; comparison by reading the face information from the personnel positioning cards can elimi-
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
nate the retrieving time, shortening the detection time; the detection location marking function can provide the frontal face image for face feature detection. (2) By use of the Mallat algorithm for 2D wavelet transformation to resolve and reconstruct images and then carrying out the generalized symmetry transformation to acquire stained face features, difficulties in detecting the in-pit personnel face features due to factors present in coal mines such as helmets and coal dust are eliminated, error reduced and face features wearing helmets with light stains acquired effectively. (3) The proposal on in-pit coal mine personnel uniqueness detection technology described in the paper is of great importance for wide application of the in-pit personnel positioning system and coal mine’s in-pit safety management because it can put an end to the violations which happened at the time entering the pit and improve the in-pit personnel positioning system.
Acknowledgments The financial supports from the National Natural Science Foundation of China (No. 51134024) and the National High Technology Research and Development Program of China (No. 2012AA062203) are gratefully acknowledged.
361
References [1] Sun JP. Technologies of monitoring and communication in the coal mine. J China Coal Soc 2010;35(11):1925–9. [2] Sun JP. Effect and configuration of six systems for safe act of rescue of coal mine underground. Ind Mine Autom 2010;11:1–4. [3] Lu GM, Li HB, Liu L. A servey on biometrics. J Nanjing Univ Posts Telecommun: Nat Sci 2007;27(1):81–6. [4] Ming AL, Ma HD. The framework and typical application for biometrics grid. J Huazhong Univ Sci Technol 2006;34:156–9 [Suppl.]. [5] Lin XR, Huang XW, Su XS, Zhou B, Dai XQ. Progress of biometric technology standardization. J Tsinghua Univ: Sci Technol 2006;46(2):194–8. [6] Wang FH, Han JQ, Yao XH. Multimodal biometric fusion approach based on iris and face. J Xi’an Jiaotong Univ 2008;42(2):133–7. [7] Liu YN, Sun JP, Su H, Na JF. Digital solution to mining image monitor system. J China Univ Min Technol 2001;11(2):204–7. [8] Sun JP, Chen W, Tang L, Liu XY, Lin HT. Face detection in the low-luminousdensity and low-resolution image within limited room. J China Univ Min Technol 2008;37(3):373–8. [9] Sun JP, Chen W, Wang FZ, Tang L, Ma FY, Li C. Recognizing empty trains in coalmine surveillance images. J China Univ Min Technol 2007;36(5):598–602. [10] Li Y, Liu XC. A method of image enhancement based on Mallat algorithm. J Taishan Univ 2006;28(3):51–3. [11] Sun JP, Li M. Life detection and location methods using UWB impulse radar in a coal mine. Min Sci Technol 2011;21(5):687–91. [12] Mallat S, Zhong S. Characterization of signals from multi-scale edges. IEEE Trans 1992;14(7):710–32. [13] Reisfeld D, Wolfson H, Yeshurun Y. Context-free attentional operators: the generalizes symmetry transform. Comput Vision 1995;14:119–30. [14] Peng JY, Yu BZ, Wang DK. Multi-scale symmetry transform with application to location of feature points on human face image. Acta Electr Sinica 2002;30(3):363–6.
Contents lists available at SciVerse ScienceDirect
International Journal of Mining Science and Technology journal homepage: www.elsevier.com/locate/ijmst
In-pit coal mine personnel uniqueness detection technology based on personnel positioning and face recognition Sun Jiping, Li Chenxin ⇑ Research Institute of Information Engineering, China University of Mining & Technology, Beijing, Beijing 100083, China
a r t i c l e
i n f o
Article history: Received 29 August 2012 Received in revised form 16 September 2012 Accepted 21 December 2012 Available online 5 June 2013 Keywords: Coal mine Uniqueness detection Recognition of personnel positioning cards Face recognition Generalized symmetry transformation
a b s t r a c t Since the coal mine in-pit personnel positioning system neither can effectively achieve the function to detect the uniqueness of in-pit coal-mine personnel nor can identify and eliminate violations in attendance management such as multiple cards for one person, and swiping one’s cards by others in China at present. Therefore, the research introduces a uniqueness detection system and method for in-pit coal-mine personnel integrated into the in-pit coal mine personnel positioning system, establishing a system mode based on face recognition + recognition of personnel positioning card + release by automatic detection. Aiming at the facts that the in-pit personnel are wearing helmets and faces are prone to be stained during the face recognition, the study proposes the ideas that pre-process face images using the 2D-wavelet-transformation-based Mallat algorithm and extracts three face features: miner light, eyes and mouths, using the generalized symmetry transformation-based algorithm. This research carried out test with 40 clean face images with no helmets and 40 lightly-stained face images, and then compared with results with the one using the face feature extraction method based on grey-scale transformation and edge detection. The results show that the method described in the paper can detect accurately face features in the above-mentioned two cases, and the accuracy to detect face features is 97.5% in the case of wearing helmets and lightly-stained faces. Ó 2013 Published by Elsevier B.V. on behalf of China University of Mining & Technology.
1. Introduction The in-pit coal-mine personnel positioning system requires a uniqueness detection function to detect the personnel identity of coal mines inside the pit and a device to be located at all pit entrances and exits to detect the recognition cards work normally and to perform the uniqueness detection [1,2]. The potential risks are present in management because in the current actual application the in-pit coal-mine personnel positioning system cannot achieve effectively detection of the uniqueness of the personnel inside the pit, nor eliminate the violations such as attendance cheat and card swiping by others. Thus, it is of great importance in developing a good and complete in-pit coal-mine personnel positioning system and promoting safety production management to study the technology on uniqueness detection over personnel of coal mines inside the pit. The biologic feature recognition technology can fulfill the uniqueness detection by comparing the unique biologic features on human bodies. Recognition technologies on palm print, fingerprint, iris and face can be used to detect the identity uniqueness of human biological features, with respect to palm print, ⇑ Corresponding author. Tel.: +86 15210198505. E-mail address: [email protected] (C. Li).
fingerprint, iris and face, and application of biologic feature recognition technologies have been discussed in some literatures in the fields such as security, attendance, ticket management, medical care and banking [3–6]. However, in comparison with other fields, coal mines are different in the facts that biologic features of in-pit coal mine personnel are easy to be stained with coal dust affecting the detection precision, and in some literature, discussions are made to the application of the image monitoring and face detection technology in the coal mine [7–9]. The research compares the main technical features, as shown in Table 1. As shown in Table 1, the biologic features in palm print and fingerprint recognitions are not suitable for the uniqueness detection over in-pit coal mine personnel because of reproducibility and need of active cooperation of subjects for feature acquisition, while iris recognition is not appropriate for wide application to a large extent because of its high cost in acquisition devices though it is hardly affected by factors present in coal mines. Face recognition may be subject to the acquisition environment and factors related to the recognized persons to some degrees, and with some modifications, it can be applied in the uniqueness detection over in-pit personnel. In this paper, in-pit personnel uniqueness detection system on the basis of a face feature detection is introduced to be integrated into the in-pit coal mine personnel positioning system.
2095-2686/$ - see front matter Ó 2013 Published by Elsevier B.V. on behalf of China University of Mining & Technology. http://dx.doi.org/10.1016/j.ijmst.2013.05.014
358
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
Table 1 Comparison of four types of biometric recognition technology. Type
Palm print recognition
Fingerprint recognition
Iris recognition
Face recognition
Detection method Human’s behavior needed or not Cost of the device Acquisition speed Affecting factor Stability Likelihood of reproduction
Contact Yes Low Fast Injury, age, tiredness High Reproducible under static condition
Contact Yes Low Fast Dryness, stain, injury High Reproducible under static condition
Non-contact Yes High Faster Adjustment of camera lens Highest Very low reproducibility
Non-contact No Low Fastest Light, age, shielding High Low reproducibility
2.1. System design The uniqueness detection, as a supplement to the in-pit coalmine personnel positioning system, can be realized with the existing functions of the latter and therefore it is proper to integrate the former into the latter. In this paper, discussion is given to the in-pit personnel uniqueness detection based on face recognition + personnel positioning card recognition + automatic detection release mode. The system is comprised of 3 modules, as shown in Fig. 1. The personnel entry module includes the smart channel controller 1, the gravity sensor with location mark. The automatic detection module includes the low-power card reader for in-pit coal-mine personnel positioning (the card recognition technology is compliant with that in the current personnel positioning system, e.g., RFID-based, ZigBee based and WiFi-based technologies), CCD cameras, the image processing host and the judgment host. The movement-limiting alarm module consists of the smart speaker and the smart channel controller 3.
Entry into the mark of detection point
Reopen the channel
Sense Personnel s entry
Close the channel
Detect the positioning card
Acquire the realtime face image
Only one card or not
Image pre processing
Yes No
Read information saved on the card
Extract face features
Output pre-stored face features Alarm against for multiple cards for one person and close the channel
Output acquired face feature
Match or not
No
Detection is over
2. Uniqueness detection system
Alarm against for card-swiping by others and close the channel
Yes Manual check of identity
Normally entry into / or exit from pit
Manual check of identity
2.2. System’s working principle Fig. 2. Coal mine personnel uniqueness detection process.
The ‘‘face recognition + personnel positioning card recognition + automatic detection release mode’’ system’s working principle is shown in Fig. 2. When the coal mine personnel arrive at the detection point and the marked location according to the marking module, the gravity sensor at the marked location detects personnel’s entry, then the sensor sends the command ‘‘start detection’’ to the smart channel controller 1 and the camera, and the smart channel controller 1 closes the channel entrance after receiving the command ‘‘start detection’’ and waits for the command ‘‘end detection’’. Two persons of coal mines and more are not permitted to arrive at the detection points at the same time.
2 3
Automatic detection module
Movement-limiting alarm module 2 3
3
2 1
1
1 Personnel entry module
Fig. 1. Design of in-pit coal mine personnel uniqueness detection system.
The low-power card reader for in-pit coal mine personnel positioning of the automatic detection module detects the information of the quantity of the personnel positioning recognition card in the area in the radius half of the length of the detection point. If the detection shows that there is one card only, then it continues to read entity and face information saved in the recognition card and sends the face information to the judgment host for feature judgment; if the detection shows that there is more than one card, then it is judged as violation ‘‘multiple cards for one person’’ and it sends the command ‘‘alarm for banned movement’’ to the movement-limiting alarm module, and as a result, the smart speaker buzzes for alarm, and the smart channel controller 3 keeps the channel closed to forbid the person with multiple cards to enter or leave the pits for manual check. While the automatic detection module is detecting the recognition cards, the CCD camera takes photos of the mark location, and sends to the image processing host for image pre-processing and face feature detection and then to the feature judgment host. The feature judgment computer compares the face information saved in the recognition card and the extracted instant face information to judge the face information in tally with each other or not; if not, it is judged as violation ‘‘card swiping by others’’ and the command ‘‘alarm for banned movement’’ is sent to the movement-limiting alarm module, and as a result, the smart speaker buzzes for alarm, and the smart channel controller 3 keeps the channel closed to forbid the person with multiple cards to enter or leave the pits for manual check; If yes, the command ‘‘release’’ is sent to the movement-limiting alarm module, and the smart channel controller 3 opens the exit of the channel to permit the
359
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
person to leave/enter the pit normally and the detection is over, and the command ‘‘detection over’’ is sent to the personnel entry module to open the entrance of the detection channel for the next detection. With integration of the personnel entry module and the automatic detection module, the system allows effectively recognizing and forbidding the violation ‘‘multiple cards for one person’’, and with comparison between the face feature information saved in the recognition card and the acquired one, it enables to effectively recognize and forbid the violation ‘‘card-swiping by others’’, shortening the time duration of detection due to retrieving information on face features. The detection accuracy is increased using the detection location marking function to provide the basic front face image for face feature detection. Overall, the system can be used for in-pit coal mine personnel uniqueness detection.
coordinates, can be expressed as: qn ¼ ðrn ; hn Þ, where @ rn ¼ logð1 þ krzn kÞ and hn ¼ arctan j @@x zn = @y zn j are expressed as the gradient strength and direction at point zn . cij is the counterclockwise angle of the link of zi and zj and the horizontal line, and the range of cij is expressed as ½0; p [13–14]. Define the set of all dotted pairs using z as links’ central point as:
kzi zj k 1 Lr ði; jÞ ¼ pffiffiffi e 2r 2pr
ð4Þ
3. In-pit coal mine personnel face feature detection
Zði; jÞ ¼ ð1 cosðhi þ hj 2cij ÞÞð1 cosðhi hj ÞÞ
ð5Þ
The factors present in the coal mines are mainly that the coaldust is prone to adhere on the face skin and miners are required to wear helmets, so the study propose a face feature detection method based on modified generalized symmetry transformation: first, apply the 2D wavelet Mallat algorithm to pre-process the face image; second, apply the generalized symmetry transformation algorithm to detect the face feature. The face features to be detected include the eyes and mouths that are not easy to be stained and the miner lights fixed on the helmet and located in the front of faces. All are of geometrically symmetric structure.
z þ z i j ¼ z 2
CðzÞ ¼ ði; jÞ
ð3Þ
Define the distance weight function and the phase weight function as:
where r and subsequently-mentioned r0 ; r1 ; r2 are the ratio of the detected image width and the image width. Define the strength of generalized symmetry transformation at point z as:
Sr ðzÞ ¼
X
Cði; jÞ
ð6Þ
ði;j2CðzÞÞ
where
Cði; jÞ ¼ Lr ði; jÞZði; jÞr i r j
ð7Þ
Define the direction of the generalized symmetry transformation as:
3.1. Mallat algorithm for 2D wavelet transformation
hi þ hj ; ði; j 2 CðzÞÞ 2
The acquired image is resolved and reconstructed using the Mallat algorithm for 2D wavelet transformation to pre-process the image. The resolution algorithm’s formula is:
uðzÞ ¼ Max/ðzÞ/ðzÞ ¼
X 8 > C ¼ hl2n hj2m ckþl;l;j > > k;n;m > l;j > > > X > 1 > > d ¼ hl2n g j2m ckþl;l;j > > < k;n;m l;j X 2 > dk;n;m ¼ g l2n hj2m ckþl;l;j > > > > l;j > > > X > 3 > > g l2n g j2m ckþl;l;j > : dk;n;m ¼
4. Test of face feature detection
ð1Þ
l;j
where g is the high-pass filter o (HPF); h the low-pass filter (LPF); and 1 2 3 k; n; m; l; j 2 Z; fck ; dk ; dk ; dk the 2D wavelet transformation ckþ1 (le1 2 3 vel 1), of which dk ; dk ; dk respectively corresponds to the horizontal high frequency (vertical edge), the vertical high frequency (horizontal edge) and the high frequencies in both directions (angle point) in the frequency domain characteristics [10–12]. The equation for the reconstruction algorithm is as follows:
ckþ1;n;m
Using the software MATALB, face features are acquired from 80 miners’ face images, where 40 are clean faces with no helmets and the remaining 40 are lightly-stained faces with helmets. Comparison was made to the result from the method proposed in the paper and the one from the method based on grey-scale transformation and edge detection. The process for the wavelet resolution method of 2D signals is shown in Fig. 3. The process for wavelet reconstruction method of 2D signals is shown in Fig. 4. The face feature detection is based on generalized symmetry transformation algorithm.
X X 1 ¼ hn2l hm2j ck;n;m þ hn2l g m2j dk;n;m l;j
Column convolution with LPF h
l;j
X X 2 3 þ g n2l hm2j dk;n;m þ g n2l g m2j dk;n;m l;j
ð8Þ
ð2Þ
Line sampling , reserve even rows
l;j
Line convolution with LPF h
Column sampling, reserve even rows
ck
Line convolution with HPF g
Column sampling, reserve even rows
d k1
Line convolution with LPF h
Column sampling, reserve even rows
d k2
Line convolution with HPF g
Column sampling, reserve even rows
d k3
ck +1
3.2. Generalized symmetry transformation algorithm As a local operator of the gradient graphics, the symmetry transformation is used to process the point symmetry of points on an image. Assume zn ¼ ðxn ; yn Þ is a random point on an image, @ where n ¼ 1; 2; 3; . . . ; N: Define: rzn ¼ j @@x zn ; @y zn j is the gradient operator at point zn and rzn , when transformed into the polar
Column convolution with HPF g
Line sampling , reserve even rows
Fig. 3. Mallat algorithm for 2D wavelet resolution.
360
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
Column interpolation column 0
Line convolution with LPF h
d k1
Column interpolation column 0
Line convolution with HPF g
d k2
Column interpolation column 0
Line convolution with LPF h
Column interpolation column 0
Line convolution with HPF g
ck
d k3
⊕
Line interpolation line 0
Column convolution with LPF h
⊕ ⊕
Line interpolation line 0
ck +1
Column convolution with HPF g
Fig. 4. Mallat algorithm for 2D wavelet reconstruction. Fig. 5. In-pit personnel face feature detection.
The miner’s light is considered as a strong symmetric round structure in detection. Based on the image width ratio, assume r0 ¼ 0:25 and the distance weight function is:
" # ðxj xi Þ2 þ ðyj yi Þ2 1 Lr ði; jÞ ¼ pffiffiffi exp 2r20 2pr
ð9Þ
The directional weight function is:
p Zði; jÞ ¼ ð1 þ cosðhi cij Þ cosðhj cij pÞÞcij 2
ð10Þ
Eye detection: the eye is divided into the round symmetric area abstracted from the pupil area and the ellipse symmetric area of the whole eyeball. Based on the image width ratio, assuming r0 is 0.05 of image width, r1 ¼ 4r0 and r2 ¼ r0 , correspondingly, the distance weight function of symmetric area is divided into two sections for expression:
h i 8 ðx x Þ2 þðy y Þ2 > < pffiffi21pr exp j i 2r2 j i ; r < r0 0 Lr ði; jÞ ¼ h i h i ðx x Þ2 ðy y Þ2 > 1 : pffiffiffiffiffiffiffiffiffiffiffiffi exp j2r2i exp j2r2i 2pr1 r2
2
1
ð11Þ
2
2
where r ¼ ðxj xi Þ þ ðyj yi Þ . The directional weight function at the pupil area is:
p Zði; jÞ ¼ 1 þ cosðhi cij Þ cos hj cij p cij 2
ð12Þ
Mouth detection: in analysis, it is considered as an ellipse area. Based on the image width ratio, assume r2 ¼ 0:3 image width, and since the mouth can open or close any time, assume r1 ¼ mr2 ; m 2 ð1; 3Þ. The distance function is:
" # " # ðyj yi Þ2 1 ðxj xi Þ2 exp Lr ði; jÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi exp 2r21 2r22 2pr1 r2
ð13Þ
The directional weight function is revised as:
p Zði; jÞ ¼ ð1 cosðhi þ hj 2cij ÞÞð1 cosðhi hj ÞÞcij 2
ð14Þ
Fig. 5 is the face feature detection result from in-pit personnel face images. It must be noted that, by applying the systemic structure and the detection method proposed in the paper, the image from the acquired features is basically the front face image with fixed sizes because the system structure in the paper limits the location of subjects receiving detection using the detection marks. Fig. 6 shows the comparison between the effect of the greyscale-and edge-detection-based face detection method and the one based on Mallat algorithm for 2D wavelet transformation and generalized symmetry transformation algorithm. Fig. 6a is an original image, and Fig. 6b is the detection result with MATLAB test from the method based on grey-scale transformation and edge detection on face. Fig. 6c is the result with MATLAB test from the method mentioned in the above.
Fig. 6. Effect comparison between the face detection results.
Table 2 Accuracy of face feature detections. Accuracy (quantity of images correctly detected) (%)
Method based on grey-scale transformation and the edge detection
Method proposed in the paper
Faces with no stains and no helmets Faces with light stains and helmets Total
100/40
100/40
20/8
97.5/39
60/48
98.75/79
Fig. 6 shows that there is great difference in positioning and detection of faces with coal dust stains and wearing helmets using the grey-scale-and-edge-detection-based method while the method proposed in the paper acquires precise face features about the miner light, eye and mouth of in-pit personnel. Table 2 shows the accuracy of face detection with the two methods. It can be seen from Table 2, the accuracy of the grey-scaletransformation and edge detection-based method is high for faces with no stain and helmets, but is very low for faces with stains and helmets, while the accuracy of the method proposed, affected little by these factors, is 97.5% with overall accuracy of 98.75%. Therefore, the method proposed in the paper is effective in face feature detection. 5. Conclusions (1) The mode ‘‘face recognition + personnel positioning card recognition + automatic detection release’’ can effectively recognize and eliminate violations like ‘‘multiple cards for one person’’ and ‘‘card swiping by others’’, rooting out any violations on pit entering; comparison by reading the face information from the personnel positioning cards can elimi-
J. Sun, C. Li / International Journal of Mining Science and Technology 23 (2013) 357–361
nate the retrieving time, shortening the detection time; the detection location marking function can provide the frontal face image for face feature detection. (2) By use of the Mallat algorithm for 2D wavelet transformation to resolve and reconstruct images and then carrying out the generalized symmetry transformation to acquire stained face features, difficulties in detecting the in-pit personnel face features due to factors present in coal mines such as helmets and coal dust are eliminated, error reduced and face features wearing helmets with light stains acquired effectively. (3) The proposal on in-pit coal mine personnel uniqueness detection technology described in the paper is of great importance for wide application of the in-pit personnel positioning system and coal mine’s in-pit safety management because it can put an end to the violations which happened at the time entering the pit and improve the in-pit personnel positioning system.
Acknowledgments The financial supports from the National Natural Science Foundation of China (No. 51134024) and the National High Technology Research and Development Program of China (No. 2012AA062203) are gratefully acknowledged.
361
References [1] Sun JP. Technologies of monitoring and communication in the coal mine. J China Coal Soc 2010;35(11):1925–9. [2] Sun JP. Effect and configuration of six systems for safe act of rescue of coal mine underground. Ind Mine Autom 2010;11:1–4. [3] Lu GM, Li HB, Liu L. A servey on biometrics. J Nanjing Univ Posts Telecommun: Nat Sci 2007;27(1):81–6. [4] Ming AL, Ma HD. The framework and typical application for biometrics grid. J Huazhong Univ Sci Technol 2006;34:156–9 [Suppl.]. [5] Lin XR, Huang XW, Su XS, Zhou B, Dai XQ. Progress of biometric technology standardization. J Tsinghua Univ: Sci Technol 2006;46(2):194–8. [6] Wang FH, Han JQ, Yao XH. Multimodal biometric fusion approach based on iris and face. J Xi’an Jiaotong Univ 2008;42(2):133–7. [7] Liu YN, Sun JP, Su H, Na JF. Digital solution to mining image monitor system. J China Univ Min Technol 2001;11(2):204–7. [8] Sun JP, Chen W, Tang L, Liu XY, Lin HT. Face detection in the low-luminousdensity and low-resolution image within limited room. J China Univ Min Technol 2008;37(3):373–8. [9] Sun JP, Chen W, Wang FZ, Tang L, Ma FY, Li C. Recognizing empty trains in coalmine surveillance images. J China Univ Min Technol 2007;36(5):598–602. [10] Li Y, Liu XC. A method of image enhancement based on Mallat algorithm. J Taishan Univ 2006;28(3):51–3. [11] Sun JP, Li M. Life detection and location methods using UWB impulse radar in a coal mine. Min Sci Technol 2011;21(5):687–91. [12] Mallat S, Zhong S. Characterization of signals from multi-scale edges. IEEE Trans 1992;14(7):710–32. [13] Reisfeld D, Wolfson H, Yeshurun Y. Context-free attentional operators: the generalizes symmetry transform. Comput Vision 1995;14:119–30. [14] Peng JY, Yu BZ, Wang DK. Multi-scale symmetry transform with application to location of feature points on human face image. Acta Electr Sinica 2002;30(3):363–6.