- Email: [email protected]
Logically Efficient Spatial Resolution Conversion Using Paired Increasing Operators
Real-Time Imaging 3, 7–16 (1997)
Logically Efficient Spatial Resolution Conversion Using Paired Increasing Operators n a typical office setting, digital documents may be handled by a variety of devices and software packages that perform scanning, printing, display, transmission, and various forms of image processing. The devices that comprise the system may each operate at a different sampling resolution and spatial resampling must often be performed to allow porting of a digital image among the various devices. The primary purpose here is to design optimal integer resolution-conversion filters based on increasing operators rather than the more typically employed non-increasing operators. Design and implementation of optimal increasing filters possess certain statistical advantages and, more importantly from the perspective of real-time imaging, there is often a great logic-cost savings in implementing an increasing as opposed to a non-increasing filter.
I
© 1997 Academic Press Limited
Robert P. Loce*, Edward R. Dougherty†, Ronald E. Jodoin‡ and Michael S. Cianciosi* *Digital Imaging Technology Center Xerox Corporation †Texas Center for Applied Technology and Department of Electrical Engineering Texas A&M University ‡Department of Physics Rochester Institute of Technology
Introduction In a typical office setting, digital documents may be handled by a variety of devices and software packages that inter-operate to form a system capable of tasks such as scanning, printing, display, transmission, and various forms of image processing. The devices that comprise the system may each operate at a different sampling resolution. To enable a comprehensive system, spatial resampling must often be performed to allow porting of a digital image among the various devices.
1077-2014/97/010007 + 10 $25.00/ri960042
Each stage of handling a digital document has associated with it one or more sampling resolutions. Digital documents are created by scanning a hardcopy or by decomposition from a page-description-language (PDL) form. Office scanners usually digitize at 8 bits/pixel and either immediately threshold or threshold after some processing and prior to transmission to another device. Creation from a PDL involves several stages, starting with generating an ASCII document with word processing software and then writing the document to a file in PDL form. Software can then decompose the PDL code to binary bitmap form. The
© 1997 Academic Press Limited
8
R. P. LOCE ET AL.
decomposer may be on a host computer or integrated into a printer. Most decomposers can render a bitmap at any of a number of spatial resolutions, while some may be limited to decompose at a fixed resolution. Because document images are usually bilevel, office printers are generally based on laser/xerographic or ink-jet technologies that are suited for making binary marks on paper. The sampling resolution for an office printer is usually 240, 300, 400, or 600 spi (spots per inch). Document bitmaps created or decomposed for one printer must often be printed on another. This can occur because scanners, decomposers, and printers on a network may have been produced by different manufacturers that design at different resolutions. Resolution conversion is characterized by the ratio of the input and output sampling resolutions. A conversion ratio is defined in terms of the pagewise horizontal and vertical sampling resolutions, N1 and N2 (in spi), of the input document and the corresponding sampling resolutions, M1 and M2, of the output document. This paper is concerned with resampling at a higher resolution, meaning M1/N1 > 1 and M2/N2 > 1. Our attention is directed to integer conversion, meaning that both M1/N1 and M2/N2 are integer ratios. A number of simple techniques are sometimes used to enable printing at different resolutions; however, trivial techniques usually do not provide satisfactory results. For instance, for 300 spi to 600 spi conversion, the correct size could be achieved by simply performing nearest neighbor scaling. Stroke width would be maintained but the capability of the 600 spi printer would not be utilized. Diagonal and curved strokes would print with the characteristic jaggedness of a 300 spi printer. It is not sufficient for a resolution conversion algorithm to produce a bitmap that is only a coarse bitmap scaled to the correct size. The algorithm should utilize the sampling resolution of the printer to best represent the underlying image. Resolution conversion should result in enhanced print with less staircasing and blockiness than would be possible on the 300 spi printer. The goal is not simply resolution conversion, but resolution conversion with enhancement. Because the methodology discussed in the present paper employs image statistics to estimate a best output document given an observed input document, it is called inferential resolution conversion with enhancement. For a thorough account of resolution conversion, both inferential and non-inferential, see Loce and Dougherty [1]. The primary purpose of the present paper is to design and implement optimal integer resolution-conversion filters
based on increasing operators rather than the more typically employed non-increasing operators. Design and implementation of optimal increasing filters possess certain statistical advantages and, more importantly from the perspective of real-time imaging, there is often a great logic-cost savings in implementing an increasing as opposed to a non-increasing filter. For introductions to morphological image processing, see [2] or [3].
Resolution Conversion For input sampling resolutions N1 and N2 and output sampling resolutions, M1 and M2, with integer ratios M1/N1 > 1 and M2/N2 > 1, let AN1,N2 and AM1,M2 be the lowresolution input and high-resolution output images, respectively. If each pixel i in the low-resolution input is mapped into L pixels, then the underlying pixel transformation is of the form (i, 1) ( i, 2 ) . i→ M (i, L)
(1)
Each pixel value AN1,N2 (i) of the low-resolution bitmap is a single sample of a region of some underlying image and the corresponding high-resolution bitmap has multiple samples for the same region. We treat these multiple samples as components of a vector defining the ith block of the output image AM1,M2 namely, AM1 , M 2 (i, 1) A M , M (i, 2 ) . A M1 , M 2 [i ] = 1 2 M A M1 , M 2 (i, L)
(2 )
Assuming there are R pixels forming the low-resolution image, for i = 1, 2,..., R, each component AM1,M2 (i, j) of AM1,M2 [i] gives the output high-resolution image value at pixel (i, j), j = 1, 2,..., L. In vector form, the input lowresolution image is written as
AN1 , N 2
AN1 , N 2 (1) A N , N (2 ) = 1 2 , M A R N1 , N 2 ( )
(3)
and, in block form, the full high-resolution image is written as
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION
9
Pixel (7,6) or AM1, M2 (7, 6) Pixel 7
Block 7
AN1, N2
AM1, M2
Figure 1. Blocking and subsample phases. Note that the pixel is indexed by block number and raster position within the block.
AM1 , M 2
A M1 , M 2 [1] A M1 , M 2 [2] . = M A M1 , M 2 [ R]
( 4)
Vector-valued mapping
For each j = 1, 2,..., L, the collection (over all i) of pixels AM1,M2 (i, j) forms a digital image that is the j subsampling phase of AM1,M2 which in vector form can be written as
ψ1 ψ2 ψ4
AM1 , M 2 (1, j ) A M , M (2, j ) . AM1 , M 2 [•, j ] = 1 2 M A M1 , M 2 ( R, j )
ψ3
(5)
Components of the vector-valued mapping
Figure 2. Window mapping
Figure 1 illustrates blocking and subsampling phases, respectively.
Y
A resolution-conversion filter, , is defined by observing pixel values in a window about pixel i in the low-resolution image and using these values to estimate the L values forming the corresponding high-resolution block AM1,M2 [i]. The filter is of the form
sampling defined by the high-resolution block AM1,M2 [i] Figure 2 illustrates a 3 3 3 window at 300 spi being used in a conversion to 600 spi (the large circle denoting the window origin). For 300 to 600 conversion, generates four pixels in the 600 spi document at each location in the 300 spi document.
Y
Filter Representation and Optimization
Ψ1 Ψ2 Ψ= , M ΨL
(6 )
where Cj(AN1,N2)(i) serves as the inferential estimate of AM1,M2(i, j). Blockwise, (AN1,N2)(i) estimates the sub-
Y
Each component operator Ψj in Eqn (6) is a translation-invariant windowed filter. This means it is defined via a Boolean window function on a fixed window W = {w1, w2,..., wn}. For each pixel i, let Wi = {w1 + i, w2 + i, K, wn + i}
(7)
R. P. LOCE ET AL.
10
be the translation of W to pixel i. Cj is defined by a Boolean function fj on W, meaning
(
)
(
Ψ j AN1 , N 2 (i ) = f j AN1 , N 2 ( w1 + i ), AN1 , N 2 ( w2 + i ), K,
)
AN1 , N 2 ( wn + i )
(8)
The jth component of the resolution conversion filter is defined at input pixel i by applying its window function to the binary input values in the window translated to pixel i. This is the standard technique for defining translationinvariant window operators. If we let x1, x2,..., xn denote the binary values in a window (translated or otherwise), then any Boolean function f on x1, x2,..., xn can be represented as a logical sum-of-products. Specifically f possesses a disjunctivenormal-form representation, f ( x1 , x2 , K, x n ) =
∑x (
p i ,1) p( i , 2 ) x2 K x np(i, n ) , 1
( 9)
i
where the “sum” denotes OR, the “product” denotes AND, p(i, k) is either “c” or null depending on whether the variable is or is not complemented, and there are at most 20 products (minterms) in the expansion. The representation can be (non-uniquely) reduced by logic reduction to a sum of products containing a minimal number of logic gates. Then f ( x1 , x 2 , K, x n ) =
p i, n( i ) p i ,1) p( i, 2 ) xi, 2 K xi, n( (i ) ) . i ,1
∑x (
(10)
i
Architecturally, the filter C defined by the Boolean window function f is implemented by the gate array characterized by f. The window is placed at a pixel and the binary values in the window are computed by the array. An important special case arises when the representation can be reduced so there are no complements in the products and the expansion takes the form f ( x1 , x 2 , K, x n ) =
∑x
i ,1 x k , 2 K x i , n ( i ) .
(11)
i
A Boolean function f possesses a non-complemented representation if and only if it forms an increasing operator, meaning that (x1, x2,..., xn) ≤ (z1, z2,..., zn) implies f(x1, x2,..., xn) ≤ f(z1, z2,..., zn). Owing to logic reduction, the complement-free expression of Eqn (11) always possesses a unique minimal form, in which case the set of product expressions is referred to as the basis of the filter. In minimal form, no set of variables comprising one prod-
uct is a superset of a variable set for a distinct product. Each product in the basis corresponds to a set of variables on a subset of the defining window, xi,1, xi,2,..., xi,n(i) corresponding to pixels wi,1, wi,2,..., wi,n(i) ∈ W. Graphically, we represent the basis by a collection of templates having the appropriate pixels painted black. If an image operator, C, is defined via an increasing Boolean function, then Ψ is also said to be increasing. Operator increasingness manifests itself in a full image in the following manner: whenever image S1 , S2, C(S1) , C(S2). For a more complete account of logical operator representation and its relevance to real-time image processing, see Dougherty and Laplante [4]. The purpose of the present paper is to design integer resolution-conversion filters based on increasing operators rather than non-increasing operators. We also desire optimal (or close to optimal) filters. This requires a definition of filter error and selection of a filter that minimizes the error. For n observation binary random variables X1, X2,..., Xn, binary random variable Y, and Boolean function f(X1, X2,..., Xn) to estimate Y, the mean-absolute error (MAE) of f is defined by the expected value
[
]
MAE f = E Y − f ( X1 , X2 , K, Xn ) .
(12)
Given that f possesses a logical sum-of-products representation, an optimal choice of f is determined by finding the representation providing minimal error. The most general approach is to assume a disjunctive-normal-form representation, find the minterms that result in minimal MAE, and then apply logic reduction to find the optimal Boolean function [5–7]. The image filter, C, defined by the optimal Boolean function then provides the optimal image filter and we define its error by MAE〈C〉 = MAE〈f〉. If, as in our case, the desired filter C is vector-valued [Eqn (6)], then each Cj is treated as a filter for the j subsampling phase of the output image and the optimization procedure is applied to each Boolean function defining each Cj. In practice, the optimal filter has to be designed by using image statistics derived from input-output image pairs, in the present case, low-resolution-high-resolution pairs. The statistical methods employed in the present paper can be found in Loce and Dougherty [1]. If it happens that an optimal Boolean function is increasing, meaning that the optimal representation possesses no complements, then the appropriate complement-free expression will result from logic reduction. However, there are different design procedures available when it is assumed that the filter is increasing so that optimization takes place relative to the complement-free representation
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION of equation (10) [8–11]. These possess certain statistical advantages [12] and, more importantly from the perspective of real-time imaging, there is often a great logic-cost saving in implementing an increasing as opposed to a nonincreasing filter. Each product in the representations of equations (10) and (11) is manifested by n(i) lines (complemented or uncomplemented) entering into an AND gate. In some cases, the difference could be the use of a few dozen products for an increasing filter compared to thousands for a non-increasing filter. Hence, even if the truly optimal filter is not increasing, it can be practically useful to design an optimal increasing filter and apply it, even though there will be some increase in MAE. If visual or OCR performance is not seriously affected by the resulting suboptimality, then the decrease in logic cost can be worth the slight MAE increase. Returning to the resolution-conversion filter of Eqn (6), the optimal filter is typically composed of nonincreasing component filters. Even here, we typically do not apply the sum-of-products representation directly; rather, an equivalent representation, known as a differencing filter, can be used to design the filter [13, 14]. Differencing representation results in more precise statistical estimates when the filter is being designed and more efficient logical implementation of the designed filter. Nonetheless, increasing representation can result in much greater logic savings. Here, we present a design method for spatial resolution conversion using a pair of increasing operators. Experience has shown that a suboptimal increasing filter often performs nearly as well as an optimal non-increasing filter in two particular situations. An image filter C is said to be antiextensive if, for any image S, C(S) , S and C is said to be extensive if S , C(S). Relative to the window Boolean function f defining C, C is antiextensive if and only if the variable corresponding to the window origin is included in each basis product term for f. When the optimal nonincreasing filter is either antiextensive or extensive, experience with document images has shown that the optimal antiextensive or extensive (respectively) increasing filter usually gives good suboptimal performance. The approach here is to represent a general operator by a pair of antiextensive and extensive operators and then optimize relative to these.
Paired-filter Representation If we consider a binary image as a set, then an arbitrary binary-to-binary image filter C possesses a decomposition
(
)
Ψ( S ) = Ψ ∪ ( S ) − S ∪ Ψ ∩ ( S ) ,
(13)
11
where Ψ ∪ ( S ) = Ψ( S ) ∪ S ,
(14)
Ψ ∩ ( S ) = Ψ( S ) ∩ S ,
(15)
In operator form,
(
)
Ψ = Ψ ∪ − Id ∪ Ψ ∩ .
(16)
Since C∪ and C∩ are extensive and antiextensive, respectively, Eqn (16) provides a decomposition of the filter in terms of extensive and antiextensive filters. The preceding decomposition is useful for designing optimal filters if the extensive and antiextensive parts of an optimal filter can be designed separately in such a way that the representation of equation (16) can be used in the reverse direction to construct an optimal filter. We prove in the Appendix the following proposition: if S and S0 are random image processes and C∪ and C∩ provide optimal MAE windowed filters for estimating S ∪ S0 and S ∩ S0 from S, respectively, then the representation of Eqn (16) provides an optimal MAE filter for estimating S0 from S. Because S ∪ S0 . S and S ∩ S0 , S, C∪ and C∩ are extensive and antiextensive, respectively. For the purposes of resolution conversion, the filter of Eqn (6) can be represented as
( (
) )
(
)
Ψ1∪ − Id ∪ Ψ1∩ ∪ Ψ2 − Id ∪ Ψ2∩ Ψ= . M ∪ ΨL − Id ∪ ΨL∩
(17)
As derived, the extensive–antiextensive paired filter is expressed via unions of erosions over two bases of structuring elements in conjunction with a set subtraction. If so desired, the filter can be expressed as a union of hit-ormiss transforms, so that it has a templating-matching interpretation; specifically, we can express its Boolean window function as a nonpositive logical sum of products. To see this representation, let f ∪ and f ∩ denote the Boolean window functions for C∪ and C∩, respectively. Since C∪ is extensive, one of the minimal products for its window function f ∪ must consist of the single term xk, where xk is the value at the window origin. Thus, f ∪ is of the form f ∪ = x k + ∑ xi,1 xi, 2 K xi, ni . i
(18)
R. P. LOCE ET AL.
12
Table 1. Number of structuring elements (templates) and number of active pixels in each structuring element, We see that the logic of a filter constructed with the above described erosion-based architecture can be very inexpensive. Extensive Filters (10 templates) C1∪ C2∪ C3∪ C4∪ Anti-extensive Filters (16 templates) C1∩ C2∩ C3∩ C4∩
# of Elements in each Filter
# of Pixels in Element 1
# of Pixels in Element 2
# of Pixels in Element 3
# of Pixels in Element 4
4 1 1 4
1 1 1 1
2 0 0 4
3 0 0 4
3 0 0 4
4 4 4 4
2 2 2 2
2 3 3 3
3 3 3 3
4 3 3 3
(a) ψ1
(a)
ψ2
ψ3
ψ4
(b)
Figure 4. Images: (a) input image at 300 spi; (b) output image at 600 spi
Since C∩ is antiextensive, every minimal product for f ∩ must contain xk as a factor. Hence, f ∩ is of the form (b) ψ1
f ∩ = ∑ x k x j ,1 x j , 2 K x j , n j .
(19)
j
ψ2
ψ3
ψ4
To obtain the window function f for CCE, we need the window function for the identity Id, which is fId = xk. Since set subtraction is equivalent to intersection by the complement, according to Eqn (9.4),
(
= xk +
∑x
= x k x kc +
i,1 x i, 2 K x i, ni
i
∑x
c xk +
∑x x
j ,1 x j , 2 K x j , n j
x kc +
∑x x
j ,1 x j , 2 K x j , n j
i ,1 x i , 2 K x i , ni
i
= Figure 3. Structuring elements composing filters
)
f = f ∪ ∧ f Idc ∨ f ∩
∑x
{i: x ∉{x k
x K x i ,ni
k
j
i,1 x i, 2 K x i , ni
i ,1 i , 2
k
j
}}
x kc +
∑x j
kj ,1 x j , 2 K x j , n j
,
(20)
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION ψ1
the last equality following from recognizing that xkxkc = 0 and that xi,1xi,2...xi,n xkc = 0 if there exists xi,l such that i xk = xi,l.
OR XOR ψ1
Example — 300 spi to 600 spi Resolution Conversion
ψ2 OR
XOR ψ2
Interleave
300 spi Input image
ψ3
Using the sampling-phase scheme of Loce and Dougherty [1], a set of erosion-based filters has been designed to perform 300 spi to 600 spi conversion. An extensive/ antiextensive filter pair was designed for each of the four sampling phases. The four pairs of erosion-based filters contained a total of 26 structuring elements that are constrained to fit within a 17-pixel window. Note that hit-or-miss algorithms have employed hundreds or thousands of templates to perform this filtering task. A summary of the designed filter is given in Table 1, where the entries in the table indicate the number of templates in each filter and the number of activated pixels in each template. The filters are logically small enough to be
600 spi Output image
OR
XOR ψ3
ψ4 OR
XOR ψ4
Figure 5. Filter architecture High res. pixel clock Counter Low res. pixel clock
Extensive Filter 4
Addess to high res. scan line buffers
Antiextensive Filter 4 Extensive Filter 3
Addess to low res. scan line buffers Counter Filter input vector paralleled to all filters
High res. line sync High res. MUX select
Counter
Antiextensive Filter 3 Extensive Filter 2 Antiextensive Filter 2
Low res. line sync Counter
High res. MUX select
Extensive Filter 1 n bit Shift register
Binary video data input (low res)
13
m Low res. Scan line buffers
Low resolution MUX
Antiextensive Filter 1 Binary filter outputs to combinatorial logic section
n bit Shift register n bit Shift register
Low res. bit alignment logic
High res bit alignment logic and MUX to high res scan line buffers
Combinatorial logic section
Binary filter output vectors
Figure 6. Functional scheme for video electronics
Output interleave section
p High resolution scan line buffers
High resolution MUX
Output latch
Binary video data output (high res)
14
R. P. LOCE ET AL.
contained in relatively small inexpensive programmable logic arrays, PROMs or ASICs. The structuring elements are shown in Figure 3. The results of applying these filters to a text image can be seen in Figure 4. The converted bitmap is significantly smoother than the input bitmap. The filtering architecture is given in Figure 5. Examination of Figures 3 and 5 shows that very few gates are required to perform the filtering operation. Fewer than 20 gates are required to implement each filter path. Therefore all four filter paths illustrated in Figure 5 can be implemented with fewer than 100 gates in total. A functional scheme for the video electronics of the overall resolution conversion module is shown in Figure 6. Preceding the filter operation is a bit alignment section similar to that found in all current template-based resolution conversion applications. This bit-alignment function is made up of a series of scanline buffers and MUXs, which can easily fit in a gate array with the filter logic. Following the filtering operations, additional Boolean logic is employed to complete the overall filtering operation. The resulting pixels are then appropriately “clocked out” or “interleaved”. The output interleave function can be implemented with two scanline buffers and a series of latches could be incorporated into a larger gate array with the filter functions. Bit alignment electronics are required to map the incoming low resolution (300 spi) binary data to the appropriate m 3 n matrix input to the filters. Interleave electronics are required to map the combined filter outputs to the high resolution (600 spi) binary output. The figure illustrates a functional architecture that would be simplified for design in relatively small inexpensive application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). A set of counters divides the higher resolution pixel clock and line sync pulses to produce low resolution clock and low resolution line sync for control of the lower resolution processing circuits. A set of scanline buffers, MUXs and shift registers form a bit alignment function for the lower resolution image. The scanline buffers in this bit alignment logic are clocked by the low resolution clock and addressed by a set of low resolution counters that are also clocked by the low resolution pixel clock. The MUXs in this low resolution bit alignment logic are controlled by the output of a set of counters clocked with the low resolution line sync and additional control logic. These MUX outputs are routed to a set of n-bit shift registers. The bit alignment
circuits form an m 3 n matrix that is input in parallel to the extensive and anti-extensive filters, where m and n are the slow and fast scan dimensions of the observation window, respectively. The individual filter binary outputs are routed to a combinatorial logic section for the XOR, AND, and INVERT functions. The outputs of the combinatorial logic section are combined and MUXed to a set of scanline buffers for the higher resolution image. This set of high resolution scanline buffers is clocked by the high resolution clock and addressed by a high resolution address counter. The output of these scanline buffers can be routed to an output MUX and latch for final output, or routed to additional image processing circuits. The high resolution MUX is controlled by a set of counters clocked with the high resolution line sync, and additional control logic. The output latch is clocked with the high resolution clock.
Conclusion We have presented a logic-efficient method for implementing spatial resolution conversion of binary digital images. First, the problem setting for spatial resolution conversion was reviewed. Block coding and Boolean logic operators were presented in the context of conversion processing for binary digital images. Logic efficiency was achieved by employing paired-increasing filters rather than the typically employed non-increasing filters. If a filter is either antiextensive or extensive, experience has shown that a suboptimal increasing filter often performs nearly as well as an optimal non-increasing filter that would require a far greater number of logic gates. The approach taken here has been to represent a general operator by a pair of increasing anti-extensive and extensive operators and then optimize each member of the pair. Visual inspection of images resampled in the manner presented here shows them to be significantly enhanced.
References 1. Loce, R.P. & Dougherty, E.R. (1997) Enhancement and Restoration of Digital Documents: Statistical Design of Nonlinear Algorithms, Bellingham: SPIE Press. 2. Dougherty, E.R. (1992) An Introduction to Morphological Image Processing, Bellingham: SPIE Press. 3. Gonzalez, R.C. & Woods, R.E. (1992) Digital Image Processing, Boston, Addison-Wesley. 4. Dougherty, E.R. & Laplante, P. (1995) Introduction to RealTime Imaging, Bellingham: SPIE Press and IEEE Press.
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION 5. Dougherty, E.R. & Loce, R.P. (1993) Optimal mean-absoluteerror hit-or-miss filters: morphological representation and estimation of the binary conditional expectation, Optical Engineering, 32(4): 815–823. 6. Dougherty, E.R. & Astola, J. (1994) An Introduction to Nonlinear Image Processing, Bellingham: SPIE Press. 7. Barrera, J., Tomita, N.S. & Côrrea da Silva, F.S. (1995) Automatic programming of morphological machines by PAC learning, Proc. SPIE Neural, Morphological and Stochastic Methods in Image and Signal Processing, 2568, 233–244. 8. Dougherty, E.R. (1992) Optimal mean-square N-observation digital morphological filters – Part I: optimal binary filters. CVGJP — Image Understanding, 55(1): 36–54. 9. Loce, R.P. & Dougherty, E.R. (1992) Facilitation of optimal binary morphological filter design via structuring-element libraries and observation constraints, Optical Engineering, 3(5): 1008–1025. 10. Loce, R.P. & Dougherty, E.R. (1992) Optimal morphological restoration: the morphological filter mean-absolute-error theorem. Visual Communication and Image Representation, 3(4): 412–432. 11. Dougherty, E.R. & Loce, R.P. (1993) Efficient design strategies for the optimal binary digital morphological filter: probabilities, constraints, and structuring-element libraries. In: Dougherty, E.R. (ed) Mathematical Morphology in Image Processing, New York: Marcel Dekker. 12. Dougherty, E.R. & Loce, R.P. (1994) Precision of morphological-representation estimators for translation-invariant binary filters: increasing and nonincreasing, Signal Processing, 40(3): 129–154. 13. Handley, J. & Dougherty, E.R. (1994) Model-based optimal restoration of fax images in the context of mathematical morphology. Electronic Imaging, 3(2): 182–189. 14. Dougherty, E.R. & Loce, R.P. (1996) Optimal binary differencing filters: design, logic complexity, precision analysis, and application to digital document processing, Electronic Imaging, 5(1): 66–86.
Appendix The aim here is to demonstrate the following proposition: If S and S0 are random image processes and if C∪ and C∩ provide optimal MAE windowed filters for estimating S ∪ S0 and S ∩ S0 from S, respectively, then the representation of Eqn (16) provides an optimal MAE filter for estimating S0 from S. The proof is performed at an arbitrary pixel z, the probabilistic assumption being that, as random image (set) processes, S and S0 are jointly strict-sense stationary. For an arbitrary pixel z, define the random variables Y • = ( S ∪ S0 )( z ) = max[ S( z ), S0 ( z )] ,
(21)
15
Y• = ( S ∩ S0 )( z ) = min[ S( z ), S0 ( z )] ,
(22)
Also, let K = S(z) and Y = S0(z), the random variable we wish to estimate. Then
(
)
Y = Y • − K + Y• .
(23)
To estimate Y, we can estimate Y • and Y• (K being known because it is the value at the center of the observation window). Since Y • ≥ K, it is estimated by an extensive filter; since Y• ≤ K, it is estimated by an anti-extensive filter. Optimal estimators of Y • and Y• are given by the filters C • and C• that minimize
[
]
(24)
MAE Ψ• = E Y• − Ψ• ( S )( z ) ,
(25)
MAE Ψ • = E Y • − Ψ • ( S )( z ) ,
[
]
respectively. The filter C, defined by
(
)
Ψ( S ) = Ψ • ( S ) − S ∪ Ψ• ( S ) ,
(26)
or, in operator form, by
(
)
Ψ = Ψ • − Id ∪ Ψ• ,
(27)
where Id is the identity operator, can be used as an estimator of Y. While C • and C• are optimal estimators of Y • and Y•, respectively, we need to demonstrate that the resulting filter C is an optimal estimator for Y. To do this, let F be an optimal estimator for Y and define F∪ and F∩ via Eqns (14) and (15). As a binary operator, F possesses the decomposition of Eqn (16). Its MAE is given by Eqn (28) [see below]. If z ∈ S, then Y • 2 F∪(s)(z) = 0; if z ∉ S, then Y• 2 F∩(s)(z) = 0. Hence, the preceding equation reduces to Eqn (29). Since the decompositions of Eqns (9) and (16) are structurally the same, MAE〈C〉 also satisfies Eqn (29) and, since, C • and C• minimize the first and second expectation, respectively (Eqns (23) and (24)), C is optimal.
R. P. LOCE ET AL.
16
((
)
)
MAE Φ = E Y • − K + Y• − Φ ∪ − Id ∪ Φ ∩ ( S )( z )
( ) [ ] = E[ Y − K − (Φ − Id )( S )( z ) + Y − Φ ( S )( z ) z ∉ S]P( z ∉ S ) + E[ Y − K − (Φ − Id )( S )( z ) + Y − Φ ( S )( z ) z ∈ S]P( z ∈ S ) = E Y • − K − Φ ∪ − Id ( S )( z ) + Y• − Φ ∩ ( S )( z ) •
∪
•
[ = E[ Y
∩
•
∪
∩
•
] ( S)( z ) ] + E[ Y − Φ
[
(28)
]
MAE Φ = E Y • − Φ ∪ ( S )( z ) z ∉ S P( z ∉ S ) + E Y• − Φ ∩ ( S )( z ) z ∈ S P( z ∈ S ) . •
− Φ∪
•
∩
]
( S)( z ) .
(29)
Logically Efficient Spatial Resolution Conversion Using Paired Increasing Operators n a typical office setting, digital documents may be handled by a variety of devices and software packages that perform scanning, printing, display, transmission, and various forms of image processing. The devices that comprise the system may each operate at a different sampling resolution and spatial resampling must often be performed to allow porting of a digital image among the various devices. The primary purpose here is to design optimal integer resolution-conversion filters based on increasing operators rather than the more typically employed non-increasing operators. Design and implementation of optimal increasing filters possess certain statistical advantages and, more importantly from the perspective of real-time imaging, there is often a great logic-cost savings in implementing an increasing as opposed to a non-increasing filter.
I
© 1997 Academic Press Limited
Robert P. Loce*, Edward R. Dougherty†, Ronald E. Jodoin‡ and Michael S. Cianciosi* *Digital Imaging Technology Center Xerox Corporation †Texas Center for Applied Technology and Department of Electrical Engineering Texas A&M University ‡Department of Physics Rochester Institute of Technology
Introduction In a typical office setting, digital documents may be handled by a variety of devices and software packages that inter-operate to form a system capable of tasks such as scanning, printing, display, transmission, and various forms of image processing. The devices that comprise the system may each operate at a different sampling resolution. To enable a comprehensive system, spatial resampling must often be performed to allow porting of a digital image among the various devices.
1077-2014/97/010007 + 10 $25.00/ri960042
Each stage of handling a digital document has associated with it one or more sampling resolutions. Digital documents are created by scanning a hardcopy or by decomposition from a page-description-language (PDL) form. Office scanners usually digitize at 8 bits/pixel and either immediately threshold or threshold after some processing and prior to transmission to another device. Creation from a PDL involves several stages, starting with generating an ASCII document with word processing software and then writing the document to a file in PDL form. Software can then decompose the PDL code to binary bitmap form. The
© 1997 Academic Press Limited
8
R. P. LOCE ET AL.
decomposer may be on a host computer or integrated into a printer. Most decomposers can render a bitmap at any of a number of spatial resolutions, while some may be limited to decompose at a fixed resolution. Because document images are usually bilevel, office printers are generally based on laser/xerographic or ink-jet technologies that are suited for making binary marks on paper. The sampling resolution for an office printer is usually 240, 300, 400, or 600 spi (spots per inch). Document bitmaps created or decomposed for one printer must often be printed on another. This can occur because scanners, decomposers, and printers on a network may have been produced by different manufacturers that design at different resolutions. Resolution conversion is characterized by the ratio of the input and output sampling resolutions. A conversion ratio is defined in terms of the pagewise horizontal and vertical sampling resolutions, N1 and N2 (in spi), of the input document and the corresponding sampling resolutions, M1 and M2, of the output document. This paper is concerned with resampling at a higher resolution, meaning M1/N1 > 1 and M2/N2 > 1. Our attention is directed to integer conversion, meaning that both M1/N1 and M2/N2 are integer ratios. A number of simple techniques are sometimes used to enable printing at different resolutions; however, trivial techniques usually do not provide satisfactory results. For instance, for 300 spi to 600 spi conversion, the correct size could be achieved by simply performing nearest neighbor scaling. Stroke width would be maintained but the capability of the 600 spi printer would not be utilized. Diagonal and curved strokes would print with the characteristic jaggedness of a 300 spi printer. It is not sufficient for a resolution conversion algorithm to produce a bitmap that is only a coarse bitmap scaled to the correct size. The algorithm should utilize the sampling resolution of the printer to best represent the underlying image. Resolution conversion should result in enhanced print with less staircasing and blockiness than would be possible on the 300 spi printer. The goal is not simply resolution conversion, but resolution conversion with enhancement. Because the methodology discussed in the present paper employs image statistics to estimate a best output document given an observed input document, it is called inferential resolution conversion with enhancement. For a thorough account of resolution conversion, both inferential and non-inferential, see Loce and Dougherty [1]. The primary purpose of the present paper is to design and implement optimal integer resolution-conversion filters
based on increasing operators rather than the more typically employed non-increasing operators. Design and implementation of optimal increasing filters possess certain statistical advantages and, more importantly from the perspective of real-time imaging, there is often a great logic-cost savings in implementing an increasing as opposed to a non-increasing filter. For introductions to morphological image processing, see [2] or [3].
Resolution Conversion For input sampling resolutions N1 and N2 and output sampling resolutions, M1 and M2, with integer ratios M1/N1 > 1 and M2/N2 > 1, let AN1,N2 and AM1,M2 be the lowresolution input and high-resolution output images, respectively. If each pixel i in the low-resolution input is mapped into L pixels, then the underlying pixel transformation is of the form (i, 1) ( i, 2 ) . i→ M (i, L)
(1)
Each pixel value AN1,N2 (i) of the low-resolution bitmap is a single sample of a region of some underlying image and the corresponding high-resolution bitmap has multiple samples for the same region. We treat these multiple samples as components of a vector defining the ith block of the output image AM1,M2 namely, AM1 , M 2 (i, 1) A M , M (i, 2 ) . A M1 , M 2 [i ] = 1 2 M A M1 , M 2 (i, L)
(2 )
Assuming there are R pixels forming the low-resolution image, for i = 1, 2,..., R, each component AM1,M2 (i, j) of AM1,M2 [i] gives the output high-resolution image value at pixel (i, j), j = 1, 2,..., L. In vector form, the input lowresolution image is written as
AN1 , N 2
AN1 , N 2 (1) A N , N (2 ) = 1 2 , M A R N1 , N 2 ( )
(3)
and, in block form, the full high-resolution image is written as
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION
9
Pixel (7,6) or AM1, M2 (7, 6) Pixel 7
Block 7
AN1, N2
AM1, M2
Figure 1. Blocking and subsample phases. Note that the pixel is indexed by block number and raster position within the block.
AM1 , M 2
A M1 , M 2 [1] A M1 , M 2 [2] . = M A M1 , M 2 [ R]
( 4)
Vector-valued mapping
For each j = 1, 2,..., L, the collection (over all i) of pixels AM1,M2 (i, j) forms a digital image that is the j subsampling phase of AM1,M2 which in vector form can be written as
ψ1 ψ2 ψ4
AM1 , M 2 (1, j ) A M , M (2, j ) . AM1 , M 2 [•, j ] = 1 2 M A M1 , M 2 ( R, j )
ψ3
(5)
Components of the vector-valued mapping
Figure 2. Window mapping
Figure 1 illustrates blocking and subsampling phases, respectively.
Y
A resolution-conversion filter, , is defined by observing pixel values in a window about pixel i in the low-resolution image and using these values to estimate the L values forming the corresponding high-resolution block AM1,M2 [i]. The filter is of the form
sampling defined by the high-resolution block AM1,M2 [i] Figure 2 illustrates a 3 3 3 window at 300 spi being used in a conversion to 600 spi (the large circle denoting the window origin). For 300 to 600 conversion, generates four pixels in the 600 spi document at each location in the 300 spi document.
Y
Filter Representation and Optimization
Ψ1 Ψ2 Ψ= , M ΨL
(6 )
where Cj(AN1,N2)(i) serves as the inferential estimate of AM1,M2(i, j). Blockwise, (AN1,N2)(i) estimates the sub-
Y
Each component operator Ψj in Eqn (6) is a translation-invariant windowed filter. This means it is defined via a Boolean window function on a fixed window W = {w1, w2,..., wn}. For each pixel i, let Wi = {w1 + i, w2 + i, K, wn + i}
(7)
R. P. LOCE ET AL.
10
be the translation of W to pixel i. Cj is defined by a Boolean function fj on W, meaning
(
)
(
Ψ j AN1 , N 2 (i ) = f j AN1 , N 2 ( w1 + i ), AN1 , N 2 ( w2 + i ), K,
)
AN1 , N 2 ( wn + i )
(8)
The jth component of the resolution conversion filter is defined at input pixel i by applying its window function to the binary input values in the window translated to pixel i. This is the standard technique for defining translationinvariant window operators. If we let x1, x2,..., xn denote the binary values in a window (translated or otherwise), then any Boolean function f on x1, x2,..., xn can be represented as a logical sum-of-products. Specifically f possesses a disjunctivenormal-form representation, f ( x1 , x2 , K, x n ) =
∑x (
p i ,1) p( i , 2 ) x2 K x np(i, n ) , 1
( 9)
i
where the “sum” denotes OR, the “product” denotes AND, p(i, k) is either “c” or null depending on whether the variable is or is not complemented, and there are at most 20 products (minterms) in the expansion. The representation can be (non-uniquely) reduced by logic reduction to a sum of products containing a minimal number of logic gates. Then f ( x1 , x 2 , K, x n ) =
p i, n( i ) p i ,1) p( i, 2 ) xi, 2 K xi, n( (i ) ) . i ,1
∑x (
(10)
i
Architecturally, the filter C defined by the Boolean window function f is implemented by the gate array characterized by f. The window is placed at a pixel and the binary values in the window are computed by the array. An important special case arises when the representation can be reduced so there are no complements in the products and the expansion takes the form f ( x1 , x 2 , K, x n ) =
∑x
i ,1 x k , 2 K x i , n ( i ) .
(11)
i
A Boolean function f possesses a non-complemented representation if and only if it forms an increasing operator, meaning that (x1, x2,..., xn) ≤ (z1, z2,..., zn) implies f(x1, x2,..., xn) ≤ f(z1, z2,..., zn). Owing to logic reduction, the complement-free expression of Eqn (11) always possesses a unique minimal form, in which case the set of product expressions is referred to as the basis of the filter. In minimal form, no set of variables comprising one prod-
uct is a superset of a variable set for a distinct product. Each product in the basis corresponds to a set of variables on a subset of the defining window, xi,1, xi,2,..., xi,n(i) corresponding to pixels wi,1, wi,2,..., wi,n(i) ∈ W. Graphically, we represent the basis by a collection of templates having the appropriate pixels painted black. If an image operator, C, is defined via an increasing Boolean function, then Ψ is also said to be increasing. Operator increasingness manifests itself in a full image in the following manner: whenever image S1 , S2, C(S1) , C(S2). For a more complete account of logical operator representation and its relevance to real-time image processing, see Dougherty and Laplante [4]. The purpose of the present paper is to design integer resolution-conversion filters based on increasing operators rather than non-increasing operators. We also desire optimal (or close to optimal) filters. This requires a definition of filter error and selection of a filter that minimizes the error. For n observation binary random variables X1, X2,..., Xn, binary random variable Y, and Boolean function f(X1, X2,..., Xn) to estimate Y, the mean-absolute error (MAE) of f is defined by the expected value
[
]
MAE f = E Y − f ( X1 , X2 , K, Xn ) .
(12)
Given that f possesses a logical sum-of-products representation, an optimal choice of f is determined by finding the representation providing minimal error. The most general approach is to assume a disjunctive-normal-form representation, find the minterms that result in minimal MAE, and then apply logic reduction to find the optimal Boolean function [5–7]. The image filter, C, defined by the optimal Boolean function then provides the optimal image filter and we define its error by MAE〈C〉 = MAE〈f〉. If, as in our case, the desired filter C is vector-valued [Eqn (6)], then each Cj is treated as a filter for the j subsampling phase of the output image and the optimization procedure is applied to each Boolean function defining each Cj. In practice, the optimal filter has to be designed by using image statistics derived from input-output image pairs, in the present case, low-resolution-high-resolution pairs. The statistical methods employed in the present paper can be found in Loce and Dougherty [1]. If it happens that an optimal Boolean function is increasing, meaning that the optimal representation possesses no complements, then the appropriate complement-free expression will result from logic reduction. However, there are different design procedures available when it is assumed that the filter is increasing so that optimization takes place relative to the complement-free representation
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION of equation (10) [8–11]. These possess certain statistical advantages [12] and, more importantly from the perspective of real-time imaging, there is often a great logic-cost saving in implementing an increasing as opposed to a nonincreasing filter. Each product in the representations of equations (10) and (11) is manifested by n(i) lines (complemented or uncomplemented) entering into an AND gate. In some cases, the difference could be the use of a few dozen products for an increasing filter compared to thousands for a non-increasing filter. Hence, even if the truly optimal filter is not increasing, it can be practically useful to design an optimal increasing filter and apply it, even though there will be some increase in MAE. If visual or OCR performance is not seriously affected by the resulting suboptimality, then the decrease in logic cost can be worth the slight MAE increase. Returning to the resolution-conversion filter of Eqn (6), the optimal filter is typically composed of nonincreasing component filters. Even here, we typically do not apply the sum-of-products representation directly; rather, an equivalent representation, known as a differencing filter, can be used to design the filter [13, 14]. Differencing representation results in more precise statistical estimates when the filter is being designed and more efficient logical implementation of the designed filter. Nonetheless, increasing representation can result in much greater logic savings. Here, we present a design method for spatial resolution conversion using a pair of increasing operators. Experience has shown that a suboptimal increasing filter often performs nearly as well as an optimal non-increasing filter in two particular situations. An image filter C is said to be antiextensive if, for any image S, C(S) , S and C is said to be extensive if S , C(S). Relative to the window Boolean function f defining C, C is antiextensive if and only if the variable corresponding to the window origin is included in each basis product term for f. When the optimal nonincreasing filter is either antiextensive or extensive, experience with document images has shown that the optimal antiextensive or extensive (respectively) increasing filter usually gives good suboptimal performance. The approach here is to represent a general operator by a pair of antiextensive and extensive operators and then optimize relative to these.
Paired-filter Representation If we consider a binary image as a set, then an arbitrary binary-to-binary image filter C possesses a decomposition
(
)
Ψ( S ) = Ψ ∪ ( S ) − S ∪ Ψ ∩ ( S ) ,
(13)
11
where Ψ ∪ ( S ) = Ψ( S ) ∪ S ,
(14)
Ψ ∩ ( S ) = Ψ( S ) ∩ S ,
(15)
In operator form,
(
)
Ψ = Ψ ∪ − Id ∪ Ψ ∩ .
(16)
Since C∪ and C∩ are extensive and antiextensive, respectively, Eqn (16) provides a decomposition of the filter in terms of extensive and antiextensive filters. The preceding decomposition is useful for designing optimal filters if the extensive and antiextensive parts of an optimal filter can be designed separately in such a way that the representation of equation (16) can be used in the reverse direction to construct an optimal filter. We prove in the Appendix the following proposition: if S and S0 are random image processes and C∪ and C∩ provide optimal MAE windowed filters for estimating S ∪ S0 and S ∩ S0 from S, respectively, then the representation of Eqn (16) provides an optimal MAE filter for estimating S0 from S. Because S ∪ S0 . S and S ∩ S0 , S, C∪ and C∩ are extensive and antiextensive, respectively. For the purposes of resolution conversion, the filter of Eqn (6) can be represented as
( (
) )
(
)
Ψ1∪ − Id ∪ Ψ1∩ ∪ Ψ2 − Id ∪ Ψ2∩ Ψ= . M ∪ ΨL − Id ∪ ΨL∩
(17)
As derived, the extensive–antiextensive paired filter is expressed via unions of erosions over two bases of structuring elements in conjunction with a set subtraction. If so desired, the filter can be expressed as a union of hit-ormiss transforms, so that it has a templating-matching interpretation; specifically, we can express its Boolean window function as a nonpositive logical sum of products. To see this representation, let f ∪ and f ∩ denote the Boolean window functions for C∪ and C∩, respectively. Since C∪ is extensive, one of the minimal products for its window function f ∪ must consist of the single term xk, where xk is the value at the window origin. Thus, f ∪ is of the form f ∪ = x k + ∑ xi,1 xi, 2 K xi, ni . i
(18)
R. P. LOCE ET AL.
12
Table 1. Number of structuring elements (templates) and number of active pixels in each structuring element, We see that the logic of a filter constructed with the above described erosion-based architecture can be very inexpensive. Extensive Filters (10 templates) C1∪ C2∪ C3∪ C4∪ Anti-extensive Filters (16 templates) C1∩ C2∩ C3∩ C4∩
# of Elements in each Filter
# of Pixels in Element 1
# of Pixels in Element 2
# of Pixels in Element 3
# of Pixels in Element 4
4 1 1 4
1 1 1 1
2 0 0 4
3 0 0 4
3 0 0 4
4 4 4 4
2 2 2 2
2 3 3 3
3 3 3 3
4 3 3 3
(a) ψ1
(a)
ψ2
ψ3
ψ4
(b)
Figure 4. Images: (a) input image at 300 spi; (b) output image at 600 spi
Since C∩ is antiextensive, every minimal product for f ∩ must contain xk as a factor. Hence, f ∩ is of the form (b) ψ1
f ∩ = ∑ x k x j ,1 x j , 2 K x j , n j .
(19)
j
ψ2
ψ3
ψ4
To obtain the window function f for CCE, we need the window function for the identity Id, which is fId = xk. Since set subtraction is equivalent to intersection by the complement, according to Eqn (9.4),
(
= xk +
∑x
= x k x kc +
i,1 x i, 2 K x i, ni
i
∑x
c xk +
∑x x
j ,1 x j , 2 K x j , n j
x kc +
∑x x
j ,1 x j , 2 K x j , n j
i ,1 x i , 2 K x i , ni
i
= Figure 3. Structuring elements composing filters
)
f = f ∪ ∧ f Idc ∨ f ∩
∑x
{i: x ∉{x k
x K x i ,ni
k
j
i,1 x i, 2 K x i , ni
i ,1 i , 2
k
j
}}
x kc +
∑x j
kj ,1 x j , 2 K x j , n j
,
(20)
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION ψ1
the last equality following from recognizing that xkxkc = 0 and that xi,1xi,2...xi,n xkc = 0 if there exists xi,l such that i xk = xi,l.
OR XOR ψ1
Example — 300 spi to 600 spi Resolution Conversion
ψ2 OR
XOR ψ2
Interleave
300 spi Input image
ψ3
Using the sampling-phase scheme of Loce and Dougherty [1], a set of erosion-based filters has been designed to perform 300 spi to 600 spi conversion. An extensive/ antiextensive filter pair was designed for each of the four sampling phases. The four pairs of erosion-based filters contained a total of 26 structuring elements that are constrained to fit within a 17-pixel window. Note that hit-or-miss algorithms have employed hundreds or thousands of templates to perform this filtering task. A summary of the designed filter is given in Table 1, where the entries in the table indicate the number of templates in each filter and the number of activated pixels in each template. The filters are logically small enough to be
600 spi Output image
OR
XOR ψ3
ψ4 OR
XOR ψ4
Figure 5. Filter architecture High res. pixel clock Counter Low res. pixel clock
Extensive Filter 4
Addess to high res. scan line buffers
Antiextensive Filter 4 Extensive Filter 3
Addess to low res. scan line buffers Counter Filter input vector paralleled to all filters
High res. line sync High res. MUX select
Counter
Antiextensive Filter 3 Extensive Filter 2 Antiextensive Filter 2
Low res. line sync Counter
High res. MUX select
Extensive Filter 1 n bit Shift register
Binary video data input (low res)
13
m Low res. Scan line buffers
Low resolution MUX
Antiextensive Filter 1 Binary filter outputs to combinatorial logic section
n bit Shift register n bit Shift register
Low res. bit alignment logic
High res bit alignment logic and MUX to high res scan line buffers
Combinatorial logic section
Binary filter output vectors
Figure 6. Functional scheme for video electronics
Output interleave section
p High resolution scan line buffers
High resolution MUX
Output latch
Binary video data output (high res)
14
R. P. LOCE ET AL.
contained in relatively small inexpensive programmable logic arrays, PROMs or ASICs. The structuring elements are shown in Figure 3. The results of applying these filters to a text image can be seen in Figure 4. The converted bitmap is significantly smoother than the input bitmap. The filtering architecture is given in Figure 5. Examination of Figures 3 and 5 shows that very few gates are required to perform the filtering operation. Fewer than 20 gates are required to implement each filter path. Therefore all four filter paths illustrated in Figure 5 can be implemented with fewer than 100 gates in total. A functional scheme for the video electronics of the overall resolution conversion module is shown in Figure 6. Preceding the filter operation is a bit alignment section similar to that found in all current template-based resolution conversion applications. This bit-alignment function is made up of a series of scanline buffers and MUXs, which can easily fit in a gate array with the filter logic. Following the filtering operations, additional Boolean logic is employed to complete the overall filtering operation. The resulting pixels are then appropriately “clocked out” or “interleaved”. The output interleave function can be implemented with two scanline buffers and a series of latches could be incorporated into a larger gate array with the filter functions. Bit alignment electronics are required to map the incoming low resolution (300 spi) binary data to the appropriate m 3 n matrix input to the filters. Interleave electronics are required to map the combined filter outputs to the high resolution (600 spi) binary output. The figure illustrates a functional architecture that would be simplified for design in relatively small inexpensive application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). A set of counters divides the higher resolution pixel clock and line sync pulses to produce low resolution clock and low resolution line sync for control of the lower resolution processing circuits. A set of scanline buffers, MUXs and shift registers form a bit alignment function for the lower resolution image. The scanline buffers in this bit alignment logic are clocked by the low resolution clock and addressed by a set of low resolution counters that are also clocked by the low resolution pixel clock. The MUXs in this low resolution bit alignment logic are controlled by the output of a set of counters clocked with the low resolution line sync and additional control logic. These MUX outputs are routed to a set of n-bit shift registers. The bit alignment
circuits form an m 3 n matrix that is input in parallel to the extensive and anti-extensive filters, where m and n are the slow and fast scan dimensions of the observation window, respectively. The individual filter binary outputs are routed to a combinatorial logic section for the XOR, AND, and INVERT functions. The outputs of the combinatorial logic section are combined and MUXed to a set of scanline buffers for the higher resolution image. This set of high resolution scanline buffers is clocked by the high resolution clock and addressed by a high resolution address counter. The output of these scanline buffers can be routed to an output MUX and latch for final output, or routed to additional image processing circuits. The high resolution MUX is controlled by a set of counters clocked with the high resolution line sync, and additional control logic. The output latch is clocked with the high resolution clock.
Conclusion We have presented a logic-efficient method for implementing spatial resolution conversion of binary digital images. First, the problem setting for spatial resolution conversion was reviewed. Block coding and Boolean logic operators were presented in the context of conversion processing for binary digital images. Logic efficiency was achieved by employing paired-increasing filters rather than the typically employed non-increasing filters. If a filter is either antiextensive or extensive, experience has shown that a suboptimal increasing filter often performs nearly as well as an optimal non-increasing filter that would require a far greater number of logic gates. The approach taken here has been to represent a general operator by a pair of increasing anti-extensive and extensive operators and then optimize each member of the pair. Visual inspection of images resampled in the manner presented here shows them to be significantly enhanced.
References 1. Loce, R.P. & Dougherty, E.R. (1997) Enhancement and Restoration of Digital Documents: Statistical Design of Nonlinear Algorithms, Bellingham: SPIE Press. 2. Dougherty, E.R. (1992) An Introduction to Morphological Image Processing, Bellingham: SPIE Press. 3. Gonzalez, R.C. & Woods, R.E. (1992) Digital Image Processing, Boston, Addison-Wesley. 4. Dougherty, E.R. & Laplante, P. (1995) Introduction to RealTime Imaging, Bellingham: SPIE Press and IEEE Press.
LOGICALLY EFFICIENT SPATIAL RESOLUTION CONVERSION 5. Dougherty, E.R. & Loce, R.P. (1993) Optimal mean-absoluteerror hit-or-miss filters: morphological representation and estimation of the binary conditional expectation, Optical Engineering, 32(4): 815–823. 6. Dougherty, E.R. & Astola, J. (1994) An Introduction to Nonlinear Image Processing, Bellingham: SPIE Press. 7. Barrera, J., Tomita, N.S. & Côrrea da Silva, F.S. (1995) Automatic programming of morphological machines by PAC learning, Proc. SPIE Neural, Morphological and Stochastic Methods in Image and Signal Processing, 2568, 233–244. 8. Dougherty, E.R. (1992) Optimal mean-square N-observation digital morphological filters – Part I: optimal binary filters. CVGJP — Image Understanding, 55(1): 36–54. 9. Loce, R.P. & Dougherty, E.R. (1992) Facilitation of optimal binary morphological filter design via structuring-element libraries and observation constraints, Optical Engineering, 3(5): 1008–1025. 10. Loce, R.P. & Dougherty, E.R. (1992) Optimal morphological restoration: the morphological filter mean-absolute-error theorem. Visual Communication and Image Representation, 3(4): 412–432. 11. Dougherty, E.R. & Loce, R.P. (1993) Efficient design strategies for the optimal binary digital morphological filter: probabilities, constraints, and structuring-element libraries. In: Dougherty, E.R. (ed) Mathematical Morphology in Image Processing, New York: Marcel Dekker. 12. Dougherty, E.R. & Loce, R.P. (1994) Precision of morphological-representation estimators for translation-invariant binary filters: increasing and nonincreasing, Signal Processing, 40(3): 129–154. 13. Handley, J. & Dougherty, E.R. (1994) Model-based optimal restoration of fax images in the context of mathematical morphology. Electronic Imaging, 3(2): 182–189. 14. Dougherty, E.R. & Loce, R.P. (1996) Optimal binary differencing filters: design, logic complexity, precision analysis, and application to digital document processing, Electronic Imaging, 5(1): 66–86.
Appendix The aim here is to demonstrate the following proposition: If S and S0 are random image processes and if C∪ and C∩ provide optimal MAE windowed filters for estimating S ∪ S0 and S ∩ S0 from S, respectively, then the representation of Eqn (16) provides an optimal MAE filter for estimating S0 from S. The proof is performed at an arbitrary pixel z, the probabilistic assumption being that, as random image (set) processes, S and S0 are jointly strict-sense stationary. For an arbitrary pixel z, define the random variables Y • = ( S ∪ S0 )( z ) = max[ S( z ), S0 ( z )] ,
(21)
15
Y• = ( S ∩ S0 )( z ) = min[ S( z ), S0 ( z )] ,
(22)
Also, let K = S(z) and Y = S0(z), the random variable we wish to estimate. Then
(
)
Y = Y • − K + Y• .
(23)
To estimate Y, we can estimate Y • and Y• (K being known because it is the value at the center of the observation window). Since Y • ≥ K, it is estimated by an extensive filter; since Y• ≤ K, it is estimated by an anti-extensive filter. Optimal estimators of Y • and Y• are given by the filters C • and C• that minimize
[
]
(24)
MAE Ψ• = E Y• − Ψ• ( S )( z ) ,
(25)
MAE Ψ • = E Y • − Ψ • ( S )( z ) ,
[
]
respectively. The filter C, defined by
(
)
Ψ( S ) = Ψ • ( S ) − S ∪ Ψ• ( S ) ,
(26)
or, in operator form, by
(
)
Ψ = Ψ • − Id ∪ Ψ• ,
(27)
where Id is the identity operator, can be used as an estimator of Y. While C • and C• are optimal estimators of Y • and Y•, respectively, we need to demonstrate that the resulting filter C is an optimal estimator for Y. To do this, let F be an optimal estimator for Y and define F∪ and F∩ via Eqns (14) and (15). As a binary operator, F possesses the decomposition of Eqn (16). Its MAE is given by Eqn (28) [see below]. If z ∈ S, then Y • 2 F∪(s)(z) = 0; if z ∉ S, then Y• 2 F∩(s)(z) = 0. Hence, the preceding equation reduces to Eqn (29). Since the decompositions of Eqns (9) and (16) are structurally the same, MAE〈C〉 also satisfies Eqn (29) and, since, C • and C• minimize the first and second expectation, respectively (Eqns (23) and (24)), C is optimal.
R. P. LOCE ET AL.
16
((
)
)
MAE Φ = E Y • − K + Y• − Φ ∪ − Id ∪ Φ ∩ ( S )( z )
( ) [ ] = E[ Y − K − (Φ − Id )( S )( z ) + Y − Φ ( S )( z ) z ∉ S]P( z ∉ S ) + E[ Y − K − (Φ − Id )( S )( z ) + Y − Φ ( S )( z ) z ∈ S]P( z ∈ S ) = E Y • − K − Φ ∪ − Id ( S )( z ) + Y• − Φ ∩ ( S )( z ) •
∪
•
[ = E[ Y
∩
•
∪
∩
•
] ( S)( z ) ] + E[ Y − Φ
[
(28)
]
MAE Φ = E Y • − Φ ∪ ( S )( z ) z ∉ S P( z ∉ S ) + E Y• − Φ ∩ ( S )( z ) z ∈ S P( z ∈ S ) . •
− Φ∪
•
∩
]
( S)( z ) .
(29)