- Email: [email protected]
The Seelinder: Cylindrical 3D display viewable from 360 degrees
J. Vis. Commun. Image R. 21 (2010) 586–594
Contents lists available at ScienceDirect
J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci
The Seelinder: Cylindrical 3D display viewable from 360 degrees Tomohiro Yendo a,*, Toshiaki Fujii b, Masayuki Tanimoto a, Mehrdad Panahpour Tehrani a a b
Graduate School of Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan Graduate School of Science and Engineering, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, Japan
a r t i c l e
i n f o
Article history: Received 2 August 2009 Accepted 10 October 2009 Available online 21 October 2009 Keywords: Autostereoscopic display Multi-view Omnidirectional Ray-space Light field Parallax barrier
a b s t r a c t We propose a 3D video display technique that allows multiple viewers to see 3D images from a 360degree horizontal arc without wearing 3D glasses. This technique uses a cylindrical parallax barrier and a one-dimensional light source array. We have developed an experimental display system using this technique. Since this technique is based on the parallax panoramagram, the parallax number and resolution are limited by the diffraction at the parallax barrier. In order to solve this problem, we improved the technique by revolving the parallax barrier. The improved technique was incorporated into two experimental display systems. The newer one is capable of displaying 3D color video images within a 200-mm diameter and a 256-mm height. Images have a resolution of 1254 circumferential pixels and 256 vertical pixels, and are refreshed at 30 Hz. Each pixel has a viewing angle of 60 degrees that is divided into over 70 views so that the angular parallax interval of each pixel is less than 1 degree. These pixels are arranged on a cylindrical surface to allow for the produced 3D images to be observed from all directions. In this case, observers may barely perceive the discrete parallax. Ó 2009 Elsevier Inc. All rights reserved.
1. Introduction One of the most important features of the ideal 3D display would be allowing multiple viewers to see 3D images from free viewpoint and view of each viewer correctly corresponds to their own viewpoint. Since the Multiplex hologram [1],which is cylinder shaped holographic stereogram, has this feature especially 360-degree of horizontal viewing angle, it has been used for art, entertainment, advertisements, and other applications. However, it can only display static images. Some kinds of swept volume displays [2] are used to display dynamic images that can be seen from free directions; however, they are not suitable for natural images and applications are limited because all objects are see-through. Multi-view display and similar ray-based approach [3–10] such as integral photography [11] are capable of viewer-positiondependent effects including occlusion. However, it is difficult to realize an omnidirectional display using conventional techniques that consist of a flat panel display and a lens array sheet[12]. In the past several years, some types of omnidirectional displays which can be classified into multi-view technique are proposed. Maeda et al. uses rotating LCD panel with small viewing angle and switch the images in many directions [13]. This technique is easy to realize all-around displaying however parallax interval is limited by the refresh rate of the LCD. Currently, the refresh rate * Corresponding author. E-mail address: [email protected] (T. Yendo). 1047-3203/$ - see front matter Ó 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.jvcir.2009.10.004
of the LCD panel is usually 60 Hz, so the frame rate of images is about 7 Hz, and images on a display panel can be changed every 45 degrees. Otsuka et al. proposed 24-view display using rotating retro-reflective screen and projecting each view image from corresponding directions by a mirror array and projectors [14]. This technique is also difficult to realize large parallax number. Tanaka et al. [15] proposed a similar display to Otsuka’s, it employs seperate 12 projectors instead of a mirror array. Jones et al. [16] also use rotating anisotropic screen and they achieved displaying 288 views per round by using a special high-speed projector to exploit time-multiplexing. However it basically displays only B/W binary images or grayscale by dithering, due to the performance of DMD (Digital Micromirror Device) used as spatial light modulator of the projector. Cossairt at el.[17] proposed very similar display to Jones’s using modified Perspecta display [2] prior to Jones. Since the display also uses DMD projector it has same problems as Jones’s. We have also proposed 360-degree display in a unique approach [18]. This technique is based on the parallax panoramagram [19]. It uses a cylindrical parallax barrier and a one-dimensional light source array, which is constructed from semiconductor light sources such as LEDs aligned in a vertical line. The light source array rotates along the inside of the cylindrical parallax barrier, and the intensity of each light is synchronously modulated with the rotation. In this paper we present principles, prototypes, and display results of our proposing display system. The rest part of this paper is organized as follows. Section 2 explains the basic method of our display. Section 3 describes the first
587
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
prototype. Section 4 describes a significant problem of the basic method and proposes the improved method. Sections 5 and 6 describe the second and third prototype exploit the improved method, respectively. Section 7 discusses improving display performances and makes comparison with Jones’s display. Finally Section 8 concludes this paper. 2. Basic method P1
The proposed method is based on the parallax panoramagram, which is a technique that allows for different images to be shown in different viewpoints. As shown in Fig. 1, the system constructed using this technique is composed of a two-dimensional imaging device (photo, LCD, etc.) and a parallax barrier, which is a barrier with many vertical slits positioned in front of the imaging device. Several different types of multi-view autostereoscopic display systems based on this technique have been developed. These systems are based on the idea of preparing two-dimensional images that correspond to each viewpoint, and by choosing and showing the appropriate image. Here, the fundamental function of the parallax barrier is to independently control the color of the light ray for each direction so that the three-dimensional image can be displayed as a cluster of rays if the resolution of the ray direction is high enough. As shown in Fig. 2, rays from any part of the display surface concentrated to a point reproduce the light spot as if it is in the air. This means displaying three-dimensional image by reproducing light rays on the display surface. Based on this, we propose a 3D display technique.
P2
Parallax Barrier Fig. 2. 3D image reconstruction by reproducing light rays on the display surface.
Angular Scanning
Parallax Barrier
Minute Light Source
2.1. Principle of display Fig. 3(a) is a schematic diagram illustrating the principle of the proposed method. We use a parallax barrier and a minute light source that moves along the parallax barrier while keeping a fixed distance from it. If the aperture width of the parallax barrier is sufficiently small, the light going through the aperture becomes a thin flux, and its direction is scanned by the light source movement. We call this angular scanning. When a viewer sees the aperture from a particular direction, the light reaches the viewer’s eyes only at the moment when the light source passes through a certain position. As shown in Fig. 3(b), when the aperture is seen from viewing point 1, the light reaches the viewer’s eyes only when the light source passes position A. When the aperture is seen from viewing point 2, the light reaches the eyes only when the light source passes position B. A pixel, whose luminosity differs for each viewing direction, can be displayed by synchronously changing the intensity with the
Moving
2
1
A
2
1
A
B
Fig. 3. (a) Light going through the aperture is scanned by the light source movement. (b) When the aperture is seen from position 1, light reaches viewer’s eyes only when the light source passes position A. When the aperture is seen from position 2, light reaches the viewer’s eyes only when the light source passes position B.
light source movement. Furthermore, one light source can display many pixels by successively passing behind many apertures. From this principle, a cylindrical multi-view display can be made of a cylindrical parallax barrier and a one-dimensional light source array. The light source array consists of semiconductor light sources such as LEDs that are aligned in a vertical line and rotate along the inside of the cylindrical parallax barrier, as shown in Fig. 4. 2.2. Image size
Fig. 1. Parallax panoramagram.
The angular range of emission from the light source should be limited so that the light does not go through two apertures simul-
588
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
and the other is a normal line to the screen surface at a point in the set. In this case, the viewing angle h can be expressed as 1
h ¼ sin
Fig. 4. A cylindrical multi-view display device made of a cylindrical parallax barrier and a one-dimensional light source array. Display rotates along the inside of the cylindrical parallax barrier with a synchronous intensity modulation.
r sin /max R
ðR > rÞ;
ð1Þ
where R is the viewing distance from the center of the cylindrical screen and r is the radius of the cylindrical screen. Although the viewing angle h depends on the viewing distance R, we can still define a 3D image area, which is independent of the viewing distance. The 3D image area is a cylindrical space within which 3D images observable from a 360-degree horizontal arc can be displayed. The radius of this area is r sin /max . Fig. 5(b) shows this area. 2.3. Resolution of images and number of parallax
taneously. Consequently, the range of the light angular scanning should be limited. Here, we denote this range as /max . Within this limit, viewers see images that are not on the entire screen but on a part of the middle region. Fig. 5(a) illustrates the visible region of the screen. This region is a set of points satisfying the condition that / is within /max . Here, / is the angle between two straight lines. One is a line between a point in the set and the viewing point,
Cylindrical Screen (Parallax Barrier) radius : r
In this section, we study the relationship between the image resolution and the parallax number with various mechanical parameters. The aperture interval p, the aperture width wa , the light source width, wl , and the distance c between the light source and the parallax barrier are of particular importance. Fig. 6 lists all of these parameters. The horizontal and vertical resolutions of images depend on the aperture interval, p, and the vertical interval of the light source array, respectively. The diameter of the image area is determined by the range of angular scanning, /max . The light source itself always spreads light within /max . Therefore, in order to prevent light from simultaneously going through two adjoining apertures, c, p, wa ; wl , and /max must satisfy the following relationship
/max 6 tan1
Viewing Point
max
Cylindrical Screen (Parallax Barrier) radius : r
3-D Image Area
max
ð2Þ
Considering the parallax interval, it is basically determined by the intensity modulation frequency and the light source moving speed because this technique uses angular scanning. However angular resolution is also limited by divergence of light going through the aperture. The causes of divergence are width of the light source and the aperture, and diffraction at the aperture. The angle of divergence caused by the former is calculated by simple geometrical optics. When the light source is positioned right in front of the aperture, the divergence angle D/ is the largest and is described as
Visible Region of the Screen
Viewing Point
p wl wa : 2c
r sin
tan D/ ¼
wl þ wa : 2c
ð3Þ
If the aperture width is sufficiently wide in comparison to the wave length of the visible light, the effect of diffraction is negligible. However it may be main obstacle to achieve narrow parallax max
Parallax Barrier
Viewing Angle Aperture Interval p
Range of Angular Scanning max
Viewing Distance R
c
Aperture Width wa
Light Source wl
Fig. 5. (a) Images are seen not on the entire screen but in the region that consists of points satisfying / 6 /max . (b) 3D image area within which 3D images observable from 360-degree horizontal arc can be displayed. This cylindrical space has a radius of r sin /max .
Emitting Part
Distance between Light Source and Parallax Barrier
Fig. 6. The mechanical parameters that determine the image resolution and the parallax number.
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
589
Table 1 Specifications of prototypes displays. 1st prototype
2nd prototype
3rd prototype
Pixel interval (mm) Number of pixels Range of angular scanning Number of parallax Parallax interval Diameter of parallax barrier (mm) Image size (mm) Color depth
22
0.5 1
11
221 32 ±30°
1254 128 ±30°
1254 256 ±30°
22
70 per pixel
Flexible
16.4° 146
61° 230
6 1° 430
/70 64 Binary
/100 128 Binary
Refresh rate (Hz) Dynamic imagery Case dimension WDH (mm)
30 No 300 200 420
30 No 450 350 500
/200 256 4-Bit for each RGB 30 Yes (10 s) 550 550 1100
interval less than a few degrees because narrower aperture is needed to reduce geometrical optical divergence. In Section 4, we analyze this problem and propose techniques to avoid it. 3. The first prototype model We developed a prototype display model using the technique explained above. This model consists of a cylindrical parallax barrier, an LED array, a drive circuit, a photo-reflector, an induction motor, and other components. The LED array, drive circuit, and photo-reflector are rotated by the motor. The photo-reflector senses the passing light by the apertures, and the intensity of the LED array is synchronously controlled with the rotation. The drive circuit has an ROM that stores binary image data instead of relying on transmission from the outside. The LED array has 32 LEDs with a 2-mm interval. The cylindrical parallax barrier has 221 apertures with a 2-mm interval and a diameter of 146 mm. The pixel interval is 2 mm both horizontally and vertically, and the parallax number is 22. The specifications, block diagram, and photograph of this display are given in Table 1, Figs. 7 and 8, respectively. Fig. 9 shows the images obtained from three adjacent viewing points. As shown in these photographs, the images for all viewing points were observed from around the display. It was demon-
Timing Control
Photo Reflector
PLL
LED Array
LED Driver
ROM
Address Counter
Parallax Barrier Slip Rings
Fig. 8. Photograph of the first prototype model.
strated that the 3D images displayed using the proposed technique could be seen horizontally from 360 degrees. 4. Improvements 4.1. Limitations of the basic method The inequality (2) indicates that when the distance c between the light source and the parallax barrier is fixed, a decrease in pixel interval p causes the range of angular scanning to decrease. This implies a trade-off between the resolution and the sizes of images. In order to increase the resolution and size simultaneously, the distance c between the light source and the parallax barrier must be shortened. However, according to Eq. (3), shortening this distance will enlarge the divergence angle of light D/ and decrease the angular resolution. In short, there is a trade-off relationship among the image resolution, image size, and parallax number. In order to improve the resolution, image size, and parallax number simultaneously, the emitting part width wl and the aperture width wa must be decreased. Assuming that the pixel interval is 0.2 mm, the range of the scanning angle is 30 degrees, and the parallax number is 50, we then have wl ¼ wa ¼ 0:01 mm. This value is roughly ten times the wave length of the visible light, and the angular resolution is lowered by the influence of diffraction at the apertures. From above, we see a fundamental limitation of the parallax panoramagram [20]. However, this limitation can be overcome by improving the scanning technique. In the following subsection, we shall propose a method for improving the scanning technique.
DC5V AC100V
Motor
Fig. 7. Block diagram of the first prototype model.
4.2. Parallax barrier movement method Here, we propose a parallax barrier movement method that moves not only the light source but also the parallax barrier.
590
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
Fig. 9. Images obtained from three adjacent viewing points.
Fig. 10 schematically demonstrates this method. As shown in the figure, when the parallax barrier moves in the opposite direction of the light source movement, the light coming through the aper-
ture is scanned as if there exists a virtual parallax barrier that is finer than the actual one. The aperture interval pv of the virtual parallax barrier is expressed as
pv ¼
Velocity: V2
V1 p ; V1 V2 0
ð4Þ
where p0 is the aperture interval of the real parallax barrier, V 1 ð> 0Þ is the velocity of the light source, and V 2 ð< 0Þ is the velocity of the parallax barrier. This equation means that the movement of the parallax barrier decreases the pixel interval to V 1 =ðV 1 V 2 Þ times the pixel interval of the stationary parallax barrier case. On the other hand, the angular scanning range and the divergence angle of the light going through the aperture depend on the distance from the light source to the real parallax barrier and the shape of the parallax barrier. This indicates that the image size and parallax number are independent of the parallax barrier movement.
Pitch: p0
Parallax Barrier
Light Source Velocity: V1 Time: t
4.3. Multiple light source arrays
pv
Scanning
Using the parallax barrier movement method, the resolution depends on the velocity ratio of the light source to the parallax barrier. Considering the rotational speed of the parallax barrier, a higher resolution is possible if the light source array rotates more slowly. However, since the rotational frequency of the light source array equals the refresh frequency of the images, the rotational frequency cannot be significantly reduced. Consequently, we adopted multiple light source arrays. In this case, if the number of light source arrays is n, an image is then refreshed n times per rotation of the light source arrays. Thus, we can reduce the rotational frequency to 1=n of the refresh frequency, which increases the displaying time allotted to each pixel to n times. Therefore, the image luminosity can be increased n times. Moreover, since the modulation frequency of each light source is reduced, the operating frequencies of the light emitting devices and drive circuits also become lower. For a very narrow parallax interval such as the super-multi-view region of the stereogram [21], this is particularly important. 5. The second prototype model
Virtual Parallax Barrier Pitch: pv
Fig. 10. Virtual parallax barrier.
To study the efficacy of the improvement method, we constructed a second prototype display model using the parallax barrier movement method and multiple light source arrays. As shown in Fig. 11, the basic structure is that the cylindrical parallax barrier rotates rapidly, and the LED arrays rotate slowly in the opposite direction. This model is shown in Fig. 12. Tables 1 and 2 provides all of the specifications of this model. The pixel numbers are 1254 with a 0.5 mm pitch horizontally and 128 with a 1 mm pitch vertically, respectively. Each pixel is displayed in a region that has an angle between the normal line and a display surface at a pixel less than 30 degrees. This region is divided into 70 directions,
591
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
1-D light source array (LED Array) (Slow rotation)
Table 2 Comparison between the current prototypes of Jones’s and ours.
Resolution of a view Number of views Image size (mm) Refresh rate (Hz) Color depth
Cylindrical parallax barrier (Fast rotation)
Our 3rd prototype
Jones’s prototype
Approx. 200 256 > 360 /200 256 30 4 Bit for each RGB
768 768 288 /130 130 15 B/W binary
rays, which are equiangularly arranged in a cylindrical shape with a diameter of 200 mm, and each has 128 LEDs at intervals of a millimeter. The cylindrical parallax barrier with a diameter of 230 mm surrounds the LED arrays. The rotating speed of the LED arrays is 56.25 rpm, and that of the cylindrical parallax barrier is 1800 rpm in the opposite direction. The barrier has 38 apertures with widths of 0.1 mm. The block diagram is shown in Fig. 13.
Fig. 11. Structure of the improved cylindrical display.
5.2. Results Fig. 14 shows photographs of the images produced by the second prototype model that were obtained from various directions.
LED Array LED Driver FIFO
LED Array LED Driver
32
Timing Pulse
FIFO
Photo Reflect Sensor
Cylindrical Parallax Barrier
140V
DC/DC Converter
Slip Rings
Image Data
5V
AC Servo Motor
Display Unit DC140V
Write Control
Memory Memory Bank #2 Memory Bank 11 Bank #0
and the intensity of the light rays in each direction is independently controllable. The angular intervals of the rays are chosen such that the tangents of all intervals are approximately the same. The cylindrical surface is filled by such pixels, and hence, 3D images are viewable from any angle. The refresh frequency is 30 Hz optically.
Read Control
Fig. 12. Photograph of the second prototype model.
5.1. Mechanism
Frame Memory Unit This display system is composed of three parts, the display unit, the frame memory unit, and the PC. The display unit has 32 LED ar-
PC/AT compatible
Fig. 13. Block diagram of the second prototype model.
592
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
Fig. 14. Examples of images displayed by the second prototype model.
We could see the 3D image naturally in any viewing distance. The images had strong depth cues of natural binocular disparity. When we move around the display, we saw the corresponding images in our viewing position. Therefore, we perceived the objects just as if they were floating in the air.
from various directions. This 3D image was synthesized from over 360 photographs of real person which angular inteval was less than 1 degree.
6. The third prototype model
Our proposed system is functionally similar to Jones’s one. In this section, we discuss and make a comparison between the two display systems. In simply term, Jones’s one has advantages in its simple structure and high resolution relatively, and ours has advantages in color reproduction and suitability for larger size display.
Cylindrical Parallax Barrier
SDRAM
LED Array LED Driver
LED Array
Photo Sensor
54
LED Driver
Photo Sensor
We also developed the third prototype which was capable of diplaying color and moving images. This model is shown in Fig. 15. Table 1 provides all of the specifications of this model. Image size have been enlarged to 200-mm diameter and 256mm height. The pixel numbers are 1254 horizontally and 256 vertically with a 1 mm pitch, respectively. To display color images, LED arrays of three colors are used. Each LED array has frame memory to enable playback of stored video image. The total amount of the memory is 6.9 GB, which stores dynamic imagery of approximately just under 10 s. The refresh rate is 30 Hz. The block diagram is shown in Fig. 16. Fig. 17 shows photographs of the images produced by the this model that were obtained
7. Discussion
SDRAM
Controller
Controller 5V, 3V Bus AC100V
SW Regulator
Slip Rings
Display Unit
AC Servo Motor
Bus Width Converter & Line Driver AC100V
AC200V
PC/AT compatible Fig. 15. Photograph of the third prototype model.
Fig. 16. Block diagram of the third prototype model.
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
593
Fig. 17. Examples of images displayed by the third prototype model.
Jones’s system consists of a high-speed video projector using Texas Instruments’ DMD (digital micromirror device) and an rotating anisotoropic reflective screen. Its mechanical structure is simple. Its resolution, color depth, and the number of views (or angular resolution) are just determined by the performance of DMD in resolution and refreshing speed. So far their prototype shows only B/W binary imagery of 288 views at refresh rate of 15 Hz. In the meaning of color reproduction, Perspecta, which is a commercialized swept volume display employs high-speed DMD projector similar to Jones’s, and Cossairt’s system based on Perspecta have achieved displaying color imagery by using three DMDs of red, blue, and green. However these displays still show binary tone in each RGB color. Of course, the enhancement of the number of tone level or angular resolution by using more DMDs could be considered, however it could need more contrivances that have not been developed yet. Image size is determined by the rotating screen size and it is difficult to make the screen larger because of its high-speed rotation for keeping refresh rate of 3D imagery. Enlargement of the screen faces significant problems as increasing moment of inertia, centrifugal force, and air resistance. At constant angular velocity, the centrifugal force increases in proportion to the diameter. The moment of inertia of a flat plate with constant thickness increases in proportion to the fourth power of scale. Practically the thickness of the screen and supporting structures need to increase with size for keeping its shape against increasing centrifugal force. Therefore the moment of inertia increases in proportion to between the fourth and fifth power and it causes increase of energy required to rotate. Although discussing air resistance is pretty complex problem, we can say that it increases at least in proportion to the cube of diameter because circumferential length, velocity, and height of the screen increase in proportion to diameter, respectively. Moreover it is extremely difficult to make the display whose diameter exceeds 3.6 m because the circumferential velocity reaches the velocity of sound at that diameter where the rotation speed is 30 revolutions per second. In contrast to Jones’s system whose performances are mainly determined by that of DMD, our system has more flexibility to design. For tone reproduction, analog driving can be applied because our system employs LED as light source. Since even ordinary LED responds to several MHz, sufficient number of levels of tone reproduction is achieved, independently from the number of views or angular resolution. Moreover approximately 10 bit linear tone is achieved by using PWM (Pulse Width Modulation) in the mechanical condition of 3rd prototype. In vertical direction, the number of pixels and pixel interval are simply same as LED arrays. In horizontal direction, pixel interval is determined by interval between slits on the parallax barrier and velocity ratio of both cylinders as described in Section 4. Circumferential velocity of LED arrays is determined by refresh rate of 3D display and interval of LED arrays, and interval of slits is determined by angular resolution and angular scanning range. Therefore horizontal pixel interval is determined by interval of LED arrays
and circumferential velocity of the parallax barrier. Here the point is that the pixel interval is constant and independent from the display size where the interval of LED arrays and slits are kept constant. In that case, circumferential velocities, not angular velocities, of both cylinders are constant. In case of constant circumferential velocity, centrifugal force decrease in inverse proportion to diameter. That could be a great help for large cylinders. Air resistance increases in proportion to the square of diameter simply because the surface area of the cylinder increases. The moment of inertia increases in proportion to the fourth power of diameter, however energy requirement increases in proportion to only the square of diameter because rotation speed is decrease in inverse proportion to diameter. Of course circumferential velocity never exceeds the velocity of sound because it is constant.
8. Conclusions To realize a multi-view 3D display that is observable from all directions, we proposed a technique that rotates a one-dimensional light source array inside the cylindrical parallax barrier. The first prototype model was developed based on the parallax panoramagram, which was improved to display in all horizontal directions by introducing mechanical scanning. We confirmed that the image it produced was observable from a 360-degree horizontal arc. However, the parallax panoramagram had some limitations on the image resolution and parallax number, which are caused by the diffraction effect at its apertures. To overcome these limitations, we proposed the parallax barrier movement method and constructed two prototype models. The 3D images produced by these models were observable from 360-degree with a smooth motion parallax. It is demonstrated that our proposed techniques have potential to display natural 3D imagery with freedom in viewing position. Moreover we reached the conclusion that our system especially has potential for large size display from discussion and comparison with Jones’s system.
References [1] L. Cross, The multiplex technique for cylindrical holographic stereograms, in: Proceedings of SPIE Annual Technical Conference, San Diego Seminar, 10, 1977. [2] G. Favalora, R. Dorval, D. Hall, M. Giovinco, J. Napoli, Volumetric threedimensional display system with rasterization hardware, in: SPIE Proceedings #4297, 2001, pp. 227–235. [3] Y. Kajiki, H. Yoshikawa, T. Honda, Three-dimensional display with focused light array, in: SPIE Proceedings #2652, Practical holography X, 1996, pp. 106–116. [4] K. Matsumoto, T. Honda, Research of 3-D display using the anamorphic optics, in: SPIE Proceedings #3012, 1997, pp. 199–207. [5] T. Sudo, H. Morishima, T. Osaka, N. Taniguchi, 3D display using intersection of light beams, in: SPIE Proceedings #3957, 2000, pp. 215–224. [6] A.R.L. Travis, J.R. Moore, N.A. Dodgeson, S.R. Lang, Time-multiplexed color autostereoscopic display, in: SPIE Proceedings #2653, 1996, pp. 154–166. [7] O. Cossairt, C. Moller, A.R.L. Travis, S.A. Benton, Novel view display based on DMD technology, in: SPIE Proceedings #5291, 2004, pp. 273–278. [8] N.A. Dodgson, J.R. Moore, S.R. Lang, G. Martin, P. Canepa, A 50 timemultiplexed autostereoscopic display, in: SPIE Proceedings #3597, Stereoscopic Displays and Virtual Reality Systems XI, 2000, pp. 177–183.
594
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
[9] H. Liao, M. Iwahara, N. Hata, T. Dohi, High-quality integral videography using a multiprojector, Optics Express 12 (6) (2004) 1067–1076. [10] Y. Takaki, A novel 3D display using an array of LCD panels, in: SPIE Proceedings #5003, 2003, pp. 1–8. [11] F. Okano, H. Hoshino, J. Arai, I. Yuyama, Real-time pickup method for a threedimensional image based on integral photography, Applied Optics 36 (7) (1997) 1598–1603. [12] Y. Takaki, T. Dairiki, 72-Directional display having VGA resolution for highappearance image generation, in: SPIE Proceedings #6055, 2006, pp. 60550X1–8. [13] H. Maeda, K. Hirose, J. Yamashita, K. Hirota, M. Hirose, All-around display for video avatar in real world, in: Proceedings of ISMAR’03 The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003, pp. 288–289. [14] R. Otsuka, T. Hoshino, Y. Horry, Transpost: a novel approach to the display and transmission of 360 degree-viewable 3D solid images, IEEE Transactions on Visualization and Computer Graphics 12 (2) (2006) 178–185. [15] K. Tanaka, S. Aoki, A method for the real-time construction of a full parallax light field, in: SPIE Proceedings #6055, 2006, pp. 605516-1–11. [16] A. Jones, I. McDowall, H. Yamada, M. Bolas, P. Debevee, Rendering for an interactive 360° light field display, ACM Transaction on Graphics 26 (3) (2007). Article 40. [17] O. Cossairt, J. Napoli, S.L. Hill, R.K. Dorval, G.E. Favolora, Occlusion-capable multiview volumetric three-dimensional display, Applied Optics 46 (3) (2007) 1244–1250. [18] T. Endo, Y. Kajiki, T. Honda, M. Sato, A cylindrical 3-D display observable from all directions, SPIE Proceedings #3957, 2000. [19] H. Ives, Motion pictures in relief, Journal of the Optical Society of America 17 (1928) 435–439. [20] T. Okoshi, Three-dimensional imaging techniques, Academic Press, 1976. [21] Y. Kajiki, H. Yoshikawa, T. Honda, Hologram-like video images by 45-view stereoscopic display, in: SPIE Proceedings #3012, pp. 154–166, 1997.
Tomohiro Yendo received the B.Eng. and M.Eng. and Ph.D. degrees from Tokyo Institute of Technology, Tokyo, Japan, in 1996, 1998, and 2001, respectively. From 1998 to 2002, he was a researcher for the Advanced 3-D Tele-Vision Project at the Telecommunications Advancement Organization (TAO) of Japan. From 2002 to 2004, he was a research fellow at Japan Science and Technology Agency (JST). He is now an Assistant Professor at Nagoya University, Nagoya, Japan. His current research interests include 3-D image display, capturing and processing.
Toshiaki Fujii received the Dr. E. degree in Electrical Engineering from the University of Tokyo in 1995. From 1995 to 2007, he was with the Graduate School of Engineering, Nagoya University. He is currently an Associate Professor in the Graduate School of Science and Engineering, Tokyo Institute of Technology. His current research interests include multi-dimensional signal processing, large-scale multi-camera systems, multi-view video coding and transmission, free-viewpoint television, and their applications for Intelligent Transport Systems. He is a member of the IEEE, The Institute of Electronics, Information and Communication Engineers, and the Institute of Image Information and Television Engineers of Japan.
Mehrdad Panahpour Tehrani received Dr.Eng. degree in Information Electronics from Nagoya University, Nagoya, Japan in 2004. From 2004 to 2007 he was with Information Technology Center, Nagoya University as a Post-Doctoral Researcher. He worked as an Associate Research Engineer with Ultra Realistic Communications Laboratory, KDDI R&D Laboratories Inc., Saitama, Japan, from 2007 to 2009. Currently, he is working as an Associate Professor at the Department of Electrical Engineering, Graduate School of Engineering, Nagoya University, Japan. His research interests are 3D image processing and communication, multiview coding, distributed source coding in camera sensor networks, and 3D media integration and communication.
Masayuki Tanimoto (M’71-SM’07) received the B.E., M.E., and Dr.E. degrees in electronic engineering from the University of Tokyo, Tokyo, Japan, in 1970, 1972, and 1976, respectively. He joined Nagoya University, Nagoya, Japan, in 1976 and started research on visual communication and communication systems. Since 1991, he has been a Professor at Graduate School of Engineering, Nagoya University. His current research interests include image coding, image processing, 3-D images, FTV and ITS. Dr. Tanimoto is the President of ITE. He was the Chairperson of Technical Group on Communication Systems of IEICE, the Chairperson of the Steering Committee of Picture Coding Symposium of Japan, IEICE Councilor, ITE Councilor and Tokai Section Chair of IEICE. He is a member of the International Steering Committee of the Picture Coding Symposium. He received the Ichimura Award, the TELECOM System Technology Award from The Telecommunications Advancement Foundation, the Niwa-Takayanagi Paper Award from ITE, IEICE Fellow Award, IEICE Achievement Award, ITE Fellow Award, and the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science, and Technology.
Contents lists available at ScienceDirect
J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci
The Seelinder: Cylindrical 3D display viewable from 360 degrees Tomohiro Yendo a,*, Toshiaki Fujii b, Masayuki Tanimoto a, Mehrdad Panahpour Tehrani a a b
Graduate School of Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan Graduate School of Science and Engineering, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, Japan
a r t i c l e
i n f o
Article history: Received 2 August 2009 Accepted 10 October 2009 Available online 21 October 2009 Keywords: Autostereoscopic display Multi-view Omnidirectional Ray-space Light field Parallax barrier
a b s t r a c t We propose a 3D video display technique that allows multiple viewers to see 3D images from a 360degree horizontal arc without wearing 3D glasses. This technique uses a cylindrical parallax barrier and a one-dimensional light source array. We have developed an experimental display system using this technique. Since this technique is based on the parallax panoramagram, the parallax number and resolution are limited by the diffraction at the parallax barrier. In order to solve this problem, we improved the technique by revolving the parallax barrier. The improved technique was incorporated into two experimental display systems. The newer one is capable of displaying 3D color video images within a 200-mm diameter and a 256-mm height. Images have a resolution of 1254 circumferential pixels and 256 vertical pixels, and are refreshed at 30 Hz. Each pixel has a viewing angle of 60 degrees that is divided into over 70 views so that the angular parallax interval of each pixel is less than 1 degree. These pixels are arranged on a cylindrical surface to allow for the produced 3D images to be observed from all directions. In this case, observers may barely perceive the discrete parallax. Ó 2009 Elsevier Inc. All rights reserved.
1. Introduction One of the most important features of the ideal 3D display would be allowing multiple viewers to see 3D images from free viewpoint and view of each viewer correctly corresponds to their own viewpoint. Since the Multiplex hologram [1],which is cylinder shaped holographic stereogram, has this feature especially 360-degree of horizontal viewing angle, it has been used for art, entertainment, advertisements, and other applications. However, it can only display static images. Some kinds of swept volume displays [2] are used to display dynamic images that can be seen from free directions; however, they are not suitable for natural images and applications are limited because all objects are see-through. Multi-view display and similar ray-based approach [3–10] such as integral photography [11] are capable of viewer-positiondependent effects including occlusion. However, it is difficult to realize an omnidirectional display using conventional techniques that consist of a flat panel display and a lens array sheet[12]. In the past several years, some types of omnidirectional displays which can be classified into multi-view technique are proposed. Maeda et al. uses rotating LCD panel with small viewing angle and switch the images in many directions [13]. This technique is easy to realize all-around displaying however parallax interval is limited by the refresh rate of the LCD. Currently, the refresh rate * Corresponding author. E-mail address: [email protected] (T. Yendo). 1047-3203/$ - see front matter Ó 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.jvcir.2009.10.004
of the LCD panel is usually 60 Hz, so the frame rate of images is about 7 Hz, and images on a display panel can be changed every 45 degrees. Otsuka et al. proposed 24-view display using rotating retro-reflective screen and projecting each view image from corresponding directions by a mirror array and projectors [14]. This technique is also difficult to realize large parallax number. Tanaka et al. [15] proposed a similar display to Otsuka’s, it employs seperate 12 projectors instead of a mirror array. Jones et al. [16] also use rotating anisotropic screen and they achieved displaying 288 views per round by using a special high-speed projector to exploit time-multiplexing. However it basically displays only B/W binary images or grayscale by dithering, due to the performance of DMD (Digital Micromirror Device) used as spatial light modulator of the projector. Cossairt at el.[17] proposed very similar display to Jones’s using modified Perspecta display [2] prior to Jones. Since the display also uses DMD projector it has same problems as Jones’s. We have also proposed 360-degree display in a unique approach [18]. This technique is based on the parallax panoramagram [19]. It uses a cylindrical parallax barrier and a one-dimensional light source array, which is constructed from semiconductor light sources such as LEDs aligned in a vertical line. The light source array rotates along the inside of the cylindrical parallax barrier, and the intensity of each light is synchronously modulated with the rotation. In this paper we present principles, prototypes, and display results of our proposing display system. The rest part of this paper is organized as follows. Section 2 explains the basic method of our display. Section 3 describes the first
587
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
prototype. Section 4 describes a significant problem of the basic method and proposes the improved method. Sections 5 and 6 describe the second and third prototype exploit the improved method, respectively. Section 7 discusses improving display performances and makes comparison with Jones’s display. Finally Section 8 concludes this paper. 2. Basic method P1
The proposed method is based on the parallax panoramagram, which is a technique that allows for different images to be shown in different viewpoints. As shown in Fig. 1, the system constructed using this technique is composed of a two-dimensional imaging device (photo, LCD, etc.) and a parallax barrier, which is a barrier with many vertical slits positioned in front of the imaging device. Several different types of multi-view autostereoscopic display systems based on this technique have been developed. These systems are based on the idea of preparing two-dimensional images that correspond to each viewpoint, and by choosing and showing the appropriate image. Here, the fundamental function of the parallax barrier is to independently control the color of the light ray for each direction so that the three-dimensional image can be displayed as a cluster of rays if the resolution of the ray direction is high enough. As shown in Fig. 2, rays from any part of the display surface concentrated to a point reproduce the light spot as if it is in the air. This means displaying three-dimensional image by reproducing light rays on the display surface. Based on this, we propose a 3D display technique.
P2
Parallax Barrier Fig. 2. 3D image reconstruction by reproducing light rays on the display surface.
Angular Scanning
Parallax Barrier
Minute Light Source
2.1. Principle of display Fig. 3(a) is a schematic diagram illustrating the principle of the proposed method. We use a parallax barrier and a minute light source that moves along the parallax barrier while keeping a fixed distance from it. If the aperture width of the parallax barrier is sufficiently small, the light going through the aperture becomes a thin flux, and its direction is scanned by the light source movement. We call this angular scanning. When a viewer sees the aperture from a particular direction, the light reaches the viewer’s eyes only at the moment when the light source passes through a certain position. As shown in Fig. 3(b), when the aperture is seen from viewing point 1, the light reaches the viewer’s eyes only when the light source passes position A. When the aperture is seen from viewing point 2, the light reaches the eyes only when the light source passes position B. A pixel, whose luminosity differs for each viewing direction, can be displayed by synchronously changing the intensity with the
Moving
2
1
A
2
1
A
B
Fig. 3. (a) Light going through the aperture is scanned by the light source movement. (b) When the aperture is seen from position 1, light reaches viewer’s eyes only when the light source passes position A. When the aperture is seen from position 2, light reaches the viewer’s eyes only when the light source passes position B.
light source movement. Furthermore, one light source can display many pixels by successively passing behind many apertures. From this principle, a cylindrical multi-view display can be made of a cylindrical parallax barrier and a one-dimensional light source array. The light source array consists of semiconductor light sources such as LEDs that are aligned in a vertical line and rotate along the inside of the cylindrical parallax barrier, as shown in Fig. 4. 2.2. Image size
Fig. 1. Parallax panoramagram.
The angular range of emission from the light source should be limited so that the light does not go through two apertures simul-
588
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
and the other is a normal line to the screen surface at a point in the set. In this case, the viewing angle h can be expressed as 1
h ¼ sin
Fig. 4. A cylindrical multi-view display device made of a cylindrical parallax barrier and a one-dimensional light source array. Display rotates along the inside of the cylindrical parallax barrier with a synchronous intensity modulation.
r sin /max R
ðR > rÞ;
ð1Þ
where R is the viewing distance from the center of the cylindrical screen and r is the radius of the cylindrical screen. Although the viewing angle h depends on the viewing distance R, we can still define a 3D image area, which is independent of the viewing distance. The 3D image area is a cylindrical space within which 3D images observable from a 360-degree horizontal arc can be displayed. The radius of this area is r sin /max . Fig. 5(b) shows this area. 2.3. Resolution of images and number of parallax
taneously. Consequently, the range of the light angular scanning should be limited. Here, we denote this range as /max . Within this limit, viewers see images that are not on the entire screen but on a part of the middle region. Fig. 5(a) illustrates the visible region of the screen. This region is a set of points satisfying the condition that / is within /max . Here, / is the angle between two straight lines. One is a line between a point in the set and the viewing point,
Cylindrical Screen (Parallax Barrier) radius : r
In this section, we study the relationship between the image resolution and the parallax number with various mechanical parameters. The aperture interval p, the aperture width wa , the light source width, wl , and the distance c between the light source and the parallax barrier are of particular importance. Fig. 6 lists all of these parameters. The horizontal and vertical resolutions of images depend on the aperture interval, p, and the vertical interval of the light source array, respectively. The diameter of the image area is determined by the range of angular scanning, /max . The light source itself always spreads light within /max . Therefore, in order to prevent light from simultaneously going through two adjoining apertures, c, p, wa ; wl , and /max must satisfy the following relationship
/max 6 tan1
Viewing Point
max
Cylindrical Screen (Parallax Barrier) radius : r
3-D Image Area
max
ð2Þ
Considering the parallax interval, it is basically determined by the intensity modulation frequency and the light source moving speed because this technique uses angular scanning. However angular resolution is also limited by divergence of light going through the aperture. The causes of divergence are width of the light source and the aperture, and diffraction at the aperture. The angle of divergence caused by the former is calculated by simple geometrical optics. When the light source is positioned right in front of the aperture, the divergence angle D/ is the largest and is described as
Visible Region of the Screen
Viewing Point
p wl wa : 2c
r sin
tan D/ ¼
wl þ wa : 2c
ð3Þ
If the aperture width is sufficiently wide in comparison to the wave length of the visible light, the effect of diffraction is negligible. However it may be main obstacle to achieve narrow parallax max
Parallax Barrier
Viewing Angle Aperture Interval p
Range of Angular Scanning max
Viewing Distance R
c
Aperture Width wa
Light Source wl
Fig. 5. (a) Images are seen not on the entire screen but in the region that consists of points satisfying / 6 /max . (b) 3D image area within which 3D images observable from 360-degree horizontal arc can be displayed. This cylindrical space has a radius of r sin /max .
Emitting Part
Distance between Light Source and Parallax Barrier
Fig. 6. The mechanical parameters that determine the image resolution and the parallax number.
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
589
Table 1 Specifications of prototypes displays. 1st prototype
2nd prototype
3rd prototype
Pixel interval (mm) Number of pixels Range of angular scanning Number of parallax Parallax interval Diameter of parallax barrier (mm) Image size (mm) Color depth
22
0.5 1
11
221 32 ±30°
1254 128 ±30°
1254 256 ±30°
22
70 per pixel
Flexible
16.4° 146
61° 230
6 1° 430
/70 64 Binary
/100 128 Binary
Refresh rate (Hz) Dynamic imagery Case dimension WDH (mm)
30 No 300 200 420
30 No 450 350 500
/200 256 4-Bit for each RGB 30 Yes (10 s) 550 550 1100
interval less than a few degrees because narrower aperture is needed to reduce geometrical optical divergence. In Section 4, we analyze this problem and propose techniques to avoid it. 3. The first prototype model We developed a prototype display model using the technique explained above. This model consists of a cylindrical parallax barrier, an LED array, a drive circuit, a photo-reflector, an induction motor, and other components. The LED array, drive circuit, and photo-reflector are rotated by the motor. The photo-reflector senses the passing light by the apertures, and the intensity of the LED array is synchronously controlled with the rotation. The drive circuit has an ROM that stores binary image data instead of relying on transmission from the outside. The LED array has 32 LEDs with a 2-mm interval. The cylindrical parallax barrier has 221 apertures with a 2-mm interval and a diameter of 146 mm. The pixel interval is 2 mm both horizontally and vertically, and the parallax number is 22. The specifications, block diagram, and photograph of this display are given in Table 1, Figs. 7 and 8, respectively. Fig. 9 shows the images obtained from three adjacent viewing points. As shown in these photographs, the images for all viewing points were observed from around the display. It was demon-
Timing Control
Photo Reflector
PLL
LED Array
LED Driver
ROM
Address Counter
Parallax Barrier Slip Rings
Fig. 8. Photograph of the first prototype model.
strated that the 3D images displayed using the proposed technique could be seen horizontally from 360 degrees. 4. Improvements 4.1. Limitations of the basic method The inequality (2) indicates that when the distance c between the light source and the parallax barrier is fixed, a decrease in pixel interval p causes the range of angular scanning to decrease. This implies a trade-off between the resolution and the sizes of images. In order to increase the resolution and size simultaneously, the distance c between the light source and the parallax barrier must be shortened. However, according to Eq. (3), shortening this distance will enlarge the divergence angle of light D/ and decrease the angular resolution. In short, there is a trade-off relationship among the image resolution, image size, and parallax number. In order to improve the resolution, image size, and parallax number simultaneously, the emitting part width wl and the aperture width wa must be decreased. Assuming that the pixel interval is 0.2 mm, the range of the scanning angle is 30 degrees, and the parallax number is 50, we then have wl ¼ wa ¼ 0:01 mm. This value is roughly ten times the wave length of the visible light, and the angular resolution is lowered by the influence of diffraction at the apertures. From above, we see a fundamental limitation of the parallax panoramagram [20]. However, this limitation can be overcome by improving the scanning technique. In the following subsection, we shall propose a method for improving the scanning technique.
DC5V AC100V
Motor
Fig. 7. Block diagram of the first prototype model.
4.2. Parallax barrier movement method Here, we propose a parallax barrier movement method that moves not only the light source but also the parallax barrier.
590
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
Fig. 9. Images obtained from three adjacent viewing points.
Fig. 10 schematically demonstrates this method. As shown in the figure, when the parallax barrier moves in the opposite direction of the light source movement, the light coming through the aper-
ture is scanned as if there exists a virtual parallax barrier that is finer than the actual one. The aperture interval pv of the virtual parallax barrier is expressed as
pv ¼
Velocity: V2
V1 p ; V1 V2 0
ð4Þ
where p0 is the aperture interval of the real parallax barrier, V 1 ð> 0Þ is the velocity of the light source, and V 2 ð< 0Þ is the velocity of the parallax barrier. This equation means that the movement of the parallax barrier decreases the pixel interval to V 1 =ðV 1 V 2 Þ times the pixel interval of the stationary parallax barrier case. On the other hand, the angular scanning range and the divergence angle of the light going through the aperture depend on the distance from the light source to the real parallax barrier and the shape of the parallax barrier. This indicates that the image size and parallax number are independent of the parallax barrier movement.
Pitch: p0
Parallax Barrier
Light Source Velocity: V1 Time: t
4.3. Multiple light source arrays
pv
Scanning
Using the parallax barrier movement method, the resolution depends on the velocity ratio of the light source to the parallax barrier. Considering the rotational speed of the parallax barrier, a higher resolution is possible if the light source array rotates more slowly. However, since the rotational frequency of the light source array equals the refresh frequency of the images, the rotational frequency cannot be significantly reduced. Consequently, we adopted multiple light source arrays. In this case, if the number of light source arrays is n, an image is then refreshed n times per rotation of the light source arrays. Thus, we can reduce the rotational frequency to 1=n of the refresh frequency, which increases the displaying time allotted to each pixel to n times. Therefore, the image luminosity can be increased n times. Moreover, since the modulation frequency of each light source is reduced, the operating frequencies of the light emitting devices and drive circuits also become lower. For a very narrow parallax interval such as the super-multi-view region of the stereogram [21], this is particularly important. 5. The second prototype model
Virtual Parallax Barrier Pitch: pv
Fig. 10. Virtual parallax barrier.
To study the efficacy of the improvement method, we constructed a second prototype display model using the parallax barrier movement method and multiple light source arrays. As shown in Fig. 11, the basic structure is that the cylindrical parallax barrier rotates rapidly, and the LED arrays rotate slowly in the opposite direction. This model is shown in Fig. 12. Tables 1 and 2 provides all of the specifications of this model. The pixel numbers are 1254 with a 0.5 mm pitch horizontally and 128 with a 1 mm pitch vertically, respectively. Each pixel is displayed in a region that has an angle between the normal line and a display surface at a pixel less than 30 degrees. This region is divided into 70 directions,
591
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
1-D light source array (LED Array) (Slow rotation)
Table 2 Comparison between the current prototypes of Jones’s and ours.
Resolution of a view Number of views Image size (mm) Refresh rate (Hz) Color depth
Cylindrical parallax barrier (Fast rotation)
Our 3rd prototype
Jones’s prototype
Approx. 200 256 > 360 /200 256 30 4 Bit for each RGB
768 768 288 /130 130 15 B/W binary
rays, which are equiangularly arranged in a cylindrical shape with a diameter of 200 mm, and each has 128 LEDs at intervals of a millimeter. The cylindrical parallax barrier with a diameter of 230 mm surrounds the LED arrays. The rotating speed of the LED arrays is 56.25 rpm, and that of the cylindrical parallax barrier is 1800 rpm in the opposite direction. The barrier has 38 apertures with widths of 0.1 mm. The block diagram is shown in Fig. 13.
Fig. 11. Structure of the improved cylindrical display.
5.2. Results Fig. 14 shows photographs of the images produced by the second prototype model that were obtained from various directions.
LED Array LED Driver FIFO
LED Array LED Driver
32
Timing Pulse
FIFO
Photo Reflect Sensor
Cylindrical Parallax Barrier
140V
DC/DC Converter
Slip Rings
Image Data
5V
AC Servo Motor
Display Unit DC140V
Write Control
Memory Memory Bank #2 Memory Bank 11 Bank #0
and the intensity of the light rays in each direction is independently controllable. The angular intervals of the rays are chosen such that the tangents of all intervals are approximately the same. The cylindrical surface is filled by such pixels, and hence, 3D images are viewable from any angle. The refresh frequency is 30 Hz optically.
Read Control
Fig. 12. Photograph of the second prototype model.
5.1. Mechanism
Frame Memory Unit This display system is composed of three parts, the display unit, the frame memory unit, and the PC. The display unit has 32 LED ar-
PC/AT compatible
Fig. 13. Block diagram of the second prototype model.
592
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
Fig. 14. Examples of images displayed by the second prototype model.
We could see the 3D image naturally in any viewing distance. The images had strong depth cues of natural binocular disparity. When we move around the display, we saw the corresponding images in our viewing position. Therefore, we perceived the objects just as if they were floating in the air.
from various directions. This 3D image was synthesized from over 360 photographs of real person which angular inteval was less than 1 degree.
6. The third prototype model
Our proposed system is functionally similar to Jones’s one. In this section, we discuss and make a comparison between the two display systems. In simply term, Jones’s one has advantages in its simple structure and high resolution relatively, and ours has advantages in color reproduction and suitability for larger size display.
Cylindrical Parallax Barrier
SDRAM
LED Array LED Driver
LED Array
Photo Sensor
54
LED Driver
Photo Sensor
We also developed the third prototype which was capable of diplaying color and moving images. This model is shown in Fig. 15. Table 1 provides all of the specifications of this model. Image size have been enlarged to 200-mm diameter and 256mm height. The pixel numbers are 1254 horizontally and 256 vertically with a 1 mm pitch, respectively. To display color images, LED arrays of three colors are used. Each LED array has frame memory to enable playback of stored video image. The total amount of the memory is 6.9 GB, which stores dynamic imagery of approximately just under 10 s. The refresh rate is 30 Hz. The block diagram is shown in Fig. 16. Fig. 17 shows photographs of the images produced by the this model that were obtained
7. Discussion
SDRAM
Controller
Controller 5V, 3V Bus AC100V
SW Regulator
Slip Rings
Display Unit
AC Servo Motor
Bus Width Converter & Line Driver AC100V
AC200V
PC/AT compatible Fig. 15. Photograph of the third prototype model.
Fig. 16. Block diagram of the third prototype model.
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
593
Fig. 17. Examples of images displayed by the third prototype model.
Jones’s system consists of a high-speed video projector using Texas Instruments’ DMD (digital micromirror device) and an rotating anisotoropic reflective screen. Its mechanical structure is simple. Its resolution, color depth, and the number of views (or angular resolution) are just determined by the performance of DMD in resolution and refreshing speed. So far their prototype shows only B/W binary imagery of 288 views at refresh rate of 15 Hz. In the meaning of color reproduction, Perspecta, which is a commercialized swept volume display employs high-speed DMD projector similar to Jones’s, and Cossairt’s system based on Perspecta have achieved displaying color imagery by using three DMDs of red, blue, and green. However these displays still show binary tone in each RGB color. Of course, the enhancement of the number of tone level or angular resolution by using more DMDs could be considered, however it could need more contrivances that have not been developed yet. Image size is determined by the rotating screen size and it is difficult to make the screen larger because of its high-speed rotation for keeping refresh rate of 3D imagery. Enlargement of the screen faces significant problems as increasing moment of inertia, centrifugal force, and air resistance. At constant angular velocity, the centrifugal force increases in proportion to the diameter. The moment of inertia of a flat plate with constant thickness increases in proportion to the fourth power of scale. Practically the thickness of the screen and supporting structures need to increase with size for keeping its shape against increasing centrifugal force. Therefore the moment of inertia increases in proportion to between the fourth and fifth power and it causes increase of energy required to rotate. Although discussing air resistance is pretty complex problem, we can say that it increases at least in proportion to the cube of diameter because circumferential length, velocity, and height of the screen increase in proportion to diameter, respectively. Moreover it is extremely difficult to make the display whose diameter exceeds 3.6 m because the circumferential velocity reaches the velocity of sound at that diameter where the rotation speed is 30 revolutions per second. In contrast to Jones’s system whose performances are mainly determined by that of DMD, our system has more flexibility to design. For tone reproduction, analog driving can be applied because our system employs LED as light source. Since even ordinary LED responds to several MHz, sufficient number of levels of tone reproduction is achieved, independently from the number of views or angular resolution. Moreover approximately 10 bit linear tone is achieved by using PWM (Pulse Width Modulation) in the mechanical condition of 3rd prototype. In vertical direction, the number of pixels and pixel interval are simply same as LED arrays. In horizontal direction, pixel interval is determined by interval between slits on the parallax barrier and velocity ratio of both cylinders as described in Section 4. Circumferential velocity of LED arrays is determined by refresh rate of 3D display and interval of LED arrays, and interval of slits is determined by angular resolution and angular scanning range. Therefore horizontal pixel interval is determined by interval of LED arrays
and circumferential velocity of the parallax barrier. Here the point is that the pixel interval is constant and independent from the display size where the interval of LED arrays and slits are kept constant. In that case, circumferential velocities, not angular velocities, of both cylinders are constant. In case of constant circumferential velocity, centrifugal force decrease in inverse proportion to diameter. That could be a great help for large cylinders. Air resistance increases in proportion to the square of diameter simply because the surface area of the cylinder increases. The moment of inertia increases in proportion to the fourth power of diameter, however energy requirement increases in proportion to only the square of diameter because rotation speed is decrease in inverse proportion to diameter. Of course circumferential velocity never exceeds the velocity of sound because it is constant.
8. Conclusions To realize a multi-view 3D display that is observable from all directions, we proposed a technique that rotates a one-dimensional light source array inside the cylindrical parallax barrier. The first prototype model was developed based on the parallax panoramagram, which was improved to display in all horizontal directions by introducing mechanical scanning. We confirmed that the image it produced was observable from a 360-degree horizontal arc. However, the parallax panoramagram had some limitations on the image resolution and parallax number, which are caused by the diffraction effect at its apertures. To overcome these limitations, we proposed the parallax barrier movement method and constructed two prototype models. The 3D images produced by these models were observable from 360-degree with a smooth motion parallax. It is demonstrated that our proposed techniques have potential to display natural 3D imagery with freedom in viewing position. Moreover we reached the conclusion that our system especially has potential for large size display from discussion and comparison with Jones’s system.
References [1] L. Cross, The multiplex technique for cylindrical holographic stereograms, in: Proceedings of SPIE Annual Technical Conference, San Diego Seminar, 10, 1977. [2] G. Favalora, R. Dorval, D. Hall, M. Giovinco, J. Napoli, Volumetric threedimensional display system with rasterization hardware, in: SPIE Proceedings #4297, 2001, pp. 227–235. [3] Y. Kajiki, H. Yoshikawa, T. Honda, Three-dimensional display with focused light array, in: SPIE Proceedings #2652, Practical holography X, 1996, pp. 106–116. [4] K. Matsumoto, T. Honda, Research of 3-D display using the anamorphic optics, in: SPIE Proceedings #3012, 1997, pp. 199–207. [5] T. Sudo, H. Morishima, T. Osaka, N. Taniguchi, 3D display using intersection of light beams, in: SPIE Proceedings #3957, 2000, pp. 215–224. [6] A.R.L. Travis, J.R. Moore, N.A. Dodgeson, S.R. Lang, Time-multiplexed color autostereoscopic display, in: SPIE Proceedings #2653, 1996, pp. 154–166. [7] O. Cossairt, C. Moller, A.R.L. Travis, S.A. Benton, Novel view display based on DMD technology, in: SPIE Proceedings #5291, 2004, pp. 273–278. [8] N.A. Dodgson, J.R. Moore, S.R. Lang, G. Martin, P. Canepa, A 50 timemultiplexed autostereoscopic display, in: SPIE Proceedings #3597, Stereoscopic Displays and Virtual Reality Systems XI, 2000, pp. 177–183.
594
T. Yendo et al. / J. Vis. Commun. Image R. 21 (2010) 586–594
[9] H. Liao, M. Iwahara, N. Hata, T. Dohi, High-quality integral videography using a multiprojector, Optics Express 12 (6) (2004) 1067–1076. [10] Y. Takaki, A novel 3D display using an array of LCD panels, in: SPIE Proceedings #5003, 2003, pp. 1–8. [11] F. Okano, H. Hoshino, J. Arai, I. Yuyama, Real-time pickup method for a threedimensional image based on integral photography, Applied Optics 36 (7) (1997) 1598–1603. [12] Y. Takaki, T. Dairiki, 72-Directional display having VGA resolution for highappearance image generation, in: SPIE Proceedings #6055, 2006, pp. 60550X1–8. [13] H. Maeda, K. Hirose, J. Yamashita, K. Hirota, M. Hirose, All-around display for video avatar in real world, in: Proceedings of ISMAR’03 The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003, pp. 288–289. [14] R. Otsuka, T. Hoshino, Y. Horry, Transpost: a novel approach to the display and transmission of 360 degree-viewable 3D solid images, IEEE Transactions on Visualization and Computer Graphics 12 (2) (2006) 178–185. [15] K. Tanaka, S. Aoki, A method for the real-time construction of a full parallax light field, in: SPIE Proceedings #6055, 2006, pp. 605516-1–11. [16] A. Jones, I. McDowall, H. Yamada, M. Bolas, P. Debevee, Rendering for an interactive 360° light field display, ACM Transaction on Graphics 26 (3) (2007). Article 40. [17] O. Cossairt, J. Napoli, S.L. Hill, R.K. Dorval, G.E. Favolora, Occlusion-capable multiview volumetric three-dimensional display, Applied Optics 46 (3) (2007) 1244–1250. [18] T. Endo, Y. Kajiki, T. Honda, M. Sato, A cylindrical 3-D display observable from all directions, SPIE Proceedings #3957, 2000. [19] H. Ives, Motion pictures in relief, Journal of the Optical Society of America 17 (1928) 435–439. [20] T. Okoshi, Three-dimensional imaging techniques, Academic Press, 1976. [21] Y. Kajiki, H. Yoshikawa, T. Honda, Hologram-like video images by 45-view stereoscopic display, in: SPIE Proceedings #3012, pp. 154–166, 1997.
Tomohiro Yendo received the B.Eng. and M.Eng. and Ph.D. degrees from Tokyo Institute of Technology, Tokyo, Japan, in 1996, 1998, and 2001, respectively. From 1998 to 2002, he was a researcher for the Advanced 3-D Tele-Vision Project at the Telecommunications Advancement Organization (TAO) of Japan. From 2002 to 2004, he was a research fellow at Japan Science and Technology Agency (JST). He is now an Assistant Professor at Nagoya University, Nagoya, Japan. His current research interests include 3-D image display, capturing and processing.
Toshiaki Fujii received the Dr. E. degree in Electrical Engineering from the University of Tokyo in 1995. From 1995 to 2007, he was with the Graduate School of Engineering, Nagoya University. He is currently an Associate Professor in the Graduate School of Science and Engineering, Tokyo Institute of Technology. His current research interests include multi-dimensional signal processing, large-scale multi-camera systems, multi-view video coding and transmission, free-viewpoint television, and their applications for Intelligent Transport Systems. He is a member of the IEEE, The Institute of Electronics, Information and Communication Engineers, and the Institute of Image Information and Television Engineers of Japan.
Mehrdad Panahpour Tehrani received Dr.Eng. degree in Information Electronics from Nagoya University, Nagoya, Japan in 2004. From 2004 to 2007 he was with Information Technology Center, Nagoya University as a Post-Doctoral Researcher. He worked as an Associate Research Engineer with Ultra Realistic Communications Laboratory, KDDI R&D Laboratories Inc., Saitama, Japan, from 2007 to 2009. Currently, he is working as an Associate Professor at the Department of Electrical Engineering, Graduate School of Engineering, Nagoya University, Japan. His research interests are 3D image processing and communication, multiview coding, distributed source coding in camera sensor networks, and 3D media integration and communication.
Masayuki Tanimoto (M’71-SM’07) received the B.E., M.E., and Dr.E. degrees in electronic engineering from the University of Tokyo, Tokyo, Japan, in 1970, 1972, and 1976, respectively. He joined Nagoya University, Nagoya, Japan, in 1976 and started research on visual communication and communication systems. Since 1991, he has been a Professor at Graduate School of Engineering, Nagoya University. His current research interests include image coding, image processing, 3-D images, FTV and ITS. Dr. Tanimoto is the President of ITE. He was the Chairperson of Technical Group on Communication Systems of IEICE, the Chairperson of the Steering Committee of Picture Coding Symposium of Japan, IEICE Councilor, ITE Councilor and Tokai Section Chair of IEICE. He is a member of the International Steering Committee of the Picture Coding Symposium. He received the Ichimura Award, the TELECOM System Technology Award from The Telecommunications Advancement Foundation, the Niwa-Takayanagi Paper Award from ITE, IEICE Fellow Award, IEICE Achievement Award, ITE Fellow Award, and the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science, and Technology.