Autostereogram

From Scholarpedia

Author: Dr. Christopher Tyler, The Smith-Kettlewell Eye Research Institute

Dr. Christopher Tyler accepted the invitation on 5 March 2009 (self-imposed deadline: 5 September 2009).

Autostereogram refers to the single-image form of stereogram that can be free-viewed to achieve a stereoscopic depth effect without artificial aids to binocular fusion.

An autostereogram (ASG) a form of stereogram that provides a 3D depth impression from a single printed image to unlimited visual angle regardless of viewing direction. The principle of the ASG is a repetitive image in which the repeated horizontal pattern is modulated in such a way that the image can be viewed with an abnormal convergence angle to generate the visual impression of a stereoscopic depth image within the space of the pattern. When viewing with normal convergence of the eyes on the physical plane, the image appears only as a flat repeating pattern. When the eyes either converge or diverge at the distance of the pattern repeat, the small differences between adjacent pattern cycles provide binocular disparities that are interpreted by the brain as differences in depth. The disparity structure may be designed to correspond to the depth map of any desired three-dimensional scene, which is perceived when the eyes are held at the appropriate convergence angle. In order to do so, however, the brain must overcome the natural tendency of the ocular lenses to focus at the convergence distance, and enable the eyes to refocus at the plane of the image.

Figure 1: 'Furrows' (1979). One of the first random-dot autostereograms (from Tyler, 1994). Converge or diverge the eyes so as to see a triplet of three red dots, then clear the blurriness while maintaining the triplet to view a field of horizontal furrows in vivid 3D.

History

The history of the ASG falls into four phases. The first phase is the early conceptual development, which began with Sir David Brewster in 1844, who conceptualized the 'wallpaper' depth illusion and its implications for the presentation of relative (3D) depth by a trick of free-viewing of a planar (2D) image. The second phase was the hand construction of coherent depth images based on deviations from repeated pattern of the wallpaper effect. This approach seems to have been first used by Masuhiro Ito in 1970, was accurately developed by Edward Trent in 1972 and was patented by Donald Peck in 1974.

Figure 2: Geometric pattern autostereogram of a crystalline structure. (Edward Trent, 1972, Bulletin of the Stereoscopic Society.)

The third phase was the development of the algorithm for providing any arbitrary depth image into random-dot ASG by Christopher Tyler in 1979 in conjunction with programmer Maureen Clarke. This technique was published by Tyler in a visual science textbook 1983, by David Stork in 1986 and by Dan Dyckman in 1990 in the computer graphics literature. These computer-generated images used the computerized random-dot stereogram approach developed by Bela Julesz in 1960 to camouflage the structure of the depth image being represented in the ASG when viewed directly. The latest phase of the ASG development is the explicit, full-color images by Magic Eye and many other commercial companies. In many cases, the designers of these ASGs have abandoned the idea of camouflaging the 2D information and synchronized the depth image with one repeat of the 2D image, allowing its adjacent repeats to degrade non-gracefully as the depth structure modifies the relative position information.

Figure 3: Structured pattern autostereogram of a heart on a ground of roses. (Magic Eye, 1993)

In addition to the random-dot camouflaged autostereogram and the non-camouflaged structured-image autostereogram, Tyler and Clarke (1979) described four other types of free-fusion figure deriving from the same concept, although none have gained comparable recognition. Each requires computational tricks to achieve the novel conceptual structure. One is the continuous-depth autostereogram, in which the depth structure cycles around on itself to give an array of overlapping depth structures as the eyes converge on different parts of the image. Another is the wallpaper autostereogram, with a cyclic pattern that matches as the two ends to allow the infinite tiling of a plane, or wall (rather than the monocular confusion that occurs at the edges of standard autostereograms. A third is the two-way autostereogram, which contains a pair of independent cyclopean depth structures when viewed at rotations 90º apart. The final one is the autolustergram, which contains a cyclopean image depicted in dichoptic luster rather than defined binocular disparities. These latter give a ghostly, evanescent appearance rather than the smooth surfaces of the autostereograms.

Explanation of the Autostereogram

(excepted from Steven Pinker's 'How the Mind Works' with minor modifications)

Stereo vision was not discovered until 1838, by Charles Wheatstone, a physicist and inventor after whom the “Wheatstone bridge” electrical circuit is named. . . . Wheatstone proved that the mind turns trigonometry into consciousness when he designed the first fully three-dimensional picture, the stereogram. The idea is simple. Capture a scene using two of Leonardo's windows, or, more practically, two cameras, each positioned where one eye would be. Place the right picture in front of a person's right eye and the left picture in front of his left eye. If the brain assumes that the two eyes look at one three-dimensional world, with differences in the views coming from binocular parallax, it should be fooled by the pictures and combine them into a cyclopean image in which objects appear at different depths. . . . The stereoscope became the television of the nineteenth century. Victorian-era families and friends spent cozy hours taking turns to view stereo photographs of Parisian boulevards, Egyptian pyramids, or Niagara Falls. . . . These technologies all force the viewer to don or peer through some kind of apparatus. The illusionist’s dream is a stereogram that can be seen with the naked eye—an autostereogram. The principle was discovered by David Brewster, the Scottish physicist who also studied polarized light and invented the kaleidoscope and the Victorian-era stereoscope. Brewster noticed that the repeating patterns on wallpaper can leap out in depth. Adjacent copies of the pattern, say a flower, can each lure one eye into f beating on it. That can happen because identical flowers are positioned at the same places on the two retinas, so the double image looks like a single image. In fact, like a mis-buttoned shirt, a whole parade of double images can falsely mesh into a single image, except for the unpaired members at each end. The brain, seeing no double image, is prematurely satisfied that it has converged the eyes properly, and locks them into the false alignment. This leaves the eyes aimed at an imaginary point behind the wall, and the flowers seem to float in space at that distance. They also seem inflated, because the brain does its trigonometry and calculates how big the flower would have to be at that depth to project its current retinal image.

. . .

Brewster also noticed that any irregularity in the spacing of a pair of copies makes them protrude or recess from the rest. Imagine that the flowers pierced by the lines of sight in the diagram are printed a bit closer to each other. The lines of sight are brought together and cross each other closer to the eyes. The images on the retina will splay out to the temples, and the brain sees the imaginary flower as being nearer. Similarly, if the flowers had been printed a bit farther apart, the lines of sight will cross farther away, and their retinal projections will crowd toward the nose. The brain hallucinates the ghost object at a slightly greater distance. If you don’t already know how to fuse stereograms, try holding the book right up to your eyes. It is too close to focus; just let your eyes point straight ahead, seeing double. Slowly move the book away while keeping your eyes relaxed and “looking through” the book to an imaginary point beyond it. (Some people place a pane of glass or a transparency on top of the stereogram, so they can focus on the reflections of distant objects.) You should still be seeing double. The trick is to let one of the double images drift on top of the other, and then to keep them there as if they were magnets. Try to keep the images aligned. The superimposed shapes should gradually come into focus and pop in or out to different depths. As Tyler has noted, stereo vision is like love: if you’re not sure, you’re not experiencing it. Some people have better luck holding a finger a few centimetres in front of the stereogram, focusing on the finger, and then removing it while keeping the eyes converged to that depth. With this technique, the false fusion comes from the eyes crossing so that the left eye sights a boat on the right while the right eye sights a boat on the left. Don’t worry about what your mother said; your eyes will not freeze into that position forever. Whether you can fuse stereograms with your eyes crossed too much or not enough probably depends on whether you are slightly cross-eyed or wall-eyed to begin with.

. . .

The trick behind the wallpaper stereogram—identical drawings luring the eyes into mismatching their views—uncovers a fundamental problem the brain has to solve to see in stereo. Before it can measure the positions of a spot on the two retinas, the brain has to be sure that the spot on one retina came from the same mark in the world as the spot on the other retina. If the world had only one mark in it, it would be easy. . . . Add more marks, and the matching problems multiply. With three marks, there are six ghost matches; with ten marks, ninety; with a hundred marks, almost ten thousand.

This “correspondence problem” was noticed in the sixteenth century by the astronomer Johannes Kepler, who thought about how stargazing eyes match up their thousands of white dots and how an object's position in space could be determined from its multiple projections. The wallpaper stereogram works by coaxing the brain to accept a plausible but false solution to the correspondence problem. Until recently, everyone thought that the brain solved the correspondence problem in everyday scenes by first recognizing the objects in each eye and then matching up images of the same object. Lemon in left eye goes with lemon in right eye, cherries in left eye go with cherries in right eye. Stereo vision, guided by the intelligence of the whole person, could head off the mismatches by only joining up points that came from the same kind of object. A typical scene may contain millions of dots, but it will contain far fewer lemons, maybe only one. So if the brain matched whole objects, there would be fewer ways for it to go wrong. But nature did not opt for that solution. The first hint came from another of Adelbert Ames’ wacky rooms This time the indefatigable Ames built an ordinary rectangular room but glued leaves on every inch of its floor, walls, and ceiling - the Leaf Room.. When the room was viewed with one eye through a peephole, it looked like an amorphous sea of green. But when it was viewed with both eyes, it sprang into its correct three-dimensional shape. Ames had built a world that could be seen only by the mythical cyclopean eye, not by the left eye or the right eye alone. But how could the brain have matched up the two eyes’ views if it had to depend on recognizing and linking the objects in each one? The left eyes view was “leaf leaf leaf leaf leaf leaf leaf leaf.” The right eyes view was “leaf leaf leaf leaf leaf leaf leaf leaf.” The brain was faced with the hardest correspondence problem imaginable. Nonetheless it effortlessly coupled the views and conjured up a cyclopean vision. The demonstration is not airtight. What if the edges and corners of the room are not perfectly masked by the leaves? Perhaps each eye had a rough inkling of the rooms shape, and when the brain fused the two images it became more confident that the inklings were accurate. The airtight proof that the brain can solve the correspondence problem without recognizing objects came from an ingenious early use of computer graphics by the psychologist Bela Julesz. Before he fled Hungary for the United States in 1956, Julesz was a radar engineer with an interest in aerial reconnaissance. Spying from the air uses a clever trick: stereo views penetrate camouflage . . . . Since a camouflaged object, by definition, is near-invisible in a single view, we have another example of the cyclopean eye seeing what neither real eye can see. The proof had to come from perfect camouflage, and here Julesz went to the computer. For the left eyes view, he had the computer make a square covered with random dots, like television snow. Julesz then had the computer make a copy for the right eye, but with one twist: he shifted a patch of dots a bit over to the left, and inserted a new stripe of random dots into the gap at the right so the shifted patch would be perfectly camouflaged. Each picture on its own looked like pepper. But when put in the stereoscope, the patch levitated into the air.  Many authorities on stereo vision at the time refused to believe it because the correspondence problem the brain had to solve was just too hard. They suspected that Julesz had somehow left little cut marks behind in one of the pictures. But of course the computer did no such thing. Anyone who sees a random-dot stereogram is immediately convinced. All it took for Julesz' collaborator, Christopher Tyler, to invent the magic-eye autostereogram was to combine the wallpaper autostereogram with the random-dot stereogram. . . . The first commercial autostereograms used coloured squiggles and the Japanese ones use flowers, ocean waves, and, taking a leaf out of Ames’ book, leaves. Thanks to the computer, the 3D shapes that are depicted don’t have to be flat cutouts like in a diorama. By reading in the three-dimensional coordinates of the points on a surface, the computer can shift every dot by a slightly different amount to sculpt the solid shape in cyclopean space, rather than shifting the entire patch rigidly. As long as the computer has sufficient resolution, smooth, bulbous shapes materialize, looking as if they are shrink-wrapped in leaves or flowers. Why did natural selection equip us with true cyclopean vision—an ability to see shapes in stereo that neither eye can see in mono—rather than with a simpler stereo system that would match up the lemons and cherries that are seeable by each eye? Tyler points out that our ancestors really did live in Ames’ leaf room. Primates evolved in trees and had to negotiate a network of branches masked by a veil of foliage. The price of failure was a long drop to the forest floor below. Building a stereo computer into these two-eyed creatures must have been irresistible to natural selection, but it could have worked only if the disparities were calculated over thousands of bits of visual texture. Single objects that allow unambiguous matches were just too few and far between.

Autostereogram Principle

The principle of the autostereogram is at the same time simple but elusive. It is not, as many first think, just a combination of the images for the two eyes from a stereopair. Neither it is the uniform shift of the classic Julesz random-dot stereogram. The key to understanding the autostereogram is to realize that it is based on the principle that a flat plane is represented as a continuous repetition of the dots (or other pattern) horizontally across the image plane at a fixed repetition interval. To change the stereoscopic depth, therefore, the repetition interval has to be varied according to the required depth at any point. In fact, the algorithm is a direct conversion of a map of the required depth throughout the image into the instantaneous repetition interval at every point in the image. There is thus complete freedom to convert any depth map into an autostereogram, as long as its depth range does not exceed that available from the initial repetition interval.

This depth-to-change-in-repetition-interval conversion may be achieved by 'looking back' to set the coloration at any point to match that of the point one repetition interval back, plus the differential shift prescribed by the depth map at that point. (Strictly, it should be the shift prescribed by the depth map at a location half of the base repetition interval back from that point.) In this way the full extent of the autostereogram can be built up from the initial vertical strip of seed dots. The only other requirement is to avoid the repetition that arises from going to dense repetitions and then switching to sparse ones at an edge. Thus, when the current look-back exceeds the previous one, the overlap should be replaced with new random dots rather than those provided by the look-back. The resulting uncorrelated appearance is an inevitable result of the binocular viewing of vertical edges.

Historical Timeline

1792. Charles Wells describes the fusion of the images from the two eyes into a single perceptual image, perceived as being located in the “cyclopean eye” centered halfway between the two physical eyes.

1838. Charles Wheatstone discovers the full properties of binocular disparity in depth perception and invents the first stereoscope, for viewing his dual-image stereograms.

1844. David Brewster rediscovers the wallpaper autostereogram and remarks that defects in the repeats form a disparity-based depth image.

1901. The neuroanatomist Santiago Ramon y Cahal invents an early form of random-dot stereogram, as a tool to study the stereoscopic depth processing of binocular vision.

1939. Adelbert Ames constructs the Leaf Room to camouflage the monocular depth cues to, in order to study the perception of the 3D shape of the room primarily from disparity cues.

1939. Boris Kompaneysky publishes the first camouflaged random-blob dual-image stereogram (for the Russian Academy of Fine Arts).

1960. Bela Julesz develops the computerized random-dot dual-image stereogram for encoding any specified disparity profile, perfectly-camouflaged.

1962. Bela Julesz and Joan Miller develop the iterative algorithm for simultaneous encoding of two independent disparity surfaces in a dual-image stereogram. When one of the two surfaces is flat, the depth of the second surface may be seen by free-fusion on one eye’s image alone, forming a random-dot autostereogram. There is no record that Julesz was aware of this single-image capability for viewing the two-surface stereograms.

1970. Masayuki Ito uses hand-construction methods to design a fully camouflaged random-dot autostereogram of protruding flat-plane squares.

1972. Edward Trent publishes complex line autostereogram images in the Bulletin of the Stereoscopic Society.

1974. Alphonse Schilling invents the stereogram triplet for either crossed or uncrossed free-fusion and hand-paints flat-plane autostereograms.

1979. Donald Peck obtains US Patent 41235502 for the block-image autostereogram technique.

1979. Christopher Tyler and Maureen Clarke develop the direct, ‘look-back’ algorithm to convert any specified disparity profile into a random-dot autostereogram.

1980. Peter Burt and Bela Julesz use the line autostereogram method to demonstrate properties of the fusion limit.

1983. First publication of Tyler and Clarke’s autostereograms, generated on an Apple II computer and a dot-matrix printer.

1986. David Stork uses a random-dot autostereogram that he devised with Chris Rocca as the frontispiece of ‘Seeing the Light’ by David Falk, Dieter Brill and David Stork, introducing the technique to the physics and computer science communities.

1991. Tyler and Clarke publish the random-dot autostereogram algorithm, together with two-way autostereograms, depth-cycling autostereograms, and autolustergrams.

1993. Publication of 'Magic Eye: A New Way of Looking at the World' by Tom Baccei, Cheri Smith and Bob Salitsky, the first of an extended series of Magic Eye books.

1994. Publication of ‘Stereogram’, edited by Seiji Horibuchi, with a variety of high-resolution, full color and image-based autostereograms and extended analysis of their historical development by Tyler and by Itsuo Sakane.

2001. Harry Potter Magic Eye autostereogram books available in Germany and many other countries.

References

Brewster, D. (1844) On the knowledge of distance given by binocular vision. Trans. Royal Soc. Edin. 15, 663-674.

Burt, P., Julesz, B. (1980) A disparity gradient limit for binocular fusion. Science, 208, 615-617.

Dyckman, D. (1994) Hidden Dimensions: Use Your Deep Vision to Solve Mazes, Riddles, and Other Perplexing Puzzles. New York: Harmony Books.

Horibuchi, S. (1994) (Ed.), CG Stereogram. San Francisco: Cadence Books.

Julesz, B. (1960) Binocular depth perception fo computer generated patterns. Bell Syst. Tech. J. 39, 1125-1162.

Julesz, B., Miller, J.E. (1962). "Automatic stereoscopic presentation of functions of two variables". Bell System Technical Journal, 41:663–676.

Magic Eye Inc. (1993) Magic eye: A new way of looking at the world : 3D illusions. Boston: NE Thing Enterprises.

Panini Books (2001). Harry Potter, Magische Bilder (Gebundene Ausgabe).

Peck, D. (1979) Stereoscopic patterns and method of making same. US Patent 4135502.

Pepin S. (2003) Adelbert Ames and the Dartmouth Eye Institute. J. Neuro-Ophthal. 23, 290-297.

Pinker, S. (1997). The mind's eye. In Pinker S., How the Mind Works. pp. 211–233.

Ramon y Cahal, S. (1901) Recreaciones esteroescopica y binoculares. La Fotografia, 27, 41-8.

Sakane, I. (1994). In Horibuchi, S. (Ed.), CG Stereogram. San Francisco: Cadence Books (pp. 38-45).

Stork, D.G., Rocca, C. (1989) Software for generating auto-random-dot stereograms, Behavior Research Methods, Instruments, & Computers 21, 525-534.

Trent, E. (1972) Stereo designs as an art form. Bulletin of the Stereoscopic Society, 37, 7-10.

Tyler, C.W. and Clarke, M.B. (1990) The autostereogram. Stereoscopic Displays and Applications, Proc. SPIE Vol. 1258:182–196.

Tyler, C.W. (1994). Random dot stereogram 2. In Horibuchi, S. (Ed.), CG Stereogram. San Francisco: Cadence Books (pp. 29-34).

Tyler, C.W. (1994). The birth of computer stereograms for unaided stereovision. In Horibuchi, S. (Ed.), CG Stereogram. San Francisco: Cadence Books (pp. 83–89).

Wade, N. J. (2003) Destined for Distinguished Oblivion: The Scientific Vision of William Charles Wells (1757-1817). New York: Kluwer/Plenum.