The figure below illustrates the three basic steps involved in the perception of a stimulus on a computer screen. The inset graphs show the time course over a 100 ms period. The first step is writing the stimulus to the video memory. A display command in the source code of an experiment instructs the program to write information to the memory on the video controller card. The second step is actually putting the stimulus on the screen, which occurs in scanning or “refreshing” the screen. This cycle will be explained in detail below, but note now that it is the limiting factor in how fast a stimulus can be put on the screen. The third step is activating the participant’s retina. Let’s consider each of these steps in turn.
Step 1 - Writing to Video Memory
When a display command is executed, it writes data into a video controller card. This card has a certain amount of memory that can be thought of as a direct representation of the computer monitor (i.e., each memory location corresponds to a screen location). While the technical details of writing to video memory are not essential to this discussion, it is important to remember that this is a distinct event in the process. In the case of one or a few letters, the step is nearly instantaneous – on the order of tens of microseconds (0.00001 seconds).
If the stimulus involves extensive graphics (e.g., filling a circle), the process of writing to video memory can take a significant amount of time (milliseconds). The amount of time necessary for this step is dependent not only on the nature of the stimulus, but also on the computer and the display card.
Step 2 - Scanning the Screen
The second step is the physical activation of the stimulus on the screen. Older, CRT type computer monitors were based on raster technology, which works in the following fashion: The monitor has three raster guns, which emit a beam of electrons to activate red, green, and blue dots of color. The position of this beam can be adjusted to accommodate several different pixel resolutions. A pixel is the smallest unit of drawing on the monitor, and it is defined by the pixel resolution of the current graphics mode. For instance, if you are using the typical VGA graphics mode, the pixel resolution is 640x480, which means that there are 640 pixels per row and 480 pixels per column. SVGA is 800x600. The raster guns are deflected to draw each pixel on the screen sequentially, from left to right, top to bottom (see Figure 1, lines in Monitor), and the guns can only draw one set of pixels (red, green, blue) at a time. The raster guns shoot electrons at the screen, or front of the monitor. The electrons activate the phosphors to emit light. As soon as the guns are moved to the next pixel, the phosphors in the previous pixel begin to dim, or decay (see Figure 2 below). The decay period can vary from one monitor to the next, but for most color monitors, the intensity of the pixel will be 10% of its maximum within 2-5 ms. The video display card sequentially scans each memory location in order, turning on the gun’s intensity based on the contents of the video memory set during Step 1.
The raster guns are continuously scanning the screen at a constant rate. This means that any given pixel on the screen is drawn with some constant frequency, known as the vertical refresh frequency. The vertical refresh frequency is the number of times per second that the entire screen will be drawn. The inverse of this measure, the refresh rate, is the amount of time necessary to draw one full screen. So, if a monitor has a 70Hz vertical refresh frequency, the refresh rate of that monitor is 1/70 of a second (= 14.2857 ms). Basically, once the scanning begins, it sweeps at 70Hz, independent of any writing to video memory operations.
Associated with the refresh rate is the vertical retrace, or top of screen event. This event occurs whenever the raster guns are moving from the bottom to the top of the screen to begin a new scan of the monitor. This event can be detected by the computer and can be used to synchronize the drawing of stimuli. By sensing the top of screen and only writing to video memory in a short period of time, the display will always occur within a fixed period of time after the top of screen event. For example, if text is displayed at the center of the screen and can be written in less time than half the refresh period (7 ms), the data will be in video memory before the display locations are scanned for the next cycle (7.14 ms after the top of screen event).
The effect of Liquid Crystal Display (LCD) monitors is basically identical to CRT monitors. Current active matrix LCD monitors work by having an electrical signal change the angle of the crystal of the LCD for each colored dot on the display. In contrast to a CRT display, where each dot is activated by an exponential function (see Figure 1 step 2), on an LCD the activation is a square wave signal (on for a given intensity during the time of the refresh). The LCD still has a refresh rate, which is the rate at which each dot is updated. Similar to the CRT, on the LCD the middle dot would be updated 7.14 ms after the first dot of the screen (on a 70Hz monitor). The effect on the eye of a LCD and CRT display are indistinguishable. See TIMING: E-Prime 2.0 Monitor Recommendations and Timing Information  for our latest test results.
Step 3 - Activating the Retina
The last step in the perception of stimuli on a computer monitor is the retinal response to the light emitted from the screen. The chemistry and neural circuitry of the retina are such that the visual system will integrate visual input over time. This is the reason that we cannot detect the raster guns scanning the monitor. The retina will average the input and produce the perception of a continuous display. For example, a stimulus which is to last for one refresh of the monitor will produce the following results: The pixels on the screen which make up the stimulus will be activated by the raster guns once, and then begin to decay. This decay process will take approximately 5 ms on a color display (see Figure 1, graph step 2) before the intensity is below perceptible levels. On LCD, the dot will be on for the duration of the refresh.
However, the eye integrates that impulse for the next 80 ms (see Figure 1, graph step 3). As an analogy, think of what occurs when you see a photographer’s electronic flash go off. Typically, a very brief flash (microseconds) is presented and is perceived as a short (e.g., 100 ms) flash. If you flash the electronic flash 70 times a second, the retina would perceive the light as being continuous. During a period of about 80 ms, the eye is integrating the energy (e.g., a stimulus on for 5 ms at 10 Lux is seen as intense as a stimulus on for 10 ms at 5 Lux).
Example Stimulus Activation
Figure 2 provides an illustration1 of the time course of the raster display on each scan, and the visual integration of the raster display. We want to display a ‘+’ (text mode) in the center of the screen. Assume the monitor has a vertical retrace rate of 14.3 ms. We will examine the sequence of events, considering only the center pixel of the plus symbol. Execution of the display command occurs at time t = 0. After about ten microseconds, the ‘+’ is written into the video memory. At some point during the next 0 to 14.3 ms, the raster guns will be aimed at the center pixel on the screen (see Raster Display peaks in Figure 2). With the stimulus in the center of the screen, the pixels for the ‘+’ are displayed after 7.14 ms. As soon as the pixel has been activated, it will be refreshed every 14.3 ms. The retinal response begins as soon as the pixel is activated for the first time, but the response is delayed in the sense that it must build up over the next 80 ms to reach a steady state (see Raster Based Visual Activation Figure 2). At 200 ms the ‘+’ is overwritten in video memory. When the raster guns reach that point (at 207 ms) where the ‘+’ was, the pixel is not refreshed. The retinal activation begins to decay as soon as the pixel is no longer visible during the last refresh, and the visual activation slowly drops off over the next 80 ms.
There are four important aspects to note about the timing in the attached graph. First, there is a variable delay of up to one refresh period from the time that the stimulus is written to the video memory and the time that the center pixel is initially activated on the screen. The second feature is the exponential decay of the light intensity of the center pixel between refreshes. Third, the retinal response is slow and smooths the bumps in the pixel activation. Fourth, the decay of retinal activation is delayed, not instantaneous. If the stimulus is removed from video memory just after the pixel has been refreshed, the pixel will decay for another few milliseconds, and the retinal activation will persist for substantially longer unless it is masked by another stimulus.
NOTE: Figure 2 is based on simulation and empirical studies reported in Busey, T. A., & Loftus, G. R., 1994, Sensory and cognitive components of visual information acquisition, Psychology Review, 10, 446-469, assuming a decay constant of 20 ms.
Figure 2 also shows a comparison of presenting material with a slide shutter projector or tachistoscope relative to a computer monitor. The lines labeled Slide Display and Slide Based Visual Activation show the activity that would be produced if the total display intensity were continuous as opposed to pulsating (i.e., like the refreshes of a computer monitor). The total visual energy of the Slide and Raster activation have been matched. Note the similarity of the activation functions*. For the participant, there is no perceptible difference. For a detailed discussion of these issues, see Busey & Loftus, Psychology Review, 1994, 101, 446-469.
*In the simulation the slide display was delayed by 2ms and the raster by 7ms. This produces functions that have matching increases in activation within a few percent from 50ms until the termination of the display. These differences are insignificant in terms of human perception (see Busey and Loftus 1994). In Figure 2, the decay of the activation for the Slide Display was somewhat slower than the Raster Display due to the last refresh of the Raster Display not being displayed as a result of clearing video memory 200ms after the write to video memory but rather than 200ms after the first display of the video.
Previous Article: GETTING STARTED: Considerations in Computerized Research