TIMING: E-Prime Timing Test Results [27520]

Windows desktop computers can support millisecond precision timing if they are reasonably configured. It is prudent to run the timing test programs whenever new hardware or applications are added to the computer. The timing test results in this section show that E-Prime can maintain millisecond precision on appropriately tuned desktop PC hardware for Pentium class machines running at 2GHz or faster. Experiments can take precision real-time input from the keyboard device, the PST Serial Response Box, or Chronos, but we do not recommend that the mouse be used as an input device for experiments in which response time at the millisecond level is the primary dependent measure. Using a quality name brand PCI audio card, E-Prime can achieve consistent low latency playback of digital audio (e.g., latency < 1 screen refresh). For experiments that require optimal and consistent playback latency, the recommended sound API varies with operating system. Please refer to the timing test summary data at the end of this section for a detailed review of the tests performed and the resulting data.
 
Applications frequently install either new background jobs that take processor cycles or new device drivers that may alter timing precision. Some hardware cards and software programs compromise the machine’s ability to provide millisecond precision. The testing of the computer should occur when the computer is initially setup or when new hardware or software is installed, but it need not be done for every experiment. The rest of this section details how the testing should be performed. If the system has already passed the performance tests described and someone else is responsible for machine configuration testing, then this section may be skimmed, although the figures should be reviewed. Once the machine passes the performance tests, it is ready to serve as a data collection station for E-Prime.
 
People tend to install multiple background applications that may, at intervals, wake up and steal cycles from other applications (e.g., the experiment), resulting in timing errors. For example, the Windows Office toolbar periodically checks the state of the machine to see if some action needs to be performed (e.g., put up a reminder alarm on the screen). These types of programs use small amounts of time, but can block or delay other programs from executing and therefore can result in timing errors. Having multiple applications running encourages the operating system to swap out parts of the experiment (see TIMING: Timing Challenges [22847]). A common problem is virus detection programs that scan all files and periodically scan the system for viruses. There is an internal timer in the virus monitoring applications that will, on some cycle, wake up and scan the system, halting activity for potentially hundreds of milliseconds. To avoid random time delays during the experiment, scanning must be disabled while running the experiment. E-Prime runs in high priority mode and will block most, but not all such background jobs.
 
A good experimental computer should have a minimum number of concurrent applications running and no other applications loaded during a data collection run. Ideally, no other applications other than E-Run should be displayed on the Windows taskbar when real data is being collected. There should also be a minimum number of background applications and utilities running (this is typically determined by what programs are on the Start menu configuration for the computer but can also be affected by various non-obvious entries located in the Windows System Registry). Pressing Ctrl+Alt+Del and choosing TaskManager, will display a list of all programs currently running on a computer. Figure 1 is a display of a completely clean computer. This is the ideal scenario and may not be possible to achieve on all hardware setups because of various drivers that may be required for proper operation of the computer hardware. The Windows Explorer application will always be listed, as it is responsible for running the Windows desktop. If other applications or utilities are listed due to the configuration of the machine, be aware of the tasks each program performs. It is important to know if these programs are indeed necessary while running an experiment, since the programs may be taking processor time.
 

Figure 1. Ideal listing of operating programs before running an experiment. Note this is the ideal minimum. Other programs will be displayed depending on the configuration of the machine.
 
Clock Test
The E-Prime Clock Test monitors the clock for a period of 10000 ms and checks to see if continuous readings of the clock ever failed to identify sequential clock ticks. E-Prime cannot completely stop the operating system from suspending an executing experiment. However, E-Prime can determine if the reading of the clock was halted long enough to skip a clock tick (e.g., if the values 2001, 2002, 2005, 2006 were returned on 4 successive clock reads, E-Prime sees the 3 ms tick between 2002 and 2005), indicating that the operating system would, in a similar situation, have delayed the experiment. E-Prime uses a microsecond precision clock for all internal time assessment. If E-Prime is unable to read the crystal clock when it needs to (e.g., to timestamp the onset of a stimulus), the precision of the timing is compromised (see TIMING: Timing Challenges [22847]).
 
Having a well-configured machine is critical for achieving accurate timing during the run of an experiment. Figure 2 illustrates the importance of the configuration of the hardware and how dramatically it can impact millisecond precision.

Figure 2. Missed ticks test of 10,000 ticks on a computer with and without the network connected.
 

The test was run on a Pentium 1.8 GHz computer with a network card. Although this computer is capable of delivering millisecond accuracy, it cannot do so with the installed network card and version of the network software. Better network cards and drivers on other machines can provide millisecond precision while supporting network activity. One of the reasons for doing long duration tests (e.g., 6 hours) is to see if a system is sensitive to the behavior of other devices on the network.
 
Figures 3 displays the timing error pattern during the six-hour time test. A perfect run would be a straight line, indicating that there was no loss of ticks during the run. Three computers were tested. Of the three computers, the fastest two (3.4 and 2.9 GHz respectively, both mutli-threaded) showed 0 missed ticks. The timing test run on the third machine (1.8 GHz, single core, running XP) shows a repeating pattern of missed ticks, settling around 10% after a larger rate on the first 2 runs ranging from 2% to 4% (Figure 3).
 

Figure 3. A time loss pattern.
 
The impact on the data of missed ticks is determined by the percentage of missed ticks. We recommend that the percentage of missed ticks be less than 0.1%. Note, a high rate of missed ticks early in the run is likely caused by the operating system stabilizing the running of the program.
 
The clock bin count test (Figure 4) gives a distribution of the delay times in table and graph form. It shows how many counts occurred for each time interval. This can be used as a diagnostic to suggest what programs are taking cycles. Below is the distribution of tick sizes for a HP laptop. There were a significant number of ticks in the first bin. Potentially, a program like the power conservation program is producing interrupts.
 

Figure 4. Distribution of tick times from a laptop computer, showing frequent interrupts.
 
Configuring computers is a complex process and may require the assistance of a technician who is experienced in installing and removing software applications and/or hardware. The main strategy is to run a test. If timing is a problem, remove programs, and possibly hardware, until the system can provide the precision needed for professional research. E-Prime provides the tools to assist the researcher in detecting configuration problems that disrupt timing prior to the implementation of experiments.

E-Prime 3.0 Timing Tests
See TIMING: E-Prime 3.0 Timing Data [24332] for E-Prime timing test results using the Black Box Toolkit.
 
Previous E-Prime Version's Timing Tests
The following articles contain information on E-Prime 1.0 and E-Prime 2.0 timing tests:
TIMING: E-Prime 1.2 Timing Data [19580]
TIMING: E-Prime 2.0 Timing Data [19579]
 
Previous Article: TIMING: Time Audit: Confirming Accurate Timing [22864]