INFO: How to configure annotations to an IDF file and use these annotations when analyzing eye tracking data in BeGaze [22441]

This article applies to:
E-Prime Extensions for SMI 1.0

Detail

Introduction
E-Prime Extensions for SMI (EESMI) supports the creation of IDF files. These files contain eye tracking data from the SMI eye tracker along with optional experiment information from E-Prime. After the experiment session is complete, the IDF file can be loaded into SMI's BeGaze data analytics software package, which is part of the SMI Experiment Suite. Using IDF files to record eye tracking data has several advantages:

• Data can be saved to the IDF file without the need to write E-Basic script
• The IDF file can be used to easily conduct qualitative visual data analysis
• Raw data and event statistics can be exported from BeGaze for analysis in other third party applications

In order to take advantage of these BeGaze features the data that is sent to the IDF file must include some type of annotation, or marker, in order to assign portions of the data to different stimuli or trials. This article illustrates how to include annotations into an IDF file and further how to utilize these annotations within BeGaze to construct a rich set of analyses on the gaze data.
 

Terminology
The following terminology is used throughout this article:

• Stimulus:  In BeGaze, a Stimulus refers to a period of recording that can be visually analyzed.  In the simplest case, a unique static picture is shown during the period.  However, a stimulus may also refer to multiple recording periods that have slightly different pictures but the same visual structure, or which share some type of coherent animation. After importing an IDF file, BeGaze displays all of the stimuli that are available for analysis on the Dashboard.  An example is shown below:

• Trial:  In BeGaze, a Trial refers to one instance of stimulus recording, either by the same or multiple participants. Thus, a stimulus can “have” multiple trials from one or more participants. Trials are shown in BeGaze’s Participants view, such as is shown when conducting a Scan Path analysis:

 
It is important to understand that these definitions of Stimulus and Trial do not necessarily map to the definitions typically used by an experimenter to describe the experiment structure.  From an experimenter’s view, multiple trials for one (BeGaze) stimulus can differ in such a way, including the nature of the visual display, that the trials should be analyzed separately within BeGaze.  This could occur, for example, when certain spatial properties of the display are counterbalanced. 
 

Image Messages and User Events are both types of Messages
Annotations to the IDF file are supported via the sending of Messages. There are two types of Messages that can be viewed within BeGaze. The first type, image messages, trigger the creation of a Stimulus and a Trial. This process is examined in detail in the remainder of this document.  The second type of message, user events, are used to mark events that occur during the recording of one specific stimulus. Technically, image messages are also user events, and therefore image messages appear along with user events in the User Event View. Image messages are associated at the 0 ms mark, while user events are time-stamped where they occur relative to the start of the trial. The User Event View is shown in the image above.  The first entry, at time 0 ms, corresponds to an image event.  The second entry, at time 2503, is a user event, such as a participant’s response. Both image messages and user events are referred to as messages within this document. 
 
IDF data recording
The following EESMI PackageCalls are required in order to generate an IDF file:

• A connection is established with the eye tracker via the SMI Open PackageCall.  
• Data recording is initiated via the SMI StartRecording PackageCall.

These PackageCalls are typically placed on the Session level Procedure.  
 
The actual data recording occurs between the StartTracking and StopTracking PackageCalls. These PackageCalls are typically placed on the Trial level Procedure.  The final step is to call the StopRecording PackageCall; this call creates the .idf file.  See Tutorial 1 in the E-Prime Extensions for SMI User Manual for details.  
 
Sending Image Messages from E-Prime to the IDF file
E-Prime provides two different methods for specifying the message to be sent to the IDF File:
 
     1. Set the vEtMessage parameter in the StartTracking PackageCall (or the corresponding StartTracking E-Basic subroutine)
     2. Set the SendImageMessage parameter for the SMI TaskEvent
 
With either method, an image message will be sent if the message ends with a graphic file extension, including “.jpg”, “.png”, or “.bmp” or with the video extension “.avi”. Conversely, a user message will be sent if the message does not end with a graphic file or video suffix. The TaskEvent method is preferred when the message is designed to indicate a new stimulus or some type of user event because the TaskEvent can be triggered very precisely. For example, a message can be sent when a new stimulus is displayed on the screen by using the stimulus object’s OnsetTime event.
 
However, regardless of whether your experiment design requires the use of TaskEvents to send user messages to annotate user events or not, it is good practice to also send an image message with every StartTracking PackageCall. Sending image messages with the StartTracking call enables the eye tracking recording to be associated with the correct (BeGaze) stimulus. The examples shown below illustrate how the image messages affect how BeGaze aligns the eye tracking data with the stimulus.  Both examples show a timeline with the same sequence of E-Prime actions:  a TaskEvent is followed by 2 PackageCalls (StopTracking, StartTracking), which are followed by another TaskEvent. The figure on the left illustrates how the stimulus is aligned when the StartTracking PackageCall specifies a user event, while the figure on the right illustrates how the stimulus is aligned when the StartTracking PackageCall specifies an image message. The eye tracking data that was recorded between the StartTracking PackageCall and the second TaskEvent in the sequence on the left is incorrectly associated with the first stimulus (Stimulus A); the corresponding data is correctly associated with the second stimulus (Stimulus B) in the sequence on the right.   If each Trial level Procedure in E-Prime begins with a StartTracking PackageCall and ends with a StopTracking PackageCall, then the association shown on the right is the expected outcome and will be achieved by sending image messages with each StartTracking PackageCall. 
 

 
Trial and Stimulus Construction in BeGaze
As demonstrated above, BeGaze creates stimuli and trials only when an image message has been sent to the IDF file.  A new trial is created whenever an image message has been sent.  The file that is identified in the image message may or may not exist; stated another way, image messages can be sent and will be represented in the IDF file regardless of whether the named image file is an actual file. However, within BeGaze, in order for separate stimuli to be created, different image messages have to be associated with a specific picture file, which can be in *.bmp, *.jpg or *.png format. Associating the image messages with particular graphics fies can be done manually with the “Manual Experiment Creation…” or automatically with “New Experiment from Folder” in the BeGaze file menu.
 

 
If done automatically, the name of the image file that is identified in the SendImageMessage call must match the corresponding file names.  Images and IDF files must be located inside the folder that is imported by BeGaze. Identical image messages within different IDF files are matched when importing the experiment data, thus creating multiple trials for the corresponding stimuli with different participants. Also, if the same image message was used more than once during data recording of one participant, multiple trials for the same participant are created. A sketch for an automatic BeGaze import example might look like this:
 

 
NOTE: It is important that images have the same size as the screen resolution during recording. It is advisable to use the Canvas.SaveImage command in InLine script to capture stimulus.