Learn how to implement your participant design in Pro Lab
When you design an eye-tracking experiment, you must decide how to distribute your participants across conditions. Usually there are two options: either you assign participants to the different experimental conditions, so that each participant experiences only one condition; or you expose all participants to all conditions and measure how their behavior changes when the experiment’s circumstances have changed. In experimental design lingo, these options are called a between-subject design (independent group) and within-subject design (often called repeated measures).
Figure 1. The test participants are randomly divided into two groups. Each group is assigned to a different condition.
Figure 2. The whole group of test participants is tested in both conditions, and thus measured twice during the experiment.
Usually, you make your design choice by weighing potential spurious effects caused by repeated testing vs. losing statistical power. In between-subject designs, each participant generates a single score per dependent variable and this design relies on an effort to randomize your participant assignment to each condition. On the other hand, in a within-subject design, each participant generates multiple scores for each dependent variable and this design relies more on the effort of counterbalancing the exposure to each condition.
Design | Advantages | Disadvantages |
Within-subject |
|
|
Between-subject |
|
|
In some cases, your choice is inherently restricted to a specific participant design. For example, if you want to investigate whether the reading performance of children with ADHD differs from that of “neurotypical” children, you must use a between-subject design, as the independent variable (diagnosis of ADHD) is intrinsically related to each child. If your objective is to determine how reading performance improves with age in a group of children with ADHD, then the choice needs to be a within-subject design, as the nature of the research question specifically requires you to follow the reading performance of each child in the group.
In short, your design choice is strongly influenced by your research question, and therefore you should consider each design according to it. In general, between-subject designs are more robust to effects caused by the repeated testing but may be affected by individual variation, whereas within-subject designs are more powerful but might suffer from repeated measures confounds (such as the participant figuring out the outcome of the experiment).
Once you have settled on your choice of participant design you will continue to work on the rest of the experiment design by creating the tasks, stimuli and defining the trial structure of your experiment. Once this is completed, you are ready to start to create your experiment in Pro Lab. In this section, we will go through some examples of how to implement different participant designs and export the relevant experimental variables (Metrics export).
When implementing a design in Pro Lab it is important to plan your data analysis, more specifically, how will you export the relevant information for you to further process your data and perform your statistical analysis. This is done by:
Take the example cited above, where your aim is to investigate whether the reading performance of children with ADHD differs from that of “typical” children. Your experiment will, most likely, be composed of single or multiple reading tasks, and your stimuli will be text excerpts or individual sentences. Both groups will be exposed to the same tasks and stimuli, and the same eye tracking performance measures will be extracted during the analysis. The group identity of the participant is related to the independent variable (diagnosis of ADHD), which is intrinsic to the participant.
In this scenario, you can implement the experiment in Pro Lab in the following way:
During the analysis you will be able to export the participant variable values in a single column, that will allow you to segment the data according to the participant diagnosis (independent variable), together with the eye tracking measures and perform the necessary statistical analysis in, for example, R or SPSS (see Figure 3).
Figure 3. Shows the implementation of the experimental variables (independent and dependent variables) and the resultant Metrics export format in Tobii Pro Lab for our first experimental design scenario. The independent variable is implemented through a participant variable, and the dependent variable using the stimuli AOI and respective measures.
In the second scenario, you would like to run a study to evaluate the impact of two alternative versions of an ad on a consumer viewing behavior. Particularly we want to know if that element distracts the viewers from reading a text element that contains the message. In this situation, the two ads differ only on the design of one element, so it is likely that you will want to avoid any viewing behavior carry over due to repeated exposure to the two ad versions. Consequently, you decide to expose each version to different participant groups. Your task design will most likely be a free viewing task with several unrelated filler ads plus the target ads. Your eye tracking measures will only be extracted from the viewing behavior on the target ads. Now let’s see how you can implement this design in Pro Lab, remember that you want to expose the two versions of the add to two groups of similar characteristics participants:
During the analysis, you can use the timeline information (your independent variable), together with your AOI based eye tracking measures to perform the necessary statistical analysis in, for example, R or SPSS.
Figure 4. Shows the implementation of the experimental variables (independent and dependent variables) and the resultant Metrics export format in Tobii Pro Lab for our second experimental design scenario. The independent variable is implemented through the timeline name or id, and the dependent variable using the stimuli AOI and respective measures.
Like in the between-subject design scenarios we will discuss how you will export the information regarding your independent variable and the measures of your dependent variable.
Let’s imagine that you want to measure the progress of a group of children with an ADHD diagnosis that are attending a 1-year long special education program. You have decided to evaluate their performance at the start, middle and end of the program (month 0, month 6 and month 12 respectively). Due to the nature of the research question, your participant design will be a within-subject design. Like in the previous example, your experiment will, most likely, be composed of single or multiple reading tasks, and your stimuli will be text excerpts. Let’s assume that due to the long time gap between the three tests, you do not expect any strong memory or practice effects on the performance. As a result, you will keep the test design constant across the three periods (including re-using the same texts) and extract the same eye tracking measures. In this scenario, your independent variable is the stage in their education operationalized in time units – month 0, month 6 and month 12. Let’s implement this design in Pro Lab:
During the analysis you will be able to export the participant variables, that will allow you to segment the data according to the 3 stages in the education program (independent variable), together with the eye tracking measures and perform the necessary statistical analysis in, for example, R or SPSS.
*Note: this step is important to be able to assign the recording to the participant and your independent variable - the test period.
Figure 5. Shows the implementation of the experimental variables (independent and dependent variables) and the resultant Metrics export format in Tobii Pro Lab for our third experimental design scenario. The independent variable is implemented by setting and individual name to each recording and tagging the recording with two participant variables. The dependent variable using the stimuli AOI and respective measures.
Now imagine that like in the previous scenario you are worried about practice and repeated exposure effects, and you decide to show a different text each time you re-sample a participant. Your participant design is still a within-subject design as each participant will contribute to the three independent variable scores, however, now you will have to counterbalance the presentation of the different texts, by dividing the participants into subgroups, where one group is exposed to the different stimulus in the following order "text A" at Month 0, "text B" at Month6 and "text C" at Month12. Another group "text B" at Month 0, "text A" at Month6 and "text C" at Month12". text and so on. Thus to be able to represent all these factors in your data export when you move your experiment design to Pro Lab you will need to use a different approach than the previous example, instead, you will:
During the analysis we can then export the time of interest related to the different texts, allowing you to segment the data by the three timelines, i.e. by text (control), and participant variable, i.e. 3 stages in the education program (independent variable), together with our eye tracking measures and perform the necessary statistical analysis in, for example, R or SPSS.