Fixed or randomised trial order? A case study of pedestrian interaction with an automated vehicle

Alam, M. S., Martens, M. H., Bazilinskyy, P.

In preparation.

ABSTRACT In a virtual reality study of pedestrian-automated vehicle interaction, the same trial order was used by mistake for every participant. A second study presented the same 40 conditions to a new group, randomising the order for each participant. We compared the two groups of 50 participants to establish which conclusions a fixed sequence alters. Participants stood at the kerb near a virtual pedestrian, held a controller trigger whenever they felt it was unsafe to cross and rated the influence of the other pedestrian and of the distance to them. The effects of vehicle yielding, the external human-machine interface (eHMI) and pedestrian position on perceived crossing safety were the same in both groups; none of 12 group differences was significant after adjustment. Overall, perceived crossing safety did not differ significantly (difference 2.34 percentage points, 95% CI $[-5.16,9.83]$). Of 600 condition-by-condition comparisons, one differed after adjustment: in the condition that every fixed order participant met first, the fixed order group rated the influence of distance 23.34 points lower (adjusted p=.011). When the conditions were ignored, judged unsafety rose by 21.76 percentage points from the first to the last ten trials in the fixed order group, against 2.88 in the randomised order group, because the sequence ended with non-yielding vehicles. As the groups were tested at different times and not randomly assigned, differences cannot be attributed to trial order alone. A fixed sequence can still support conclusions about average factor effects, but not about single early conditions or change over the session.