Facebook or Fakebook?: The effect of simulation on location privacy user studies
Iain Parris, Fehmi Ben Abdesslem, Tristan Henderson School of Computer Science, University of St Andrews, St Andrews KY16 9SX, UK
1. INTRODUCTION There is a well-known divide between users' actual pri-
vacy behaviours and their responses to survey questions . In an attempt to reduce this divide, monitoring and ques- tioning users in situ, during day-to-day life - using tools from psychology such as the Experience Sampling Method (ESM)  - is a popular research method. The results are data concerning real behaviour in real systems.
But what if the real system does not yet exist, and is non- trivial to build? Some researchers instead simulate systems: ask experimental participants ESM questions in situ about how they would use such a system if it did exist.
Our research question is: for location privacy user stud- ies, does simulating a system change the behaviour of par- ticipants compared to using a real system? Do real social consequences change the study results?
2. METHODOLOGY To study location-sharing behaviour in the online social
network Facebook, we recruited 40 undergraduate students (17 male, 23 female) who were active users of Facebook. Each participant was asked to carry a Nokia N95 smartphone for one week and offered £50 for participation. Due to hard- ware constraints, we ran the experiment over two different weeks with two runs of 20 participants.
At the start of the experiment, participants were asked to group their Facebook "friends" into lists, to which they would like to be able to disclose similar amounts of informa- tion. In addition we added two lists for "everyone" (publicly- viewable information) and "all friends" (all of a user's Face- book friends). The phones sensed participants' locations every five minutes and uploaded these to a central server. Based on these sensed locations, participants were sent ESM questions asking whether, and with which lists of friends, they would be willing to share their locations and photos of their current activities. Their responses were collected and uploaded using the phones.
Participants were randomly divided at the start of the ex- periment into two groups of 20 participants: the real group experienced real publishing of their location information to their chosen friend lists, while the simulation group experi- enced simulated publishing, where information was not dis-
Figure 1: Question response rate. The response rates are similar for the simulated and the real groups. (Median: 43% for simulation group; 42% for real group.)
closed to any friends, regardless of user preferences. To control for differences between runs, half of the partic-
ipants in each run were assigned to the simulation group and half to the real group. When reporting results, we combine results from both runs. Participants...