The coursework for Usable Security and Privacy (2022-2023) comprises two parts and contributes 17% to your overall grade. The deadline for submission is 24 March at 12:00pm.
PART A: EVALUATE AND RE-DESIGN A SECURITY OR PRIVACY TOOL
In this part of the coursework, you will select a security or privacy tool, identify its primary use case, security requirements, and likely causes of user error, and then re-design several of its screens to reduce the chances of user error. You can work with other USec students to complete steps 1-3 (below) of Part A. However, each student should submit an individual report.
Step 1: Pick a tool
Select one of the following tools to evaluate. The list of tools is restricted to ensure that the selected tool is likely to have sufficiently interesting features to do the coursework. If you would like to evaluate a different tool, please email the Lecturer or the TA.
· NoScript
· UBlockOrigin
· PrivacyBadger
· Mailinator
· Tor Browser
· Off the Record Messaging
· Helios e-voting system
· YubiKey (only select if you already own one)
· ProtonMail
· Any email encryption program (Enigmail, Mailvelope, etc.)
· Any password manager (LastPass, OnePassword, KeePass, Chrome’s default manager, etc.)
Step 2: Determine the correct usage of the tool and any unrecoverable or invisible errors
Start by thinking about who is supposed to use your tool. Who are they? What knowledge do they possess? Next, write down one or two high-level tasks that your tool is supposed to support and that you want to focus on. For most tools, this will be obvious, but for those that have multiple features, writing down the tasks will help you think about what the tool is really supposed to do. For example, a cookie manager might perform the high-level tasks of 1) easily deleting unwanted cookies without deleting wanted cookies (i.e. opt-out and Google login), 2) letting the user view and edit the content of cookies, and 3) showing which companies are tracking the user. It is okay to modify these tasks as you work through the rest of your coursework.
Conduct research on your tool and determine the correct way to use it to obtain the main security or privacy benefits for the selected high-level tasks. We recommend that you look at the documentation of the tool and any tutorials or guidance available online. In particular, you need to find any steps or actions that a user must take, or they risk not obtaining the security or privacy benefits of the tool. You should also look for any potentially unrecoverable or invisible errors that a user can make.
For example, if a user opts out of BlueKai tracking because they do not like the company’s privacy policy, they delete their cookies to be sure and then visit a page that uses BlueKai as a tracker. They will be invisibly tracked by BlueKai while also believing that they are safe. An unrecoverable error might be that, if the user tries to send an email with private data to someone outside their company, the company security email plugin warns them about potential data leakage, but they do not understand the warning and decide to send the email anyway, causing a data breach. The first error is invisible with the user never learning it and the second is unrecoverable in that they cannot undo the action. The invisible or unrecoverable nature of these types of errors potentially makes them very serious, depending on the risk concerns of the user.
Step 3: Use a framework to identify and describe potential errors
First, try interacting with your tool as if you were a new user trying to accomplish your identified tasks, noting any parts of the interaction that seem confusing or might confuse others. In particular, any point in the interaction where a user has to be careful to avoid making an invisible or unrecoverable error. Then, go back and explore action options that you did not try the first time around doing the same.
Write up a report of the more serious potential errors you identified. To explain the sources of potential errors, use either the NEAT/SPRUCE framework or the Framework for Reasoning about the Human in the Loop. You only need to use one of these two frameworks. We have attached the two frameworks below for you to view and read.
Step 4: Re-design an interaction
Take screenshots of at least two of the problematic interactions and redesign them using your favorite image editor. In your design, ensure that you avoid previous mistakes by helping potential users follow the correct interaction naturally. Ensure that your design does not produce new errors by following the NEAT\SPRUCE or Human in the Loop Frameworks to describe your new design.
· Name the tool you selected.
· The high-level tasks you selected (a few sentences).
· 1-2 paragraphs on what a correct interaction with your tool looks like. (step 2)
· 1-2 paragraphs on the most critical potential errors users might make while interacting with the tool using one of the frameworks.
· The new interactions design screenshots and the discussion on how they will solve the current problem while keeping the interface in line with one of the frameworks.
· Any students you discussed steps 1-3 with?
Frameworks
· Reeder, Rob, Ellen Cram Kowalczyk, and Adam Shostack. “Poster: Helping engineers design NEAT security warnings.” SOUPS, 2011
· Cranor, Lorrie F. “A framework for reasoning about the human in the loop.” (2008).
PART B: PROVIDE AN ANALYSIS AND RECOMMENDATIONS FOR COOKIE OPT-OUT PAGES
In this part of the coursework, you will write an analysis interpreting the outcomes of a survey conducted to explore the concepts of cookies and behavioral advertising.
In the Usable Security and Privacy (2019-2020) course, the staff wrote and ran a survey, which we will be referring to as USec2019. The survey consisted of demographic questions, scale questions, questions from prior research, and additional questions formulated by the staff. To build the survey, they used the results from two prior studies: Turow2009 and McDonald2010. Turow et al. studied public perceptions of behavioral advertising using a structured telephone interview with a representative subset of the US population. A year later, McDonald et al. conducted a more in-depth online survey using Mechanical Turk, which also comprises mostly the USA population.
In the 10 years between Turow2009 and USec2019, quite a few things changed. There has been a large public discussion on the use of data by large companies, such as Facebook. Multiple public education campaigns have been conducted, such as the Cyber Street Wise by the UK government, as well as the creation of multiple privacy-protecting tools by groups such as the Electronic Frontier Foundation. The number of people who have grown up using the Internet and Social Media from a young age has increased drastically. An 18-year-old today has much more experience with technology than someone of comparable age in 2010. In this part of the coursework you will be making three tables which are described below, and then analyzing those tables in the context of McDonald2010. Your overall research question is:
In what ways have users’ cookie knowledge and opinions about behavioral advertising changed between 2010 and 2019?
Create the following three tables and include them in your report
Demographics – Create a table summarizing the age and gender demographic information from all three surveys. If it helps with the analysis, other demographic information may also be included.
Attitudes – All three surveys asked the same question about whether users were interested in seeing ads, news, or discounts targeted at their interests. Create a table comparing the results of these questions from the three surveys.
Knowledge – Knowledge – McDonald2010 section 4.2, Table 3 describes a set of questions used to test users’ knowledge of cookies. The same set of questions appears in the USec2019 survey. Extend McDonald2010’s Table 3 with data from the USec2019 survey.
NOTE: The original survey data of Turow2019 and McDonald2010 are not available. Students are required to refer to the respective papers for the tables needed for the analysis.
Write an analysis:
Write an analysis approximately one page long. Your analysis should focus on interpreting the USec2019 results in the context of the McDonald2010 paper. The focus of the write-up should be on how people’s attitudes and knowledge have changed since 2010. You should also include a discussion of generalizability and the ways in which these surveys can and cannot be compared.
This is a critical-thinking question. There is no single right or wrong way to analyze the data. The focus of this question is your ability to interpret the information in tables.
References:
[McDonald2010] McDonald, Aleecia, and Lorrie Faith Cranor. “Beliefs and behaviors: Internet users’ understanding of behavioral advertising.” Tprc, 2010. Available at “https://www.researchgate.net/publication/228237033_Beliefs_and_Behaviors_Internet_Users%27_Understanding_of_Behavioral_Advertising”
[Turow2009] Turow, Joseph, et al. “Americans reject tailored advertising and three activities that enable it.” Available at SSRN 1478214 “https://repository.upenn.edu/cgi/viewcontent.cgi?article=1138&context=asc_papers.” (2009)
[USec2019] The survey conducted by Usable Security and Privacy (2019-2020) course staff in 2019.
Submission Link: Coursework Submission