Case Study - Internal Security Process Improvement

Overview

COMPANY

Analog Devices Inc.

MY ROLE

UX Researcher

RESEARCH METHODS

Focus group, Survey

TOOLS

Miro, Microsoft Forms

** My case study is based on an internal process and I can’t share certain details or visuals related to it.**

BACKGROUND

As part of the product development process, products teams were responsible for following an internal security process. The process was developed by the Product Security Assurance (PSA) team to better assist product teams with their development process. One of the first steps of this process required product teams to complete a questionnaire on the Product Life Environment (PLE) website, which consisted of 6 "yes or no" questions that helped determine the level of security that would be implemented in a new product.

Many product teams struggled to completing the questionnaire and failed to understand the importance of the overall security process.

I was asked by the PSA team lead to step in and determine what issues product teams face with the questionnaire and ultimately the entire internal security process.

Approach: I facilitated a focus group of 4 engineers from different market verticals and used those insights to conduct a survey for a larger sample size. The survey was open for one week and received 18 responses.

RESEARCH GOAL

Understand users' approach to answering the questionnaire and get their suggestions on improvement based on what issues they face during this process.

PART 1: FOCUS GROUP

KEY TAKEAWAYS

  • Users struggled to understand the value of the questionnaire and how it affected the security process as a whole.

  • The questions required further clarification to avoid confusion.

  • The impact of the score associated with the level of security in the product was unclear.

PROCEDURE

The focus group was hosted online on Microsoft Teams and I served as the facilitator during the 60 minute session. They were given 15 minutes to fill out a set of questions in Miro and then proceeded to read over the other participants' responses to vote on the ones they agreed to.

RECRUITING INTERVIEWEES

The PSA team lead provided me with a comprehensive list of product teams that had completed the questionnaire. From that list, I emailed 7 engineers from product teams had gone through the process in the past two years. I explained to them that I was conducting a research study to help improve the questionnaire associated with the process by facilitating a focus group. 4 of those engineers were able to get back to me and I set up an online focus group for them as we were all from different geographical locations.

PART 2: SURVEY

RESEARCH GOAL

Based on the feedback I received from the focus group participants, I designed a survey to gain insights from a wider range of engineers across the organization. The results helped me determine what actions needed to be taken to improve the security questionnaire.

KEY TAKEAWAYS

  • Every individual seemed to have different interpretations of the questions because of the lack of examples included.

  • There was a noticeable pattern with which questions the users struggled to understand and successfully answer without requiring assistance or changing their responses to it at a later date.

  • Product teams did not understand the impact that the score had on the entire product development process or what the next steps were depending on the score they received.

PROCEDURE

The survey was conducted using Microsoft Forms and was open for one week. There were two sets of groups that the survey was sent out to:

  1. Projects that had a high security exposure (based on the score they received after completing the questionnaire).

  2. Projects that had answered "No" to questions 2 and 5 on the questionnaire.

I reached out to about 30 technical leads from this list, 18 of which had completed the survey.

SURVEY QUESTIONS

  1. How familiar are you with the security assessment questionnaire?

  2. How did you come across this questionnaire?

  3. Did you need any assistance with completing the questionnaire? This includes any help you may have received from the PSA team. If you did, please list their names and project roles.

  4. Please list any suggestions you may have for improving the current security questionnaire.

INITAL ASSUMPTIONS

  • Product teams learned about the questionnaire either through reading the product development process document or through the PLE site.

  • Teams that struggled with the questionnaire failed to keep their product development on track and faced major delays with the release.

The results here disprove the PSA team’s initial assumptions about where product teams learned about the questionnaire.

NEXT STEPS

Actions Needed from the PSA Team

  • Every individual seemed to have different interpretations of the questions because of the lack of examples included.

  • There was a noticeable pattern with which questions the users struggled to understand and successfully answer without requiring assistance or changing their responses to it at a later date.

  • Product teams did not understand the impact that the score had on the entire product development process or what the next steps were depending on the score they received.