Research Planning

Kick-off Meeting

At the start of the project, I lead a kick-off meeting to align stakeholders on goals, expectations, and logistics.

I asked probing questions to learn more about the context and objectives. From that, I determined three main research goals:

1. Identify user friction across touchpoint flows

2. Evaluate content users expect to find

3. Understand what nudges or notifications users would want to receive

I realized I would need to employ two research methods to achieve these goals:

• An unmoderated usability study would allow me to identify user friction

• A survey would allow me to evaluate user expectations and understand preferences for nudges or notifications

Test Plans

Next, I drafted two test plans for each method. These test plans delineated the study objectives, participants, methodology, tasks, and questions.

I asked my fellow researchers for feedback on my test plans. They gave me considerations for how I could add clarity and context to some of the questions. For instance, I rephrased Task 3 after my colleague pointed out that study participants might not realize they need to look for a chatbot instead of staying on the home tab.

The images below show the original task (on the left) and the revised version (on the right).

After applying my colleagues’ feedback, I set up a meeting to review the test plans with the stakeholders. During the meeting, I walked them through the structure of each study, explained the rationale for every task, and asked for their input. This collaborative approach got everyone on the same page and created a space where the stakeholders could voice their ideas and opinions. Notably, one of the Product Managers expressed that it would be important for the Product team to learn about the users’ expectations for the To-Dos feature, so I added a couple tasks to address this.

Study 2

Survey Content Testing

The second test I conducted was a survey intended to inform the future state of the Alight Worklife® app.

Unlike the usability study, the survey would capture a variety of user preferences and expectations for what kinds of content and notifications employees wanted to have featured on Microsoft Teams.

Programming

I programmed and launched the survey in UserZoom using my test plan as a guide.

I screened for participants using the same criteria and age groups as the unmoderated usability testing (see above); however, this time I sourced 200 total participants. A larger sample size was needed to capture diversity of opinion and to generalize the results. While 200 is a healthy and meaningful size, I made it clear to stakeholders that my findings should be taken as directional and informative. A statistically significant sample would have required thousands of respondents and was out-of-scope for this project.

Analysis

I used two approaches for analyzing the survey data.

For single choice, multiple choice, and rank order questions, I visualized the data by creating and formatting charts in Excel. This allowed me to show a clear breakdown of the results and distill meaningful findings.

The image on the right shows an example of a chart I created for a rank order question.

For open-ended questions, I grouped together common themes using Miro. I leveraged the AI features in Miro which reduced the time it took for me to sort through the large amount of open-ended responses.

I created charts to quantify and visualize open-ended response questions which had a wide array of ideas shared.

The graphic below shows my process of grouping responses in Miro (sticky notes) and transforming them into a chart that was easy to understand.

Presentation

Lastly, I crafted a presentation which included the top findings from the survey and a comprehensive breakdown of all the charts I put together.

Both images below are samples from the presentation I delivered to stakeholders. The slide on the left highlighted the top findings from the study. The slide on the right displayed the results from Question 3.

Results

My research findings had several implications for the future state of the app.

The stakeholders decided that the To-Dos and Account Summary should be displayed prominently because users rated them as the most important features.

Furthermore, the survey uncovered expectations for which features and information employees wanted to see on Microsoft Teams. The Product Lead said that this will inform promotional content within the Recommendations and Banner sections.

Participants also shared a variety of notifications they would like to receive on Microsoft Teams, such as enrollment reminders and changes or updates, and we found that they had a variety of expectations for how often they wanted to receive notifications. These insights will be invaluable when the Product team integrates notifications in the future.