National Lottery of Ireland Case Study

Introduction

About the project

Loved by many, the National Lottery of Ireland is the most popular lottery provider in Ireland, and is also partially owned by the state of Ireland. My objective in this project is to improve the user experience on the current Lottery.ie App. To do that I will first conduct some research to gather data, empathize with the user and define what is the problem I'm trying to solve. Secondly, I will start my ideation process making sense of the data gathered and starting to give shape to a possible solution. Finally, I will create and test this solution using prototypes and usability testing.

Exploring The Lottery.ie App

Affinity Diagram

The research phase was started by a team of 3 people analyzing the Lottery.ie app and writing down their observations on a post it notes and putting those into a wall. With all the post it notes on the wall, we were then able to map out and categorize the problems into five different categories. This is an excellent method to organize ideas and create an overview of the current product. 

Other Research Conducted: SWOT, App Reviews, Empathy Map, Competitor Analysis,Competitor Analysis 2

Understanding the audience

Stakeholder and Target Audience Map

Who are the people involved with the product? and Who are the current and possible audiences? are two questions that try to bring the focus of the design process to the user. Trying to answer these questions, I have created a stakeholder and a target audience map.

Proto-Personas and Personas

Initially, I have created proto-personas that were based on the assumptions of who the users are and what are their pains, gains and objectives. Those points were later validated through user research and the personas were updated with the data gathered from the research to become more solid and true to the actual Lottery.ie App user. For example, the user base was previously thought to be older in age, but in fact, through a survey I learned that most users are working-age adults.

Scenarios and Storyboard

Scenarios and storyboards were created to understand the current experience of the user when using the Lottery.ie App.

Conducting a Survey

A quantitative research method was chosen to validate the assumptions made about the demographic, and also to find out more about the user's experience with scratch cards. There were 98 respondents in total that answered a total of 9 questions on Google Forms about the user's age, educational background, platform of choice, and their usage of scratch cards. These are some of the biggest insights of the research:

User Interviews

After the survey, I have conducted a set of three semi-structured user interviews. There was a simple requirement to be able to answer the interview: having played scratch cards before. The semi-structured interview format was chosen because there were some core questions that needed to be answered but also, I would like to explore the "whys". 

"I don't know what an Instant Win is."

• Learn more about real-use of scratch card playing.
• What are people's thoughts about the ethics of scratch cards?
• How is the experience of playing scratch cards like today?

The Problem Statement

Through research I was able to discover the main problems I want to focus on the redesign of this the Lottery.ie App.

Social/Fun

The social aspect of playing scratch cards was firstly mentioned on the Affinity Diagram and later validated through the Survey. By asking users about their social scratch cards play, it became clear that one of the gaps in the product is that users can't currently gift scratch cards to friends and family as they would do in real-life. 76.5% of users reported having gifted a scratch card to someone, but none of the scratch card providers analyzed offer this service for a digital scratch card.
Moreover, users reported not having a favorite scratch card and suggested more relevant themes for scratch cards. Most users recommended having TV Shows, Sports or Artist related scratch card. Nowadays most of the digital scratch card game selections are very generic.

Fairness and Transparency

Through the analysis of the App Reviews on the Play Store and also User Interviews, I was able to identify another problem that seem to be concerning for players. Many players don't fully trust the Lottery.ie App. On the interviews users reported not knowing their real chances of winning when playing scratch cards.

Easy to Play

Through product analysis, comment review and user interviews, it was identified that there are current many friction points between the user and actual gameplay. With a quick scan on the App comment section reveal the lack of game and platform features that would make players life much easier.

Following the Point of View quote structure (User + needs + because), we end up with the following problem statement:

Adults who gamble need to have a better online experience with scratch cards because today the game doesn’t feel fair, and isn’t fun or easy to play.

The Project Plan

To organize what needed to be done in which phase and order I have created a project plan document where I put everything that needed to be done. The project had a duration of approximately 7 weeks was divided in two phases. For the first phase of the project I focused on collecting data and understanding and defining the problem to be solved. In the second phase of the project I went from ideation to the creation of a prototype and testing.

User Research Planning

Survey

A survey was chosen as my quantitative research method because it is cheap, quick and it can be done easily online. The objective of the survey was to validate my assumptions about demographics and the usage of scratch cards. The structure of the Survey consisted in 8 closed questions and 1 open question. The Survey was created on Google Forms and then was posted to two Facebook groups: The Real Dun Laoghaire and Scratch Cards From Around The UK.

In this research I found that the Lottery.ie is the most popular place people play scratch cards. The second is Lottoland and third is Paddypower. On the UK the most popular is the National Lottery UK. Those four providers are the ones that I'm going to be using for Competitor Analysis later on.

• Age-group was younger than expected.
• Mobile phones are the primary method of play.
• Users don't like generic scratch cards.
• Sports and TV Shows are amongst the most desired themes.
• 3/4 of the scratch card players have gifted a scratch card before.

User Interviews

The semi-structured user interview was chosen as a qualitative research method to be able to find user motivation and why they play scratch cards. To conduct my interviews I first prepared an Interview Guide with a quick introduction and all the questions I wanted to ask the interviewees. The choice of a semi-structured interview was based on the fact that although I had in my mind that I needed to answer, I still wanted to explore the topic and ask follow up questions as they were relevant.

To organize and code my interviews I have user the Dovetail App. I have created an interview template that was then duplicated for each participant.

User Interview Session 1
Interview 1 Date: 24/03/2020 Consent Form

User Interview Session 2
Interview 2 Date: 24/03/2020 Consent Form

User Interview Session 3
Interview 3 Date: 24/03/2020 Consent Form

All recruited participants have played scratch cards before. Interviews took approximately 20min each and I have conducted 3 interviews in total, being two in person and 1 remote interview. All participants signed a consent form for the interviews and for the usability test.

Ideation

Creating tangible solutions

Now that we already have an understanding of who the user is, what are their needs and what are the main flaws in the current product, we can start the ideation phase and think about how to give shape to our solution.

In the ideation phase, we will look at all the data collected so far, the problem statement, and try to output our ideas into a more tangible and real-world solution.

Solutions to the Problem Statement

Based on the research done so far and our problem statement, I have created a list of desired features for each of the problems identified on the App.

Visual Research

Before putting ideas into paper, I have gathered some inspiration and ideas that could add value to my final design. Fingerprint login, catalog lists and payment methods are functionalities that are well accepted by many users in various apps. Making a similar mechanic for the Lottery App would add this additional layer of usability, and also respect the Consistency and Standards Heuristic from Jakob's 10 Usability Heuristics.

Wireframing

Paper and pen were my tools of choice for the creation of the first wireframe. This would enable me to quickly fix incorrect flows and elements without focusing on visuals. The screens were based on the current Lottery.ie App, with the fixes and extra features implemented.

After I was happy with the first draft of the wireframe, I have started to introduce more detail and think about the user flows of the new solutions. For example: How will the social feature of gifting scratch cards work? or How can the new Registration/Login process work? This new round of wireframing led to the following designs:

The wireframes were then turned into digital using InVision Studio. During this phase I have introduced the icon and colours of the design, and also created the prototype. The colours were based on the current brand of the Lottery.ie App, where the green tone represents the Republic of Ireland, a strong component of the brand. To build this digital prototype I have used visual references from the current App, and also components from the competitor analysis.

In this digital hi-fi prototype, I have introduced the following improvements to tackle the problems highlighted on the user research and mentioned on the problem statement:

• Fingerprint Login (Ease of Use)
• Social Registration (Ease of Use)
• Location and age confirmation step (Ease of Use)
• Better Categorization/Titles (Ease of Use)
• Introduction of Recently Played (Ease of Use)
• Reveal All Scratch Boxes (Ease of Use)
• Try a game without logging in (Ease of Use)
• Alternative payment methods (Ease of Use)
• Gift a Friend (Fun/Social)
• Introduction of more relevant Scratch Card themes (Fun)
• Faster Gameplay (Fun)
• Clear Paytable (Fairness/Transparency)
• No Min. Deposit Limit (Fairness/Transparency)

Conducting Usability Testing

Current Lottery.ie App

The first round of usability testing was conducted on the current Lottery.ie App before the wireframes were built. The testing on the current app allowed me to find pain points that users didn't talk about during interviews, and also establish metrics in which I could compare to the new prototype afterwards. The metrics I chose to evaluate the current app was: time, fun, transparency and usability through a System Usability Scale (SUS) score.

I have conducted 3 pilot tests before running the test with real users. Having pilot tested the testing sessions helped me elaborate the tasks and fix problems with the app and accessibility.

The usability testing was conducted using the Lookback.io App. Participants used their own mobile phone to download the Participate app and were asked to record their screen and audio. The remote sessions were conducted live with me as the moderator. There were 4-6 tasks that were given to the users, and also on the end of the session they were asked to respond to the SUS questions.

Session Results

Mean time of completion for each task:

SUS Score:

Fun and Transparency Score:

• Time: How much time does it take for the user to complete a task?
• Fun: Self-reported agreement or disagreement (1-5) with the statement: "My experience on the Lottery App was entertaining and fun"
• Transparent: Self-reported agreement or disagreement (1-5) with the statement: "I feel the Lottery App is fair and transparent"
• System Usability Scale Score: Measured through a calculation involving 10 questions about usability.

The results of this usability testing reveal that users are taking 1+ minutes even to do the simplest tasks. The SUS calculation result revealed a poor score of 44.1 to the app usability. Overall users felt neutral about the app transparency and fun. Those were the main pain points identified on the app:

The pain points highlighted on the usability testing sessions of the current app were very important to help me decide on the desirable features for the redesign of the app. Problems like the username input and password only surfaced on the usability test. For example: users can't use foreign characters like (^) on their username. They are also now allowed to create a strong password with more than 8 characters or using special characters, making all user accounts vulnerable with weak passwords.

Usability Testing on the Prototype

After the creation of the hi-fi prototype, I conducted other remote usability sessions on the prototype to be able to compare the current app with the prototype.

Session Results - Comparison

Mean time of completion for each task:

New SUS Score:

Fun/Transparency Scores:

The results of the prototype usability testing already show a major improvement with a much quicker and easier experience. Some journeys on the app are now 9x quicker than they were before. The SUS score also indicates a major improvement in usability when comparing to the current app. To be able to compare the metrics in a fairer way is necessary to level the prototype experience with a native app experience with real-life internet speeds and loading times.

List of improvements for the next version of the prototype:

• Change Login: I have noticed through user testing that some users didn't click on "Login" when asked to register. Instead they were clicking on the burger menu icon to look for a registration link. On the next prototype version I have updated the link to "Login/Register".
• Add better feedback on the gameplay: During gameplay users felt there wasn't enough feedback during the game results. In the next iteration "You Win" and "You Lose" messages could be added to add to the feeling of playing a real game.
• Add/Subtract Scratch Cards: When sending a scratch card as a gift, users should be able to add or subtract scratch cards from the card on the bottom of the screen. Currently on the prototype there is no way a user can add or delete a scratch card selected to gift someone.
• Scratch Individual Boxes: During testing, users also tried to scratch individual boxes instead of just using "Revealing All". This funcionality would be available on the final product, but due to technical constraints it couldn't be implemented on the prototype.

Conclusion and Next Steps

Did the solve the user problems?

I believe the main problems identified through research where tackled in this redesign and definitely put the app on the right path to solve the user problems.

Many other solutions could be introduced to increase the fun and entertainment of the App. One of them that couldn't be implemented due to time constraints and scope is the Augmented Reality functionality so players could scan their physical scratch cards and enable digital features on them.

Also, the auto-play feature would add to the overall game experience and fun of the app, but also wasn't implemented on this redesign. However, the social component was implemented to address the lack of fun of the current Lottery App. Enabling the users to send and receive scratch cards from friends adds another layer of entertainment to the app and makes it more interesting for the user.

Features like social registration and fingerprint login can add major efficiency to the app by removing 3 pages of unnecessary form filling. Even simple changes like changing separating "Instant Win" from "Scratch Cards" can have a significant difference between the user finding their favorite game or giving up and quitting the app.

The problem of complicated gameplay and difficult tasks on the app received a major help with the introduction of many features and the removal of clutter. The improvement on the time it takes to complete the tasks is also proof that the app is now much easier to use.

The Lottery.ie App currently shows the winning chances for winning any prize, but that could be misleading to some users. To make the app more transparent I have introduced a very prominent pay table on the game window that shows the user's chances of winning for each prize available on the game. Knowing the real changes of winning, is less likely that a user will suffer from addiction issues or feel wronged by the game.

Other ways we could remove dark patterns and make the game fairer and transparent is removing the requirement for a user to deposit the minimum amount of 10 euros. Users should be able to play only one scratch card if they want to. The default value for the deposit amount should be 0, and not 20 as it currently is.

Adding a step to the registration process to make the user aware of the responsible gambling features available could be another good way to make the app fairer to the user, and avoid problems with gambling.

What did we learn?

User Research

Having different types of research was essential to get rich data from the users. Each type of research was able to give different answers for the same product. The Survey was a good way to get a better grasp at the demographics and quantitative data about device usage and scratch card playing. The user interviews highlighted some of the user perceptions about the product and what they like or dislike on the current experience. Usability testing was great to get the details of the user experience using the app and where are the main pain points.

Conducting remote usability testing can be tricky when the administrator is the same person that is observing, taking notes, writing the completion times and explaining the tasks. Something that could be helpful next time is wait until the test is over to go over the video and then take the notes, observations and times later or have another person taking notes.

According to Jakob Nielsen and Don Norman, it's recommended to have at least 5 people involved on a usability testing session to catch 85% of the errors. The tests conducted on this study used only 3 subjects for each usability test due to COVID-19 and time complications. To get better results of the usability sessions conducted in this study it would be recommended to conduct at least two more tests for each.

Ethics

For the ethical aspect of the research, all participants have signed a consent form to confirm their agreement to the participation of the research and data collection. The usability testing sessions video recordings and audio are stored safely on Lookback.io.

Talking about gambling ethics on the app, it is always a risk that some users will become addicted to the game and develop gabling issues. In order to prevent that, the app does give the user tools to manage spending, control gameplay and support for people that may be suffering from gambling addiction. Some of the resources available for users are access to Gamblers Anonymous, setting a spending limit, and spreading awareness. Click here to read the National Lottery full report on Responsible Gambling.

During user research, the main user concerns were around transparency about chances of winning. I couldn't identify concerns around gambling addiction and users that were

Future Work

For the future, I would like to develop the prototype further, introduce the missing flows and making a fair comparison to the current app. For the usability sessions I would like to bring 2-3 more participants to find more of the errors and pain points. Another research that could be done in the future is Diary Studies to gather longitudinal data and find how users using the app, and if they are likely to develop gambling addiction and how to stop that from happening. It would also be interesting to test how are people going to use the new social feature of gifting scratch cards, and how can we enhance other aspects of the social layer on the app.