This week we got to analyze gaze and heat maps from an eye tracking data from a usability study. The data was very valuable in conjunction with the videos of the participants to identify their challenges and opportunities that we have. The videos were also great as you can hear some recommendations and frustrations during the think out loud sessions. I learned this week if you simply look at the data it could lead you one direction, however, if you use both resources together (video and map data) it will take you into the correct path for making your design decisions and recommendations. It is vital after doing the usability eye tracking sessions to have a retrospective with the participant to gain further qualitative data from the study. Sometimes the data can show the participant seeing something on the screen, but they may not perceive it at all, ie. the ketchup bottle effect.
Eye tracking has been around for many years and has been a great tool for UX researchers. With technology getting better with the quality of equipment, improved accuracy and most importantly affordability; UX researchers are starting to implement this type of testing more and more. Eye tracking is great to see where the users are looking and gazing at, however, do not rely on the pure data alone. It is crucial that at the end of your usability test session to review the information with your participant and ask post-task questions to get more in-depth qualitative data. This will help you pinpoint areas to focus upon as opportunities and also to ignore areas that the user may be glazing around the page. You may also find areas where the user was maybe struggling to locate the information or action needed even though it could be staring them in the face, known as the ketchup bottle effect, and completely miss on what they are looking for. Finding the data patterns great in the eye tracking study, however putting that in context with the user’s actual feedback will help you determine the necessary changes that are needed
The past two weeks we got to dive deeper into mobile usability testing and mobile gestures. After looking at several devices to use for mobile usability testing, I like Mr. Tappy product. The main reason I like this device apparatus is that it is lightweight, you can still hold our device as you naturally would, it comes with magnet or suction to hold your device in place; which can also be turned to landscape mode quickly, and that it comes with an HD camera. Having the users hold the device and not be stuck on a large bulky apparatus as others I saw on the market which makes you feel as though you are at the store using a display unit. Those testing devices do not allow the user to interact with it naturally and for us to clearly capture a participant’s gestures on the touchscreen accurately. Our main goal is to capture how a participant would interact with the device, website, or app as they would at home to identify their challenges and help us find opportunities to create a better experience.
What is mobile first design? Shift your approach by thinking about the user experience and functionality on a mobile device first before you start to create your website, application, etc. for the desktop experience. For many years we, designers, were taught and trained to design for the desktop first and then figure out how to transfer that experience on to a mobile device second.
However, since early 2014 the number of mobile devices and spent on mobile devices surpassed desktop devices1 and as the smartphones continue to advance at a rapid rate, this trend will continue to grow. Thus, we need to shift as designers and organizations to design for the small constrained area on a mobile design to get the essential information on the screen, as well as the bare functionality that we want users to do and experience or actions we want users to take.
Here are some additional reasons to why we should design mobile first2:
• There are over 1.2 billion mobile web users worldwide
• In the U.S., 25% of mobile Web users are mobile-only (they rarely use a desktop to access the web)
• Mobile apps have been downloaded 10.9 billion times
• Mobile device sales are increasing across the board with over 85 percent of new handsets able to access the mobile Web
Ran into a situation when I was looking at usability testing quantitative data, which by the numbers showed the users were able to complete their tasks easily and quickly. However, when I reviewed each video to get some qualitative data they seemed to be complete opposites. Watching the users struggle in one of the tasks and hearing their frustrations was more informative vs. just looking at the numbers. If I had only looked at the numbers, I would not have been able to make the better UX design recommendations for the stakeholders. Hearing the users points of frustrations and identifying them will give us opportunities that will have a direct impact on the business’s financial bottom line. So my takeaway from this study is never solely to rely on the data numbers given to you and always ask to watch some usability testing in person or the videos.
This week I had done an unmoderated remote usability study (URUT) on American Eagle Outfitters website. I checked out the website myself before the study with the participants and saw some challenges from a user experience standpoint. Thus I had identified some opportunities for a better experience. While setting up the study with my objectives in mind, I created several tasks and questions for the URUT. There are many different approaches when it comes to conducting a usability study and when is the right time to run one type of research vs. another kind? There are five main reasons why you would want to consider URUT:
• Large Sample Size
• Dispersed Audience
• Small Budget
After my participants had completed their tasks from the study, several of them confirmed the same opportunities I had also identified when shopping on the American Eagle Outfitters website. That was excellent news to me as it validated the same challenges I saw in the site, and the data from the study will be used in my report to present to the stakeholders for my recommendations to the website to enhance a better user experience.
Continuing my thoughts on the growing field of UX and the tools that we use in powering our user research. I understand that everyone has a budget and you want to make the dollar stretch as far as you can, however, are you doing your research justice if you use a tool, for example, Amazon’s Mechanical Turk? Just because it’s very cheap and you can get a large pool of participants for your user research are you getting the right data that you need to make the right decisions to make your product the very best it can be? Or is simply good enough is what you’re going after? These are a few questions you may want ask yourself when conducting your research and when is the right time to get the right participants for the critical tasks and problems you’re trying to validate. Don’t get me wrong I’m all about maximizing my budget, however I would rather do something once and get it done right vs. having to explain to my stakeholders why I need additional funds to do the same research over again and have them lose confidence in me doing the study and the UX process as many stakeholders in the real world are not exposed or familiar with it at all in the first place.
So keep this in mind when you are out there looking for a website to help you with face to face or remote user research to find the right tool and company to aid you in this process. Ethnio website and their customers that they have whom all have used their tool gives me great confidence in the data that I would gain from their tool. This would help me in my report to all stakeholders, and when many of them want to learn or watch the process, I feel tools and website such as Ethnio would provide validation of the UX process to the business.
Well for the last two weeks I’ve had the opportunity to work on my final paper for Usability Report and I kept struggling with how many pages should my report be? From my experience, less is more concept; I saw many examples of reports 20, 40, or more pages long and I’m thinking to myself what kind of stakeholder has that much time to read a report that long? I guess it all depends on your audience that you are working for, however for me the pace of design sprints, design iterations, and also being tasked to create the end product, less is more. Less is more is great because it allows you to focus on short-term goals and create simple action items from the usability test sessions. The best part about usability testing is that users will tell you what they want and help you drive the product into the right direction for better user experience in creating something that is usable. Getting an unbiased group to test your product will also save you a lot of time, money, and resources. The key is to bring in the testing at the right time during the formative part of the process and then you can also do a summative test to help further validate your designs. Doing the usability report is the start in the right direction for better user experience.
I’ve never been a moderator for any usability test before this week.
I am very critical of the performance, and I felt that I was worse than I thought it would be after watching and listening to the test session several times. I found myself speaking too fast and saying “umm” often which I need to get corrected. With practice, I will need to slow down and talking more naturally to the user, will help fix these two most critical areas. I did feel that I was unbiased, I let the user operate the website as she would regularly use it without me helping her. I let the user speak more than I did so that I would not lead her too much other than what tasks needed to complete.
I would in the future want to ask and engage with the user a bit more; however, that is also dependent on the type of testing we are doing. In this summative test with the given tasks to the user and watching her use the site as well as getting her feedback out loud was our primary goal. The thing that surprised me the most is getting the technology and promoting the user to do certain tasks was a bit of a surprise challenge that I thought it would be. So I needed to do some dry runs a couple of times before getting started to make sure the flow of the test went as smooth as possible.
Overall, I learned that being prepared is crucial in moderating a usability test and I believe practicing in front of a group of people at work with more stakeholders, group critics, etc. will help me be more comfortable talking to users in testing.
One of the main things that we UX practitioners have to do at some point or another is qualitative or quantitative research studies. No matter if we are in user-centered design, design sprint, or simply talking with our stakeholders this kind of information is extremely helpful to persuade the audience to whom we are presenting.
“Qualitative research studies can provide you with details about human behavior, emotion, and personality characteristics that quantitative studies cannot match.”
“Statistical analysis lets us derive important facts from research data, including preference trends, differences between groups, and demographics.”
Having a basic understanding of each is critical when you have a project in which you need to perform one of the studies or if you need to perform both: “While quantitative and qualitative research approaches each have their strengths and weaknesses, they can be extremely effective in combination with one another.”
The conclusion is that we are looking to answer questions to better help solve the problems that can better enhance the product or service in which we are testing. After all, we only want to create a better user experience so that the user will have an easier and more enjoyable time using that product or service.