Introducing usability testing to a business that doesn’t understand its value
For the first six months after joining my current employer I must have explained the value of usability testing a hundred times. Each time I was met by a dismissive attitude and told that design within the company was not mature enough for it.
This cycle went on until we were mid way through a project and I become concerned its usability was not quite what it should be. I felt that during the translation of designs from paper prototypes to production code we had lost enough subtle but critical design elements that outsiders to the project team would struggle to understand what the interface was trying to communicate to them.
Voicing these concerns to the stakeholders was met with the response that “No-one is working on it any more, the _[functional]_ requirements have been met”. I felt that the business didn’t understand the pain users would feel, I wanted to push back and make sure they did.
I had read that the best way to make the case for usability testing is to let your users do it for you. So I prepared as you would for any in person usability test and just did it. I had no access to actual users or any budget for incentives so I opted to run sessions with my family and produce a highlights video to present back to the business. In doing so, this is what I learnt.
Record in as much detail as possible
During sessions valuable observations come so thick and fast that it is impossible to keep up with the participant whilst producing meaningful notes. It is also impossible to convey to stakeholders the full severity of certain issues if they have not witnessed them first hand.
Recording the screen, the users facial expressions and the audio in the room makes the viewing experience as close as possible to a first hand experience. I found a laptop web cam and microphone worked well.
Always do a dry run
The first piece of recording software I tried worked perfectly for short test videos, but longer videos failed to save. After trying several solutions I found BB Flashback Express to be reliable and to produce great results.
Running through the tasks with someone else also highlighted subtle tweaks I could make so that future sessions would run smoother.
Start on a neutral page
The tasks I wrote started with the browser window open on the first page the user would see after logging in to the system. I thought this would save time but instead found that it disorientated participants by removing the additional context that navigating to the page would provide, and also took away their permission to interact with the system.
Note the time of significant events
Using a stopwatch or clock to note the time that significant events occurred would have greatly reduced the amount of time I had to spend reviewing recordings.
Be cautious of the computer set-up
During the sessions I gave participants my laptop but didn’t provide an external mouse, forgetting that my settings disable tapping on the track pad. This unfamiliar set-up meant there a few instances where participants tapped on the track pad expecting it to perform an action and thought the system was not responding, when in actual fact the computer was configured not to do anything with the tap event.
Run the tests in a private space
Because of the casual nature of the sessions and my existing relationship with participants there was one instance where I allowed a previous participant to sit in and observe another session. This was a mistake as I found them to be disruptive; talking over the participant and putting words in their mouth. The ideal set-up would be a single facilitator and single participant in the room.
I edited the recordings down to a three minute highlight video and presented it at the end of sprint demo. I expected business stakeholders to object to the small sample size, to the participants not being an accurate representation of our user base, or to the scenario and tasks only representing my interpretation of use cases. No-one voiced any objections, the video was so powerful at expressing the pain felt by participants that any flaws in the study were the last thing on anyone’s mind.
This first session did not raise many usability issues that had not been raised internally already, but I found that the message held a lot more weight with stakeholders when it was seen through the eyes of a user. The video held enough weight that the business recognised the need to usability test regularly, I have taken this momentum as an opportunity to introduce regular usability testing into our sprints to ensure it is a part of every project I am involved in.