The first task undertaken for the usability testing was the development of an initial recruitment questionnaire (Kirsty Kift) which was placed on SurveyMonkey and distributed via an online announcement to students. In addition to being a recruitment tool for Stage 1 testing, the questionnaire was used to determine a rough baseline of students' familiarity with the library services, various mobile devices, and whether they had access to wireless internet at home (a requirement for Stage 2). We received over 40 respondents although several of those did not leave contact information, making it impossible to recruit them.
Of those remaining, approximately 20 were recruited for the Stage 1 rapid usability analysis (identified by Kirsty, contacted by Sue Adcock). They were asked to sign up for one of several time slots for "testing groups" of 4-6. Before these groups met, Juliet Hinrichsen, Amanda Hardy, and Paul Grove developed a plan for a task-based cognitive walkthrough of the Locate interface using their own devices to identify any early issues.
Using the sample personas from the "JISC Usability Studies Final" document and the user tasks from the initial Locate development project, we assigned Helen (Amanda) to the status of "novice device user, novice library services user" and Monica (Paul) to the status of "expert device user, expert library services user" to suit both the descriptions in the personas and the experience of the people enacting the walkthrough.
Videos of the tasks were recorded and made available to the project's developer. Samples may be uploaded to a YouTube channel if there is demand.
The walkthroughs (and subsequent reports) were useful. They identified many early issues with the currently existing website's performance on the target mobile devices and flagged items for consideration in development. For example, not all of Locate's webpages could be "zoomed" on the Apple devices which made using those pages difficult. Interface elements (such as "view in new window" buttons) were not always present on the HTC. Several accessibility notes were made as well. Please see the reports for the complete information.
After the cognitive walkthroughs were completed, the rapid usability analysis "testing groups" took place. Students gathered from the initial SurveyMonkey questionnaire respondents (in groups of 4-6) were asked to bring and use their own mobile devices (as a convenient solution to delayed procurement of project devices). Testing groups were led by Library/eLU staff in pairs (Kirsty, Theresa Morley, Paul, Amanda). At the testing groups, students were asked to complete a "baseline survey"(Amanda/Kirsty) to determine some demographic and experiential data. We intend to administer this survey to the remaining participants at the end of the project to see whether some of their usage habits have changed. These surveys were also used to help identify potential persona assignments for those students willing to participate in Stage 2.
The testing group activity was in three parts: Completion of basic library tasks using the website (establish familiarity), completion of the same basic library tasks on the user's mobile device (feedback to the Library developers on this stage to be written on a handout), and a brief "focus group" at the end to discuss as a group their experience, feedback and any issues. Focus group feedback was collected by the facilitators and typed up.
It was useful to talk directly to the students about their experience and what they might like to do (or not) with the mobile version of the website. Some of them the project members had anticipated (the ability to renew books) and others we had not (linkage to the university's new managed printing system). Some of the suggestions are likely to be technologically complex to implement but all suggestions were collected and will be evaluated for priority and feasibility.
And a summary of the biggest priorities as provided by Theresa:
the following seem to be the biggest priorities to come out of last weeks’ testing:· Getting a simple search screen and results screen that limits scrolling/zooming (with clear availability info)o Users frustrated with slowness at times – simpler info = quicker?· Getting an option to login to My Account – primarily to renew books and see books out on loan (renewing higher priority than requesting).o Lots of login issues on mobile devices that caused frustrationsOther issues that could be included in future development:· Opening online resources in a new window automatically (SFX, ebooks etc)· Students would like to be able to search for articles and add them/save them for future reference (Article search was very slow/froze up in testing)· Some students mentioned a way to get help (icons, tips etc) – Ask A Librarian link?· Viewing loan history (linked with My Account)· Paying fines / receiving overdues
