Summary of Contextual Inquiry Results

The contextual inquiry took place over the week of August 15-19.

The original plan was to recruit 4 students as testers, chosen to match four general "profiles" developed for the purpose of usability testing and to cover a range of user capabilities. The four profiles were:

EXEX - expert at mobile device, expert at library services

NONO - novice at mobile device, novice at library services

EXNO - expert at mobile device, novice at library services

NOEX - novice at mobile device, expert at library services

The students were sorted into these profiles based on the self-reported survey and questionnaire data.

Four students were signed up, but two did not show up to the initial device checkout appointment. A third student was brought in ad-hoc to try and make up some of the difference, but a fourth was not found in time.

I have assembled the relevant profile data for each contextual inquiry tester and made them anonymous: Profile Tester 1 | Profile Tester 2 | Profile Tester 3

The devices were checked out at the Monday session and students were asked to sign consent and release forms. They were also briefed on their tasks and desired focus for testing, outlined in the Student Tasks (final) document.

Over the course of the week, Amanda Hardy checked in with the Moodle module and posted several loose prompts for feedback. The students developed their own reporting format and left some comments in that way, which were collected and brought to the final interview session.

The final interview session was loose and relatively conversational, but followed the general themes in the Individual Interviews Notes document.

Generally speaking there was very little contextual feedback even with prompting and direct questioning, most of what the testers commented on were functionality issues. The full list of derived issues and contextually-significant data, as well as some thoughts as to why so little was found, is collected in the Contextual Inquiry Results document.

Summary of Final Testing - Functionality Aspects

This post will summarise the key issues/comments about functionality issues that came from the final testing of the full, mobile and mock-up versions of Locate. Testers were given iPads, iPhones and Android phones over a period of four days to test Locate on the devices and were asked to contribute to a Moodle discussion board as well as attending a Focus Group to discuss their findings. The Focus Groups were recorded and the results below come from those recordings. A further post regarding use of the devices in different environments will be posted separately.

Summary of Testing

The testers were asked to look at the 3 different versions of Locate (full web version, basic mobile version, and the mock-up created on the basis of the earlier testing) and to test how well they worked on various devices: iPad, iPhone and HTC (Android). Testers were asked to complete similar tasks to those completed in the earlier testing to look at functionality, usability, colour-schemes and uses of Locate in various locations.

Comments and Issues arising from Locate and Mobile Locate Testing

·         Mobile version of Locate very limited (some testers would prefer to use full version even though there are zooming/scrolling issues).

·         One tester preferred the mobile version but would like some features added (‘simpler the better’).

·         Testers were confused by having the section descriptions available on the mobile version (e.g. Books, Journals & Media, Article Search, Subject Databases etc) – these need to be hyperlinked if they are going to be available (will mobile Locate have these options available??).

·         One tester suggested having information appear only as you needed it (collapsible menus, sliding etc) so that all the information is not immediately in front of you but is still accessible.

·         Mobile version does not give enough information about availability – want to know how many copies are available – frustrated that couldn’t click on book to find information. One tester expected the Available at text to be a hyperlink – also recommended red text for not available and green for available (as already used in Locate full version).

·         One tester noticed a large white space after the end of the search bar in the mobile version which looked a bit odd – perhaps extend the search bar to reach across the page? (Would depend on whether the device is held horizontally or vertically?)

·         One tester suggested having an auto-detect function that would recognise which type of device you are using and take you to the appropriate version – however also mentioned the need to have a link between the two versions so that users can move between the two.

·         Issues around zooming in and out on various devices (maybe need to test these to get a definitive idea of which ones are working and which ones aren’t??).

·         In the mobile version the ‘Previous’ and ‘Next’ buttons are stuck in the corner and are very close together – difficult to click on the right one.  

·         In the full version the ‘Editions’ link is very far off too the right and it is possible that users won’t see it all (need to check mobile version r.e. ‘Editions’ link and tester didn’t try this).

·         Functionality mentioned that would be useful:

o   Availability information (no. of copies etc)

o   Requesting a book

o   Looking at account info (and paying fines – Paypal)

·         Issues surrounding linking to other library resources e.g. EBSCO (outside of library control) are seen as library issues. Also issues around logging in to non-Shibboleth resources. (iPads easier than phone screens)

·         Issues with using mobile phones to read e-books (scrolling left-right turns pages rather than moving across the page).

LocateMe Mock-Ups

It is possible that there were some misunderstandings about the purpose of the Mock-Ups in the testing. Some testers were confused about what they were supposed to be looking for/testing. Possibly due to the time lapse between the original testing and the final testing some testers may have forgotten the issues that they deemed most important at the beginning of the project – and that these were things that we had worked on to create the mock-ups.

Some comments:

·         Suggestion to change Login to My Account (tester: “login for what?”)

·         Suggestion that there may be ‘surface target’ issues with the drop down menu (e.g. books, fines, requests etc) especially on small screens.

·         Also noted the search bar still does not go all the way to the end – suggestion to replace with magnifying glass to save horizontal space.

One tester received an error when testing the mock-up (iPhone)

LocateME: Creating a virtual mock-up.

Methods used: Paper based wireframing, Balsamiq mock-up - wireframing software, Guerrilla usability testing and Adobe Flash ActionScript.

After analyzing the comments regarding using Locate on mobile devices from the feedback sheets, we identified one of the main *key functionalities and began working towards creating a ‘virtual mobile environment’ that would incorporate the identified component effectively within the Locate mobile interface.

 *Getting an option to login to My Account – primarily to renew books and see books out on loan (renewing higher priority than requesting).

Paper based wireframing.

We created two separate walkthrough scenarios which demonstrated how the Locate mobile environment could display the ‘My Account’ functionality within the web application.  The initial walkthrough mock-ups were created using paper, pen and post-it notes.

A screen copy of our current mobile environment was captured and printed.  We tailored the screen captures and positioned the ‘My Account’ option within several places throughout the walkthrough. 

Scenario 1: Access ‘My Account’ from the initial home screen via the ‘Sign in’ option.

Scenario 2: Access ‘My Account’ during or after executing a search.

Guerrilla usability testing.

We randomly approached 30 library users and asked each of them if we could have 5-10 minutes of their time, whilst briefly explaining to them what we were trying to achieve.  Each participant was handed the first page of the wireframe walkthrough and asked to access their ‘My Account’ information from the mobile environment mock-up.  We then presented each participant with the next paper mock-up in accordance to which option and direction they took.

All comments were noted throughout the process, and the participants were also each given the chance to offer additional feedback at the end of the session.

Wireframe screen captures with Post-it notes:

Balsamiq development.

The findings from the initial paper based wireframe guerrilla usability testing were subsequently fed back into the design process using the ‘Balsamiq’ wireframing software.  The two scenarios were mapped out using Balsamiq, fine tuned and printed as graphical based printouts.  We then ran the previous tests, but this time we used a different random selection of library users; again allowing each participant to determine their own way through the task. 

Scenario 1: Access ‘My Account’ from the initial home screen via the ‘Sign in’ option:

Scenario 1: Access ‘My Account’ during or after executing a search:

 

 

Key Comments:

(1) “I normally expect to click on a toolbar at the top, or bottom.  The login option is in the right place.”

(2) “The web pages are simple and easy to get around.”

(3) “Smooth, logical and easy to use.”

(5) “Can the font be different colours, if you are logged in?”

(6) “The layout is intuitive and fresh.”

(7) “The information on the screen is not overwhelming.”

(8) “Is the ‘My Account’ link above the item just for that item?”

(9) “The word login is old fashioned.  Sign in is the normal option.”

(10) “It is very much like Sainsbury’s. It works like other similar mobile search pages.”

Flash and ActionScript : Virtual mobile device.

We took our findings from the paper based and the Balsamiq wireframe walkthroughs and used the information to create a virtual mobile walkthrough that demonstrates the two predetermined routes that allow the user to obtain their ‘My Account’ information. 

Adobe Flash: Adding user interaction:

Adobe Flash: Adding 'My Account' functionaily:

You can view the final virtual mobile interactive environment by clicking on the link below:

IMPORTANT: Flash version 9 or above is required to view the interactive mobile environment.

Virtual Mobile Device Mock-up.

The initial stages of testing Locate on smart phones and tablet devices.

The initial walkthroughs by the instructional designers and learning technologist, using the latest mobile devices, allowed us to evaluate the interface and identify specific issues to the development team. The group analysis was recorded and made available to the development team for a further discussion in a scheduled LocateMe meeting.

A detailed report overview of the analysis videos can be accessed online: The five cognitive walkthrough reports.

The first video created analysed Locate using the iPad on the university WiFi network - and its unique feature of zooming in and out:

The second video to be included in the blog uses the iPhone. The task of using the iPhone (on the 3G network) was to search for a book (Operations Management) in the main university collection and find where this was located:

The final video to be referenced makes use of an ebook on the iPhone (using the 3G network):