We are often asked by our clients to find out whether their customers would use or need a Smartphone or tablet app to access their functionality or content. Of course the answer depends on the business and the target user group but it often raises the question of whether a native app, web app, or mobile optimized website is the most appropriate approach where the business wants to facilitate use of mobile devices.

Mobile websites are the simplest of the three as they are simply web pages that are designed to be accessed on devices with smaller screens. Native apps are device specific and are downloaded and installed via services like Apple’s App Store. Web apps are somewhere in-between as they are accessed via a device’s browser, are cross platform and offer improved functionality over mobile websites but do not leverage the full power of the device in the way that native apps do.

The pros and cons of each approach are too numerous to list here, however, in depth understand the context in which their users need to interact with them on mobile devices is often an overlooked consideration. Native apps are great for providing a slick, rich and integrated experience. However, if your customers access your content from mobile devices rarely or are more likely to use web search to find you, developing a native app may not represent the most cost effective solution and choosing instead to develop a mobile-optimised website would be more appropriate. Simply having a native app will not enhance the overall experience if users are not likely to download it in the first place. Companies like Amazon take the approach of catering for all eventualities but most online businesses do not have Amazon’s user base or deep pockets and must choose what to implement.

We believe that establishing context of use and the needs of users with respect to mobile device access is an essential when planning how to facilitate mobile device user interaction. This will make sure that the business understand when, where and what users access on their mobile devices and ensure that the most appropriate, and cost effective, implementation is used.

We regularly test websites where internal stakeholders of our client organization are a primary, if not sole, user group of the interface that we are engaged to test.

Testing with internal stakeholders can often provide much deeper insights into an interface as they typically have a stronger level of engagement with the interface being tested and better understand content and the organization as a whole. Stakeholders are invariably excited to participate and appreciate the opportunity to share their perspective, however, recruiting these participants involves the following considerations:

• Ensure that you are getting the right staff members and not just any staff members. It is important that recruitment specifications are used to focus recruitment on the staff members that you want to participate. In general, staff participants should be forthcoming and willing to participate, should have appropriate levels of technical expertise and should use the interface that is being tested
• Experts in a system under test can provide useful feedback, however, often don’t encounter usability issues as they are too familiar with the interface. Feedback from expert users is often best elicited in focus group or survey type research
• Ensure that the interface is relevant to the individual staff member. Often some members of a team or department use an interface while others in the group do not. It is important that the participants are, or will be, using the interface
• If testing a new interface, ensure that the staff members have not had significant time to learn the interface. It is often best to enter a usability session with a participant that is new to the interface but who is familiar with the organization
• Do not include staff members that participated in the design or implementation of the interface or in research related to the interface
• When using external recruiters to find internal participants, it is important that the client contact communicates to potential participants that they will be contacted by somebody from outside of their organisation and the reasons for this

It is highly productive and encouraging when we have the project teams view usability testing sessions. Design & development teams find it easier to digest things when usability issues are highlighted directly by the user than receiving it in an observation report from an external consultant. However, there are few things to mind when you are observing a testing session:

1. Never draw conclusions from observing one or two participants- Observations and findings should always be looked in total and not concluded based on the behavior of a single participant.
2. Keep the decibel level low in the observation room – Even the best sound proofing cannot guarantee for 100% noise reduction. Though participants are well aware that they are being observed, the worse thing for them is to actually hear discussions in the testing room.
3. Observe what participants do and give less emphasis for what they say that they do – Participants tend to get overtly critical or extremely positive during testing sessions. Take notice of actual behavior and less notice to their extreme comments.
4. Hold the urge to ask questions during a testing session – This can be unnerving for the participants and can also be leading in some cases. 
5. Never go astray from the initial objectives set out for the test sessions – Though it is quite tempting to ask participants ‘How do you want to fix the problem?’, Avoid asking such design specific questions. To make testing sessions more effective, let the user to play his/her role rather than asking them to do the designer’s role.

At UsabilityOne we have extensive experience, not only in evaluating search functionalities across a wide range of websites, but also in helping our clients gathering user requirements and designing the user experience for their search functionality.

Search is an important user interface element in any website. Research shows that around 10% of users will elect to use search as their primary means of locating content within a website. However our experience indicates that most users will turn to search when they have been unsuccessful in locating content via a site’s navigation structure. In addition, when it comes to search, Google sets users’ expectations to a high standard.

So, how do we evaluate the performance of a site search, from a usability perspective? Well, this question can be addressed from two separate angles: firstly, from a classic usability evaluation perspective we can compare how the site search performs against search best practice and observe users searching the site. Secondly, we can look at Key Performance Indicators (or KPIs) to monitor how the site search is performing over time.

At UsabilityOne we suggest the following seven KPIs that relate to search:

1. Average searches per visit
2. Percent of visitors using search
3. Percent of zero results searches
4. Percent of zero yield search
5. Results Page views per search
6. Search result to site exits ratio
7. Percent of search refinements

Unfortunately, the above KPIs do not monitor how well suited to the query the search results are. Some qualitative methods need to be put in place in order to monitor both the appropriateness of the search terms used and also the relevance of the results returned.

The story behind each of these quantitative and qualitative measures will be published in our next newsletter.

One of the most important benefits of user testing is the opportunity it provides our clients to observe real users interacting with their product or website. This never fails to be an enlightening experience for stakeholders and really helps to engage them in the process.

We expect our clients to experience some surprise when confronted with how users actually interact with their website, however something else that we are increasingly observing is stakeholders’ surprise at how their website actually works and the information it provides. Whilst our clients are always very knowledgeable about their business, it seems that keeping track of how their own website works and fulfils business needs can slip to the back burner.

Large and intricate websites and complex business structures make keeping track of even a small part of a website a challenge and makes it easy for changes made to a website to slip under the radar of those who best understand the usability or business requirements.

Consequently, we have found that usability testing, and in particular clients taking the time to observe sessions, can really help stakeholders to re-connect with their own website, their users and allow them to re-evaluate how the website aligns with their business goals. This is a somewhat understated benefit of user testing but it is one what we feel is becoming increasingly relevant.

iPad meets user: UsabilityOne tests the iPad

The iPad has finally landed in Australia. Here at UsabilityOne, we’ve been following the commentary on the iPad from afar, and decided to get our hands on one so we could make up our own minds.

iPad – user testing:

So just how usable is the iPad? To find out, we conducted some usability research of our own. We asked users to complete a number of tasks that use the native apps, as well as some browsing activities.

Our preliminary findings indicate that the iPad is inherently usable. The average score attained by iPad users on our UsabilityOne Task Completion Index (TCI) compares extremely favourably with the average score for the hundreds of interfaces we have tested here at UsabilityOne.

The reaction to the iPad amongst our testing participants, which include a mix of iPhone and non-iPhone users, has been overwhelmingly positive. Users thought the device looks very slick, providing a good overall experience. The web browsing experience was particularly positive, as it feels more immersive due to the touch interaction.

Downside? People are struggling to see exactly how it will fit into their lives. It clearly is not a replacement for a desktop or laptop computer. In fact, you require one of these devices to transfer and manage content on the iPad. Mostly, the iPad has been perceived as a luxury device rather than a must-have. The most common use-cases our participants have put forward include: web browsing while sitting on the couch or using it to watch video content, things that you can already do with a laptop or netbook. Other scenarios suggested include use of the iPad in the kitchen as a recipe book or as an electronic photo frame. The touch QWERTY keyboard is seen as reasonable for typing in search phrases, URLs and short emails, but no more than that. Nobody could conceive of using the iPad to create or edit documents.

We also observed that the way in which people held the iPad is likely to impact on how they use it. Users commented that it is actually much heavier than it looks. In order to use it to watch video or read e-books for a long period of time means that – unless you have forearms like Popeye – you will need to put it down or prop it up in some way. Obviously this will impact on the most common usage scenario suggested by our test participants (i.e. sitting on the couch).

UsabilityOne’s take on the iPad

So what do we think of the iPad? Whilst a nice piece of kit, the iPad has several obvious limitations.

The keyboard limits the iPad’s potential as a business tool, and the inability of the iPad’s Safari browser to display Flash content clearly impacts on its utility. Whilst it is great as a video device, the continued decision by Apple not to support Flash means the iPad is not compatible with catch-up TV services such as the ABC’s iView. The same goes for all of the other free-to-air broadcasters’ catch-up TV services. In the UK, the BBC created a version of their iPlayer which is compatible with iPhones, and it is interesting to see that the ABC has just announced the release of an app for the iPad that will allow users to access ABC content such as iView.

Given the boom in mobile browsing, many websites are now presented in a format which is compatible with mobile phone browsers, providing an optimal experience on those devices. Our experience thus far with the iPad indicates that websites presented on its Safari browser are not modified in any way; they are the same as you would see when accessing them on a laptop or desktop computer. As touch provides the only means of interaction with websites on the iPad, there are some question marks over the compatibility of traditional website design and the iPad. Just as with the iPhone, target areas (i.e. text links, navigation) can be difficult to hit on the iPad without first zooming in. However, a consequence of zooming in is that some content drops out of view, which suggests that it is important to be even more conscious of ensuring proximity between related controls and content. Perhaps there is even an argument for creating iPad versions of websites.

Like our participants, we think the iPad would be nice to have but the fact that it does not replace your desktop or laptop computer makes it hard to justify outlaying the cash to buy one.

So, is the iPad a game changer? Its immediate success, and the fact that competitors such as Microsoft, Dell, Sony and HP are preparing their own releases into the tablet market suggests it may well be. There are great hopes that the iPad can save the newspaper industry and that it will find a niche in education as a replacement for traditional text books. It is a lot of expectation for one little tablet device. The true success of the iPad, as with the iPhone, will lie in the development of apps that exploit its capabilities and fit into people’s lifestyles. We will be eagerly watching this space.

Want more info on the usability of the iPad? Keep an eye out here for some detailed findings on this research in the near future.