Google User Experience Research

Last Thursday evening I went to a talk hosted by the UKUPA at LBi. The following is a reworking of my notes from the event. As always, they may not be accurate, typos may exist and it may not all make sense. Sorry this is late, some confusion about whether I could publish. Well the UKUPA is aware of these notes and there’s nothing controversial. So I publish in the name of openness (and usefulness).

Some background about the presenters

Presented by Molly Stevens from the New York office and Jens Riegelsberger who works in London. Jens works on Google Maps. Before working for Google he worked on Microsoft’s Xbox 360. He got into usability ten years ago. Molly is a user experience researcher who’s been with Google for three years working on products for advertisers. She has a Masters from Georgia Tech University.

What is user experience at Google?

Some can’t understand what research and design goes into Google products. What do you really need to do to a search box? The point is, the box has stayed the same for all these years. The focus for Google is on simplicity and the core task. The User Experience department is a small part of Google, the number of people employed in it is in the low three digits. For comparison, Product Management is larger – in the low four digits and Engineering larger still – in the low five digits.

Google’s approach

The most important thing is be agile for your users. A lot of what they do is data analysis. For example, with Google Analytics they conduct a lot of A-B testing, just like the functionality in Google Website Optimiser. They invite people into user testing labs to observe and test. They also do in-the-field observations. For example at Victoria Station in London, they did ad-hock observations of how people navigate. They also look at how people navigate shops etc.

Adwords (case study)

Molly displays a picture of an advert-laden door in a street in Cairo. This is the original Adwords. Targeting users to give the most relevant content possible is important for Google. They go into offices to see what kind of data people need as they use their products. They notice the equipment people use, computers, notebooks etc. Context is important.

Users don’t often appreciate their own behaviour. A lot of people ask ‘who really clicks on ads’ in Google products? People do, they just don’t know it. Some background: The system is designed for realtime; it’s a pay per click model of advertising; with Adsense, bloggers and publishers can make money on their websites with ad blocks; ads are made relevant to the content of the page.

In 2007 they started a project to update Adwords, it hadn’t been updated since it was built. Since 2003 they’d added more tools like Ad Scheduling. Additional reports and tools were coming into the interface but the framework wasn’t there to support it. User experience was asked to understand what the experience is now, plus it needed to be as usable as possible around the world. They focussed on three things:

Lab studies
Field visits (to understand context)
Task analysis

They travelled the world to get to diverse markets. Visited large advertisers and small businesses and other users of their service. One example – they went to a small Canadian business that GeoTargets Ads to Canada (picture shown of small mom-and-pop business), the owner finds it works.

The researchers came up with some key tasks. They focused on the task that advertisers spend the most time on. In the new Adwords interface, things that came out of this:

ROI – needed to be clear to the users, so they could tell what they were getting from it
Efficiency – with the task and analysing ROI
Clarity – how clear can we make it about exact costs

Design principles in addition:

Speed
Guidance – important to be given at the right time
Consistency – so users are thinking about the activity not navigating the interface

The redesign took two years from 2007. In 2009 they visited Google advertising agencies and search marketers in Mexico City and used them as a test model. They looked at two things:

User experience now
What’s happening in the field

They found Mexico was similar to other markets but some of the pain-points were exaggerated. E.g. If you’re in Mexico city the traffic is really bad, also it’s also difficult for advertisers to target people in their local vicinity based on the technology available. However even new people with little marketing experience were happy with Adwords. Google expected more unhappiness, but couldn’t find it.

Some things they saw:

One of the larger advertisers set their language to English(US) which meant the date format defaulted to US format in the interface. So there were some things they were able to fix on the product
In Mexico there were less people educated in the web, there’s a much less robust tech infrastructure and level of knowledge about how to do things effectively. Google identified a need to find a way to get this knowledge into the population.

Changes to Adwords from this work – they set up a navigation panel and gave deeper links. They also added a help widget that gives relevant content for the topic of the page.

Google Maps for mobile (case study)

Jens presented this case study starting with explaining a field trial from two years ago. The UK and Germany had poor uptake of mobile maps. They went to Hamberg, Munich, Manchester and London. They had six people in each location using the maps software on their normal phones. They had install parties. Where phones don’t come with it pre-installed, getting maps onto the phone is the biggest hurdle (depending on the phone, of course some have app markets).

In the study they asked to record the participants’ searches and sometimes even called them to get the context of the searches. Through this they got detailed information on the goal, plus what did and didn’t work so well.

They went back after two weeks. They found that some small issues get larger by rubbing people up the wrong way over time. One such feature was that the map reverts to default view every time you start it. This upset people on their mobiles since they may want to start the map where they left off.

They set competitions to test the speed of the app vs. the real world equivalent – for example, go to a petrol station and ask someone directions. From this they realised minimising app startup time was important.

Zoom function. The engineers assumed people would learn the maps interface (‘They’ll find the zoom button the second time they use the app’). Using these real world tests they could show people had difficulty finding the button on repeat use. The interface had to be changed.

Jens’ favourite bug story was of a Manchester participant who tried to pull up Manchester Airport, however in maps she found herself teleported to Manchester Airport in New Hampshire (US). App teleporting, or ‘warping’ users over to the US has been improved but it’s still not perfect. Importantly they combined real usage with detailed logging to find these problems.

Driving directions in India (case study)

The question, what’s the best way to use Google Driving Directions to show which way to go? Even with complete data a list of turns would not be good. The initial research on this happened in India and the US, they spent a long time looking at human way-finding. From this research they came up with a proposal that was literally test driven in India through the busy traffic using landmarks. The final product was launched in November 2009. They decided to includ confirmation landmarks in directions (as well as turnings), but these weren’t necessarily places where you change direction.

Google Maps India uses a Wikipedia model, maps and landmarks are contributed by users. Google chose to give colloquial directions to people to make it clear in the same way you’d give directions to a friend, this took a lot of engineering.

Five things they’ve learned

20% products are important to Google. Molly and Jens work on the usability of other people’s 20% products. For example, an engineer worked on search insights looking at the top five things in rising search trends. They turned this turned into flu trends, they found this correlated with actual cases. Countries are now using this to track diseases. In fact it was used in Mexico during the swine flu outbreak.
Prototyping. Important to do. Sometimes using a paper prototype is good or engineers hack something together.
Launch early and listen.
Observe in the field. A recent study in Tokyo showed that a large number of users take pictures of PR codes (not sure what this is?). Instead of using maps on their phone, they bring up Google Maps on a computer and take a picture of the screen with their mobile.
Everyone at google is a gatherer. The usability guys collate. Being a user researcher is often like being a doctor, you diagnose.

Q and A

Q. With the 20% projects, what’s your process of selection?
There isn’t one.

Q. Where do designers fit in this?
(Jens and Molly only mentioned designers once in the presentation). They work so closely with the designers they forgot to mention them. They are a part of the same team as the Interaction Designers etc. Jens and Molly only emphasised engineers because they wanted to stress their importance. Besides, everyone on the Product team came from an industrial design background.

Q. Road names change, some have two, some don’t have any – how does Google Maps handle that?
This happens in India quite often, it also apparently happens in Greece. Every road can have multiple names in Google Maps. Google also like to emphases other aspects like landmarks.

Q. Questioner likes the idea of 20% projects, are they peer reviewed?
There are project fairs (where employees can pitch for help with their project) but no review board. 20% projects are bottom up, if it has a good technology it might grow naturally.

Q. Do 20% projects get shoved to the bottom of the pile? Do employees end up spending 2% of time on them? What happens when things get busy?
It varies from person to person. You can bank up time so the deadline for your main project is passed you can sometimes spend weeks on your 20% project.

Q. Does Google prefer usability stats over qualitative insights?
The most effective is a combination of the two. For example, the teleporting maps problem in Manchester. This was taken back and found to have happened more often than they would have liked so things were changed.

Q. Do you ever take a lead in the research strategy? Meaning, is it always lead by engineers?
The Google Maps India landmarks research was a good example of a significant project (over two years) that was driven by the Product team.

Q. Field studies. You mentioned one big one, but do you think you spend enough time in the field?
Field studies happen in parallel and in multiple locations. Jens actually feels he spends too much time away.

Q. How do you keep up with all the products that Google produces?
We don’t. The Product team are a limited group.

Posted on Thursday 25 February 2010.

Posted in google, notes, usability | Add a comment »