A search happens whenever a user looks at any GUI: the process of scanning your interface to determine where to click, tap, or look is a conceptual task known as Visual Search. Many factors shape a user’s visual search, for example:

• Habituation
• The salience, number, and legibility of features
• The user’s level of expertise in interfacing with GUIs in general
• Latent biases based upon other/similar sites that they know well
• The user’s language

And even though there are plenty of tools, like user testing, heat maps of user interactions, eye tracking studies – there is no silver bullet for understanding exactly where an individual user will look for things when interfacing with your GUI.  Still, assuming visual search is a search task like any other – what insight might we gain from thinking about visual search the way we think about other forms of search?

These are the steps that the user goes through in doing a Visual Search:

1. Identify the search space. You might go to Google to search the Web. You may go to Twitter to see what tweets are trending.  Both are searches driven by your own high-level goals, but the search spaces themselves are very different.  In the visual search of your GUI, the user’s search space will vary too – largely depending upon the user’s proficiency within your site, and their own conceptual biases.  If they’ve never been to your site before, the search space will be your overall page (or regions of it).  But if they’re familiar with your GUI, the search space may be confined to particular components (like your sidebar nav as an entry point, or the login block) that your user believes will contain the information they’re looking for.  This is important to keep in mind when you’re designing for new versus return users: an expert/returning user’s reduced search space helps make Visual Search easier to do, but introduces a preconceptual bias in terms of where the user will tend to look be looking.
2. Create the query. Users will be looking for one of two things: a specific artifact that they believe exists within the search space, or artifacts of a certain type that they believe to exist.  For example: I may want to revisit a specific article on cnn.com that I was looking at 15 minutes ago to see what comments have popped up.  Or, I may simply go to the cnn.com home page to look for new articles that are interesting: I know articles exist there, but I’m not yet sure if there are any interesting ones. These are two very different types of Search, often supported through complementary Browse/Search user flows.
3. Examine the resultset.  The resultset in a Visual Search is comprised of the particular interface features that were notable and which caught the user’s attention as they scanned the aforementioned search space.  Unlike explicit search operations within which the user invokes a query to have a set of results returned to them, Visual Search is more of an implicit search where the user extracts features out of their visual field. The factors impacting notability of any particular feature are the focus of a significant body of research – wikipedia is a great place to start.  But some key factors include: salience (i.e., how bright is its color?), positioning (i.e., is it in the expert user’s search space?), and noteworthiness or subjective interest. The resultset is essentially the subset of UI features that stand out to the user, the elements that catch her eye.
4. Iteratively refine the search.  Like other forms of search, Visual Search is iterative. Based upon the relative quality of the results, a user may reflect upon and change their query: “Did I find what I wanted? Are my options interesting?” But interestingly enough, I may also reflect upon my search space, and change that.  If I return to the cnn.com homepage but I don’t find the article where I’d found it before, I may shift my search space to include the entire page (not just the page component where the article used to be) and I might also change my query (from looking for the specific article to looking for a category that I think might contain the article.)  A user’s visual search has built-in mechanisms for “adjusting” their mental models in terms of shifting the search space/query.  This is both an opportunity and a risk – it might help returning users discover and identify new features, but it could also mean that user will change their search space to someone else’s site, in order to find what they are looking for.
5. Select a result.  Once the user finds a result that matches their query best, they will select it.  In visual search this doesn’t necessarily mean that someone clicks on something – it might just be that the user looks at it, or shift her search space to focus on that particular component.  The important thing is that the user has made the decision to focus some of their attention on that part of your GUI.

Have you ever thought about Visual Search this way? Do you find it useful in terms of how you might evaluate your designs?  Identifying the Search Space seems like a particularly important step to think about, especially in terms of a user’s long-term usage of our GUIs – something we all hope to support.

Search is a fundamental aspect of life. Scavenging for food, seeking shelter, the quest for love. In user Human-Computer Interaction, “Search” describes a specific, iterative workflow:

• The user enters a query
• The search is invoked
• The result set is reviewed
• The user refines their query or exits the search

Within that structure, an iterative user process of refinement emerges, where users react to the results that come back and subsequently refine their search. Results, result parameters (like sort order), and result metadata (like number of items matching, number of pages) each educate users and give them insight into the relative ‘quality’ of the search engine or document set.

But: search is a fundamental aspect of life. The conventional definition we use in designing Information Systems can be a risky conceptual shortcut to use. How so? Well for one, search usage even within a particular product will evolve:

The protracted search
A few days ago, I had a great conversation about the nature of search, and the obvious reality that some forms of search are much more complicated than others. We were essentially asking ourselves the question: what makes a search complicated?

• One dimension is how educational it needs to be – i.e. how much does the user need to learn about the solution space or the search tools to be able to specify what they are looking for.
• A second dimension is how personal the search is. A search is often a quest. It can include processes of self-discovery, needs assessment. It can involve making difficult decisions about which path to take. Consider the search for your perfect job. Or possibly even more complex: the perfect house. The finding involves endogenous processes of needs assessment, not just scanning and reacting to exogenous cues.

The impact of these factors is that the search becomes an experience that spans a much longer duration, and multiple sessions of actual use. Over time, the user will re-engage with your search tools differently: their queries will shift or change as they better understand the search domain. Their needs and desires may shift. They may look for results they had saved to pick up where they left off. They may develop new frustrations based upon things they’ve learned. Consider a search related to more nuanced decisions, like the specific attributes of your ideal home, something I explored in my thesis research. These searches are inevitably extended over longer periods of time.

In a complex search ecosystem, careful consideration must be given to each of these nuances: how can you phase and tune the relative salience of affordances to help users discover, learn, and pick back up where they left off?

Next Up: Visual Search
Even if you’re not designing a search engine, every single user of your site will invoke a search: Visual Search. It’s the process of scanning your UI to understand product capabilities, and how things are organized. Remember that your product is a place. Visual search is how users navigate our places: it’s how they find their way.

Summary: A focus on ecology is not yet part of most UX practices. But maybe it should be.

Background

Considering that UX design is a relatively young and highly applied domain, I suppose it’s not surprising that designers haven’t really gotten around to discussing the broader ecological impacts of our work, even though others have been. We’re more product-centric and rarely take time to do deep analysis of the virtual landscape we collectively craft. And yet, our work is becoming increasingly ubiquitous: We’re beginning to see that the topology and human impacts of the UX that we have been collectively creating are quite significant and poorly understood. In that light, it seems strange that we haven’t been taking time to step back and reflect upon the ecological impact of our work. Instead, we tend to operate according to a discrete set of values:

• Aesthetics. “Beautiful is good.” Not aesthetics in the philosophical sense; rather, a concerted and persistent focus upon visual style and the emotional impact of our UX. For example, minimalism as demonstrated by the Metro UI, versus the “metaphorical physicality” that Apple has often employed.
• Usability. “Simple is good.” Ease of use. Focusing on how intuitive a UX design is to an end user – the standardization of features, leveraging established design solutions, employing constrained choice, presenting a clear hierarchy of concepts. Discourse here often includes quantitative analysis and measurability because the validation of usability becomes the rationale for design decisions.
• Persuasion. “Obedient is good.” Persuasive Design. This is actually quite prevalent because helps our UX align with the business goals that most practicing designers are tasked with achieving. It entails the employment of the specific design solutions that are most likely to result in the user doing the specific things you want them to do. For example, tuning the salience and positioning of a button that you want the user to click. Maybe you’re doing something noble, like building a fitness app that aims to encourage the user to go to the gym. But maybe you’re just designing a trial registration form that aims to drive customer acquisition.

So what does it mean?

One thing that’s interesting here is how much overlap there is between these discrete thought centers, and how they chain together into what is probably the fundamental goal of most UX practitioners and organizations: design elegant solutions that are easy to use and that grow the business. In fact, this is a nearly ubiquitous mission statement:

• Design elegant solutions (Aesthetics)
• That are easy to use (Usability)
• And that grow the business (Persuasion)

But what’s also interesting is how potentially myopic it is. While the goal is of course to employ best practices, leverage standards, and to operate within existing stylistic trends, these foci emphasize one-offs – each design is an independent whole, crystallized, autonomous. We design tools and places, without giving broader focus to the ecological impacts of our work, and how we shape the virtual topography.

UX Ecology

So, what if we focused upon the ecological impacts of our work as well?

• Harmony. How does our work fold into the user’s life experience and processes? Do the user experiences we create disrupt the virtual ecosystem? If so, are they an improvement or detraction?
• Balance. How does our work in virtual/mediated ecosystem balance with the physical ecosystem?
• Sustainability. Do the products we design create untenable ecological shifts in the virtual world? What about the physical world. Texting while driving, anyone?

Our mission statement might then read:

To improve user experience through designs that efficiently complement today’s human experience, while being easy to maintain in the future.

• To improve user experience through designs (Harmony)
• That efficiently complement today’s human experience (Balance)
• While being easy to maintain in the future (Sustainability)

What vs. How. Or, Mobile Site vs. Responsive Design

I was intrigued by several of the strong disagreements to Jakob Nielsen’s recent post detailing his findings about the usability of mobile sites. To me, it seemed that Nielsen was simply leveraging data to argue that a design for a mobile audience should be optimized for mobile use.  It would be hard to disagree, right?  Most of the critical responses stemmed from people who were asserting Responsive Design as the correct methodology for implementing a mobile experience: see here, here, and here.

So what’s going on here? Isn’t responsive design supposed to facilitate mobile optimization? Why would its proponents take issue? Let’s take a few moments to unpack this debate. It’s interesting because strawman arguments are leveraged on both sides…

Nielsen’s Strawman: talking about “Sites” instead of User Experiences

From my reading, and his own points of clarification Nielsen’s root findings have nothing to do with any specific implementation. His assertion is simple: a mobile experience should be optimized.  He provides some guidance on how to optimize for mobile, and notes that this optimization should include “cutting” out features and content.  But his language strays into the realm of suggesting how to build your site.  His summary is:  “Two designs, two sites, and cross-linking to make it all work” and he talks specifically about redirecting users between two URLs.

Nielsen’s strawman is thus to use the concept of building two separate sites to convey his point, instead of talking about designing two separate experiences.

Sites may be easier to talk about and describe, but they are the wrong way to characterize his findings if he’s not concerned with implementation.  As soon as you talk about linking between two URLs, practitioners will think about implementation. Yet as he notes, Nielsen’s findings are not concerned with how you build your site; they are concerned with what the UX you create is like.

Responsive Design’s Strawman: talking about “User Experience” instead of implementation

Proponents of responsive design countered that users expect all features and content of a site to be present on mobile and that building two separate sites is bad for content, and more difficult to maintain.  Further, they assert it treats your mobile users as “second class citizens.”  Thus there should be no  “cutting.”

Several parts of this argument are strawmen: they’re talking about content strategy and user experience in place of what responsive design is actually about: tactics for implementing a single code stack within which layouts are automatically adjusted to provide cross-platform UI optimization.

The focus of general discourse on responsive design, as an implementation methodology, should be on how you can create optimized mobile experiences using responsive techniques, not what those experiences should include from a content or usability perspective.

The Takeaway

In short, these are two entirely different areas of focus, and it’s counter-productive to mix up the two. Both Nielsen’s findings and responsive techniques are relevant to mobile optimization, in terms of streamlining the presentation of content.

The language used by both sides is fundamentally what is at fault. Discourse on responsive design should not preoccupy itself with content strategy or the usability behind what a mobile user experience is like; rather, it should remain focused quite specifically on the how, on the implementation. Likewise, research into the usability of mobile sites shouldn’t make assertions about implementation. Interestingly though, I think the whole disagreement, and myriad others like it, would be avoided if UX knew how to talk about Places and not just Sites.

Previously, I’d written about a peculiarity of our lexicon in Web UX design, which is that we tend to talk about the things we create as sites – or parts of sites – rather than places.  I see this as being an historical oversight; an omission in our conceptual vocabulary, but I’m not sure why it happened.  Maybe it occurred because we were operating in an evolving virtual landscape, where everything was inchoate and abstract.  Whatever the reason, an example would be our tendency to talk about Web pages as being parts of Web sites. That is like describing your roof as a feature of your lawn.  The whole concept of house – of place – would be missing from your words.

If you’ve humored me this far, I suppose you are thinking “So what?”  To put it simply, I’m wondering if operating without a solid notion of Place has had an impact upon how we framed – and continue to frame – design issues on the Web:

The relationship of a place to its site.  One of the most important considerations in architectural design – particularly in the modernist and subsequent movements – is the relationship of the place that you’re designing to the site that contains it.  How do people discover and observe the building before entering it?  How do they approach it, enter it, and navigate through it? How do they experience the site from within the building itself?   A classic example often given is the relationship of the Parthenon to the Acropolis in Athens, Greece: the site, its vistas and entry points informed the position and proportions of the building. In the virtual realm of the Web, this inquiry would translate to the positioning of elements, salience of elements, and how users perform way finding within the site.

The relationship of a place to neighboring sites.  Nothing is done in isolation.  How a building responds to the other buildings and sites around it is just as important as how the building responds to the site that holds it.  On the Web, this consideration would include all points of discovery, traffic volumes, integrations and APIs used, entry points, re-entry points, exit points, and the UX of any integrated products – or even the influence of a related product’s UX.

Gestalt: The nature of the place itself. Our “websites” as wholes are generally comprised of a set of content blocks – header, body, footer, and common navigational elements.  Best practices, standards, and guidelines help to inform the design of these individual features. But regardless of the fonts, colors, visuals and branding, what is the nature of this place in the users experience? The whole is greater than the sum of its parts.  This inquiry is hard to talk about, harder to represent, and too often, overlooked. It is also the key to good design. On the Web, I’d suggest that it starts with recognizing what the whole actually is – a Place in a site.  (Note: often and increasingly w/ mobile apps – we’re creating Tools and other types of things, not only Places – the two are distinct.)

What’s the consequence of designing places without talking about places?  The Web is a complex, messy, expansive and ever-growing world.  Within that landscape, UX is in need of more reflection, more planning.  None of the 3 points above are revolutionary – these are all things that are done today – but far too often they are done in an ad-hoc, informal, and near-sighted way.  Words are merely words, but perhaps if we put more focus upon the relationship of the Web Places we create to the Web Sites that contain them, we’d reflect more deeply upon the broader impact of our designs upon our user’s experience.

I find it curious that collections of pages within a specific domain became known as parts of a “Site”.  While the site may be where we create, it is the wrong metaphor for what we create. As an architect, I’ve always understood a site to be the spatial location where a structure may be built. While the site is a particular space, the structure or building that we put there subsequently creates a place within that space. Raw experiences or events can also create places out of spaces. Place is a distinct concept from space because it’s not agnostic – creating a place involves the setting apart of a particular space for some specific use or purpose.

A place is what’s created; space just is. A site is a particular space, and provides both opportunity and context.

It seems to me that “Site” was originally an apt metaphor for describing a particular URL, which provided a location identifier for a particular hosted environment, made accessible over the worldwide web. These sites were truly spaces, and presented us with the opportunity to build something new therein. But as the technology (and the consumer market) took shape, we simply extended the site metaphor to include all of the stuff that we were creating within our sites: pages, service endpoints, or whatever.

This extension of the site metaphor was a significant event because it precluded us from talking about what we were doing in the correct way. These days, a web designer would generally tell you that pages are the constituent parts of a site… a space. The notion of place is missing. That is like describing your home without ever using the word “house.” As designers, we are and have always been creating places – carving and defining tools and features to fit some specific use. We operate with purposeful specificity, a specificity that in this case our language obscures.

Next: Part 2 covers the impact of this obfuscation upon how we describe our work, and how we frame design problems.

I recently designed a few Icons in omniGraffle for wireframing, and threw them into a stencil (below) so that you all can reuse them if you’d like.  I’ll be expanding this into a more complete library over time, but right now there are a few renditions of a gear icon I was developing.  These are all vectors so they’ll scale nicely.  Feel free to reuse / distribute!

Free download here

Diagrams are important because they represent and force consideration of the broader context within which a web site or service is situated.  In doing interaction design, you have to think about more than your site.  For example, consider traditional architecture.  Lets say you’re designing a building – obviously, it is important it is to consider the site where your building will be located.  It influences a buildings form, positioning, materials, and construction, zoning limitations.  Architects don’t build sites. They react to and build within them.  Similarly, a web experience is only one context, one that is situated within the complex virtual and physical workflow of your customer’s experience.  Customer experience is your site – a collection of pages is not.

Conceptual Diagramming provides a mechanism for representing the broader context that you’re designing within.  The experiences and touchpoints that you diagram will inform the limits of the user experience you ultimately design. The diagram is a versatile representation for articulating flows, modalities, alternative paths, as well as significant and natural transitions within that site.

To help you get started, I’ve posted an Omnigraffle stencil and template for Diagramming

This is an OmniGraffle Template and Stencil for drafting basic State Diagrams for interaction design and touchpoints for user experience as a part of experience design. It’s based upon a drafting style that I’ve been evolving, give it a try and let me know how it works for you. I’ll post versioned updates as the system matures.

State Diagram Template & Stencil bundle for OmniGraffle:
Click here to download

While I was at MIT, I developed a Shape Grammar that describes Louis Sullivan’s system of ornamentation. Using the Grammar as a tool, it is possible to calculate (generate) each of Sullivan’s designs, as well as to generate entirely new designs following his system.  

Enjoy ~ and let me know what you think!