Notes on Human Computer Interaction

This post is a collection of my notes on Human-Computer Interaction (HCI). It is not intended to be a comprehensive guide or a systematic review, just a collection of useful articles, links, videos, books, webs that I found during the last years and that I have been using during this time and that I think they could be useful for someone else. There are also Twitter accounts, blogs, LinkedIn groups, etc.. that could be interested for any person interested in HCI.

Since my personal interests are in the healthcare area some of the comments are focus on that area.

Cognitive and psychological
Some time ago, I wrote this entry “Technology is the Enabler… but Design is the Engager” which summarizes perfectly the importance of the design. I’m not sure but I guess it is a quote from Daniel Kraft although I also listen it on this talk by Aaron Sklar founder of Healthagen, a firm aimed aimed at “using design to fade technology into the background”. It is not enough to build something good and useful, it should be appealing and understandable by the users if we want that they use it frequently and properly. Aaron together with Lenny Naar founded this interesting initiative: Prescribe Design (“a network of healthcare natives and design professionals collaborating to re-imagine medical products and services”). On that post I also included a TED talk from Dan Ariely called “Are we in control of our decisions?“. That talk opened my eyes and I realized the importance of design and interaction to build successful product, services and interventions.

Predictably irrational book coverDan Ariely suggested that often, actually more often that we thought, human behavior follows a totally irrational direction and we do so in a repeatable and predictable way. One of the most interesting statements was:

“wake up in the morning and we feel we make decisions. We wake up in the morning and we feel we make decisions. We wake up in the morning and we open the closet and we feel that we decide what to wear. And we open the refrigerator and we feel that we decide what to eat. What this is actually saying is that much of these decisions are not residing within us. They are residing in the person who is designing that form.”

This theory is more detailed in his book Predictably Irrational, one of the most interesting books that I’ve read in the last times and that really helps to understand the decision-making process and how external inputs could influence in this process and in our behavior. The entry in the wikipedia has a good summary of the book. I found this book extremely interesting and useful and I tThe design of everyday things coverotally recommend it. As I read it, I realized the importance of a good HCI design in order to build succesful products, systems, intervention or whatever. This short video with an interview to Don Norman is a good summary of the ideas on the book.

General resources
There are a lot of useful resources in the Web about HCI, one of the most useful that I found during these years was this MOOC about Human-Computer Interaction in Coursera taught by Scott Klemmer. Now, this course has evolved into a Coursera specialization called Interaction design. I only took the course and not the specialization, it seems that they have expanded a little bit the content and the scope of the original course.

A quite interesting blog to follow is the blog of the Nielsen Norman Group (NN/g) a well known computer user interface (UI) and user experience (UX) consulting firm, founded in 1998 by Jakob Nielsen, Donald Norman and Bruce Tognazzini. One of my favorites or most useful articles on this blog are:

About the founders of the (NN/g), Jakob Nielsen is focus on usability and web usability and I’ll talk about him later discussing about how to measure usability. Meanwhile, I recommend his talk about “Mobile Usability Futures” during a Google Talk event and this other talk comparing the Web UX 2016 vs 2004.

On the other hand, Don Norman has carried out a most extensive work on interaction and user-centered design. He is the author of an absolute must-read book on interaction and design “The design of everyday things“. From the wikipedia:

“One of the main premises of the book is that although people are often keen to blame themselves when objects appear to malfunction, it is not the fault of the user but rather the lack of intuitive guidance that should be present in the design.”

He is the author of more books about HCI and interaction, the second one that is normally recommended (still on my wish list) is “Emotional Design: Why We Love (or Hate) Everyday Things” (here you have a TED talk on the topic 3 ways good design makes you happy). This topic is also linked with affective computing, an emerging area, specially due to the growth of the wearable technology which offer a novel opportunity for this field. From my point of view, the Affective Computing tools will impact and redefine many other areas and specially the the HCI. In one of the foundational books of this area, Affective Computing, written by Rosalind Picard, professor at the MIT,  presents many examples of how, by giving computer the ability to detect and interpret affective information the interaction with humans would be transformed. Some of the example proposed in the book are the development of

  • a cAffective Computing coveronsumer feedback (“gathering of physiological signals corresponding to responses such as frustration while a person is interacting with a product”)
  • the educational applications (“There is a saying in athletic training: No pain, no gain. The same saying applies to learning, although the pain is not demanding physically, but mentally”)
  • the classroom barometer (“This application involves constructing a simple device that accumulates the responses of the students in the room, displaying them up front for all to see. When a student is confused, his response anonymously helps make the barometer level climb.”)
  • the Audience Performance (“Successful entertainment induces emotion. Although “emotional inducement” may sound subversive, the fact is that people deliberately engage in it all the time, whether through selection of a piece of music to calm down, through watching a comedy to lighten up, or through participating in or watching an exciting sporting event.”)
  • music applications (“Listening to What you Like Often a radio station or a particular CD or tape is selected to match a person’s present mood, or to modify the mood they are in.”)
  • agents that learn your Preferences  (“If the system does something that annoys you, you have no way of letting it know short of hunting for some parameter you can change to try to get it to stop that behavior. Often, figuring out how to adapt the system to do what you want is too much trouble, so instead you adapt yourself to it.”)
  • learning when to interrupt  (“My husband and I were awakened the other night by his beeper loudly announcing that its battery was low. We keep pretty regular hours, and it should be relatively easy to redesign this device to be more considerate by simply having it look at its internal clock. But this solution does not apply to most problems of interruption, which are much harder to solve—such as whether the beeper or phone should alert you to a particular caller. For certain callers, the answer might be clearly yes or no”)
  • Sensitive Toys and improvement of the communications channels (evolve the emoticons)

Recently, I found some people that have made a very interesting work that somehow is related to the HCI field:

I liked the talk of Luke Wroblewski during the Conversions@Google 2015 about Mobile Design Essentials (part 1 and 2). And Conversions@Google 2016 “Obvious Always Wins”. You can also follow Luke on Twitter and G+, where he constantly uploads new content.

Dieter Rams, an German industrial designer who worked for Braun and Virsoe. His approach and belief in “less but better” design. Take a look at his portfolio with products from the ’60 to the ’80 and actually they will look familiar to you. He suggested also ten principles for good design:

  • Good design is innovative
  • Good design makes a product useful
  • Good design is aesthetic
  • Good design makes a product understandable
  • Good design is unobtrusive
  • Good design is honest
  • Good design is long-lasting
  • Good design is thorough down to the last detail
  • Good design is environmentally-friendly
  • Good design is as little design as possible

John Maeda (he is actually quite active on Twitter), author of the book “The laws of simplicity”  (and the website He’s book is also interesting about how to create “simplicity”. He proposes some 10 laws and 3 keys to achieve that.

  • Law 1: Reduce (The simplest way to achieve simplicity is through thoughtful reduction)
  • Law 2: Organize (Organization makes a system of many appear fewer)
  • Law 3: Time (Savings in time feel like simplicity)
  • Law 4: Learn (Knowledge makes everything simpler.)
  • Law 5: Differences (Simplicity and complexity need each other.)
  • Law 6: Context (What lies in the periphery of simplicity is definitely not peripheral.)
  • Law 7: Emotion (More emotions are better than less.)
  • Law 8: Trust  (In simplicity we trust)
  • Law 9: Failure (Some things can never be made simple)
  • Law 10: The One (Simplicity is about subtracting the obvious, and adding the meaningful)
  • Key 1: Away (More appears like less by simply moving it far, far away)
  • Key 2: Open (Openness simplifies complexity)
  • Key 3: Power (Use less, gain more.)

You can also take a look at his talk on this TED event.

Another interesting video is this TED talk of Tim Brown, CEO of IDEO about design thinking, he is also the author of “Change by Design : How Design Thinking Can Transform Organizations and Inspire Innovation“, a book on my wish list.

Finally, I present here some articles or links with guidelines or ideas for design on more specific areas:

Understanding the user and design for people’s needs

Something that is agreed among designers (in the wider sense) is that people’s tasks, goals, and values should drive the development. That means working with users throughout the process and putting them, their work, and their environment in the center of the design and paying attention to people’s abilities and situations. Actually, there is a well-extended framework of processes (not restricted to interfaces or technologies) in which the needs, wants, and limitations of end users of a product, service or process are given extensive attention at each stage of the design process called User-centred design (UCD).

User-centred design can be characterized as a multi-stage problem solving process that not only requires designers to analyse and foresee how users are likely to use a product, but also to test the validity of their assumptions with regard to user behavior in real world tests with actual users at each stage of the process from requirements, concepts, pre-production models, mid production and post production creating a circle of proof back to and confirming or modifying the original requirements. Such testing is necessary as it is often very difficult for the designers of a product to understand intuitively what a first-time user of their design experiences, and what each user’s learning curve may look like.

It’s actually an idea that comes from long, in 1981 the Bell Labs produced this video: Designing For People, Human Factors Research where they talk about how to incorporate human factors in the design process.

Also, there is an ISO standard for Human-centred design for interactive systems (ISO 9241-210, 2010) which is basis for many UCD methodologies, where  users, tasks, and environments are considered right from the beginning and included into the whole product cycle. Actually, there are many UCD approaches such as the SAP User-Centered Design (UCD) Process and also many implementations depending on the approach you want to follow. Anyway, from my point of view there are a few key steps that you should follow, choosing the appropriate tools or methodologies on each one that better fit on your workflow and that connect you with the final users. The key steps are:

  1. Research and analyze: Understand the context and the users needs
  2. Propose design solutions
  3. Evaluate
  4. Iterate

Basically, there are many methods to involve the users on the steps 1, 2 and 3 and by iterating this process it is possible to come up with a final product. I found this video which is quite illustrative about how the IDEO team works to redesign a shop cart, it is only quick example but I guess it transmit the message.

  1. Research and plan: Understand the context and the user needs

The very first step is to understand the context and the users you are defining for. It is important to understand the users, their context and to identify the unmeet needs. Actually, this simple idea it is not always so easy. There are many methodologies and tools to do that, both quantitate and qualitative, choosing the right ones will depend on the availability of resources and the kind of the development.

“You Can Observe a Lot Just by Watching”

—Yogi Berra

As this quote from Yogi Berra that I took from the Coursera HCI course says, observing your users provide you with a lot of information about their habits, their context, how they actually interact with the given system or product and it allow you to get an idea of their needs. Something more structured would be to carry out focus groups with different stakeholders, not necessarily the final users and not necessarily with a homogenous profile of participants. Nevertheless, it is important to have a clear, common and unique target in the focus group, e.g. in the healthcare field even if you are designing an application for patients, it is useful to carry out a focus group with patients, clinicians and healthcare providers. Even better, you can carry out two different focus groups, one to identify the needs of the patients and other one to identify the “clinical” goals. Here you have a good read about “The Use and Misuse of Focus Groups” written by Nielsen. To complement the focus groups and the observation of the users and their behaviors, it is possible to gather information from personal interviews or questionnaires.

If you need more quantitative data, in the last years, with some colleagues at the UPM we have been using the Analytic Hierarchy Process (AHP) as a method to elicit user needs. TGlobal weights of the different user needshis method is quite useful in complex environments (such as the healthcare field). Essentially, the AHP breaks down a complex problem into simple pairwise comparisons and a groups of experts in the domain are requested to evaluate each of these comparisons. The maths behind the AHP automatically calculate the inconsistency in the user responses (this is something normal in a complex decision-making process) and feedback him about how to reduce the inconsistency is his responses. You can take a look at this Open Access work as an example “Using the Analytic Hierarchy Process (AHP) to understand the most important factors to design and evaluate a telehealth system for Parkinson’s disease” or this other “OB CITY – Definition of a Family-Based Intervention for Childhood Obesity Supported by Information and Communication Technologies” where I and other colleagues used AHP to figure out the user needs. At the end, you get the list of your user needs with an aggregated weight according to the user responses. This is useful not only for to make decisions on the design of the system but also to evaluate already existing systems. After the research phase it is necessary to put all the information together, personally, there are four tools that I like, and that can be used all together or just some of them depending on the application.

User journeys, I found this article quite useful in the definition of user journeys and their application on User research.
This article from Sarah Gibbons at Nielsen group, make a comparison of different UX Mapping Methods (Empathy mapping, Customer journey mapping, Experience mapping, Service blueprinting) which is quite insightful.

User journey mapping is one of the tools that UX pros use to find and address the weakest points in customer experience with the product. It helps visually articulate the key touchpoints where users interact with your product and analyze the user experience at each stage. The map itself looks different for each project, that’s why some professionals prefer to refer to it as “journey modeling”. Depending on its goal a user journey may have various sections, but generally it highlights typical user behavior step by step for each channel (website, physical store, call center, mobile app, etc) or simply highlights user’s actions, thoughts and ideas at each step.

Storyboard it is also an useful tool for HCI, here you can find some templates to design your own storyboard.

Storyboarding is a common technique in HCI and design for demonstrating system interfaces and contexts of use. […] As computing becomes increasingly integrated into the environment, blurring the distinction between the system and its surrounding context, it is imperative to depict context explicitly in storyboards. In this paper, we present two formative studies designed to uncover the important elements of storyboards. These elements include the use of text, inclusion of people, level of detail, number of panels, and representation of the passage of time.

Persona, my favourite guide to understand what a Persona is and how to use this tool, is this guide published in Smashing Magazine and divided in Part 1 (A Closer Look At Personas: What They Are And How They Work) and Part 2 (A Closer Look At Personas: A Guide To Developing The Right Ones)

A persona is a way to model, summarize and communicate research about people who have been observed or researched in some way. A persona is depicted as a specific person but is not a real individual; rather, it is synthesized from observations of many people. Each persona represents a significant portion of people in the real world and enables the designer to focus on a manageable and memorable cast of characters, instead of focusing on thousands of individuals. Personas aid designers to create different designs for different kinds of people and to design for a specific somebody, rather than a generic everybody.

Use Cases, there is a good description on the

A use case is a written description of how users will perform tasks on your website [or system].  It outlines, from a user’s point of view, a system’s behavior as it responds to a request. Each use case is represented as a sequence of simple steps, beginning with a user’s goal and ending when that goal is fulfilled.


2. Propose design solutions

Prototyping is a strategy for efficiently dealing with things that are hard to predict. And, even better than generate one prototype,generate as many as you can.

“The best way to have a good idea is to have lots of ideas.”

-Linus Pauling

Parallel prototyping is a HCI methodology where different prototypes are designed in parallel. When designers create multiple distinct prototypes prior to committing to a final direction, several important benefits arise as it was suggested by Hartmann et al. 2008 (see “Design as exploration: creating interface alternatives through parallel authoring and runtime tuning”): 1) a more complete understanding of a design space; 2) more effective, efficient decision making; 3) discussing multiple prototypes helps project stakeholders a better communication and 4) presenting multiple alternatives in user studies facilitates participants’ ability to understand design tradeoffs and offer critical feedback.

The type of prototype used will depend on the nature and status of the system, from low-level mock-ups (Balsamiq is my favorite tool to do that) to functional prototypes or Wizard of Oz for more advanced phases of the development. Parallel prototyping is even useful to design auditory cues, for example this work that we did a couple of years ago “Designing auditory cues for Parkinson’s disease gait rehabilitation” (Examples).

Also, brainstorming sessions are a typical recomendation at this point. To organize this kind of sessions I like to use MindMaps, there are many tools for Desktop or online to do that. A couple of them that I like are online: MindMeister and Coggle.

3. Evaluate

In the UCD involving users in the evaluation is essential, nevertheless, their time and the tests with users require a significant effort, for that reason, it is important to arrive at that point with a good prototype. For that reason, I find extremely useful, tools such as the Heuristic evaluation, created by Nielsen. It is an usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process originally developed in collaboration with Rolf Molich in 1990 (Nielsen and Molich, 1990). A further refined from a based factor analysis of 249 usability problems ( Nielsen 1994) to derive a set of heuristics with maximum explanatory power, resulting in this revised set of heuristics (Nielsen 1994b)

Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the “heuristics”). It is important to note that, this is not an usability test, it helps you to identify common and major usability problems quickly and effortless. Also, in my experience, it is useful to have this list of heuristics in mind while designing, not only interfaces but any system. This is the list of the 10 heuristics:

1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation

Normally, for an Heuristic evaluation session it is recommended to involve three to five evaluators since one does not gain that much additional information by using larger numbers.

Measuring the User Experience coverAfter removing the major issues with the Heuristic evaluation it is time to involve your users and to carry out tests to see how the interact with your system, which is his actual behavior when they use your system. Again, there are many methods and tools, and they one that you need will depend on the specific test that you are interested on. My favorite book on this topic is Measuring the User Experience (and its website). It is a very pragmatic guide that helps you to identify the tools that better fit on your problem and guide you to use them and to analyze their results.

The table beyond is from that book and it is summary of the type of tools and methods that you should use to address specific usability problems. Again, the book is focus on usability of websites/GUI but most of the ideas can be extrapolated to other systems.

Measuring User Experience

Two tools that I would like to highlight because I have used them extensively:

System Usability Scale, it is a simple and quick scale but its results are widely accepted and it provide a quick overview of the users acceptance.

The affinity diagram, as described by Beyer & Holtzblatt. It is a bottom-up method that helps to organize comments, generate design ideas and to raise relevant issues from a large set of opinions.

Finally, most of the tools presented here have been designed or used in the context of UX for websites and GUIs. Nevertheless, most of the ideas and most of the methods can be extrapolated for their use with ottools for wearability assessmenther kind of systems. But, a couple of years ago we need to assess the aceptance of wearable a Body Area Network of wearable sensors, so we come up with a set of already validated tools to do that such as the CRS (Comfort Rating Scales), REBA (Rapid Entire Body Assessment) and the Borg CR scales. For more details you can check the original work “Wearability Assessment of a Wearable System for Parkinson’s Disease Remote Monitoring Based on a Body Area Network of Sensors“. It is an Open Access article so there you can read the description of the tools and teh references to the original description.

Human behavior change

BJ Fogg's Behavior ModelThis is a section that I’ll update with more detail in the future since it is something that I’ve started to explore quite recently.  Human behaviour change is a field of study with special interest for myself in the healthcare area. The idea is try to create a system to modify some unhealthy behavoirs and to promote new or more healthier ones. Actually, this kind of approaches have been used in practice for a long time for stop smoking or weight loss. There are many theories that try to create a framework to design this kind of interventions, in the last years I have started to work with two of them, trying to design mHealth interventions for Human behavior change in the health area. I particularlly like the Fogg Behavoir Model:

The Fogg Behavior Model shows that three elements must converge at the same moment for a behavior to occur: Motivation, Ability, and Trigger. When a behavior does not occur, at least one of those three elements is missing. […] The FBM highlights three principal elements, each of which has subcomponents. Specifically, the FBM outlines three Core Motivators (Motivation), six Simplicity Factors (Ability), and three type of Triggers. The subcomponents define the larger elements. For example, in the FBM the word Ability refers to the how the six Simplicity Factors work together in the context of a Trigger.

This models has been largely used in many works and health areas and there are also some works usign ICT solutions based on it.

According to Fogg, persuasive technology uses seven strategies to influence behavior: reduction, tunneling, tailoring, suggestion, self monitoring, surveillance, and conditioning.

  • Reduction – simplifies a task that the user is trying to do.
  • Tunneling – guides the user through a sequence of activities, step by step.
  • Tailoring – provides custom information and feedback to the user based on their actions.
  • Suggestion – gives suggestions to the user at the right moment and in the right context.
  • Self-monitoring – enables the user to track his own behavior to change his behavior to achieve a predetermined outcome.
  • Surveillance – observes the user overtly in order to increase a target behavior.
  • Conditioning – relies on providing reinforcement (or punishments) to the user in order to increase a target behavior.

Some related articles:

Aesthetics for Design
Japanese rock garden

This is another section that for sure I’ll update in the future, bascailly it is aimed at providing a collection of aesthetics styles for design that have been succesful on the past or that traditionally have been considered as appealing or interested: