More broadly, this research involves comparing analyst behaviour and processes across domains (intelligence analysts, journalists, bioinformaticians, aerospace engineers). Thankfully there are now countless industries and research areas working with large datasets.

The research group I belong to at UBC is one of many that develop applications that facilitate these processes, often in the form of interactive information visualisation tools.

“The purpose of visualization is insight, not pictures” - Ben Shneiderman (1999)

These tools are built for providing insight into the data. In the following visual conceptual map, I’ve decided to unpack this term and its related concepts (a work-in-progress!).

Given these relationships, I’m beginning to frame my research with the following high-level questions:

1. What constitutes insight, or a unit of discovery? When is insight the means hypothesis generation) or the ends (hypothesis validation)? Can we establish a shared interpretation of insight that traverses domains?
2. How do existing data analysis and visualization practices facilitate insight via different types of problem-solving strategies? Where do they break down? How is collaborative analysis and problem-solving supported?
3. How does one develop expertise as an data analyst?
1. How does a novice analyst initially explore a dataset? How is this different from how an expert analyst would explore a dataset?
2. How does collaboration between novice and expert analysts, or between novice analysts, effect the development of expertise?
3. Are there commonalities across domains with regards to how one learns to use information analysis tools and techniques?
4. How does one learn domain-independent analysis tasks (those that span multiple tools and techniques)?

I’m now five months into my PhD. The next step involves putting these high-level questions into action, using them to frame concrete questions that I will put to data analysts in different domains.

AUTHOR’S NOTE

This post (and the previous post) was adapted from an assignment I submitted for a graduate course in qualitative research methods at UBC.

epistemology: the theory of knowledge, esp. with regard to its methods, validity, and scope.

I’ve found that Human-Computer Interaction [HCI], a multidisciplinary field of study, inherits the dominant epistemologies of the disciplines that comprise it. I admit this is what drew me to the field, the opportunity to wear many hats: the cognitive scientist, the industrial/graphic designer, the computer scientist, and the ethnographer. Each role asks a different set of questions dependent on the current research goal and the stage of a project. As a result, knowledge can be discovered at one stage, while it must be constructed at another.

The cognitive scientist

As a cognitive scientist, I am interested in issues relating to perception, cognition, and behaviour. An understanding of these issues is critical for justifying the design of future technologies, or for explaining behaviour with existing technologies. I could ask questions about low-level sensory perception, such as “when an individual is presented with a simple [visual, auditory, or tactile] stimulus, what is their response?”.

Or I may ask about mid-level processing, about the recognition of patterns, of aggregates of simple stimuli (i.e. “Does the set of points in a scatter plot suggest a correlation between variables X and Y?”). Answering questions at these levels often have direct implications for a technology’s usability: how efficiently and correctly can it be used?

Finally, I may ask questions about higher-level processes, those spanning minutes or longer, about divided attention and an individual’s (in)ability to perform multiple tasks concurrently (i.e. “How do software developers effectively write code while maintaining instant messaging conversations?”).

I’ve been fortunate to participate in several research projects that ask these types of questions. In each case, the typical means by which these questions are answered is via post-positivist hypothesis testing and controlled laboratory experiments. In this sense HCI researchers inherit the objective epistemology of cognitive psychology.

Limitations of objectivity

Several of my recent research projects in this highly post-positivist vein have pertained to issues of multi-tasking. During the course of these projects I have begun to question the appropriateness of an objective post-positivist stance.

A significant component of my M.Sc research was devoted to studying user performance on early prototypes of a self-administered computerised web-based cognitive testing application, to be taken at home by older adults as an initial screening test for age-related dementias. Our research group acknowledged that the home is unlike a quiet clinic office, the setting where cognitive testing is presently conducted. In particular we assumed the home setting to be one in which distractions and interruptions could inhibit performance on such a cognitive test, a test that demands full attention and involves time-sensitive responses. Foreseeing these problems, we decided that a deeper understanding of the interaction between user age and interruption type. We believed this to be necessary for the purposes of designing interventions for preventing, detecting, and mitigating interruptions within the application itself. We selected an approach grounded in prior research in psychology and HCI: we conducted a controlled laboratory experiment in which research participants from different age groups performed cognitive testing tasks, interleaved with short interrupting working-memory tasks, intended to simulate a range of possible interruptions which might occur in the home.

Our results were unexpected and largely inconclusive. Upon reflecting on this outcome I’ve realised how a strictly post-positivist perspective limited what we could study and how we could study it. For instance, we could not guarantee a shared understanding of what cognitive testing meant to our different groups of participants (I expect this to be a more sensitive and significant topic for our oldest group). Moreover, we did not have a shared understanding of what constituted an interruption, how it may occur in the home, and how one may respond to it naturally (as opposed to in an experimental setting), particularly when one’s cognitive health (a sensitive matter, of course) was being assessed.

I now acknowledge how higher-level cognitive processes, such as multi-tasking and handling interruptions, are highly dependent on context, on a mutual understanding of that context and the processes involved, that these must be constructed with our research participants. While I maintain that issues of low-level sensory perception and mid-level pattern recognition ought to be studied with a post-positivist perspective, the study of higher-level cognitive processes may call upon a mixture of theoretical perspectives.

The designer

An understanding of what is aesthetically pleasing is an important asset for the HCI researcher or practitioner. I argue that this too, requires a mixture of epistemological perspectives.

As we design interface technologies intended for human interaction, we rely on a history of established graphic and industrial design guidelines, as well as familiar interaction techniques, those that are known to provoke a positive response from users. Many of these positive responses are supported by experimental findings in the cognitive sciences. In a sense we know what is considered pleasing, elegant, intuitive, or natural because it correlates with what is efficient and accurate. We objectively know which guidelines and techniques work based on what sells, on market research and survey studies.

However when working in a new medium it becomes necessary to challenge the aforementioned constructs: of what is pleasing, intuitive, etc. Handheld touch screens and large shared public displays are recent examples of how new technological mediums have redefined what was previously considered to be an intuitive interaction, just as hypertext, the mouse, and the graphical interface did in decades prior.

Once again, the HCI practitioner, in their role as a designer, must be able to shift their epistemological perspective based on the medium they are currently working in. This could entail a constructionist perspective, observing individuals and groups as they engage with a new medium, as well as discussing and negotiating what these constructs have come to mean in light of their experiences.

The ethnographer

ethnography: the scientific description of the customs of individual peoples and cultures.

A pivotal role of the HCI researcher-practitioner, one in which I’m currently assuming in my PhD studies, is one of an ethnographer. We exhaustively study users and their day- to-day context before designing a technological application. Similarly, we also study these users and their contexts after these applications have been deployed.

Requirements analysis is a process that begins when a client, often a representative for a group of employees or customers (the target users), approaches you with an ill-defined problem, a perceived need for a technological intervention.

It is our task to study the existing workflow of these individuals and establish a common understanding of these tasks and their context. We then negotiate the requirements of a technological intervention, which problems or needs it will address, and the relative importance of these needs. A mutual understanding must also be reached in terms of whether such an intervention will replace an existing workflow, add to workflow, or if instead it will provide an alternative to the workflow.

At the other end of the project timeline, following the deployment of a technological application, we study whether and how users’ workflows have changed. We ask whether the intervention was successful, as well as what constitutes success and failure for each party involved. Similarly we examine whether the application was perceived to be useful, at the same time determining what constitutes the constructs usefulness and uselessness (in other words, utility).

Before development and after deployment, it is apparent, at least to me, that the HCI researcher-practitioner must maintain a constructionist perspective. While I expect that an objective survey approach could accumulate a large amount of information regarding workflows, use cases, application requirements, and perceived utility, too much is lost. There are simply too many constructs that need to be negotiated.

Final Thoughts

At a glance, it appears messy and if as though HCI researcher-practitioners, myself included, suffer from split-personality disorder. We adopt an objective perspective for one project and a constructionist perspective for another. Often this perspective changes within a project. This is a result of the multidisciplinary nature of the field. I’m not worried about this. We may construct a use case that requires developing a novel interaction technique, one that must be evaluated objectively. If anything, I view this ability to shift perspectives as adaptive behaviour.

Author’s Note

This post (and the following post) was adapted from an assignment I submitted for a graduate course in qualitative research methods at UBC.

You can stream it and download it for free at wearedashing.com. We’d love to know what you think, so why not leave us a note while you’re there.

Hello Cosmonaut EP by WeAreDashing  

• Written and produced by DASHING © 2011
• Mixed and mastered by Jesse Karr at the Hive Creative Labs in Burnaby, BC
• Dashing is Daniel Puzzuoli, Eric Lajoie, Matthew Brehmer, Peter Beshai, Ray Wang, and Thomas Hazelton

Dashing – Hello, Cosmonaut EP Teaser by WeAreDashing

Available at archive.org here (20 MB zip).

1. Inevitabilities
2. Daylight Narrows
3. The Mercury Drops
4. Passengers
5. Floodlights