Yvonne Rogers and John Long – Diablog

Yvonne 1 (11 May 2014):

John Hi!

Here are my MSc student reflections. What do you make of them?

Date of MSc: 1982-83

Project Title: An Exploration of Compatibility Problems Found in Everyday Situations

Pre-MSc Background: BA Psychology, University of Wales

Pre-MSc View of HCI/Cognitive Ergonomics:

I first came across Ergonomics when I took a third year option in my undergraduate degree at Swansea University. We were introduced to the notion of man-machine interfaces and the importance of understanding people (from a cognitive, organisational and social perspective) when evaluating how effective technologies were for work settings. I became fascinated from then on with understanding how people and computers could work together in new and symbiotic ways.

My early experience of computers occurred whilst there were big changes afoot; first, I started learning to program using a mainframe and punch cards, then started using a workstation for doing stats tests; and then spending hours on a BBC microcomputer running psychology experiments but also playing lots of video games. HCI was just emerging as a field and I had no idea what it was.  But my journey moving from a non-interactive machine to a highly enjoyable user experience set me up for understanding what makes for a bad and good interface. That has stayed with me ever since.

My third year undergraduate project was concerned with measuring different forms of information processing for cognitive and motor tasks when under the influence of alcohol and caffeine. It involved asking a number of students to drink a large amount of vodka and orange early in the morning followed by a cup of strong coffee to see how their interaction affected their motor and cognitive performance. The findings from this study were surprising; dispelling the myth that coffee sobers you up. Instead I found it made reaction time worse. Even more surprising, was receiving the Undergraduate Award for best dissertation at the Ergonomics Conference in the following year. This recognition spurred me on to greater things; wanting to know more about how human performance is affected by context.

In sum, I really didn’t have much of an idea for what I had signed up for when embarking on the Masters Course in Ergonomics at UCL. But instinctively I knew it was right for me.

Post-MSc View of HCI/Cognitive Ergonomics:

The Ergonomics MSc at UCL opened my eyes to the value of studying many different subjects rather than only delving deeply into one. Every day, we traipsed to a different London college to study the various contributions to Ergonomics; for example, studying lighting at the Bartlett, physiology at Chelsea College, biomechanics at the Royal Free and Cognitive Psychology at Birkbeck College.  Being exposed to so many different areas and cultures (‘old school’ Birkbeck was quite different from ‘new medical school’ Royal Free) could be overwhelming at times. But it paved the way for new insights, instilling in me why and how multidisciplinarity is central to HCI and Ergonomics when trying to frame questions and generate new ideas in the context of understanding the relationship between people and technology. I was also able to study a few subjects in more depth, such as cognitive psychology and organizational psychology. This enabled me to explore more theory, learn how to model users and conduct experiments to investigate the usability of user interfaces.

What has stuck with me most from my time on the Masters degree are my fond memories of the many visits we went on as part of the course to industrial places, such as Wall’s factory (where they make sausages), a now extinct coalmine in the Midlands and a control centre in the London Underground. We learnt so much more about real people, work and machinery than you could ever put across in a lecture.

Subsequent-to-MSc View of HCI/Cognitive Ergonomics

After obtaining my Masters degree I became increasingly interested in technology, interfaces and interaction design. I knew I wanted to continue studying after completing the course. I got a job as a research demonstrator and begun my PhD in earnest, investigating the cognitive, semiotic and aesthetic properties of graphical representations, with a particular focus on iconic interfaces. It was exciting to be at the start of a new zeitgeist.  I was inspired to think about future interfaces – having battled for so long with command-based interfaces.  The field of Human-Computer Interaction (HCI) came into its fore and I became part of that movement, exploring how to augment and extend a diversity of human experiences with new technologies. While I continued to have an interest in Ergonomics, for me, the action and excitement was now in HCI.

Additional Reflections

In September 2011, I took up the directorship of UCLIC, following in Professor Ann Blandford’s footsteps. She had done an excellent job during the previous 6 years overseeing the HCI and Ergonomics Master’s course, keeping it up-to-date, while expanding it to match the changes taking place in the field. UCLIC has grown and changed considerably since when I remember it as the old Ergonomics Unit back in the 80s. In the beginning there were about 15 students each year on the course. Now, there are between 30-50 students per year from all over the world. I am always amazed at the backgrounds, skills and previous experiences of our students. This includes music, media, philosophy, computer science, languages, psychology and history of art. It makes for an eclectic and vibey mix.

There is a world of difference when looking back between my time on the course and the current course.  For one, the student experience is very different. The course is more integrated in what and how it teaches the different strands of HCI and Ergonomics. Technology is central to everything, from the way we teach, what we teach and how the students learn. Many of the modules are more practice-based. The students have access to fantastic online learning resources. They also learn how to use a number of software tools that are industry standard so they are better equipped to go into the world of UX.

Sadly the visits are no longer – it is simply too impractical, time-consuming and expensive to organize for 50 students each year.  One legacy that remains is the course being available to students who want to study it part-time. We still get a number of students who work in a diversity of industries taking this route. It is one of our strengths to be able to mix a full-time with part-time student experience, so both can benefit.

At first it felt strange to be on the other side of the fence at UCL with such strong memories of my time here before; being the professor and the director now instead of the student. But it did not take long for me to fit into my new leadership role. My vision is to continue to grow UCLIC and evolve and update the Masters course to meet the ever-changing needs of industry and academia. Right now we are in the middle of revising the whole course. We have lots of discussions about how we might achieve this. As part of that process, we want to introduce more design thinking, physical computing and prototyping. It is a joy to address the many challenges and take up the opportunities that come our way while retaining the legacy, specialness and quality of the old Ergonomics Masters course. (Date 25 April, 2014)

John 1 (24 may 2014):

Yvonne Hi!

Great Reflections – MSc student to UCLIC Director! Can’t go wrong. Also, lots of interesting then-and-now contrasts: EU to UCLIC; Ergonomics to HCI; and original MSc to the present one. All good diablog fodder for the future, no doubt. However, for now, I’ve opted for your challenges.

You identify three challenges – growing UCLIC; revising the MSc; and satisfying industrial and academic masters. Surprisingly perhaps (or not), we had the same challenges, when I took over the EU in 1979. I think we met them (in all modesty, of course). However, there were also some serious residual problems. Of interest to you today? If so, your thoughts please.

Your first challenge is to continue to grow UCLIC. Fine; but how and by how much; to what end; and in what institutional form (for example,(sub-) department; institute or other?) We grew the EU research from scratch to 15 or so researchers and PhD students. At the same time, the number of MSc students increased threefold to over 30. So, grow we did. However, we notably failed to establish a promotional path from EU researcher to (Psychology) department member. As a result, the continuity of researchers on ‘soft’ money suffered much to the cost of the research. Any thoughts?

YR: Growing pains? A research centre needs to keep growing to keep on its toes and be able to address the new challenges afoot in research and industry. However, there is the danger that it can grow too much or too quickly. UCLIC is already in a healthy state with a good balance of faculty, researchers and students. Therefore, I see any further growth as being incremental. Since arriving, we have added 2 new lecturers and two teaching fellows to the mix. That has helped take the weight of existing staff so they can apply for more grants and do more research. It also means we can offer a wider range of courses and provide the students with more face-to-face time, without feeling harried.

I would like to see us expand further over the next few years based on our needs and successes – rather than simply grow bigger based on the old adage more is more. If we are successful in obtaining some substantial grants then we will look for new posts. Areas we are looking into for new funding include digital health, personal data/data analytics and the Internet of Things.

YR: The perennial problem of soft money. Unfortunately, the British system has a poor track record of making researchers permanent that begin on soft money. However, I always think it is much better for people to move on to another institution, after completing their post doc and PhD posts. The trouble is that there are very few lectureships or permanent research positions advertised in HCI and so it is much harder for the bigger pool of HCI PhD students that graduate each year.

JL:  We first integrated HCI into the applied sciences of multi-disciplinary Ergonomics as Cognitive Ergonomics (that is, applied Cognitive Psychology). However, as the EU  conducted its HCI research and practice as engineering (design-oriented models and methods), HCI was developed first as an MSc option and then as a specialist HCI MSc degree (albeit with Ergonomics as minor). The residual problem was two-fold. First, how to integrate HCI, as engineering with Ergonomics as applied science(s)? Second, how to integrate the Human element of HCI with the Computer element both in terms of models and in terms of methods? Any ideas?

You mentioned you are evolving and revamping  the Masters course, particularly in terms of design thinking, physical computing and prototyping. These directions are a good fit for Computer Science, where UCLIC currently resides. But what about Psychology – the other UCLIC parent?

YR: Which direction? This is a difficult question. HCI is now multi-faceted. I think it is just as important to learn about design and social science skills (e.g. ethnography) as it is to have a good understanding of psychology and computer science. We already offer a good balance of cognitive science/psychology and ergonomic courses. What we don’t offer in my view, ironically, is enough computer science in our courses. Our move into physical computing is an attempt to redress this balance by offering a hands-on course where students will learn basic electronics, programming, prototyping and further design skills that will enable them to build functional systems that can be tested and situated in the real word.

We are also thinking of offering specialized Masters Courses besides the existing HCI and Ergonomics, such as HCI and digital health; HCI and data analytics; HCI and Security. I think having a deeper understanding of an application area or methodology that HCI can be applied to may be attractive to students who want to work in those areas.

JL: Your third challenge is how to meet the ever-changing needs of industry and academia. We met industry’s needs in two ways. Our MSc students never lacked for jobs. The EU attracted considerable industrial funding and acquired a reputation for the quality of its research. The residual problem here was in maintaining an HCI research output (for example, HCI models and methods) in the face of industry’s desire simply to outsource its development work to the universities. It was a permanent struggle. We met academia’s needs by producing ‘active’ RAE researchers and HCI publications of some originality and quality. The residual problem, here, was that although our publications appeared in top-rated HCI journals (like Interacting with Computers), they did not appear in top-rated Psychology titles (like Psychological Bulletin). The former was considered to be ‘applied’ and the latter ‘theoretical’ – to the latter’s benefit. Our RAE profile inevitably suffered as a result.  Any ideas?

YR: To be or not to be? Most of our students go into industry after completing their Masters degree. We have many companies who come to us throughout the year offering very attractive UX positions. There is a shortage of good UX professionals and we help to fill that gap. We also cater for those who want to continue with their studies and pursue a PhD. We think we do a good job of preparing them for understanding and being ready for what is in store for further academic study. Several supervisors also help them to write up their Masters projects as CHI papers after the course has finished – which several have been accepted.

To answer the second part of your question: for the last REF we were submitted as part of Computer Science rather than Psychology. Certainly, that has had an impact on where we publish. There are several members of the current Computer Science REF panel who have a HCI background or leaning towards, so it is clearly an area that is considered central to computer science. I think we have a very strong return this year (the majority of our submissions were in top HCI journals and conferences) but the price has been that our presence in the psychology literature has suffered. But this is symptomatic of the way HCI more globally has been heading for the last few years – and where it has to be said sits most comfortably. There are very few Psychology departments now in the world that offer HCI degrees. It does not seem to fit alongside their other applied courses in psychology. Which in my view is a great shame as understanding people and behaviour continues to be integral to much of what HCI is about. Somehow, our relationship with technology has slid down the pecking order of what is considered central in psychology; it is generally viewed as equipment by which to measure and diagnose rather than as tools to augment and empower people.

JL: Lastly, aspects of meeting the three challenges obviously interacted. As concerns the successes, the growth of research facilitated the development of the MSc as HCI, and its turn to engineering and design increased its attraction to industry. Unfortunately, interaction also occurred between aspects of the failures. The turn-over of researchers on ‘soft’ money did not support either the continuity of developing HCI theory as models and methods or the publication of papers to satisfy academia.

As a final aside, how are you going to retain the ‘legacy, specialness and quality of the old Ergonomics Masters course’ – your words. Even I would not have had the temerity……

YR: Legacy, specialness and quality OK, that turn of phrase was maybe a bit cheesy but it was sincere and heartfelt. The reputation and branding of the Ergonomics Master is second to none. A few weeks ago another former graduate from the old course emailed me and wrote about how instrumental it had been to his illustrious career

John (18, July, 2014)

Yvonne Hi and thanks for this. Looks like the diablog might be taking off…….

A couple of brief comments, before moving on to the Liu et al CHI’14 paper challenge that you suggest.

You are certainly rising to the three challenges of growing UCLIC; revising the MSc; and meeting the needs of industry and academia. Interesting, though, how similar residual problems, as those experienced by the EU, remain: respectively soft money and the promotional path for researchers; the integration of the human and the computer elements of HCI; and the position of HCI with respect to Psychology and Computer Science, as concerns the RAE (but not the provision of MSc students to industry).

As it happens, I would have been pleased to follow up these problems in greater depth and with a view to identifying possible solutions. For example, as concerns the second challenge of how the human and computer might be better integrated, by increasing the amount of Computer Science in the MSc, along the lines that you suggest. Alternatively (or in addition), an attempt could be made to attract, and to cater for, more software engineers on the course. Some of my best research was carried out with such people (for example, Adam Stork – see this website).

These residual problems could also be considered with respect to their impact on the quantity and quality of the HCI research (funded by both public and private grants), which results. For example, how to maintain the quality of research in the face of industry’s desire to outsource its R&D to the universities. This might link up nicely with the issues, concerning the possibility of research leading practice, which you raise in your recent ‘HCI Theory’ book.

However, looks like we are moving the diablog on, which is also fine by me. Although, it is interesting to note, that Liu at al’s paper also raises issues, concerning the types and nature of HCI research. You challenge me to say what I think of their ‘brash’ (key-word based) claims , that there seems to be no well-defined way to study a new technology in the context of HCI and that this is resulting in it becoming a fragmented field. It may be a bit late in the day for me to rise to new challenges; but I will do my best (once I have read the paper with the due care and attention, which it deserves). Watch this space; but don’t hold your breath.

OK, now it is my turn to start a new dialog.

YR: There was a paper presented at the CHI conference earlier this year in Toronto that caused quite a stir. It was called  “CHI 1994-2013: Mapping Two Decades of Intellectual Progress through Co-word Analysis” by Yong Liu, Jorge Goncalves, Denzil Ferreira, Bei Xiao, Simo Hosio and Vassilis Kostakos – all from Finland. They had mapped the CHI output by analysing the keywords chosen by authors to represent their papers for the last 20 years from 1994 to 2013. They used hierarchical cluster analysis, strategic diagrams and network analysis on co-words.

The paper itself is written in a rather dry form of bibliometrics. One wades through pages of graphs, stats and some inferences they make about what has happened in the field during these two decades of CHI papers. But then towards the end, where they start to draw conclusions based on their analysis, it completely changes gear. The discussion and conclusions they present seem out of the blue and are controversial and highly debatable – and is what caused the stir during their CHI presentation.

They argue, based on their bibliometric analysis, that:

“the only tradition in HCI is that of having no tradition in terms of research topics”.

“when a new technology comes along it seems that researchers start from scratch leading to relatively isolated research themes”

“current HCI research is lacking motor themes and will also lack motor themes in the near future.”

They conclude by saying that there seems to be no single well-defined way to study a new technology in the context of HCI and this is resulting in it becoming a fragmented field.  My challenge to you is: What do you think of such brash claims – given they are based on a bibliometric analysis of the keywords authors add to their papers to get noticed in the blogosphere– does it really give us a true picture of the history of HCI over the last 20 years of HCI? How else might we take stock?

John (14/8/2014)

Yvonne Hi!

Here’s my attempt to rise to your challenge, concerning Liu et al’s CHI ’14 paper. 

     – Is the paper interesting?

Members of the HCI research community, like any other, need different perspectives, within which to view their own work and to relate it to that of others. Familiarity with the field of HCI, through journals and conference papers, provides researchers with  such perspectives. For example, in my case, I am well aware of developments in the areas of multi-tasking; cognitive modeling; and user-interface design. Less familiar, however, with eye-tracking; virtual reality; and digital libraries. However, the relations between these research topics are implicit, for example, multi-tasking and cognitive modeling, including their change over time.

The Liu et al paper, then, is interesting, because it offers an additional perspective within which researchers can locate their work. The perspective is expressed in the form of research themes and their relations, based on the keywords of published CHI papers. The perspective has the advantage of being explicit. It offers us, then, an additional opportunity to reflect on our own work with respect to that of others. The survival of Fitt’s Law as a theme might have lessons for us all.

     – What are the paper’s strengths?

First, it collects together a complete list of research topics, as identified by the keywords in CHI-published papers from 1994 to 2013. Second, it analyses the research topics into clusters, for example, hypertext; input devices etc. Third, it proposes dimensions to relate the clusters, for example, density; centrality; frequency; time (1994-2003 and 2004 – 2013); and core-periphery. 

In conclusion, the paper provides a novel and explicit perspective for researchers to locate and relate their work to that of others.

     – What are the paper’s weknesses?

First, the keyword source is limited to one conference – CHI. Second, the keyword-content relationship of papers is not assessed; but only assumed. Third, the assignment of keywords to papers is certainly driven in part by a number of different biases, for example, to relate more to emerging or popular research themes and to appeal to CHI reviewers and audiences. Fourth, there is no explicit framework for relating the research themes analysis to the conclusions (see also later). Last, there is a rather cavalier use of key concepts, such as ‘paradigm’; ‘discipline’; ‘tradition’ etc. Such terms need to be grounded, for example, paradigm following Kuhn, or newly defined.

In conclusion, these weaknesses are damaging to the paper’s creditability as it stands, although they might possibly made good in the future.

     – What are the paper’s conclusions?

‘First…..the only tradition in HCI is that of having no tradition in terms of research topics’.

The use of ‘tradition here is at best curious and at worst misleading. It is neither defined (see also earlier), nor is it explained, if and how it relates to the dimensions of centrality, density, and time. If ‘tradition’ here means something like ‘continuity’, then the conclusion would appear false. Research topics such as ‘usability’, ‘user interface design’, ‘cognitive modeling” etc self-evidently have a long ‘tradition’ (continuity), which is unlikely to suddenly come to an end. If ‘tradition’, as used here, means something else, then it needs to be defined, along with the level  of description of its use, and to be related to the requirements for HCI discipline progress. Emerging, declining and merging research topics, as identified by the data, would be expected in any developing discipline, including HCI.

Second, ‘HCI has a long enough history for knowledge to accumulate……when a new technology comes along, it seems researchers start from scratch….’

Although I happen to think that HCI has a poor record of generalising and so incrementing its knowledge base (see also later), the above conclusion in no way follows from the research theme key-word analysis. The latter makes no reference to ‘knowledge’ or indeed its acquisition or its application by practitioners. Further, methods used on ‘old’ technology (like keyboards) are often used on ‘new’ technology (like touch screens), for example ‘time and error analysis’.

Third, ‘…..different approaches or perspectives are adopted, when studying a new technology, leading to a relative fragmentation within HCI.’

Again, it is unclear how this conclusion follows from the research topic key-word analysis. The relationship with the dimensions of centrality and density, as well as with core-periphery analysis (which might be thought to be relevant) are not referenced. Further, the different approaches or perspectives are neither defined nor identified. HCI is indeed fragmented, in my view,; but that has more to do with the absence of agreement as to its scope and its discipline problem, than different approaches per se.

Fourth, ‘There seems to be no single well-defined way to study a new technology in the context of HCI’.

Again, this conclusion does not follow from the research topic key-word analysis. The latter makes no explicit reference to ‘ways of study’, only to research topics and their relations. Further, it fails to rationalise the need for a ‘single, well-defined way to study new technology’. What is indeed missing from HCI is some agreement on the scope of HCI and its discipline problem. Their absence militates against the generalisation and so the incrementation of its knowledge. How can  HCI knowledge, for example models and methods, be evaluated, compared and incremented in the absence of agreement, that they are promoting different design solutions; but to the same problem (see also later, concerning Newman’s paper). However, the absence is not, and cannot, be made good by research theme key-word analysis, as it stands. An additional bridging framework is required ( for which, see later).

     – Does the paper really give us a true history of HCI over the last 20 years of HCI?

I am not sure that I believe in the notion of ‘true’ history. Some historical accounts are less false than others perhaps (as Popper might have said). Some throw light on different aspects of history. Within its limits (see Weaknesses earlier), the paper throws some light on the development of HCI research topics over the last 20 years. It does not provide an overall history of HCI, as it lacks a framework for linking HCI research topics to HCI discipline concerns and requirements.

As it happens, I published a paper in the proceedings of HCI International ’97 entitled: Twenty-Five Years of HCI: growth without Progress? The paper makes three main claims:

1. HCI, as a field, has grown in the last twenty-five years (that is, 1972-1997), as attested by the growth in computing technology; design; and research.

2. HCI, as a discipline, has made only limited progress in the last twenty-five years, as attested by the failure of HCI research to acquire HCI knowledge to support with some sort of guarantee the HCI design by practitioners of humans interacting with computers. HCI design practice remains essentially one of ‘trial and error’.

3. Only if the relations between research knowledge, design and humans interacting with computers are specified and implemented, can knowledge be validated and so incremented, such that progress in HCI, as a discipline, be made, as opposed to mere growth in HCI as a field (and indeed as a community).

The paper concludes by proposing a validation framework (conceptualisation; operationalisation; test; and generalisation of HCI design knowledge and practice) to make good the deficiency.

Interestingly (and fortuitously perhaps), it is precisely this sort of framework, which is missing from Liu et al’s paper (as indeed their research topic analysis is missing from mine – although see Newman later).

Note to Yvonne: you might like to complement the above by characterising HCI growth and progress from 1997-2014…..(or not, of course).

     – How else might we take stock of the history of HCI?

In one sense, the answer to this question is easy. Relate research theme key-word analysis to a framework of HCI discipline concerns, such that conclusions, based on the former, can be made about the latter. The problem, of course, is the implementation……

The only work (present company excepted, of course), which comes anywhere near this, in my view, is a paper by William Newman entitled: A Preliminary Analysis of the Products of HCI Research, Using Pro-Forma Abstracts. Newman’s analysis is of research products, caste in the form of pro-forma abstracts. His framework is that of the (HCI) engineering design process.

It is worth citing the complete Abstract of Newman’s paper, for the purposes in hand:

“A classification scheme for the products of engineering research is described, involving three principal categories of product: improved modeling techniques, solutions and tools. These categories can be linked to the contributions they make to engineering design. A set of ‘pro-forma’ abstracts are proposed as a reliable means of identifying the three categories. A preliminary sample of published engineering papers indicates that normally at least 90 percent of papers fall within these three categories. For recent CHI and InterCHI conferences, however, only 30 percent can be thus categorised. The remainder appear mostly to describe ‘radical solutions’ (solutions not derived from incremental improvements to solutions to the same problem), and experience and/or heuristics gained mostly from studies of radical solutions. Some comments are made about the reasons for these departures from normal engineering research practice.”

Well, that’s it for this contribution, at least. I enjoyed pulling the bits and bobs together, although I found it hard going……

 A final thought….. is Newman’s sort of work worth updating and re-doing, perhaps with different pro-forma abstract products and a different framework? If so, how would you do it? I smell an MSc student project here….. Joint supervision?

Have a good summer! 

John