Solving class design problems: towards developing Engineering Design Principles

ABSTRACT

Current HCI design knowledge is rarely specified adequately to support its validation. However, such validation is important, since it offers promise for the re-use of design knowledge, to solve design problems, similar to previously solved problems. This paper proposes a strategy for the development of HCI design knowledge, leading to validation. This knowledge is to be expressed ultimately as HCI Engineering Design Principles, which comprise validated knowledge, supported by performance guarantees. The strategy initially requires the specification of class design problems, and their solutions. A partial operationalisation of the strategy, for e-commerce transaction systems, is reported. The operationalisation resulted in the specification of a class design problem and solution, which are exemplified in this paper. The specification will be developed in future research, to construct Engineering Design Principles.

KEYWORDS

HCI, Engineering, Design knowledge, Validation, Engineering Design Principles

RESEARCH NEED

Validated Human-Computer Interaction (HCI) design knowledge offers the benefit of reducing system development costs for the solution of design problems, similar to previously solved problems. However, the validation of HCI design knowledge has been limited, if compared with related disciplines, e.g. Software Engineering (Sutcliffe, 2002).  It has been argued that this deficit is due to the lack of (explicit) conceptualisation of design knowledge, to solve design problems. This under-specification limits testing, and so validation (Long 1996).

This paper describes the initial and partial operationalisation of a research strategy, intended more adequately to specify HCI design knowledge, to support validation. Dowell & Long (1989) state that “established engineering disciplines possess formal knowledge: a corpus of operationalised, tested and generalised principles. Those principles are prescriptive, enabling the complete specification of  [classes of] design solutions before those designs are implemented.” To support validation, design knowledge must be expressed explicitly (conceptualised), to enable its application to design (operationalised), so its relationship with interactive system performance can be established (tested). On the basis of this relationship, the class of design problems for which this performance is achieved can be specified (generalised) (Dowell & Long 1989).

The conception of Engineering Design Principles (EDPs), proposed in Cummaford & Long (1998), specifies the knowledge representations, sufficient to support development and validation of EDPs. This conception is shown in Figure 1.

Figure 1: EDP conception

The conception comprises models, required by HCI as an engineering discipline (HCIe), following Dowell & Long (1989). The conception expresses the HCIe general design problem in terms of a domain model; a user model and a computer model, which together comprise the interactive worksystem (IWS) model; and a statement of inequality between actual and desired performance (Pa ≠ Pd). The domain model comprises objects, which have attributes having particular values. The work to be performed is expressed as domain object attribute value transformations, collectively termed the product goal. The user and computer are characterised by structures and behaviours. These structures are physical, and abstract, and they support behaviours, which are physical, and abstract, and effect transformations in the domain. Performance is expressed as task quality (Tq), (i.e. how well the work is effected) and IWS costs (i.e. the workload incurred in effecting the work). IWS costs comprise user costs (Uc) and computer costs (Cc), and can be physical and mental.

The EDP conception specifies relations between its representations, which can be validated. First, the user and computer models contain sufficient structures, to support behaviours, which carry out the task-goal structure (TGS), whilst incurring IWS costs. Second, the task goal structure, achieves the product goal, for some Tq. The EDP conception thus relates statements of performance to the models, supporting operationalisation, and so, validation. EDPs comprise conceptualised, operationalised, tested and generalised design knowledge, which supports the diagnosis and prescription of a class of design solutions (CDS), which satisfies a class of design problems (CDP). Thus, the specification of design problems and design solutions, at the class level, is a pre-requisite, and so, necessary process of EDP development. To support the ultimate specification of EDPs, there is a need iteratively to identify CDPs and their CDSs, and extract both commonalities between them, and between the commonalities themselves. The latter constitute an EDP, which applies to all design problems within its scope. EDPs offer the possibility to specify, then implement design solutions to ‘hard’ (determinate) design problems.

EDPs may be contrasted with HCI design patterns (Borchers 2001), which focus on a part of a design solution, rather than the complete solution. A complete solution offers more promise for performance guarantees, as the application scope is more constrained.

In this paper, the process of CDS development is reported, to illustrate the application of an EDP development strategy. The result, comprising class representations of the user, computer, domain and TGS for the CDP and CDS, is exemplified (due to space limitations) by the single task ‘find total price’.

DEVELOPMENT STRATEGY

Two strategies for EDP development were described in Cummaford & Long (1998). The ‘initial instance’ strategy (Stork & Long, 1994) generates specific design solutions to specific design problems, then abstracts commonalities, between a problem and its solution, then between these commonalities. This approach is ‘bottom up’. This strategy was contrasted with the ‘initial class’ strategy, proposed by Cummaford & Long, in which ‘promising’ candidate CDPs are identified, by abstraction of commonalities from (constructed) design problems, and corresponding CDSs constructed (where possible). This strategy thus constrains EDP development effort by focusing on domains, which offer ‘promise’ for class design knowledge (although there is no guarantee). This approach is ‘top down’.

SELECTION OF PROMISING CLASS

This research conducted a review of four e-shops, from a wider range, to evaluate similarities in product goal, users and devices, comprising the IWS, and Pa and Pd. All shops support a similar product goal, namely, the exchange of goods for funds, when certain criteria apply. The user groups appear also to be similar, as members of the general public, having knowledge of using websites, and making payments with credit cards (or similar payment technologies). The e-shops sell different goods (e.g. books, CDs, tea, etc.), but comprise similar structures, e.g. ‘virtual shopping cart’, which support similar behaviours, e.g. ‘display subtotal’. Pd is also assumed to be similar for the shops, i.e. to enable all users (fulfilling the criteria), to complete a purchase (high Tq), with minimum workload (low Uc). The similarities between the product goal, IWS, and Pd, for the e-shops reviewed, are considered sufficient to offer promise for the development of a CDP and CDS (and so ultimately an EDP) for e-shop transaction systems.

PROCESS

The following process specified the CDP and CDS. The stages are shown in Figure 2. Of the 6 SDPs represented, two e-shops were reviewed, for which SDPs were specified, the remaining two e-shops were only reviewed (i.e. as potential SDPs), and other e-shops were available for review more generally (i.e. 5 …n). CDP and CDS specification involves first specifying SDPs [1], by testing existing systems and identifying instances for which Pa ≠ Pd. The CDP is then specified, by abstracting commonalities from them [2]. The resulting model is evaluated, by checking that the class user and computer models can be operationalised to complete the TGS for each SDP [3]. Existing HCI design methods are applied to the CDP, to specify a CDS [4]. To evaluate Pa of the CDS, it is decomposed into SDSs, to enable testing by the target user group [5]. Pa is abstracted from the SDSs’ performances [6]. A complete CDP and CDS were specified by the present research, using this process. The complete CDP and CDS models however, are not presented here, due to space limitations. However, complete models, relating to a particular goal of the TGS (‘find total price’), are presented, to exemplify the process completely. The exemplification demonstrates how the models are related to performance, as required by the CDP and the CDS. The requirement of the EDP conception in what follows, for each stage of the process, precedes exemplification by the models of the partial operationalisation of the EDP strategy.

Figure 2: CDP & CDS specification process

1: Specify SDPs

RequirementFirst, a promising class of systems is identified, on the basis of (informal) similarities in work performed. Second, examples of such systems are selected for testing. If these systems achieve a desired level of performance, they constitute specific design solutions (SDSs). If Pa ≠ Pd, they constitute specific design problems (SDPs). The SDP representations specify the domain objects, sufficient to characterise the work performed by the IWS, to achieve the product goal, by operationalisation of the TGS. The IWS, comprising user and computer models, specifies the structures, sufficient to support behaviours, to achieve the TGS. Performance, expressed as Tq and Uc and Cc, is established empirically.

InstantiationIn this research, of the four e-shops reviewed, two – Amazon.com and Barnes & Noble.com, were selected for testing. The e-shops achieved similar product goals, but exhibited different behaviours. The testing required the users to attempt to complete a range of tasks. The latter ensured that relevant aspects of the transaction were evaluated, e.g. removing items from the order, as well as adding them. Actual Tq was lower than desired, as not all users completed the tasks. Uc were inferred from user behaviours[1] and verbal protocols, and enumerated in a Costs Matrix (Cummaford & Long 1999). The SDPs are not included here, due to space limitations, but differences between the SDPs and the CDP are reported later.

Requirement

Following SDPs specification, commonalities are abstracted to construct the CDP. This abstraction comprises common aspects of the SDPs, to provide an initial CDP expression. The domain model is also abstracted, to express the product goal in terms of domain transformations. The TGS is then similarly abstracted. The IWS model is likewise abstracted from the SDPs. As class users cannot be tested as such, Pa for the CDP is derived from the SDPs tested.

Instantiation

Here, the CDP domain model was abstracted from the SDP domain models tested. The domain transformations to achieve the product goal were then specified. The product goal is presented here as text, with an example of its domain transformations. The TGS, to achieve the product goal was then specified, by abstraction from the SDP TGSs. The U-class and C-class, comprising the IWS, were then specified, to contain the minimal set of behaviours to enable the domain transformations to effect the TGS. Performance was then derived by calculating the mean performances of the SDPs tested.

Domain model

Only those components of the domain model, sufficient to specify the total price of the goods ordered, are presented here (see earlier). Domain objects (e.g. ‘user’) have affordant attributes (e.g. total price), the values of which are transformed by the IWS. Dispositional attributes comprise information, which is needed, in order to complete the product goal (e.g. user location), but their values are not transformed.

The domain model was the same for the two tested SDPs, and so was synthesised as the CDP. However, the reviewed e-shops sold different goods (e.g. books, CDs, tea, etc.). The goods were therefore specified generally as ‘homogeneous physical goods, of which multiple units of multiple items are typically ordered’. This specification constrains instances of the CDP to transaction systems, which support collation of an order, containing multiple items, and which incur shipping (i.e. delivery) price, as part of the total transaction price.

Figure 3: Partial domain model for transaction processing

Product goal

The product goal comprises tasks, which must be supported, under specific conditions. Here, the design solution must likewise support the exchange of goods for payment in real-time. In addition to enabling transactions, the system must support the user in gathering pricing information to inform later purchasing decisions. The product goal for the class of physical goods transaction systems is specified as:

1. Transfer ownership rights of goods from vendor to customer
2. Transfer goods & shipping price from customer account to vendor account
3. Transport goods from current location to customer’s address

when the following conditions hold:

1. User desires to complete a transaction in response to offer from vendor
2. User has sufficient funds in appropriate format
3. User address is in legal domain of contract for vendor

In the CDP, the product goal is expressed in terms of transformations of domain object attribute values, which are presented using the syntax ‘domain_object/attribute [value]’.  For example, the first statement in the product goal is expressed as:

1. Transform item/owner from [vendor] to [customer] for all products for which product/ordered is [true].

The relationship between domain transformations, and IWS behaviours to effect these transformations, is represented in the TGS.

Task-goal structure

Here, the TGS was abstracted from the TGSs of the two SDPs tested. A range of shopping tasks was specified, to support comparison of costs between the CDP and CDS. These tasks included ordering single and multiple units of goods, deleting goods from an order, finding total price, and completing the transaction. Only Task 4 in the TGS is shown, as the exemplification of the CDP models, establishing performance. Task 4 requires the domain transformation: ‘Transform user/total price from [unknown] to [amount]’.

Figure 4: Task-goal structure: Task 4

User model

Here, the user model (U-class) was abstracted from the user models in the two SDPs tested. The mental behaviours were specified by positing the minimal set of behaviours, sufficient to effect the TGS, to achieve the product goal. User behaviours were specified analytically, and then evaluated, using verbal protocols from the SDP tests. Here, only user behaviours, sufficient to achieve Task 4 of the TGS, are specified.

Figure 5: Partial user model

Computer model

The computer model was abstracted from the computer models of the two SDPs tested, as for the user model. The computer model contains the minimal set of behaviours, to achieve the TGS.

Figure 6: Partial computer model

Performance

Performance is abstracted from testing the SDPs, instances of the CDP. Tq is expressed as the percentage of successful attempts to complete the product goal. IWS costs are expressed as the number of behaviours performed. Costs are measured, using the Costs Matrix. Cc are not reported here, due to space limitations.

Task quality

80% of tasks were successfully completed during SDP testing. Tq is less than desired, that is, 100%.

User costs

Uc are incurred, whilst learning to achieve the product goal (‘set-up costs’), or during execution of the TGS behaviours (‘runtime costs’). Acceptable setup costs are essentially zero, as target users are able to complete a transaction with no previous training (they are specified as having used the internet previously, and having experience of using credit-card-style payment technologies). Pd is for lower runtime Uc than the SDPs. An increase in Cc is considered acceptable, if this supports a reduction in Uc, or an increase in Tq (as required earlier).

Runtime costs were calculated using the Costs Matrix, one of which was constructed for each user interacting with each SDP. The matrix was completed by giving a score of 1 for every observed or inferred user behaviour. The mean values for users were then calculated (shown in Table 1). Costs support diagnosis of ineffectiveness in the SDPs, and enable comparison with potential design solutions. The tasks in Table 1 constitute the TGS to achieve the product goal. Similar tasks were used during evaluation of the CDS, to support direct comparison of performance. Tasks 1-3 involved ordering goods; Tasks 4 & 6 were ‘find total price’; Task 5 was ‘delete items’; and Task 7 was ‘complete transaction’.

Table 1: CDP Costs Matrix

The CDP, here, comprises all the representations, specified in the EDP conception, and as such, is considered acceptable .

3: Evaluate CDP

Requirement

To ensure that the CDP is sufficient to characterise SDP behaviours, it is evaluated analytically, with respect to the TGS for each SDP. The IWS behaviours of the CDP are operationalised, to achieve the TGS, for each SDP. If the CDP IWS achieves the TGS, then the CDP is retained. If there are insufficient behaviours, then SDP / IWS differences must be re-synthesised. If they are too dissimilar to support synthesis, then a CDP cannot be abstracted.

Instantiation

The evaluated e-shops exhibited different behaviours, e.g. present subtotal on all screens, or only on the ‘virtual shopping cart’ screen. These differences resulted in a similar product goal, but with different performances. However, the aspects of the work, resulting in high user workload, were similar. They were included in the CDP. The CDP user model and domain model were operationalised analytically, to check that the TGS in each SDP could be achieved. The user model contained sufficient behaviours to achieve the TGS in each of the SDPs tested. The CDP was therefore retained.

4: Specify CDS

Requirement

Existing HCI design knowledge and methods are used to specify the CDS. The design stage specifies a TGS, which is achievable by the IWS, whilst incurring acceptable IWS costs and attaining desired Tq. Whilst performance can only be established by testing SDSs, instantiated from the CDS, analytic methods can be used to establish the likely performance of the CDS, prior to testing.

Instantiation

In this case, MUSE, a method for usability engineering (Lim & Long, 1994) was generally used, to design a CDS, which achieved Pd. Substantive HCI design knowledge was also recruited in the design. In particular, the TGS, required to effect the product goal, was informed by mercantile models, which specify transactions as having distinct phases, described as Negotiation, Agreement and Exchange (Kalakota & Whinston 1996). The outcome of the redesign was a re-engineered TGS. An example for the task ‘find total price’, is shown in Figure 7. During CDP testing, calculation of total price, during order collation, was found to be a high-cost behaviour. The CDS TGS differed from the CDP TGS (Figure 4), in that shipping options may be selected at any point in the interaction, hence supporting calculation of shipping costs by the computer, during the Negotiation Phase of the transaction.

Figure 7: Task 4 of the CDS Task-goal structure

The CDP TGS required credit card details to be entered by the user, before the computer displayed the total price, including shipping. The CDP testing indicated that this interaction reduced Tq, as some users were unwilling to enter this information, prior to knowing the total price. The CDS TGS presents the total price throughout the Negotiation Phase, enabling the user to complete a transaction, based on the total price, including shipping price (and so to achieve Tq). IWS costs, incurred whilst effecting the TGS, and the Tq achieved, were measured by testing SDSs, instantiated from the CDS. This testing is described in the next stage of the process.

5: Specify SDSs

Requirement

To evaluate the CDS, it is necessary to re-express it as specific design solutions (SDSs). As there are no class users per se, SDSs must be designed, to enable CDS testing. It is necessary to instantiate more than one SDS, to abstract CDS commonalities in performance.

Instantiation

Here, to evaluate the CDS, it was instantiated as two SDSs, to enable testing with specific users. The two SDS computer models were similar to the SDP computer models, but featured controls to allow the shipping options to be selected at any point during the Negotiation Phase. Two SDS prototypes ensured class performance.

6: Evaluate CDS

Requirement

The CDS performance is abstracted from the performances attained by each of the SDSs. If Pa = Pd, then the CDS is acceptable for the CDP.

Instantiation

Here, ten users were tested on each of the SDS prototypes. Analysis by means of the Costs Matrix indicated that user workload was acceptable in both prototypes, and all users could complete the product goal (the specified criteria having been met). The mean user costs are presented in Table 3. The costs were lower than those of users interacting with the CDP systems. Desired Tq was achieved. The CDS was therefore an acceptable design solution, for the CDP.

Table 2: CDS Costs Matrix

Review

The process presented above was instantiated here, to specify a CDP and CDS for e-commerce transaction systems, as a pre-requisite for EDP development. The specification of a CDP was considered promising for the development of a CDS. This CDS was then specified, and evaluated by testing SDSs, instantiated therefrom. The results indicated that the CDS achieved Pd, and was thus was an acceptable solution to the CDP.

DISCUSSION

This research raises both general and specific issues. Specific issues will be addressed first and general issues second. The specific issues are ordered, following the process stages, shown in Figure 2.

Specifying SDPs

Four e-shops were reviewed, to inform selection of systems for SDP specification. These e-shops exhibited sufficient similarities to show promise for class development. However, the number of systems tested, to inform CDP development, was somewhat limited (i.e. two). Testing more systems would give greater confidence in abstracted similarities between instances. However, the work described here is intended to demonstrate the possibility of specifying CDPs and their CDSs, as an initial and partial operationalisation of the ‘initial class’ strategy of EDP development. Once the process of developing CDPs and their CDSs has been shown to be feasible, such additional development efforts may then be justified.

The tasks, selected for inclusion in the TGS, included a range of typical shopping activities. The tasks are sufficient to characterise any transaction performed on e-commerce sites, although the number of times each task is completed will vary for each transaction. It was, however, considered desirable to standardise the tasks here, to support systematic comparison of the SDP systems, both with respect to each other, and to the SDS prototypes. Varying sub-task frequency, however, should be considered in future work.

SDP testing involved users entering payment authorisation details, supplied by the researchers. Whilst actual payment was thus simulated, users reported that they would be uncomfortable entering their own payment authorisation, before knowing the total price payable. Tq was thus based on a simulation, with its inevitable differences with actual transactions, and so might be questioned. However, such ineffectiveness has been also shown to occur for e-shops generally (Spool 2002).

User behaviours were assumed to be equivalent in terms of workload (Uc). Whilst these behaviours undoubtedly involve user workload, the relative workload, of each, may well not be equal. Empirical evidence for differential workload between behaviours could be integrated into the Costs Matrix. Physical costs were enumerated by counting directly observable user behaviours, but abstract costs were inferred, from tasks performed by the user, and from verbal protocols. A standardised set of encoding criteria informed identification of user abstract costs, which assumed the cost of encoding each page of information to be equivalent to the cost of mental (structure) activation. The criteria, and the equivalence of costs, incurred during behaviours and activation of associated structures, could usefully be explored further in future work.

Specifying the CDP

Pd for the CDP was specified relative to the Pa of the SDSs tested (i.e. increase Tq, reduce Uc). Whilst it is possible, in principle, to state Pd as absolute values, such absolute values were not employed here. Future EDP development should support address of this issue.

Evaluating the CDP-SDP relations

Analytic evaluation of the CDP involved checking that sufficient structures and behaviours were present in the user and computer models, to perform the TGS of each SDP. This assessment supported evaluation of the CDP user and computer models. In EDPs, the CDP is used to establish its applicability, and so this process was considered necessary, in order the better to inform the development of EDP models.

Specifying the CDS

MUSE was selected, from a range of existing HCI design methods, as it supports explicit specification of user and computer behaviours, as a system and user task model, achieving domain transformations for some level of Tq, whilst incurring Uc and Cc.. These representations of IWS, work, and performance allowed the resulting MUSE models to support specification of the CDS models, with minimal re-expression. Substantive knowledge, recruited during this stage of the process, was taken from the existing literature.

Specifying the SDSs

The SDS prototypes exhibited features, which were ported from the SDP systems tested. For example, the two SDP systems exhibited different types of ‘virtual shopping cart’, aspects of which were included in the SDS prototypes. This inclusion ensured that differences in performance between the SDPs and SDSs were due to differences in the CDP and CDS, rather than to specific features of the e-shops and SDS prototypes.

Evaluating the CDS

The EDP conception, upon which this work is based, includes explicit representations of user and computer structures, which support behaviours (following the HCIe conception – see earlier). User and computer structures are not reported in this paper, due to lack of space, but structures were specified for both the user and computer models. That behaviours may be enumerated as an expression of costs, without reference to structures, suggests that structures are not a necessary component of the models, for the purpose of measuring workload, when CDP and CDS structures remain unchanged. However, structures offer a more complete characterisation of the IWS, and are necessary when CDP and CDS structures change (that is, when structure ‘set-up’ costs are not zero).

Mapping the CDP to CDS

Whilst the CDP and CDS cannot be tested directly, the differences in performances of their instantiations indicate that the CDS achieved more effective performance than the CDP. Subjective reports of workload (Likert scales of task difficulty completed by users after testing) indicate that the SDS prototypes incurred less costs in general (as perceived by users) than the SDP systems.

General issues

This initial and partial operationalisation enabled evaluation of the research process followed. The evaluation, however, suggests the need for better specification of that process, such that subsequently developed CDPs and CDSs are likely to be at the same level of generality, and are appropriate to construct EDPs. Model granularity was not an issue here, as they were specified by the same researchers. However, better specification of the process is needed, to support different researchers in specifying CDPs and CDSs at a commensurate and appropriate level of generality.

Whilst the CDP class identified appears promising, there is no guarantee that development will result in the construction of EDPs. Class problems may not have class solutions, and this relationship cannot be known in advance, that is, when the CDP is initially specified. In addition, to construct EDPs, it is necessary to have more than one CDP and CDS to abstract commonalities between these CDP/CDS pairs. In order to achieve this abstraction, further CDPs, and their CDSs, will need to be specified. The initial operationalisation of the ‘initial class’ strategy will therefore continue, with the aim of constructing EDPs.

The initial and partial operationalisation of the ‘initial class’ strategy presented here, thus, offers promise for the eventual development of EDPs, as validated HCI design knowledge, supported by performance guarantees. Whilst this goal has yet to be achieved, this initial and partial operationalisation indicates that the specification of class design problems and solutions is possible.

ACKNOWLEDGMENTS

This work was partially supported by an EPSRC research studentship.

REFERENCES

Borchers, J. (2001). A pattern approach to interaction design. John Wiley, Chichester, UK.

Cummaford, S.J.O. and Long, J.B. (1998). Towards a conception of HCI engineering design principles. Proc. ECCE-9, ed. Green et al; Limerick, Eire.

Cummaford, S.J.O. and Long, J.B. (1999). Costs matrix: Systematic comparison of competing design solutions. In Proc INTERACT 99, ed. Brewster et al; Edinburgh UK.

Dowell, J. and Long, J. (1989). Towards a conception for an engineering discipline of human factors. Ergonomics, 32, 1513-1536.

Kalakota, R. and Whinston, A.B. (1996) Frontiers of electronic commerce. Addison Wesley, Reading Mass. USA.

Lim, K.Y and Long, J.B. (1994) The Muse method for usability engineering. Cambridge University Press, Cambridge, UK

Stork, A. and Long, J.B.  (1994). A specific planning and control design problem in the home: rationale and a case study. In Proceedings of the International Working Conference on Home-Oriented Informatics, Telematics and Automation. University of Copenhagen, Denmark. 419-428.

Spool, J. (2002). The customer sieve. Online report, available at world.std.com/~uieweb/Articles/customer_sieve.htm

Sutcliffe, A. 2002 On effective use and re-use of HCI knowledge. In Human – Computer Interaction in the new millennium. J.M. Carroll (Ed); Addison Wesley.