March 30-31, 2007: Cambridge, MA


Call for Papers



Session Chair: Wayne Lutters, National Science Foundation

Design Guidelines for System Administration Tools Developed through Ethnographic Field Studies
Eben M. Haber, IBM Almaden Research Center, John Bailey, IBM Almaden Research Center

Information Technology system administrators (sysadmins) perform the crucial and never-ending work of maintaining the technical infrastructure on which our society depends. Computer systems grow more complex every year, however, and the cost of administration is an ever increasing fraction of total system cost – IT systems are growing harder to manage. To better understand this problem, we undertook a series of field studies of system administration work over the past four years, visiting a variety of enterprise and large university sites. One of our most compelling observations was how often the tools used by system administrators were not well aligned with their work practices. We believe that this misalignment was the result of administration tools designed without a complete understanding of the full context of administration work. To promote the design of better tools, this paper describes system administration work in more detail based on examples from our field studies, outlines the dimensions along which enterprise sysadmins differ significantly from other computer users, and provides a set of guidelines for tools to better support how administrators actually work.

Deciding when to trust automation in a policy-based city management game: Policity
Kenya Freeman Oduor, IBM Software Group, Christopher S. Campbell, IBM Almaden Research Center

As businesses and governments strive to improve productivity and deploy more elaborate IT systems, the need for complex systems management grows. Completely automated systems are not yet a reality, so the benefits that automation offers can only be achieved through collaboration with human operators. The question, however, is what factors influence decisions about how this human-computer relationship will be coordinated? A great deal of research has pointed to trust and perceived reliability as key factors in whether automation will be properly used, misused, or disused in systems management. To explore this question, we conducted an experiment in which an automated decision aid presented suggestions or policies to participants while they managed a simulated city (i.e., Policity). The goal was to maximize the health of the city’s population by adding hospitals, housing, businesses and other facilities and services. Participants were randomly assigned to conditions where the decision aid performed with varying (i.e., high or low) reliability levels. Results showed that users’ perception of the decision aid’s reliability directly influenced their trust in the decision aid. Consequently, the relationship between users’ perceived reliability and the decision aid’s (actual) reliability had a direct effect on human performance. Population health suffered when the decision aid’s suggestions were disused and misused compared to when they were appropriately used. Additional results and implications are discussed.

Towards an Understanding of Decision Complexity in IT Configuration
Bin Lin, Northwestern University, Aaron B. Brown, IBM T. J. Watson Research Center, Joesph L. Hellerstein, IBM T. J. Watson Research Center

In previous work we laid out an approach to quantifying configuration complexity [3]. In that earlier work, we explicitly focused on complexity as experienced by expert systems managers, and thus looked at straight-line configuration procedures, ignoring the complexity faced by non-experts as they have to decide what configuration steps to follow. Decision complexity is the complexity faced by a non-expert system administrator—the person providing IT support in a small-business environment, who is confronted by decisions during the configuration process, and is a measure of how easy or hard it is to identify the appropriate sequence of configuration actions to perform in order to achieve a specified configuration goal. To identify spots of high decisionmaking complexity, we need a model of decision complexity for configuring and operating computing systems. This paper extends previous work on models and metrics for IT configuration complexity by adding the concept of decision complexity. As the first step towards a complete model of decision complexity, we describe an extensive user study of decision making in a carefully-mapped analogous domain (route planning), and illustrate how the results of that study suggest an initial model of decision complexity applicable to IT configuration. The model identifies the key factors affecting decision complexity and highlights several interesting results, including the fact that decision complexity has significantly different impacts on user-perceived difficulty than on objective measures like time and error rate. We also describe some of the implications of our decision complexity model for system designers seeking to automate the decision-making and reduce the configuration complexity of their systems.

Session Chair: John M. Carroll, Pennsylvania State University

Cube Management System: A Tangible Interface for Monitoring Large Scale Systems
Elliot Jaffe, Hebrew University of Jerusalem, Aviva Dayan, Hebrew University of Jerusalem, Amnon Dekel, Hebrew University of Jerusalem

Data Centers and Network Operation Centers focus on the challenge of monitoring thousands of devices. They attempt to quickly identify unusual or aberrant behavior. Once the device or problem area has been identified, it is handed off to another team for further investigation and repair. In this paper we present a monitoring system for large-scale installations which uses a tangible interface to reduce the cognitive demand on the administrators and which provides a natural mechanism for handling multiple failures and for delegating tasks among several administrators.

Activity-based Management of IT Service Delivery
John Bailey, IBM Almaden Research Center, Eser Kandogan, IBM Almaden Research Center, Eben Haber, IBM Almaden Research Center, Paul P. Maglio, IBM Almaden Research Center

Growth, adaptability, innovation, and cost control are leading concerns of businesses, especially with respect to use of information technology (IT). Though standards such as the Information Technology Infrastructure Library (ITIL) offer the potential for cost savings through the use of formal processes and best practices, such top-down approaches tend to be either highlevel – often far removed from the actual work – or low-level – often inflexible given the rapid pace of technology and market change. We conducted field studies to examine work practices in IT service delivery. Our results suggest that unstructured work activities comprise a significant and vital portion of the overall work done by people in IT service delivery. These activities include negotiating work items and schedules, seeking and providing information and expertise, and using and sharing custom tools and practices. Unstructured activities are conducted in parallel to formal, structured IT service processes, but are not well supported by existing integrated tooling. Thus, they are not easily accounted for and rarely result in reusable assets or feedback to improve the formal IT processes. Based on these findings, we propose an administrator workspace aimed specifically at blending structured and unstructured work activities to support effective, reusable, and quantifiable IT service delivery.

IT Ecosystems: Evolved Complexity and Unintelligent Design
Jim L. Lentz, IBM Software Group, Terry M. Bleizeffer, IBM Software Group

Modern enterprise IT systems consist of many specialized functional components, often designed by multiple vendors, interconnected in a plethora of permutations to accomplish different goals. An increasingly large number of technical specialists support these systems. Designers of system administration and management tools for these environments must address complexity issues arising from variations in system architectures and topologies, integration between new and legacy systems as well as internal processes and organizational culture. This paper describes aspects of variability within and between IT environments and discusses approaches for managing complexity.

Session Chair: Wendy Lucas, Bentley College

Network-Centricity: Hindered by Hierarchical Anchors
Steve Abrams, University of California, Irvine, Gloria Mark, University of California, Irvine

Network-centricity is a concept under consideration as a useful paradigm for complex organizational operations, combining the strengths of bureaucracy with the innovative possibilities afforded by the ongoing explosion of information and communication technologies. Network-centric work (NCW) is that in which the activities associated with work are conducted via informal selfdirected networks of people, occurring within an environment enabled by technological and organizational infrastructure. NCW cuts across boundaries within and between organizations and engages participants with more regard for their expertise and motivation than their formal roles. Network-centric organizations embrace NCW alongside bureaucracies oriented to providing the resources and articulating the vision to which the NCW is to be oriented. Network-centricity is motivated by a desire for rapid adaptation and flexibility to changing circumstances. However, in an ethnographic study of a distributed team deployed by a large corporation seeking to benefit from a network-centric approach, we found that the work of the distributed team was hindered by some team members "anchoring" to bureaucratic work practices instead of supporting network-centric practices. We identify several such anchor points and the ways in which they impeded network-centric work.

Managing technology use and learning in nonprofit community organizations: Methodological challenges and opportunities
Cecelia Merkel, Pennsylvania State University, Umer Farooq, Pennsylvania State University, Lu Xiao, Pennsylvania State University, Craig Ganoe, Pennsylvania State University, Mary Beth Rosson, Pennsylvania State University, John M. Carroll, Pennsylvania State University

We are investigating how to empower nonprofit community organizations to develop the information technology management practices required to carry out their civic goals. We highlight our methodology of working with nonprofit organizations through three case examples from the field. These examples illustrate that nonprofit organizations are able to and can indeed sustain their IT management practices through various methodological techniques. These techniques—such as scenario development, technology inventory assessment, and volunteer management practices—emphasize the importance of long-term critical planning and design skills. Based on our fieldwork, we enumerate lessons that may be valuable for community stakeholders, designers, researchers, and practitioners.

Supporting Expertise Awareness: Finding Out What Others Know
Christian Dörner, University of Siegen, Volkmar Pipek, University of Siegen, Markus Won, University of Bonn

This paper presents an innovative approach to solve the problem of missing transparency of competencies within virtual organizations. We based our work on empirical studies to cope with the problem of competence finding in distributed organizations. Former studies have shown that central storage of expertise profiles is inappropriate due to missing flexibility and high costs of maintenance. The focus of our approach is to support peripheral awareness to become aware of the available competences in organizations. Our approach runs along two lines: making expertise-related communication visible for all members of an organization and visualizing competence-indicating events in collaboration infrastructures. We verified this approach by the evaluation of a prototypical implementation.

Session Chair: Gloria Mark, University of California, Irvine

Looking for Trouble: Understanding End-User Security Management
Joshua B. Gross, Pennsylvania State University, Mary Beth Rosson, Pennyslvania State University

End users are often cast as the weak link in computer security; they fall victim to social engineering and tend to know very little about security technology and policies. This paper challenges this view as derogatory and unconstructive, arguing that users, as agents of organizations, often have sophisticated strategies regarding sensitive data, and are quite cautious. Existing work on user security practice has failed to consider how users view security; this paper provides content on and analysis of end user perspectives on security management. We suggest that properly designed systems would bridge the knowledge gap (where necessary) and mask levels of detail (where possible), allowing users to manage their security needs in synchrony with the needs of the organization. The evidence for our arguments comes from a set of in-depth interviews with users with no special training on, knowledge of, or interest in computer security. We conclude with guidelines for security and privacy tools that better leverage existing users knowledge.

User Help Techniques for Usable Security
Almut Herzog, Linköping University, Nahid Shahmehri, Linköping University

There are a number of security-critical applications such as personal firewalls, web browsers and e-mail clients, whose users have little or no security knowledge and are easily confused, even frustrated by menus, messages or dialog boxes that deal with security issues.While there are evaluations of existing applications and proposals for new approaches or design guidelines for usable security applications, little effort has been invested in determining how applications can help users in security decisions and security tasks. The purpose of this work is to analyse conventional and security-specific user help techniques with regard to their usefulness in supporting lay users in security applications.We analyse the following help techniques: online documentation, context-sensitive help, wizards, assistants, safe staging and social navigation, and complement these with the tempting alternative of built-in, hidden security. Criteria for the analysis are derived from the type of user questions that can arise in applications and from definitions of when a security application can be called usable.Designers of security applications can use our analysis as general recommendations for when and how to use and combine user help techniques in security applications, but they can also use the analysis as a template. They can instantiate the template for their specific application to arrive at a concrete analysis of which user help techniques are most suitable in their specific case.

Important Dates:

February 27, 2007:
Advance Registration Ends

March 29, 2007:
Web Registration Ends

March 30-31, 2007:
On-Site Registration