The information world is a very different place in 2003 than it was a few years ago. It is possible, for example, to find information more quickly and easily than ever before, using new tools, and drawing on sources of information unavailable or even nonexistent not so very long ago. People seeking answers or providing them now have many more options for "question negotiation", in the broadest Tayloresque sense . Consulting an information professional continues to be one of those options. In fact, such professionals can now be much more accessible via these new tools and technologies.
There has been a great deal of discussion in the professional literature about the reference interview and question negotiation. Among the most important are Taylor's notions of question negotiation (encompassing self-help and a variety of other modes of satisfying information needs). As yet, however, no universally accepted and satisfying definition exists for the term "question negotiation".
We therefore propose the following as a conceptual definition for "question negotiation":
an interaction between a person with an information need and an information service; its purpose is to refine the information need so that it can be usefully responded to by the information service.
Given that definition, we now investigate modes or channels in which question negotiation occurs, and how the technology of those modes affects the tasks and opportunities of the digital reference professional and, perhaps, vice versa.
1. Characteristics of Some Modes of Digital Reference
In this section, we'll examine synchronous and asynchronous modes of conducting a digital reference transaction, with an eye for their affects on digital reference practice.
One of the difficulties in studying the reference interview in the past has been the ephemeral nature of the interview. Since traditional reference interviewing has taken place in person or on the phone, no objective record existed unless the interview was recorded or transcribed. Even then, one is left to wonder whether the recording process itself materially affected the nature of the interview.
In the digital environment, recordssuch as transcripts and email archivesexist and can be of great use in studying the reference interview. To date, however, little has appeared in the research literature to describe the nature of these interviews. Abels' oft-cited article examined purely email interview exchanges [Abels, 1996], and a number of "how we do it" articles have appeared in the practitioner literature.
A more recent study examined the range and scope of digital reference services in public libraries in 2002, and some findings are reported below [Janes, in press].
Findings from the Janes study show that the proportion of public libraries offering reference services via email has risen sharply, almost tripling in a two-year time frame, but it is still lower than the proportion of academic libraries offering such services in 1999. The increase in digital reference volume over all types of libraries is dramatic and may perhaps be related to new methods of question submission. While the proportion of libraries providing email addresses is roughly comparable to what has been observed in the past, the percentage of libraries offering detailed forms (anything more than the simple form questions) has doubled, largely at the expense of the very bare-bones, simple forms. In addition, the number of libraries offering real-time services (chat, instant messaging, call center software based) now exceeds 20%, and a similar number provide a page summarizing their services and offering guidance as to which type of reference service might be more appropriate.
Further examination of the web forms used for digital reference yields this list of the most frequently observed questions asked on web forms. (Nearly all forms asked for name and email address.)
A number of other questions were asked, but none more than 5 times. It is interesting to note omissions here: the rationale or motivation behind the queryone of the main reasons for doing a reference interviewwas asked on only 3 forms, as was the preferred answer type (factual, sources, etc.). The number of items or amount of information desired was asked only once. Most of the frequently asked questions, then, are functional and personal (name, email address, phone number and so on), rather than based on the nature of the query itself. These questions serve more as a profiling activity than an interview aimed specifically on refining the nature of the query.
Only very cursory examination has been made of the web pages describing and presenting chat-based services. The following are subjective observations (from the same study referenced previously), and more substantial analysis of these pages may well yield more sophisticated and trustworthy conclusions. However, these glimpses into what those pages say and do are intriguing:
General Characteristics for All Modes
Another source of potentially revealing evidence about the reference interview in the digital environment is that of comments made by librarians. The DIG_REF listserv, maintained by the Virtual Reference Desk project, is probably the best-known and most widely trafficked listserv in this area. While a comprehensive survey of the DIG_REF discussions on the reference interview is beyond the scope of this article (and would be a fascinating and worthwhile venue for further research endeavors), a discussion thread from October of 2001 seemed to encapsulate many of the issues of recent concern to reference librarians about the nature of the reference interview and how it's changing. A posting by Bernie Sloan on October 22, 2001, began the discussion by asking the following: "I'm looking for practitioners' perspectives on how the reference interview is affected by the medium it's conducted in…From your experience, how does the digital reference interview differ from the traditional face-to-face reference interview in the library?"
A large number of responses to Sloan's question were posted over the next several days. Among the discussion points of interest here are:
Regarding asynchronous (email, web form based) interviewing:
Regarding synchronous (chat, call center software) interviewing:
From the above, we can begin to draw a few conclusions. In the asynchronous world of digital reference, there is more concern about the structure of the inquiry, in the use of forms and guidelines for email messages but that is a phenomenon seen less frequently in synchronous services. Asynchronous service transactions also take more time to complete, and this increase in time needed permeates the process: more time (and, perhaps more thought) is required to formulate and submit a query; time may elapse before the query is read and responded to; and more time is required to formulate and send a response. Considerably more data is collected regarding an inquirer's personal characteristics than is collected about the nature of the inquirer's query itself, and the use to which this personal information is put is not known.
It appears that people designing synchronous digital reference services are attempting to make them more like traditional services in the use of a real-time interview and response process. In general, the services provide hours of service and give fuller descriptions of what the service entails as well as its parameters. Time also seems to be an important concern, although here it is the pace and flow of the interaction that is discussed.
Overall, the thinking and discussion about synchronous services reflects an attempt to adapt reference practice to a new set of technological environments. This is also, of course, the case in asynchronous environments. Perhaps the most intriguing question raised is whether the most fruitful and appropriate approach is to adapt reference practice to the technological domain, or vice versa.
Several issues have emerged as central to a greater understanding of the development of techniques that efficiently and effectively identify and refine users' information needs. In general, this is a period of feeling through how best to develop and deploy reference services in an emerging technological and information environment, as well as how to offer those services in a professional and high-quality manner. In the process, new phenomena, such as the disappearing questioner and the "probe query"  described above, are being observed.
Among the most important matters arising recently in the literature and in the previous discussion are:
We see then that digital technologies have affected digital reference professionals in several ways. However, it appears that the partnership of digital reference and technology is a marriage of convenience at best, and there are clearly strains in adopting practice to the currently available technologies. As digital librarians attempt to fit their familiar practices to a new world, they may find the digital reference practices inadequate, frustrating, or difficult. Perhaps we should consider instead how the technology can be adapted to and support librarians' needs, rather than have the technology dictate how librarians are to meet those needs.
There is much work to be done in developing effective ways to satisfy user information needs and assist digital reference professionals to manage questions. Several organizations have begun this work, and examples are discussed below.
2 Technology to Support Digital Question Negotiation
Although technology is obviously an important aspect of the question negotiation discussion, to this point technology has been seen primarily as a communications medium, a networked means to solicit and respond to user queries.
It might be worthwhile, however, to think about how the process of question negotiation, again broadly understood, might otherwise be supported by technological means. This leads to a new question:
In what ways could automation augment the digital reference professional's capabilities in question negotiation?
Email was originally used for digital reference, more as an extension of the familiar use of email than as a new method of outreach by librarians or a new way of thinking about reference work. As the Web gained greater acceptance, email became a principal tool in the arsenal of the digital reference librarian. (It was at this stage that the notion of "digital reference", "virtual reference" or "electronic reference" began to be discussed as a new kind of service.) In the last several years, web forms have found widespread use, adding richness and complexity to what were previously simple exchanges of personal email.
Services have evolved, and there is growing recognition of the need for supporting the intake, management, response and archival functions of reference services. One of the first special purpose solutions was QRC (produced by the Internet Public Library), which was designed to carry out the functions necessary to manage digital reference transactions. These functions included:
(See the article by Lagace and McClennen for further details [Lagace & McClennen, 1998].)
With varying degrees of success, other services have attempted to adapt commercial software to the needs of reference. In the last two years, 24/7 [McGlamery, 2001], LSSI [Coffman, 2001] and a variety of new vendors and players are entering the new digital reference software marketplace with standalone products and, more recently, with software that is interoperable with their integrated library systems. Rather than adapt existing software for new processes, however, other organizations have begun to create automation solutions that are driven by digital reference practice. What follows is a brief description of four such automation initiatives and an overview of how they support question management in digital reference services rather than impose technologically related constraints.
The most visible software development effort of late has been undertaken by OCLC as part of their contribution to the QuestionPoint service [Kresh, 2000; Quint, 2002]. An outgrowth of the Collaborative Digital Reference Service initiated by the Library of Congress several years ago, QuestionPoint is an attempt to build a global collaborative service drawing upon the skills, expertise and resources of many libraries and librarians. The QuestionPoint resources are intended to provide support to libraries in responding to questions beyond the scope of their local resources. This software is still in development, but it already encompasses many features familiar to users of QRC or 24/7, as well as many new ones.
One intriguing development, and a centerpiece of the QuestionPoint service, is its routing function. The designers of this service continue to develop and refine algorithms for sending a query to a particular service based on its subject matter and the collection and service strengths of members, as well as deadline, load, geography and a variety of other factors. Work on this function is ongoing, and while there have been some early difficulties, it seems to hold promise as an important and necessary feature of any large-scale cooperative servicestreamlining the repetitive and time-consuming tasks of question routing.
Full automation is an intriguing concept, very much in the long tradition of artificial intelligence and expert systems. (For more information, we refer the reader to Richardson; Alberico & Micco; and McCrank who have provided useful discussions about automating the reference process [Richardson, 1995; Alberico & Micco, 1990; and McCrank, 1993].)
In an article published in 2001, Kwok, Etzioni and Weld discuss MULDER, which they describe as "the first general-purpose, fully-automated question-answering system available on the web" [Kwok et al., 2001]. MULDER was designed to answer what these authors call "factual questions", and what most librarians would call "ready-reference questions", such as "Who was the first American in space?" or "What is the second-tallest mountain in the world?" While an extensive discussion of the system is beyond the scope of this article, a brief description follows.
The MULDER system has three components. The first is a retrieval engine that sits on top of the document collection and handles retrieval requests. In the context of the Web, this is a search engine that indexes web pages. The second component is a query formulation mechanism that translates natural-language questions into queries for the information retrieval (IR) engine in order to retrieve relevant documents from the collection, i.e., documents that can potentially answer a particular question. The third component, answer extraction, analyzes these documents and extracts answers from them.
In testing their system, Kwok, Etzioni and Weld used questions from the TREC-8 text retrieval competition and compared their system with the performance of Google and AskJeeves. MULDER outperformed both in terms of recall and "user effort" (actually a word distance metric) to achieve given levels of recall. Thus, although there has been no apparent attempt to extend this work or adopt it in any production-level service, MULDER shows promise for implementation and use in a practitioner's setting.
A major challenge to this system would be dealing with questions that require answers with high levels of synthesis and questions of broad scope, rather than factual questions within a narrow scope. And yet, broad scope, highambiguity questions are common to many digital reference service organizations that may find themselves in increasingly unfamiliar support scenarios.
Most digital reference services can be described within the VRD project's 5-step model , including question acquisition, triage, expert answer formulation, tracking and resource creation. Until recently, most of these steps, from the most repetitive tasks to the complex synthesis of answers, were performed by human intermediaries:
Several years ago, the Virtual Reference Desk project (VRD) at the Information Institute of Syracuse (IIS) created a software tool designed to relieve humans of some of the repetitive and low-level tasks involved in providing digital reference. Among the features of this software are:
More recent development of QABuilder Software includes further automation including:
Experts can now receive email notification of new or follow-up questions rather than having to log on to view available questions.
This system can match questions to experts based on audience and categories, and can route questions automatically to expert queues, or to the Administrator intermediary via email or QUIP.
Experts Answer Formulation
Experts can use this software to claim questions, to save them to finish at a later time and to send and receive private messages with administrators. They can also send problem questions to administrators along with comments and submit answers to the administrator for training assistance. Finally, this system allows administrators to customize automated outgoing emails and create pre-formulated answers to use when answering common questions.
QABuilder Software enables real-time reporting on question activity, turnaround times, expert performance, user activity, and most viewed questions. It also checks for orphaned (i.e., questions with no matching experts) or overdue questions and allows viewing of question threads.
This version of the software allows administrators to customize over 20 service-level options and to create unlimited categories and audience levels. Administrators can update profile information (change categories and audience levels), manage patron, expert, and administrator accounts, and customize all registration forms.
These automation initiatives described above represent some of the ways that automation is supporting the digital reference process, but many other organizations are working on these problems too, and even more sophisticated methods are being developed. The following description illustrates a research initiative that seeks to further relieve human intermediaries of repetitive tasksin this case, sophisticated decision making regarding the routing of questions.
The National Science Digital Library
Like QuestionPoint, the National Science Digital Library (NSDL)  is interested in automating question routing. However, the NSDL researchers working on this initiative (researchers from the University of Massachusetts at Amherst, Syracuse University and the Wondir Foundation) have slightly different goals from those of QuestionPoint.
The NSDL Question Triage project seeks to enhance the NSDL by merging its information retrieval (IR) and digital reference components. By combining these functions, the team hopes to automatically determine whether incoming questions can be answered by online resources or need to be answered by human experts, and in the case of the latter, which expert. In this way, human intermediators can handle questions beyond the scope of the current IR system. Therefore, high context queries, or those queries requiring expert judgment where even the best IR systems may fail, can be routed and handled by experts.
If a query is determined to need human intermediation, the system will use inductively developed profiles of available experts to route the user's query to the most closely matched online expert.
The end result of the research described here will be an automatic process that takes a user's query and produces a multi-dimensional classification of the query that can be used as the basis for refining it, directing it to specific online resources, or directing it to human experts. This system would be able to address questions of higher ambiguity and context, further simplifying the experts' tasks and improving scalability.
Software is an important part of digital reference service development and maintenance and provides technological support for the processes of taking in and responding to inquiries.
We have pointed out that technology is sometimes the unintentional driver for software design in automated digital reference systems and has changed the role of digital reference professionals: they now must manage increasing numbers of questions from users with various levels of technical and searching experience, who have less time to search and who ask repetitive questions.
We note, too, that technology can be driven by digital reference practice, and we have briefly described automated components of some systems that complement the ways in which people are already managing question negotiation. Various systems have used automation to receive, route and answer questions. We believe that future initiativesif based on the needs of the digital reference professionalwill result in systems that are easy to use, high quality, and effective in reducing the number of repetitive or low-level tasks carried out by digital reference professionals.
Successful automation will allow more time for human intermediators to do the job that computers currently cannotprovide complex and highly synthesized answers to ambiguous or broad questions from their users.
Many thanks to Lorri Mon, who provided invaluable assistance and conducted the research study discussed in Section 2. The results of that research will be published under her name. Thanks also to the subscribers to DIG_REF, specifically the people who participated in the reference interview thread last October, in particular Sara Weissman at the Morris County Library, Camilla Baker at Canisius College Library, Pauline Lynch of AskERIC and Patricia Memmott of the Internet Public Library.
 Taylor's article (1968) is easily one of the most influential and cited works in the library literature, and describes four levels of question formulation (visceral, conscious, formalized and compromised needs) and the five filters that librarians use to help understand information needs (determination of subject, objective and motivation, personal characteristics of inquirer, relationship of inquiry to file organization, anticipated or acceptable answers).
 The "probe query" may not be all that new; it's described as a vague initial question that is followed by more specific ones. This is not unusual in face-to-face or telephone reference encounters, when people ask quite broad questions, either to feel out the librarian or based on a belief that all like information is located together and all they need to do is get to that place and they can find it themselves. Taylor (1968) among others describes this phenomenon [Taylor, 1996].
 The National Science Digital Library (NSD) is a National Science Foundation program to create a digital library for mathematics and science. The home page for the NSDL is at <http://www.nsdl.org>.
[Abels] Abels, Eileen, "The E-mail Reference Interview", RQ, 35, 345-358, 1996.
[Alberico] Alberico, Ralph and Mary Micco, Expert Systems for Reference and Information Retrieval, Westport CT: Meckler, 1990.
[Coffman, 2001] Coffman, Steven, "We'll Take It from Here: Further Developments We'd Like to See in Virtual Reference Software", Information Technology & Libraries 20 (3), 149-153, September 2001.
[Coffman, 2002] Coffman, Steve, "What's Wrong With Collaborative Digital Reference?", American Libraries, 33(11), 56-58, December 2002.
[Dervin] Dervin, Brenda and Patricia Dewdney, "Neutral Questioning: A New Approach to the Reference Interview", RQ, 25, 506-513, 1986.
[Janes et al.] Janes, Joseph, David S. Carter, and Patricia Memmott, "Digital Reference Services in Academic Libraries", Reference and User Services Quarterly, 39, 145-150, 1999.
[Janes, 2001] Janes, Joseph, "Digital Reference Services in Public and Academic Libraries", in Evaluating Networked Information Services: Techniques, Policy, and Issues, Charles R. McClure and John Carlo Bertot, eds., Medford, NJ: Information Today, Inc., 2001.
[Janes, in press] Janes, Joseph, "Question Negotiation in an Electronic Environment".
[Kwok] Kwok, Cody, Oren Etzioni, and Daniel S. Weld, "Scaling Question Answering to the Web", ACM Transactions on Information Systems, 19, 242-262, 2001.
[Lagace] Lagace, Nettie & Michael McClennen, "Questions and Quirks: Managing an Internet-Based Reference Service", Computers in Libraries, 18 (2), 24-27, February 1998.
[Lankes, 1998] Lankes, R. David. Building & Maintaining Internet Information Services: K-12 Digital Reference Services. ERIC Clearinghouse in Information & Technology, Syracuse University, Syracuse, New York IR-106. 1998.
[Lankes and Kosowitz] Lankes, R. David, and Kasowitz, Abby. The AskA starter kit: How to build and maintain digital reference services. ERIC Clearinghouse on Information and Technology, Syracuse University, Syracuse, NY. 1998.
[McCrank] McCrank, Lawrence J., "Reference Expertise: Paradigms, Strategies, and Systems", Reference Librarian, 40, 11-42, 1993.
[McGlamery] McGlamery, Susan, "Creating a Consortial Chat and Collaborative Browsing Service", presentation given at Building the Virtual Reference Desk in a 24/7 World, Library of Congress, 2001. <http://www.loc.gov/rr/digiref/webcasts/mcglamery/mcglamery.html>.
[Quint] Quint, Barbara. "QuestionPoint Marks New Era in Virtual Reference", Information Today, June 10, 2002. <http://www.infotoday.com/newsbreaks/nb020610-1.htm>.
[Richardson] Richardson, John V., Knowledge-Based Systems For General Reference Work : Applications, Problems, And Progress, San Diego: Academic Press, 1995.
[Taylor] Taylor, Robert S., "Question-Negotiation and Information Seeking in Libraries", College and Research Libraries, 1968, 178-194.
Copyright © Joseph Janes and Joanne Silverstein