This is the report of a workshop held at the National Science Foundation to examine the concept of a digital library to support undergraduate education in the fields supported by the NSF. The working name of the proposed library is "the SMETE Library", an acronym for Science, Mathematics, Engineering, and Technology Education. An earlier study examined the feasibility of such a library; this workshop emphasized the impact on education.

A broad consensus emerged that a comprehensive SMETE Library would be a major resource for undergraduate education and that the NSF should be urged to implement it. Moreover, the outline of an implementation plan emerged that is practical and cost-effective. The workshop agreed that, in order to be successful, the library must be comprehensive, it must be well managed, and it must be funded in a manner that emphasizes service to the educational community. Throughout the workshop there was a sense of urgency. The time to start is now, while digital libraries and the web are still in flux.

Section A of this report describes this vision; this section is a synthesis of the recommendations from the final session of the workshop. Section B is a review of the workshop, describing the membership, the agenda, and the process by which the conclusions were reached.

The report is available on-line at http://www.dlib.org/smete/public/report.html.

TABLE OF CONTENTS

The Internet has the potential to transform undergraduate education, but only a fraction of that potential is now being realized. Some of this gap lies in the maturation process that is part of any transition, but a larger part is the result of fragmentation. Resources of great value are not being used because students and faculty do not know about them, or do not know how to use them.

The NSF has funded the development of many of these resources. Some materials were developed explicitly for undergraduate education; other materials that were created for research purposes are also valuable for students. Additional materials are being created by the commercial sector, by private individuals, and by other government agencies.

While great efforts have been placed on creating materials, less attention has been given to organizing them, maintaining them in the long term, helping people find them, and training people how to use them. For example, a faculty member who is planning a course has only the most rudimentary tools to discover what materials are available or whether they have proved effective in other courses. A student who is researching a topic is forced to choose between general-purpose web search services and commercial databases designed for scientific and technical research. Neither faculty nor students can safely rely on resources that might be withdrawn without notice, or change subtly overnight.

This situation can be seen as a problem, but it is also an opportunity. The increasing maturity of the web and recent research into digital libraries, combined with the energy and enthusiasm with which collections and services are being made available over the Internet, have reached a critical mass. The resources exist. The critical missing link is a framework and infrastructure to maximize their impact on education.

To meet these needs, the workshop articulated a vision of a new type of library, the SMETE Library. This is envisaged as a comprehensive library of the digital resources and services that are available for undergraduate education in science, mathematics, engineering, and technology. The key word here is "comprehensive". Faculty are very specific in wanting a single place where they and their students can discover, use, and possibly contribute a wide range of materials.

"Comprehensive" does not imply uniform and most certainly does not imply centralization. The SMETE Library will be a federation of library services and collections that function together to create a digital learning community. Organizationally, the SMETE Library will consist of a small central operation with a wide range of partners. Some of the services and collections are already well organized; for these, the SMETE Library will be a gateway. Others exist but are poorly organized; for these, the SMETE Library will stimulate the creation of specific services. Some materials are fragmented, unorganized, or hard to find; in these cases the SMETE Library will build library services and may even manage specific collections. Across all these areas, the SMETE Library will provide tools to help faculty and students find and use materials, with services to assist them in evaluating quality and appropriateness.

The SMETE Library will take a broad view of science and technology, and of scientific education. The primary audience is faculty and undergraduate students, but there is no hard distinction between the needs of high school students, undergraduates, and graduate students, nor between students in formal programs, independent learners, and the general public. The SMETE Library could serve them all. The members of the workshop see a particular need to organize and provide access to materials that have been developed specifically for education, but undergraduates need access to other materials as well, especially research literature and the experimental data that is increasingly available over the Internet. The SMETE Library will strive to be a gateway for all these materials. Finally, the SMETE Library will have a variety of financial models for access to the materials; some content will be free of charge while other materials will be available on a commercial basis.

The SMETE Library will contain and provide access to a wide range of materials, some of which have not been collected in the past or have been difficult to discover. The range includes curricula and courseware materials, computer programs, the results of educational research, scientific research reported both formally in journals and informally in web sites, raw data for student activities, and multimedia image banks. Further, the SMETE Library will provide services for authors and instructors such as annotation, evaluation, and peer review of donated materials. For students and faculty, it will offer the capability to search for desired information by subject area, to have access to scientific data sets, and to interact with peers. Faculty, students, and other clients such as independent learners will be able to participate in forums. Interdisciplinary activities, lifelong learning, and the process of education will all benefit. In this way, the SMETE Library will be much more than the sum of its parts, and will promote change and innovation in scientific and technical education at all levels.

The workshop participants were enthusiastic about the educational benefits of a national, federated, virtual digital library that will support students, faculty and authors through high quality collections, tools and services. To achieve maximum impact, it should cover all of science, mathematics, engineering and technology, with varying ports of access (by discipline, by type of user, by access purpose). The goal is maximum technical, intellectual and financial flexibility for all users. Guiding principles for the SMETE Library are that it should:

• be driven by educational and scientific needs,


• facilitate educational innovations,


• be stable, reliable and permanent,


• be accessible to all (though not all materials will be free),


• build on, and leverage past and current work in courseware libraries, digital library research, and successful commercial sites,


• be adaptable to new technologies.

As the principal goal is to improve the quality of and increase access to undergraduate education, the primary users must be students and instructors. However, the materials and services that the library will provide should be of great value to everybody with a serious interest in science and technology, including the research community, K-12 teachers, the public at large, and everybody involved in continuing education.

The educational benefits and national impact of the SMETE Library will be both to improve the quality of education and to improve access to information. Rather than being limited by a single textbook, students and faculty will be able to draw on a much wider range of scientific and educational resources. A quality review system will allow the user to have an appropriate level of confidence in the material being used. In addition to basic course material and reports of research, students and faculty could also access:

• images, visualizations or animations,
• examples and case studies from industry and research laboratories,
• exercises, examinations, examples and problem sets,
• digital videos of experts, technology practice in industry, and laboratory experiments,
• simulation software that could either be downloaded or run on more powerful machines,
• data sets for inquiry-based learning.

The SMETE Library will have an impact on every aspect of scientific and technical education. The only limits are the breadth of the collections, and the imagination of faculty and students. The collections and related services would allow faculty to learn about and incorporate diverse learning processes, including collaborative learning. Faculty could engage in professional training and learn about best practices through discussion groups and on-line workshops. Students could supplement their own learning with:

• diagnostic examinations and related supplemental educational material,
• discussions on topics with students and researchers from other universities and institutions.

The SMETE Library will encourage the dissemination of research into educational methods. It will also facilitate the involvement of industry and government laboratories in the educational process. Whereas some universities benefit from guest speakers from industry or government in the classroom, not all schools are able to arrange such visits. The digital library, enabled by new information technologies, would provide a forum for real-time video or voice communication to a wider range of learners. These virtual lectures and discussions could be captured and then added to the library for later access.

The SMETE Library would also facilitate cross-institutional sharing of educational resources. The ultimate goal is the development of a community of scientific and technical educators who use the library for cross-disciplinary and cross-institutional collaboration. Access and discussions with authors and prior users would be possible, along with an archive of past reviews and discussion of materials in the SMETE Library. The collections could be annotated and linked to these discussions and reviews.

The digital library also opens the opportunity for students at different institutions to work on joint projects or experiments, perhaps sharing and adding to the same data set and its analysis. This would also promote physical resource sharing, as students and instructors may have varying access to high-end instrumentation, computational capabilities, data collections, and technology.

For the SMETE Library to achieve its potential, it is critical that the library be accessible by all members of the SMETE community (especially undergraduate students, faculty, and authors). The library could have its greatest impact by improving the knowledge base and instructional resources of those undergraduate educational institutions that historically have lacked good library and computing facilities. The SMETE Library provides an opportunity to level the playing field for students and faculty whatever their institutional affiliations, but effort is needed to achieve this goal.

The workshop strongly recommends that accessibility should be an explicit part of the SMETE Library mandate. This requires a two-part strategy. The first is that the library should be realistic in its technical expectations. Since accessibility can be limited by a range of factors, including network bandwidth, availability of computers, and costs, the library must be designed to accommodate a wide range of users and be realistic about the technology that they use. In particular, to serve the off-campus needs of faculty and students, the library should provide most core services and material based on modem level communication. However, not all SMETE Library services need be limited to the lowest common denominator of the current capabilities of students and faculty. Technology is improving rapidly, and the library must grow with it. One approach is to provide multiple version of content that can be adapted to a range of bandwidth options.

The second part of the strategy is that the SMETE Library should work vigorously with concerned individuals and organizations, including the NSF and other federal and local agencies, to ensure that all students and faculty have good Internet access. Modern scientific and technical education requires that all faculty and students have computers and telecommunications, with the training to use them effectively.

The SMETE Library will be a team effort comprising four groups: the National Science Foundation, the SMETE Library central team, the SMETE Library partners, and associated research projects. Each group has a specific and distinct role.

The leadership of the National Science Foundation is a vital component of the SMETE Library plan. The NSF brings prestige that will stimulate the community to share and work together. The foundation will provide oversight, serve as a catalyst for coordination among existing digital library initiatives, and encourage the fullest possible interaction and collaboration between the foundation’s directorates and other government agencies. The foundation through its funding and evaluation functions can ensure that the SMETE Library will be sustainable, reflect current best practices regarding standards, and be a cost-effective mechanism for enhancing quality education in science, mathematics, engineering and technology for undergraduates in the United States. Actions by the foundation can also serve to bridge the gap between rapidly-changing technology and much of the educational arena.

However, the workshop also urges a more proactive role for the NSF. Many of the resources that will form the SMETE Library are the products of work that the foundation funds. These resources include web sites on which research results are reported, curriculum materials, and scientific data sets. The foundation can request or require that research projects which create these resources meet certain standards in disseminating them. The two most important standards are commitments to maintain materials for specified periods of time, and the provision of basic metadata that can be used for information retrieval. The foundation can also work with other federal agencies to ensure that their products are structured in a manner that makes them easier to access and use by faculty and students.

The NSF can promote the success of the SMETE Library by structuring initiatives across directorates such that they foster dissemination of research and educational findings utilizing the SMETE Library. Using its broad leverage and the strength of the peer review process, the foundation can work to ensure that materials developed for the library will be recognized for quality and that such recognition will be accepted within all educational communities.

The SMETE Library has both central and distributed parts. The workshop participants envisage a small but active central part with both organizational and technical functions. Planning and implementation of the SMETE Library will be guided by a central team that will recruit other partners and manage a small set of central services. It will also ensure ongoing evaluation of the strengths of the service provided, the breadth of the audience, the impact of student use on the learning process, additional needs of faculty and students, and future partners that might strengthen the enterprise.

At the heart of the SMETE Library will be a central area, technically a small web site. This will be the place where users begin their search for information and learn about the tools and services that are available. The objective is that it will be so good that every faculty member and student will maintain a bookmark to it. The reference area will provide guides to the resources that are available: collections, services, and collaborative areas. Collectively, these resources will cover all fields that are relevant to undergraduate scientific and technical education. The central team will manage this reference area. All other resources will be managed by partners. Hence, key tasks for the central team are to establish guidelines and to identify, coordinate, and work with the correct set of partners.

During the workshop several suggestions were made about possible forms for the central team, but no strong consensus emerged.

The SMETE Library will be a federation. The site to which the NSF lends its name and towards which it directs its primary marketing will be considered the central site, but it is unlikely to be the biggest or most heavily used. Other sites on the Internet, that are independently managed but have formal relationships with the SMETE Library, will be chosen as partners. Potential partners in the federation include digital library projects funded by the NSF or other government agencies, such as the National Engineering Education Delivery System by UC Berkeley and its partners, the Physics e-Print Archive at Los Alamos National Laboratory, and the Netlib repository of mathematical software. The partners constitute the distributed component of the library.

Partners will have various levels of participation. Core partners will commit to providing an agreed upon set of services. They will operate within guidelines on indexing and metadata standards, quality control, and procedures on availability and archiving of materials. Initially, the SMETE Library might plan for a small number of core partners who take responsibility for major areas of science, technology and education. Some will be existing, high-quality services for whom affiliation with the library is mutually beneficial, but other partnerships will have to be established. For example, an important task is to organize, index, and archive curriculum materials. These might be managed by a set of discipline-based partners. Many of the SMETE Library features discussed in the workshop related to the services and collections that will be provided by the core partners.

The challenge is to establish mutually beneficial arrangements between the SMETE Library and the partners. Partners will provide services to the SMETE Library. In return, some partners will receive funding; others will receive less tangible benefits. Each partner will be different and success will depend upon effective leadership. Professional societies, commercial publishers, and software and Internet vendors are all potential partners. The SMETE Library must have an organizational structure that recognizes the members of the federation as true partners. There will a process for shared decision making.

The SMETE Library is not a research project, but it will be greatly strengthened by a parallel program of research. The workshop recommends a continuing program of research linked to the SMETE Library. In addition to its programs in digital libraries, the NSF would sponsor research in technology of particular relevance to education, the impact of technology on the learning process, the effectiveness of digital learning communities, and how computer scientists, the library community, and faculty, students, and high school teachers can work together to optimize learning opportunities. However, it is important that these research activities are clearly separated from the implementation of the library itself.

The first steps for this program are already underway. The NSF is the lead agency in the Digital Libraries Initiative, and the NSF has already provided substantial funding for research that will contribute to the aims of the SMETE Library.

This section describes some of the services and collections that are envisaged to be in the SMETE Library. It should be considered as indicative of the breadth of the library, but not as a detailed plan.

A primary goal of the SMETE Library is to provide integrated and effective access to a wide range of materials, including but going well beyond the types of materials found in a conventional library. SMETE should include a comprehensive collection of the materials that might be used for education. They include, but are not limited to:

• Educational materials. On-line courses and course components; application modules; student activities; syllabi of successful courses; tests and assessment tools.
• Research publications. Scientific and educational research papers; pre-prints; conference proceedings.
• Tools and resources. Tools for graphing, calculation, symbolic integration, and data analysis; simulations and animations; images and video; data sets for analysis.

To select materials for inclusion, the SMETE Library will build on and link to existing resources, such as professional societies, commercial publishers, research archives, and local web sites developed in conjunction with NSF-supported projects. The library will survey existing materials to identify gaps, and may commission new materials or encourage NSF to solicit proposals for the purpose of filling gaps. In addition, the SMETE Library will encourage researchers and practitioners to develop materials of the quality expected by the library, thus bringing contemporary research and practice into the educational mainstream.

Workshop participants recommended that priority be placed on establishing guidelines for the SMETE Library materials and services, so that content in the central and distributed sites will have established quality and other characteristics that will promote their use and sharing among users. An example of a guideline would be the metadata terms used to describe content, such as subject, title, discipline, concept, keywords, and prerequisites. By using a consistent set of terms for cataloging the content in the central site and distributed sites, searching catalogs would be easier and more efficient.

Workshop participants suggested the following areas for establishing guidelines:

• Quality. Promote development of content with high scientific or educational impact.
• Discovery. Enable users to find content in a collection easily and precisely.
• Access. Simplify installation and presentation of content. This might include installation instructions with software and hardware dependency information if the content requires more than a standard web browser.
• Customization. Enable users to change the content to match specific educational needs.
• Interoperability. Enable content services to share information and common behaviors, such as reporting tracking data in a consistent way or accessing databases of source data.
• Ease of use. Promote development of intuitive user interfaces.
• Reusability. Enable users to combine content from different sources.
• Collaboration. Enable shared application and user-to-user communication.
• Archiving and stability. Commit to retaining stable versions of materials and not withdrawing them without warning.

In order for the guidelines to be effective, they must be used by as many of the distributed sites of the SMETE Library as possible, but the need to be comprehensive is in tension with the desire for guidelines. The breadth and depth of commitment to the guidelines by a partner site will be one of the characteristics that determines the strength of the relationship between the organization hosting the site and the SMETE Library. For example, it might be decided that all core partners must be committed to the guidelines for search query interoperability, whereas, a partner that was simply linked to the SMETE Library central site might not follow these guidelines.

Users would benefit from a branding program, by being able to identify sites that include content that meet the SMETE Library guidelines. The simple identification of support for the SMETE Library guidelines at distributed sites may prove valuable in promoting the guidelines and the support of the distributed the SMETE Library concept. Some members of the workshop suggested the development of a logo and marketing terms, such as "A SMETE Library Core Collection".

As many workshop participants noted, nearly every item proposed for the digital library already exists somewhere on the Internet, but the lack of organization makes finding materials and separating the wheat from the chaff daunting tasks. Tools and materials will have a major impact on education only when the typical faculty member or student, pressed for time, can quickly locate materials and assess their quality and applicability. The SMETE Library will meet this need by:

• developing indexes and catalogs,
• providing standards for the selection and review of library materials,
• promoting standards and practices for authors of materials.

The SMETE Library will need to provide some form of indexes or catalogs to supplement existing commercial services and the web search programs. The recommended approach to indexing is to associate metadata with each item in the collections. The metadata will follow specific guidelines. (For example, the Educause Instructional Management Systems Project is developing specifications that might be used for this purpose.) An indexing program can assemble this metadata and build searchable indexes for the library.

This procedure can be established for core collections, but not for all collections. Hopefully, most of the other collections will have some metadata that follows the emerging Internet standards in this area. However, it must be accepted that the SMETE Library may not have a comprehensive catalog, beyond the core collections.

Workshop participants expressed a wide range of views regarding whether, or to what extent, educational materials should be reviewed prior to library inclusion. All agreed that it is difficult with educational materials to predict which will be found useful by other faculty or students. The peer review process of professional journals is not a good model for much of the material likely to be considered for a digital library. Quality and ease of use have subjective aspects that may be promoted more effectively through establishment of a broader range of review processes and best practice guidelines, such as guidelines for instructional and user interface design informed by research in distributed learning.

With this in mind, the participants recommend that quality of materials be addressed by developing a dynamic and on-going quality assurance program of tagging certain materials with a record that might include: description and documentation of the materials, lists of users with comments and post-use reviews, formal reviews and other assessment data, and means of communication with the author. Materials that contain this information can be identified as SMETE compliant. In addition, it was suggested that the SMETE Library consider establishing a limited number of yearly awards for excellence. These recommendations will provide several levels of quality assurance, ranging from materials that are untested and unreviewed, up to fully reviewed and award-winning materials.

The SMETE Library will provide authors and publishers with tools and guidelines that will encourage the creation of materials that fit within the SMETE Library framework. The objective is to make compliance with the guidelines so straightforward that it will become routine with authors, publishers, and others who manage collections.

These tools and practices will support standardized submission, review, and user annotation of materials. They might also include group and multimedia authoring tools, along with developer feedback and Internet link maintenance mechanisms.

The SMETE Library must be more than just a repository of materials, with catalogs and indexes. To meet the needs of faculty and students of the twenty-first century, the workshop participants recommend that library materials be organized and linked to provide a dynamic, interactive environment rather than a static collection of isolated documents. They should emphasize interactive engagement over a passive learning environment, and allow for user modification where possible. Some participants even considered that environments for collaboration might emerge as the central theme of the SMETE Library.

The SMETE Library will provide an interconnected set of services that, together with system level services, will support the use of electronic modes of finding, accessing, and acquiring skills, information, and knowledge in all disciplines, both for faculty and students. Faculty services will provide registration, search engines, and training, to support the discovery of materials and the tracking of usage. These services will also ease the adoption and adaptation of materials accessible through the library and support the discussion and collaboration that fosters the sharing of information necessary for the effective and efficient use of SMETE Library identified materials. Student services will include the maintenance of a list of locations of appropriate applications tools, easy search facilities for finding review materials in introductory and remedial course content, and support for collaboration among students on large, inter-campus projects. Training is a theme that runs though many of these services.

Stability and archiving are different aspects of the same topic. Teaching requires that materials be stable. Faculty must be confident that resources incorporated within a course do not change in the time period between planning a course and its delivery. The flexibility of the Internet leads to instability. Resources are often withdrawn with no notice. Worse, versions may change in ways that are incompatible with educational plans. We envisage the SMETE Library providing procedures, technical standards, and notification systems to address this problem.

The archiving question is longer term. Many of the most useful educational materials are the results of projects that have a finite life. The SMETE Library archiving function is to identify resources that have long term benefits and work with partners to preserve them for the long term. Long-term preservation is not a trivial task in a time of technical change. This is a topic that would benefit from associated research.

The workshop was not explicitly asked to consider how the SMETE Library should be implemented, beyond the question of overall priorities. Several themes, however, were repeated throughout the discussions.

As the previous sections have made clear, getting started is urgent. The fragmentation of resources and the lack of organization are growing worse. The volume of materials that will need to be brought into a systematic framework or will languish unused without the SMETE Library grows daily.

The first step is a clear statement of leadership from the NSF, followed by the establishment of a process that will lead to the creation of the central team and core partnerships. This process should include people with expertise in the various content areas, scientific and technical education, and digital libraries. The objective is to flesh out the recommendations of this report, establish partnerships, and begin the process of developing quality control mechanisms, setting standards for selection and archiving of materials, and so on.

The SMETE Library provides a service; it is not a research project. The recommendation is that the technical work of the library, including the core services and collections, be carried out by a number of partners under central coordination. Eventually the SMETE Library organization will include:

• overall leadership by the NSF,
• a central team that has operational responsibility for central functions, such as recruiting and working with partners, and the development of guidelines for organizing the resources,
• core partners who take responsibility for specific parts of the SMETE Library. These might be organized by disciplines, or by other attributes,
• other partners who contribute to the library in numerous ways. They could include societies, commercial organizations, colleges and universities, etc.,
• an associated research program, carried out jointly with several divisions of the NSF and other federal agencies.

The NSF will have the responsibility of ensuring that the right organizational structure is established and that the core partners are appropriately responsive to the SMETE Library goals.

Before the meeting, the following materials were circulated to all workshop participants via e-mail and posted on the SMETE web site:

The following agenda was circulated before the workshop. Minor alterations of timing were made during the actual meeting.

The following brief was provided to the workshop by the NSF.

The workshop began with introductory remarks as follows:

Norman Fortenberry, Division Director
Division of Undergraduate Education
National Science Foundation

Luther Williams, Assistant Director
Directorate for Education and Human Resources, NSF

Frank Wattenberg, Program Director
Division of Undergraduate Education, NSF

Faculty, teachers, software developers, publishers, and a host of others across the United States and around the world are contributing to the development of rich and engaging new environments for active, inquiry-driven learning. This revolution draws much of its vigor from the contemporaneous technological revolution that has put enormous computational power on desktops and in backpacks; that has created networks that bridge distance, discipline, institution, and even culture; and that has created a new generation of powerful, flexible, and inexpensive experimental equipment. The same technology that is playing a major role in the improvement of learning and teaching in the best classrooms and laboratories also has the potential to extend the best education to all students.

The World Wide Web is a medium made for this message. In its brief existence, it has demonstrated extraordinary potential to both disseminate rich learning resources and to support the new communities required for their effective use. The richly linked architecture of the Web matches the richly linked architecture of knowledge, supporting multidisciplinary perspectives and links between research and education. The open architecture of the Web allows us to use a browser as an orchestra conductor, coordinating learners' work with a powerful set of resources and tools. But, the Web has shortcomings � developers often squander precious time with multiple reinventions of the same wheels; it is often difficult to find high quality and appropriate material; and interplatform and intertemporal interoperability are often more rhetoric than reality. The very pace of technological innovation often outruns reflective analysis of its use in science, mathematics, engineering, and technology (SMET) education.

Although the World Wide Web has had a major impact on education, it is a first generation effort, and SMET education is only one of its many uses. In some sense, we are here today to talk about a second generation effort � a true digital library � in which SMET education is the primary focus.

Creation of a national digital library for SMET education presents the challenge of overcoming the Web's weaknesses without losing its strengths � providing selectivity without excessive screening, providing a measure of stability and reliability without being overly restrictive in its operating rules. Such a library should provide an open, robust, and reliable framework for significant advances in SMET education at the same time as it serves as a vehicle for dissemination and study of the best and most current SMET education content and practice. It must have powerful mechanisms to help users find reliable, high quality, and appropriate resources. The success of such a library will be measured in terms of its effectiveness in multiplying the impact of the best work in SMET education and research; its ability to foster and support new communities working with shared resources and tools; and the extent to which it provides a framework for progress in which new developments are able to build on a solid foundation.

You are here as potential users of a national digital library for SMET education to articulate the features that you want in your library. It is important clarify the institutional and human features that are needed as well as the technological features. You are here not only to represent the needs of your disciplines, but also to apply your expertise in these disciplines to education and to the library. Our disciplinary expertise will play a key role in the success of this enterprise.

We have an opportunity to make a real difference. By building on investments by NSF and others in fundamental network and digital library technology, we can build scaffolding for the future � a future in which scientists, mathematicians, engineers, and experts in education collaborate and build on each other's work, improving the quality of SMET education and extending the reach of the highest quality SMET education to all students.

The first major topic addressed by the workshop was the scope and features of a possible SMETE Library. The topic was introduced by William Y. Arms, chair of the workshop. The participants then divided into breakout groups who discussed this topic separately, before reporting back to a full plenary session.

The following statement of the topic was provided to the breakout groups. This statement was also mounted on the web site before the workshop.

The first day of the workshop concentrated on defining the scope and features of the SMETE Library. Workshop participants broke out into five groups, each chaired by a member of the steering committee. The working groups met in two one and one-half hour sessions, with all participants reconvening in the afternoon to summarize the results of their discussions. The results of these discussions are presented below.

There was wide agreement among the workshop participants that the SMETE Library should be open to the widest possible range of students and faculty. As in the model of existing libraries and web resources, the library should serve as a venue for the discovery of information but it should also become a forum where the educational process is mediated and expanded by direct faculty to student, student to student, or faculty to faculty interactions. This new, interactive aspect of the library will offer one critical means for developing a virtual community of users. It is anticipated that this high degree of interactivity will facilitate broad collaborations across traditional disciplinary boundaries.

It was noted as important that the authors of the materials produced for inclusion in the library will also constitute a critical component of the overall audience. Direct contact between the authors and users will permit authors to incorporate feedback from the user community into future revisions of the materials that either expand the content or correct errors. In addition, it is very likely that many faculty as well as undergraduates themselves will author materials such as term papers or research reports that will be posted to the site. The seemingly distinct titles of faculty, student, and author will be blurred by the library.

A third component of the audience that will be addressed by the SMETE Library will include members of the research community and lifelong users. The library offers the unprecedented opportunity for researchers to archive and share the raw data from their research directly with students and other interested faculty. Research results can be checked by students and faulty alike, or combined with observations or methods from other disciplines to forge new conclusions. This aspect of the library will serve to close the perceived gap between teaching and research that exists for many faculty and students as well as the public at large. A vital, active SMETE Library could become a site of choice for access by lifelong users or others such as government or private employees seeking specific information about science, engineering, math, or technology education.

The workshop participants agreed that the materials to be included within the library should be comprehensive and include all materials related to undergraduate scientific and technical education. For example, the collection should include traditional materials such as course syllabi and problem sets, as well as links to journal and commercial publishers’ web sites. Faculty and students alike may wish to include lectures notes or lecture outlines, and some faculty might wish to exchange quizzes and exams in a confidential manner. Other non-traditional materials such as expanded data sets (with some of these drawn from the raw data archives), student papers, computer routines such as Java applets for displaying certain interactions, assessment tools, bibliographic records, digital images, animations, and video should also be included. It was of particular importance that materials in the collection be in a form that would facilitate the ready adoption or adaptation of the content by faculty or students to their own particular needs. In order to increase the usefulness of the library, links to specific sites will need to be maintained and updated, and certain sorts of data such as raw data sets will need to be archived in order to ensure their longevity.

Participants generally agreed that the emphasis of the library should be placed on collecting new digital materials rather than simply digitizing the existing collections of traditional libraries. This emphasis should be expansive in scope, and include new technologies that permit the digitizing of three-dimensional objects (e.g., statues, sculptures, buildings, skeletal elements, molecules) for study and manipulation by the user.

Participants noted that future advances in digital technology are certain to permit a critical expansion of the scope of the SMETE Library, and the library should be responsive to these developments. One critical role for NSF will lie in identifying those areas of the collection where its impact could be significantly improved by the solicitation of specific materials. These materials could be commissioned by a request for proposals that would exactly meet this need.

The SMETE Library should provide a set of services in order to facilitate the effective and orderly cataloging of materials and its efficient discovery and retrieval. Some of these services include:

• a standardization of taxonomies for categorizing information that cuts across discipline-specific cataloging systems,
• search engines based on the standard taxonomy as well as discipline-specific keywords and topics.

Participants of the workshop agreed that the exact location of the materials was of secondary concern to the idea that the library should appear to the users as an integrated body of information. Access to any of the individual bodies of information should be possible from multiple entry points to minimize the number of links between sites. There was broad support for the idea that the SMETE Library consist of a constellation of independent sites that are managed by a central facility. It was agreed that the collection should be stable and permanent, and that emphasis should be placed on the permanence of links so that access is guaranteed.

The organization of the SMETE Library should easily facilitate the efficient discovery and retrieval of materials as well as interaction with the authors of the materials as well as with students and faculty who have or are currently using the materials. Some of these key attributes should include:

• search features that include title, author, keywords, platform, content, educational goals, date of production,
• availability and a listing of any services or recommendations for adaptation and adoption,
• bibliographic records,
• use history,
• user profiles that enable automated notification of new materials or updates to existing materials of interest,
• links to related courseware (private and commercial),
• on-line training in the use of the materials,
• human help/training in the adoption or adaptation of the materials.

The issue of quality was one of particular interest among the workshop participants. It was noted by many of those present that one of the strengths of the web in its present configuration is that it is available to anyone who wishes to post any content. One drawback, however, is that much of the material is of limited or no utility. Participants generally agreed with the notion that some form of rating system should be used with the goal of permitting users to better identify those materials that will be of particular value. Some of the possible rating systems include:

• peer review of materials,
• testimonials,
• reviews from users (faculty, students),
• learning assessment results.

One the key attributes of the library is the unprecedented level of collaboration that it will facilitate among all members the SMETE community. Key features that will encourage collaboration across the different library audiences include:

• student chat groups,
• links to other users of the specific materials,
• links to the author of the materials,
• a history of use that details educational level and utility.

The library should not be permitted to become a simple repository for teaching materials but should instead be designed so that it will evolve into a dynamic entity that integrates its materials into an educational framework that has at its center faculty and student input into the educational process. In this way the SMETE Library will become the vehicle for the development of a community of faculty and students that will lead the country into the next millennium.

(This summary was written by John Kappelman, University of Texas, Austin.)

The second major topic addressed by the workshop was the priorities to be given to the various features of the SMETE Library that were identified by the first group et of breakout groups. The topic was introduced by three short talks by members of the Steering Committee.

Synthesis, an NSF engineering education coalition (www.synthesis.org), has been at the forefront of engineering education reform since 1990. NEEDS is the distributed architecture developed by Synthesis to enable new pedagogical models based on Internet-mediated learning environments. NEEDS currently catalogs and distributes courseware and other instructional technology developed nationally and internationally to provide a resource where both instructors and learners can search, access, and download educational materials. In addition, NEEDS supports a multi-tier courseware evaluation system including a national award competition -- the Premier Award for Excellence in Engineering Education Courseware. The presentation will provide a brief description of NEEDS and our lessons learned over the past eight years.

Each bibliographic record in NEEDS describes the pertinent information about the courseware, in the same manner that traditional on-line public access catalogs provide information on books (i.e., title, author, publisher, subject heading, keywords, etc.). A user can search for courseware by entering terms into a World Wide Web form -- Title, Author, Subject Heading, Keywords, or Platform. NEEDS then performs a Wide Area Information Service (WAIS)-indexed full text search to provide a ranked list of courseware. (The higher the ranking, the closer the courseware matches the requested query.) The user can then view the bibliographic record that describes the courseware and provides hyperlinks to download the courseware for different platform(s). NEEDS goes beyond the traditional on-line library catalog by providing recursive search capabilities and additional guiding and organizing structures. Hyperlinks on the courseware record provide access to related information indexed along multiple axes (i.e., author, publisher, subject heading, and courseware series). Materials held in NEEDS are diverse -- content ranges from single topics that can be covered in a few minutes to fully integrated, term-long courses. One of the most powerful concepts supported by NEEDS is courseware modularity. NEEDS has the capability of cataloging courseware as well as the individual elements (e.g., images, videos, and text) that comprise the courseware. Courseware elements provide a vehicle for continued re-use of content material beyond the life-span of any particular courseware module. The Virtual Disk Drive Design Studio courseware will be shown as an illustrative example.

NEEDS has been driven by feedback from our user community in engineering (faculty, students and staff). Based on this feedback, quality review of courseware, author-initiated cataloging, and extended searches have been added to the functionality. NEEDS has benefited by always building upon international standards for data description (USMARC and emerging metadata descriptors developed by the Educom Instructional Management Systems (IMS) project) data access (http and the World Wide Web), and production database technology (using structured query language (SQL) relational database to store and index courseware records). Through collaboration with the research community, NEEDS has also been able to adopt new technologies as appropriate. Future implementations will focus on: expanding the user community, providing author feedback on use/download statistics, adding feature reviews and introducing new modes for on-line discussions.

As a community college mathematics instructor whose students can benefit from new computer-based mathematical tools and inquiry-based teaching methods, who authors instructional material, who is active in professional societies and in faculty development, I am very interested in the possibilities of the National Digital Library. I would like the NDL to produce and maintain standards so that I could easily access, review, adapt and adopt the variety of new materials that have been sponsored by the NSF and other organizations that are now available in electronic form. It would be helpful to me to have standards for written text, graphics, mathematical and scientific symbols, methods of changing instructional materials, formats for specifying equipment, hardware, and software requirements for instructional materials, and for managing updates for software and hardware. With these standards, I would be able to modify and adapt these new materials in developing my courses (as I do now with traditional textbooks). Making these materials available to my students would be greatly facilitated by efficient and powerful tools for searching for materials, for modifying and authoring materials, for creating animations and simulations, and for complicated computations (graphical, symbolic, numeric, and combinations of such computations).

To increase the rapidity with which such materials are disseminated among the faculty of higher education institutions, it would also be nice to have collaboration tools that would allow easy communication with students and peers, allow the sharing of graphics and symbol-laden documents and large documents, and allow for instructional materials to be created by groups of faculty members spread across the Internet. Of course, all of these services should focus on the improvement of student performance.

As a community college instructor, I am aware of many difficulties that community college faculty will face in using the powerful capabilities of computer technology in the coming century. Though we all know the physical and financial problems that such faculty face in terms of outdated equipment, slow telecommunication lines, and expensive software, we do not often talk about how difficult it is to gain access to training and to peer level discussion of technical and academic issues surrounding the use of technology. The National Science Foundation should make available the resources and means for community college faculty to gain access to the equipment and training that will be necessary for the dissemination of National Digital Library materials and services to community college students (as it did in the initial states of the use of the Internet).

From the presenter's perspective, the NSF SMETE Library would include a combination of content and services delivered via a distributed, virtual presence. The content itself would use open interoperability specifications, such as the IMS Specifications. A strong central presence would be required to provide core services that other affiliated organizations were not able to or were not interested in providing. NSF would, via the central site, provide the means to achieve improvements in quality and access including requirements, development, research, organizational relationships, policy with NSF award winners, and quality control.

The IMS Initiative addresses a class of software known as instructional management systems or learning server that generally runs on a server and distributes educational or training content to learners on a network, supports collaboration, and records student performance data. Examples of this type of software include, Lotus LearningSpace, Web Course in a Box, TopClass, and others.

If examples exist, then why does the IMS Initiative exist? There are many reasons including the fact that for interactive content there is little interoperability, i.e., the ability to take content from multiple authors and run it, unchanged, from multiple learning servers. Given this lack of interoperability development, costs are high, as publishers have to support multiple proprietary learning server architectures. This increased cost forces publishers to focus on high payoff content areas such as introductory math and English, which have high enrollments nationally. A related problem is the difficulty in finding content. If it is difficult to find, then reuse is problematic at best. Additionally, the lack of an effective electronic commerce solution for small pieces of content threatens the viability of a marketplace in such content.

The solution to these problems is the IMS Specifications, which are a comprehensive set of rules designed to enhance communication between on-line learning systems. The IMS Specifications describe the critical interchanges between users, faculty, content publishers, and service providers, such as schools.

The Initiative's goals are to build the Internet architecture for online learning by making freely available specifications and "starter" software available to developers. The project is composed of 32 investment member organizations (Australia and Virginia Tech have joined since the presentation) including major University systems, branches of the US Government and most of the major technology, platform, and authoring tool companies. The project has an international presence with affiliated centers located in the UK and Australia.

The elements of the scope of the specifications that relate to the SMETE-LIB effort includes Profiles, Meta-data, and Content.

Profiles are collections of information about learners and instructors, and are under their control. The types of information include personal, performance, portfolio, and preferences. Performance information granularity is not specified and, therefore, can include information found in course grade books to transcripts. Preferences include data that an authoring tool, learning server, or content might use to provide a more customized learning or content authoring experience.

IMS Meta-data terms are used during a cataloging or authoring process to describe the intent of educational tools or content. The terms include Attribution, Catalog Entry, Concept, Contact, Coverage, Description, Difficulty, Discipline, and InstallationRemarks. Dublin Core terms are incorporated into the dictionary of IMS terms. IMS Meta-data allows for specialization of terms beyond the base set by different discipline and industry groups. Currently, IMS and NIST (National Institute for Standards and Technology), have created a repository to store the collections of sets of meta-data terms. Over time, it is anticipated that this collection of sets of meta-data terms will become distributed.

The IMS Meta-data system will eventually include a full set of programmatic interfaces allowing authoring tools and content repositories to work with multiple sets of meta-data terms, and leverage learner and author profiles to simplify and customize the process of cataloging, authoring, and finding content.

The IMS Content specifications provide for interoperability of performance and assessment data, course structure, and meta-data. Future areas of interoperability include course sequencing, notification, digital commerce and others.

The key intersections between functionality, likely to be present in a SMETE Library, with IMS includes Meta-data, discipline-specific metadata taxonomies, content repository services (such as distributed searching) and service value-chains. Value chains would include those workflow processes that involve multiple organizations with roles in supporting an effective learning experience for students. IMS-defined characteristics of content likely to be of interest to users of a SMETE Library include capturing data about student use, enabling collaboration, and tracking data about students to enable customization of learning experiences.

Suggestions for how to get involved in the IMS Project included requesting IMS compliance from instructional technology vendors, providing input and requirements, using IMS meta-data, and advocating the development of discipline-specific meta-data.

Tryg Ager
Lead of Digital Library Pilots and Prototypes projects
IBM Almaden Research Center
San Jose, CA

Alice Agogino
Professor of Mechanical Engineering
University of California, Berkeley
Berkeley, CA
(Member of the Steering Committee)

William Y. Arms
Vice President
Corporation for National Research Initiatives
Reston, VA
(Member of the Steering Committee)

Daniel E. Atkins
Dean, School of Information
Professor, Electrical Engineering and Computer Science
University of Michigan
Ann Arbor, MI

Nelson Baker
School of Civil & Environmental Engineering
Georgia Tech
Atlanta, GA

Robert V. Blystone
Dept. of Biology
Trinity University
San Antonio, TX

Greg Bothun
Department of Physics
University of Oregon
Eugene OR

Alan Cairns
7308 48th Ave. N.E.
Seattle, WA 98115

Lillian N. (Boots) Cassel
NSF/EHR/DUE
Arlington, VA

Sally Chapman
Chemistry Department
Barnard College
New York, NY

Ching-chih Chen
GSLIS
Simmons College
Boston, MA

Norman Chonacky
The Evergreen State College
Olympia, WA

Carol A. Christian
Director, Office of Public Outreach
Space Telescope Science Institute
Baltimore, MD

John V. Clevenger
Professor of Chemistry
Truckee Meadows Community College
Reno, NV

Melanie M. Cooper
Professor of Chemistry
Director of Undergraduate Laboratories
Clemson University
Clemson, SC

Brian P. Coppola
Department of Chemistry
The University of Michigan
Ann Arbor, MI

Claire Cornell
College of Liberal Arts
College of Business Administration
Graduate College
University of Iowa
Iowa City, IA

Gregory Crane
Associate Professor
Director, Perseus Project
Department of Classics
Tufts University
Medford, MA

Patricia A. Cunniff
Dean, Division of Sciences, Mathematics and Health Technology
Prince George's Community College
Largo, MD
(Member of the Steering Committee)

Ben Domenico
Unidata Deputy Director
Boulder, CO

Dave Douglass
Pasadena City College
Pasadena, CA

Jason M. Edington
Laguna Niguel, CA 92677
(Student at Cal Poly, San Luis Obispo)

Wade Ellis, Jr.
Mathematics Department
West Valley College
Saratoga, CA
(Member of the Steering Committee)

Kenneth E. Foote
Department of Geography
University of Texas at Austin
Austin, TX

Edward A. Fox
Professor, Dept. of Computer Science
VPI&SU (Virginia Tech)
Blacksburg, VA

Robert G. Fuller
Dept. of Physics and Astronomy
University of Nebraska - Lincoln
Lincoln, NE

Ladnor Geissinger
Mathematics Department
University of North Carolina
Chapel Hill, NC

Forouzan Golshani
Department of Computer Science and Engineering
Arizona State University
Tempe, AZ

Daniel Gomez
Prince George's Community College
Largo, MD
(Student at Prince George's Community College)

Mario J. Gonzalez
Vice Chancellor
Telecommunications & Information Technology
The University of Texas System
Austin, TX

Judith Gradwohl
Smithsonian Without Walls
Smithsonian Institution
Washington, DC

Peter S. Graham
Associate University Librarian
Rutgers, The State University of New Jersey
New Brunswick, NJ

Kurt Gramoll
Aerospace and Mechanical Engineering
University of Oklahoma
Norman, OK

William H. Graves
President, COLLEGIS Research Institute
Senior Vice President, COLLEGIS
Research Triangle Park, NC

Steve Griffin
W.P. Technology Research
COLLEGIS Research Institute
Washington, DC
(Member of the Steering Committee)

Joseph A. Heim
Industrial Engineering
University of Washington
Seattle, WA

Warren W. Hein
Associate Executive Officer
American Association of Physics Teachers
College Park, MD

William C. Jennings
Professor and Chair
Electrical, Computer and Systems Engineering Department
Rensselaer Polytechnic Institute
Troy, NY

J. A. Kampmeier
University of Rochester
Rochester, NY

Professor Lawrence J. Kaplan
Department of Chemistry
Williams College
Williamstown, MA

John Kappelman
Professor, Department of Anthropology
The University of Texas at Austin
Austin, TX
(Member of the Steering Committee)

James H. Keller
Associate Director
Harvard Information Infrastructure Project
Cambridge, MA

Randy Knight
Physics Department
California Polytechnic State University
San Luis Obispo, CA
(Member of the Steering Committee)

Deborah L. Knox
Department of Computer Science
The College of New Jersey
Ewing, NJ

Robert Kozma
Principal Scientist
Center for Technology in Learning
SRI International
Menlo Park, CA

Robert L. Lichter
Executive Director
The Camille and Henry Dreyfus Foundation, Inc.
New York, NY

Marcia C. Linn
Graduate School of Education
University of California at Berkeley
Berkeley, CA

Joan K. Lippincott
Associate Executive Director
Coalition for Networked Information
Washington, DC

Lynda Byrd Logan, Ph.D.
Dean of Learning Resources
Prince George's Community College
Largo, MD

Roxanne Baxter Mendrinos
Foothill College
Los Altos Hills, CA

Susan Millar
LEAD Center
University of Wisconsin, Madison
Madison, WI

Joseph Monroe
Dept. of Computer Science, College of Engineering
North Carolina A&T State University
Greensboro, NC

Beth A. Montelone
Associate Professor, Division of Biology
Kansas State University
Manhattan, KS
(Member of the Steering Committee)

Lang Moore
Department of Mathematics
Duke University
Durham, NC

Jan Olsen
Vice President
Wells College
Aurora, NY

Charles M. Patton
Independent Consultant
Eugene, OR

Scott Plous
Associate Professor of Psychology
Department of Psychology
Wesleyan University
Middletown, CT

Mark Resmer
Sonoma State University
Rohnert Park, CA

Brian Roberts
Office of the DeanCollege of Liberal Arts
The University of Texas at Austin
Austin, TX

Debra Rodriguez
Prince George's Community College
Largo, MD
(Student at Prince George's Community College)

Harry Roy
Professor of Biology, Biology Dept.
Rensselaer Polytechnic Institute
Troy, NY

Carol Scheftic
University Center for Teacher Education
California Polytechnic State University
San Luis Obispo, CA

Elaine Seymour
Director, Ethnography & Evaluation Research
The University of Colorado, Boulder
Boulder, CO

Richard J. Shakarchi
Dept. of Psychology - Social Area
The Ohio State University
Columbus, OH.

Mark Sheffield
Manhattan, KS
(Student at Kansas State University)

James H. Stith
Director of Physics Programs
American Institute of Physics
College Park, MD

Eric W. van Ammers
Wageningen Agricultural University
The Netherlands

Frank Wattenberg
Division of Undergraduate Education
National Science Foundation
Arlington VA
(NSF Liaison)

Gabriele Wienhausen
Vice Chair for Education
Department of Biology
University of California , San Diego
La Jolla, CA 92093-0355

Bonnie Wilson
Digital Library Specialist
Corporation for National Research Initiatives
Reston, VA

Jack M. Wilson
Dean of Faculty (Acting)
J. Erik Jonsson '22 Distinguished Professor
Rensselaer Polytechnic Institute
Troy, NY

Donald J. Wink
Associate Professor
University of Illinois at Chicago
Chicago, IL

Kate Wittenberg
Columbia University Press
New York, NY

Wayne Wolf
Department of Electrical Engineering
Princeton University
Princeton, NJ

Lee L. Zia
University of New Hampshire
Concord, NH

Dean Zollman
Department of Physics
Kansas State University
Manhattan, KS

To explore the linking of digital library research efforts and testbeds for undergraduate education, NSF's Division of Undergraduate Education will provide a total of $500,000 for planning and study projects in FY 1998. Successful applicants are expected to demonstrate high potential to advance undergraduate science, mathematics, engineering and technology (SMET) education. Three types of proposals are of interest: practical digital library applications for SMET education, technical studies of digital library capabilities, and general policy studies.

The full announcement is at http://www.nsf.gov/pubs/1998/nsf9863/nsf9863.htm

For further information, please see: Special Opportunity for Applications and Testbeds for Undergraduate Education.