Volume 5 Number 5
DOI: Current Status and Outlook
Norman Paskin, Director
The International DOI Foundation
Over the past few months the International DOI Foundation (IDF) has produced a number of discussion papers and other materials about the Digital Object Identifier (DOIsm) initiative. They are all available at the DOI web site [DOI], including a brief summary of the DOI origins and purpose [DOIover]. The aim of the present paper is to update those papers, reflecting recent progress, and to provide a summary of the current position and context of the DOI. Although much of the material presented here is the result of a consensus by the organisations forming the International DOI Foundation, some of the points discuss work in progress.
The paper describes the origin of the DOI as a persistent identifier for managing copyrighted materials and its development under the non-profit International DOI Foundation into a system providing identifiers of intellectual property with a framework for open applications to be built using them.
Persistent identification implementations consistent with URN specifications have up to now been hindered by lack of widespread availability of resolution mechanisms, content typology consensus, and sufficiently flexible infrastructure; DOI attempts to overcome these obstacles. Resolution of the DOI uses the Handle System®, which offers the necessary functionality for open applications. The aim of the International DOI Foundation is to promote widespread applications of the DOI, which it is doing by pioneering some early implementations and by providing an extensible framework to ensure interoperability of future DOI uses. Applications of the DOI will require an interoperable scheme of declared metadata with each DOI; the basis of the DOI metadata scheme is a minimal "kernel" of elements supplemented by additional application-specific elements, under an umbrella data model (derived from the INDECS analysis) that promotes convergence of different application metadata sets. The IDF intends to require declaration of only a minimal set of metadata, sufficient to enable unambiguous look-up of a DOI, but this must be capable of extension by others to create open applications.
The Foundation intends to put into place a business model to support DOI allocation and use, probably based on a cost recovery model financed by registrants allocating DOIs via one or more registration agencies, with DOI usage free of charge, and declaration at the point of registration of a minimal "kernel" of metadata with each DOI.
Although many challenges remain to be resolved, a concept of the desired development path and long term aim is now clear.
1. Origin and Aim
The DOI was an outgrowth of a program of the Association of American Publishers to develop tools to enable management of copyrightable materials in an electronic environment. In order to protect something, it is first necessary to uniquely and unambiguously designate what that entity is. The DOI therefore began as a practical initiative in unique persistent naming, as the first part of a fuller implementation to include tools to manage the persistently named entities themselves.
At its simplest, the DOI system offers:
- a persistent identifier of intellectual property; and
- a mechanism to resolve that identifier to some useful information or service.
The second of these two steps associates the identifier with some specific information; the simple way in which this was done was to route to a publisher’s web site. The DOI therefore becomes a specifier for routing to an occurrence of a piece of material on a publisher’s web site, and many of the prototype demonstrations and current practical implementations of DOI embody this functionality. However, the intent of the DOI is not for this to be the end-point, since restricting all uses of material to routing via the publishers web site is neither possible nor productive. The DOI is intended as a public identifier for use in many applications and for local uses, and as with other information identifiers [Paskin1] the DOI should be independent of specific applications -- an identifier that can be freely used in many contexts and by many users.
2. Persistent Identification
Uniform Resource Names (URNs)
The DOI provides a persistent identifier ("name") for a resource or entity. This allows the designation of the entity directly, in contrast to the URLs used by the web, which designate a location at which an instance is held. Thus DOIs allow an infrastructure for managing digital objects independent of locations. The need for persistent names has been long recognised in the development of the Internet, and requirements for Uniform Resource Names have been specified [URN]. These requirements provide a set of principles, which DOI takes as a fundamental starting point:
- Global scope: A name has a global scope, which does not imply a location. It has the same meaning everywhere.
- Uniqueness: The same URN will never be assigned to two different resources.
- Persistence: It is intended that the lifetime of a URN be permanent. That is, the URN will be globally unique forever, and may well be used as a reference to a resource well beyond the lifetime of the resource it identifies or of any naming authority involved in the assignment of its name.
- Scalability: URNs can be assigned to any resource that might conceivably be available on the network.
- Legacy support: The scheme must allow support of existing legacy naming systems, if these satisfy the other requirements described here.
- Extensibility: Any scheme for URNs must permit future extensions to the scheme.
- Independence: It is solely the responsibility of a name issuing authority to determine the conditions under which it will issue a name.
The general form of a URN is urn:nid:nss, where nid represents a defined namespace identifier (e.g., doi) and nss a namespace-specific-string within that nid. Conceptually a URN may optionally include a specific scheme identifier (si), as urn:si:nid:nss. In each case, moving from left to right is moving down a hierarchical naming structure. For example, here is a typical DOI:urn:doi:10.1000/123456789
In this example, "doi" is a namespace identifier (nid); 10.1000/123456789 is a namespace-specific-string (nss). Where the context is clear, it is conventional to write only the namespace-specific string and refer to it as the DOI. As the example shows, the DOI is in two parts separated by a slash (/). The part before the slash (10.1000) is called the prefix and is administered by the authority that creates and manages the DOI; these authorities are in turn assigned by a DOI registration agency (currently, the International DOI Foundation itself).
The concept of unique identifiers for information is not a specifically web-based or Internet-based concept. Such identifiers may have many uses -- for example, the URN framework is equally applicable to telephone numbers or ISBN numbers for books, etc. [Paskin2]. The Internet Engineering Task Force (IETF) and World Wide Web Consortium (W3C) use the term URI (Uniform Resource Identifier) as the overall scheme for standardising unique identifiers in designated namespaces [van der Werf]: URIs encompass both URLs and URNs. URIs are conceived as capable of identifying resources which may include abstract entities. DOIs are consistent with the requirements laid out for URNs and URIs.
Issues in implementing URNs
Whilst the principles of persistent naming are clear, implementation of wide-scale applications on the Internet is limited (despite the URN concept being developed from 1991 onwards). The problems are not with the fundamental concept but with the practical availability of mechanisms or standards. These obstacles include:
- Resolution mechanisms:
A resolution system takes a URN and returns a list of services or instances of the information identified by the URN, commonly one or more URLs 1.
(1) URNs need widespread deployment and ready availability of resolution systems, with accompanying registration processes for resolution systems and for URN namespace identifiers. One proposal for doing this, which has been developed through the IETF, is the Naming Authority Pointer (NAPTR), based on a new DNS resource record. It suggests using the existing global DNS infrastructure. It has not been widely implemented: at present users have to be directed to the resolver in some other way such as via proxy. Eventually, other approaches may be needed for reasons of scalability. URI registration processes are currently not controlled adequately.
(2) Browsers must be able to recognise URN entries typed into the browser windows as transparently as they currently do for URL entries. Existing deployments of URN require dependence on http either directly or as proxy mechanisms. This requires support within browsers for mechanisms to access resolution discovery systems and resolvers.
- Typology of content:
Even if a persistent naming scheme is available, content management issues arise: should different names be assigned to different entities, and how is "different" defined? [Paskin1]. This requires an understanding of when persistent names should be designated, and to what entities. Further, it requires consensus from a wide community to ensure interoperability. The skills necessary for content management of intellectual property (essentially, traditional library and publishing skills) are different from those of designing technical infrastructure, such as routing schemes, and have seldom been combined. (They are now coming together under the general rubric of digital library activities.) The analytical understanding to answer these questions is only now becoming widespread (e.g., [IFLA], [INDECS]), allowing content providers to understand the issues involved in assigning identifiers to different formats, different versions and different categories of intellectual property.
- Single-point or multiple resolution:
Commonly, there are several instances of the intellectual property identified by a URN, or several associated services. It is desirable that resolution should be able to return many or all of them. Unfortunately, resolution systems that are grafted onto today's web technology usually return only a single URL. This limits the flexibility and intelligence that can be built into URN uses on the web, and this has limited their attraction. Without multiple-resolution possibilities, it is logically impossible to build an automated system able to deliver different outcomes to a resolution request dependent on user-specified requirements. A single-resolution system can only deliver a "hard-wired", pre-determined outcome, and this substantially limits the possibilities and benefits of building URN systems. Single-point resolution can still be useful, as has been shown in the case of Persistent URLs [PURL] or URN deployments such as the current DOI. It can be used to provide persistent naming even if a URL moves (e.g., the content changes ownership), or it can lead to a manual version of multiple resolution. For example, the American Chemical Society uses the existing DOI system of single-point resolution to resolve a DOI assigned to a scientific article to a web page. This offers a choice of delivery formats and other services, which then requires user intervention. But clearly multiple-resolution would offer the possibility of fully automating such routings and open up enormous possibilities for creative solutions and services. Nesting of DOIs (DOIs resolving to other DOIs), plus multiple resolution, allow an intelligent network for managing resolution of identifiers to appropriate data types.
3. Resolution of DOIs
The DOI conforms to the URN requirements and has the potential to deal with the major obstacles noted above as currently inhibiting implementations of persistent identifiers. The DOI provides not only a persistent name (identifier) but also a resolution system, using the technology of the Handle System. The Handle System may be considered URN compliant. It offers additional possibilities and features, including several that are not yet being used for DOIs [Handle].
Handles offer multiple-resolution capability, but when the DOI concept was launched, it was felt advisable to limit the introduction by making a simplification of the DOI Handle implementation to a single-point resolution, specifically one DOI resolving to one URL. The reason for this was that the potential audience was almost wholly unfamiliar with the issues described here, and in order to begin DOI assignment and deployment, the single routing system (DOI to publisher web site) was considered a logical starting point. An expansion to use the full Handle capability was soon recognised as a desirable development and was made a long-term goal of the DOI’s development [DOI paper1]; it is a necessary development if the full potential is to be realised. We envisage that the IDF will in the future support the development of some specific applications which will use multiple-resolution capabilities (DOI registrants would then have the option of endorsing such applications by providing relevant data).
A browser plug-in for native support of Handles is available, but has to be installed by the user. Since this cannot be a requirement of use when used on the web at present, the DOI to URN resolution has to use existing protocols. The Handle developers, and the DOI Foundation, are working with browser manufacturers to encourage native support of the Handle protocol. Meanwhile, a proxy server is used which communicates with the world at large in http, and with the special Handle System tools using the Handle protocol. Thus, to the outside world the current DOI syntax (e.g., 10.1000/123456789) appears translated directly into http, e.g., http://dx.doi.org/10.1000/123456789.
Once browser support is available (directly or via plug in), a resolver must be directed to an appropriate resolution system: the registration of resolution discovery systems, and of specific namespaces within them, is another area of current Internet development which the DOI initiative is monitoring closely. It is possible that commercially important URNs or namespaces may require governance structures similar to domain naming authority [ICANN].
Since currently DOIs (i) are treated as a single-point resolution system and (ii) use a http proxy, much of the current DOI functionality could be carried out by simple redirection tools such as PURLs (persistent URLs). This is true of the current implementation but not of the fuller long term aim:
- PURLs are just single redirect servers (another level of indirection) whereas Handle is a direct protocol.
- Handle has multiple-resolution capability and, hence, can build "intelligent" clients; that is, it is possible to define client-side mechanisms which can return one or more resolution results (data types/values) from a list of many, by qualified input of the DOI. Prototypes of such mechanisms have been built using DOIs, and similar functionality (using a repository mechanism) exists in other Handle implementations.
- Handle is scalable.
- DOI is a URN (follows the requirements of URN) and, hence, is useful in other contexts than the web as an information identifier, a persistent name.
- Handle is a (globally) managed system whereas PURL is a local implementation and needs local technical support.
The DOI has a practical aim. We want applications using DOIs to be widely implemented. Two successful examples of identifiers with established practical uses are ISBN and the UPC/EAN bar code. Both have global reach and significant business development built on them, because both had early applications in which they were able to deliver cost savings that outweighed the costs of implementation. Equally, neither was built in the space of a few months [ISBN, Smithsonian].
Whilst there are many interesting functions and services which can be envisaged from further development of the DOI, the DOI Foundation's emphasis is to stimulate the building of applications that provide some compelling reasons for early adoption. As noted earlier, there are no "killer applications" of URN persistent identifiers yet, and there are some reasons for this. Just because someone can do something does not mean they will do it -- a lesson brought home in recent years from many failed internet micropayments systems [Crocker]. There must be a motivation or "killer application".
A specific application: Reference linking
The Foundation intends to provide a framework for others to build applications for DOIs, rather than build them centrally. Nevertheless, in the early days the IDF must take a pioneering role in encouraging and supporting the building of such applications. The leading application currently under development for the DOI is persistent relevant linkage between pieces of information, which is being developed by a group of Foundation members. Linkage has been a goal of information architects for many years, from Vannevar Bush through Ted Nelson to the hyperlinked WWW. We are now in a position to realise that vision; persistent identifiers such as the DOI are a key component. Since the DOI provides the persistent names by which resources and entities may be referenced, then it is an obvious step to use the DOI as a persistent link mechanism between two such entities. Such a scheme would have many advantages: it would be immediately implementable, automatable, extensible to any resource type, extensible to other applications, build on open public identifiers, and allow future extensions in a seamless fashion to future DOI capabilities such as local resolution and content type resolution. A pressing need for such linkage, which is the first crucial aspect of this problem, is linkage from citations in scholarly journals to the paper cited, in an electronic environment.
A subgroup of IDF members is developing an implementation of this in collaboration with IDF's technology partner [CNRI], with expressions of intent from some major publishers including Elsevier Science, Academic Press, Springer Verlag, Wiley, and several US professional society publishers. Alternative mechanisms for reference-linking already exist (bilateral agreements, or identifiers assigned by services such as PubRef): reference linking using DOIs has the potential to become a killer application if publishers and users see the additional functionality offered by an open public identifier to be worthwhile. A draft specification for comment on the topic of DOIs used for reference linking [Reflink] was recently published and is currently being revised and simplified.
General applications: Management of content
Beyond a specific demonstration like reference linking, we need to put into place a framework for open application building using DOIs. DOIs are intended to facilitate content management. Management of information implies the facilitation of transactions. Transactions occur as a result of the interplay of three things: objects (the subject of the transaction); people (the parties to the transaction); and agreements (the terms of the transaction). Interoperability requires a common framework and vocabulary for transactions. If the DOI is to become more than a single-point routing system, it requires the development of a common infrastructure for persistent naming and metadata, built upon a common foundation [DOI paper 1], to enable services as described in a paper by John Erickson [Erickson].
In developing this framework for successful open applications, the IDF has worked closely with the INDECS project (Interoperability of Data in E-Commerce Systems) [INDECS]. The basis of the INDECS activity is to provide a common metadata framework to support E-Commerce in intellectual property. The framework calls for unique identification (of intellectual property, of parties, and of agreements) as a core competence. The DOI has the potential to be the unifying identifier for intellectual property in such a scheme, and will be endorsed by INDECS.
Both INDECS and the DOI assume that respect for copyright, as codified in current legislation, deserves support; and they offer the means of automating and adding efficiency to the process. Copyright-related transactions may be paid transactions, or transactions with some other measure of value: all transactions may be seen as "E-Commerce" since those transactions of intellectual property which are licensed as free (for example through libraries) represent a reduced, limited case within a particular agreement.
Scope: What the DOI identifies
An important distinction is "what the DOI identifies" versus "what the DOI resolves to". In a single point resolution system (one DOI resolving to one URL), it may seem tempting to consider that these are synonymous: surely if I click on a DOI and obtain an entity, then that is what the DOI identifies? This is not so: it is easiest to see this in a multiple-resolution model, where the distinction becomes obvious: the DOI identifies the input to the resolver and the resolver has many different possible outputs or resolution destinations. What the DOI identifies remains unchanged (persistently identified) whereas what it resolves to may be changed. The DOI identifies the resource or entity named and used as input to the resolver; the DOI does not identify the output.
Having made that distinction, we can move on to discuss "what the DOI identifies" in more detail. Content typology -- the analysis of intellectual property entities in terms of all the attributes which are necessary for unique specification - has been carried out in theoretical terms, in particular by the library community in the context of cataloguing requirements, and, practically, in the world of music publishing and trading, from which the DOI has benefited by including music organisations in its membership. The majority of the initial DOI community (traditional publishing organisations) came to this concept relatively recently. At the time the DOI was being conceived (1995-97), there was recognition that the growth of digital publishing of text was creating the need to recognise a change from single to multiple manifestations of a common underlying abstract work (a commonplace in the music publishing world [CIS]). This is a distinction seen in logical analyses such as IFLA’s "Functional Requirements of Bibliographic Records" [IFLA], but also as an operational distinction.
For example, a journal publisher has a production line which is editing articles for publication, and the publisher will use a number to identify the entity being processed. In a print-only world, that process resulted in a single published entity (a printed manifestation). In a print-plus-digital world, the production line will bifurcate at the end and produce two (or more) entities (e.g., a printed article and an HTML file). Those two entities are related (they are "the same article"). The identifier that is on the production line item and carries forward into both the published entities (telling that they have something in common) is an identifier of the work. This recognition, which drew on existing practical analysis and implementation in the music world, led to proposals for identifiers of underlying abstract works such as PII (Publisher Item Identifier) by the STI group, and the possible extension of the proposed music ISWC (International Standard Work Code) to text items within ISO TC 46 SC9, and is now an essential tool in, for example, discussions of persistent reference-linking [Reflink]. Subsequently, these initiatives have converged with DOI considerations of identifiers and their associated metadata.
The analysis of content typology led to a DOI scope definition consistent with the aim of the DOI: intellectual property. This led to the decision made at an early stage [DOIpaper1] that a DOI identifies any Creation, defined as "a thing or event in which intellectual property rights may exist". (In this decision, the DOI became a "Digital Identifier of Objects" rather than an "Identifier of Digital Objects".) By embracing physical and abstract creations (such as books and musical compositions) as well as digital ones, the DOI at a stroke broadened its potential usefulness enormously, so that it may facilitate e-commerce of any kind for all intellectual property through common mechanisms. This has great attractions for those like publishers who find themselves now trading electronically and simultaneously in physical, digital and abstract property, but it also carries a cost. Of necessity the DOI must ensure that such flexibility does not bring with it a crippling ambiguity, and this places a heavier burden on the metadata which is bound to accompany a DOI to make clear precisely what it is identifying.
Whilst there is general agreement among different approaches on the need to distinguish various forms of creations, a common vocabulary is not agreed upon: the DOI initiative shares its vocabulary with the INDECS analysis [INDECS], in which the primary types of Creations are:
- Manifestation: a tangible creation made of atoms or bits: any type of physical or digital creation. Identifiers can be attached to manifestations physically (like a barcode) or embedded digitally (like a watermark);
- Performance: a spatio-temporal creation, which may or may not be recorded in a manifestation. It is particularly important to distinguish between the musical work and its performance;
- Work: an abstract creation whose existence is only revealed through a performance or manifestation.
(Note that Manifestations and Performances are both "expressions" of Works in this terminology. This terminology is slightly different from, but not inconsistent with, the IFLA reference model.)
There are some complex intellectual property legislation implications of these practical analyses, and with rendering content in particular manifestations from a common source. Patrice Lyons has pointed out that, while a unique, persistent identifier such as Handle or DOI may be used to identify a wide variety of resources, from a copyright perspective the use of such identification schemes to identify a logical entity, a data structure, in which a Work or other information is embodied, apart from the information itself, is of special interest, particularly in connection with emerging new forms of expression: a report of American Bar Association Committee 702 has suggested that consideration be given to specifically define "digital works" ("literary works…such as knowledge structures…that are capable of behaviour when processed"). [Lyons] The mapping of intellectual property definitions to the terminology used in identification schemes is a current activity of the International DOI Foundation (work shared with INDECS).
5. Metadata associated with a DOI
The need for metadata
The original concepts of Uniform Resource Names assumed that any URN persistent identifier would resolve to a standard metadata record called a URC (Uniform Resource Characteristics); it has subsequently become clear that this is too simplistic, and URC discussions have been dropped by the IETF. As with content management issues, metadata is a much bigger subject that just the UR* implications of resource description: see the INDECS description of "INDECS Connections" for a view of some of the many initiatives that need to be considered.
Why do we need to associate metadata with a DOI? Consider the wider implications of having such a persistent identifier. If the DOI is seen as "just" a single-point resolution routing system (from a DOI occurrence to a single URL at a publisher-maintained web-site which is the sole source of the entity), then there is little need for interoperable metadata, as any related information about the object so identified can be held within that "proprietary" web site. The need for interoperability arises in the wider vision of DOIs as persistent names (identifiers) for content entities, and which can be used independent of the source of that content (as noted earlier, restricting all uses of material to routing via the publishers web site is neither possible nor productive). Metadata is essential to commerce as it must be possible to process transactions via unique identifiers without recourse to physical inspection of the items being traded -- which may be either inconvenient or impossible, as for abstract and digital manifestation entities. Multiple-resolution schemes for URNs necessitate the existence of some basic metadata associated with the resolution process, in order to provide parameters for an intelligent selection process. Once the DOI is seen as a potential unifying identifier which can be used to access seamlessly a variety of entities from unrelated sources, and to access different instances of the same resource at different sources (using multiple-resolution, local resolution, and building services dependent on the persistent identifier), then the need for a common vocabulary and data model for DOI metadata is clear.
The wider vision described here is, of course, not something which can be devised by one single organisation, and the DOI initiative certainly does not aim to design such a universal E-commerce scheme! More modestly, we aim to put into place an extensible framework which will enable others to use the basic DOI concept as a building block for such efforts. Currently we envisage a minimum set of metadata being declared along with each DOI allocated, as is the normal practice with other successful identifiers like ISBN. This is not currently the case; DOIs are registered with only a few elements of administrative data. This minimum set would normally be sufficient for unambiguous identification of what the entity is (that is, to "look up" the DOI by analogy with a telephone number directory: registrants would have an incentive to register metadata since the more metadata is public, the better the chances of a successful look up2). Such a set of metadata would not be sufficient alone for more sophisticated search and discovery systems provided by added value services.
Adding metadata to enable look-up is relatively simple. But the metadata required for "look-up" of a DOI is only a subset of the (potentially infinite) metadata that may be associated with an entity for the purposes of enabling other applications. It is not the intention of the DOI system to collect or collate all such application metadata; but we wish to encourage widespread use of DOIs, and it is therefore sensible to encourage interoperability and efficiency by re-use. In the future, the look-up metadata may be re-used or extended to build applications by the registrant or by other parties, and so should be consistent with such wider potential uses by others. Designing the metadata scheme for this wider potential extensibility is more tricky; how can we be sure that the metadata elements we select as the simple starting point "look up set" to be associated with DOI entities will suffice for all content types and applications?
Principles of metadata
Just as the principles of persistent identification from the URN work are useful in designing the DOI as identifier, it would be helpful to have a similar logical framework which we could apply to guide us in designing the associated metadata scheme to guarantee such consistency. Fortunately, the development of the DOI has coincided with the publication of just such a set of principles, from the INDECS (Interoperability of Data in E-Commerce Systems) project. We are able to use this logical framework in the same way as we use the URN principles: as a guide which enables a structured approach. The alternative is an ad-hoc or "informed guesswork" approach driven by the need of specific applications with the hope that other, yet undefined, applications will find the same set of metadata terms equally useful. The principles that INDECS has identified are:
- Unique identification: every entity needs to be uniquely identified within an identified namespace;
- Functional granularity: an entity needs to be identified only when there is a reason to distinguish it;
- Designated authority: the author of metadata must be securely identified;
- Application independence: metadata structures should be independent of any technology platform;
- Appropriate access: everyone will need access to the metadata on which they depend (appropriate privacy is a necessary counterpoint).
Some of these logical principles of metadata offered by INDECS resonate with earlier requirements articulated within the DOI development process: in particular the two principles which INDECS considers to be the most fundamental. The INDECS principle of unique identification was an implicit principle of the assignment of DOIs. The INDECS principle of functional granularity is a restatement of the guideline made within the DOI community that "a DOI may be assigned to whatever a registrant chooses" (that is, at any level of analysis depending upon the need of the registrant). The INDECS analysis is also shedding light on related issues such as the need for persistence of metadata, and the use of XML and RDF schemas.
The IDF has affiliated with the INDECS initiative, and as the relevance of the INDECS activity became clear the IDF has moved to a position of being a financial supporter and collaborator of INDECS. We have chosen not to "re-invent the wheel": if an existing initiative or standard offers significant synergy with DOI aims, we have adopted that pragmatically (as in the adoption of Handle resolution technology, and URN principles for the identifier itself). INDECS offers the prospect of a common framework which can accommodate a wide variety of current metadata activities, ranging from the Dublin Core resource description community to a wide variety of content sector activities (libraries, music, books, etc.), and therefore offering the possibility of community and "political" acceptability. Just as the adoption of URN principles does not commit DOI irrevocably to any generic future URN standards developments, so INDECS affiliation does not necessitate DOI registrants to be INDECS participants; but in each case there should be good reasons for any divergence from the fundamental approach.
The DOI metadata kernel; DOI Genres
The full INDECS analysis of metadata has offered a solution to the problem of what metadata elements to specify for a DOI to both enable unambiguous "look-up" (which is, of course, logically simply one application) and be re-usable as part of a wider set of metadata for other applications. DOIs may apply to any Creation, and therefore to a wide variety of entities (abstract works, physical and digital manifestations, etc.) and content types (video clips, text, etc.), yet should be interoperable and extensible in a multi-media environment.
To solve this, we have found it useful to start from the basis of considering potential applications in general terms, and then define what these should have in common (a minimum common metadata requirement). The interoperable common metadata set is defined as a minimum required "kernel" of metadata elements which applies to any DOI. Specified "extensions" to this kernel may then additionally apply to some, but not all, DOIs characterised as similar in some way (a "DOI genre"). A working definition of a genre is "A set of creations for which particular additions or qualifications to the DOI metadata kernel are mandated, either for general purposes or limited to specific functional requirements as designated by the Genre."
Table 1 lists the proposed DOI minimal metadata kernel elements. These eight minimal elements may be qualified or added to in any degree: the extensions for a DOI genre may be additions (for example, a specific genre might add a ninth element such as "audience") or they may be qualifications (for example, in a possible genre for reference linking between Works, the top level minimal kernel element of identifier is "qualified" into a multiple of more detailed enumeration elements: work identifier, related manifestation (rm) article identifier, rm journal identifier, rm publication date, rm page number, and so on.). So a DOI Genre may be defined, for example, for journal articles in general, or restricted (as now) to the purposes of linking journal articles containing references to one another.
Table 1: DOI minimal kernel elements of metadata*
Element (top level)
Possible genre qualifications
This is the key for this record
A class of resources with common characteristics defined by the IDF community
From DOI Genre tables
A unique identifier (e.g. from a legacy scheme) applied to the resources
Any string but must include an identifier type, e.g. ISBN
Mandatory unless the DOI Genre extension rules specifically exclude it
A name by which the resource is known
Any alphanumeric string
Mandatory unless the DOI Genre extension rules specifically exclude it
The primary structural type of the resource
The process by which the resource is made
The name or identifier of the primary agent(s)
Identifier or Name from an agreed Genre namespace
The specification of Primary Agent (normally but not necessarily the creator) is determined in the extension rules for each DOI Genre.
The role(s) played by the primary agent(s)
Role from an agreed Genre namespace
*In addition to the minimal kernel metadata elements (and any extensions for a specific DOI Genre), administrative data such as registrant, date of registration, record version number, etc. is compulsory.
Both the DOI kernel metadata and any extensions (qualifications or additions) fit within the framework outlined by the INDECS metadata model, which enables any other conforming metadata scheme to be used in the design. It is not necessary to adopt the full INDECS model for this purpose; the DOI metadata elements follow the metadata principles such as unique identification, and can be mapped to most other metadata models using the INDECS framework. Attributes can be specified at varying levels of depth, depending upon need, ranging from a simple label (text string) through a defined class, a relation, or part of an event structure. The underlying data model is capable of accommodating any entity at the highest level of specification, but a specific application or Genre may not need such detailed specification. Some of the DOI metadata elements can be "transformed" from the highest event-based level of the model into a "flattened" form (e.g., "origination" in the minimal kernel).
DOI Genres and applications
The Genre approach is now being developed further in practical applications (including the IDF's own look-up requirements), where there is a need to derive specification standards and metadata registration procedures for the specific elements.
It seems likely that Genres will be defined functionally; that is, in terms of an application. In order to take advantage of the application so defined, a registrant will commit to registering (declaring) some metadata elements; this common set will ensure interoperability of DOIs from multiple sources in that application. As the element set conforms to the overall data model scheme of the kernel, other applications can become interoperable (i.e., applications can be written to take advantage of multiple Genres). One such Genre already in development is a DOI Genre for using DOIs in reference linking, as noted above. This application-focussed approach may seem at first sight a paradox, in that metadata should be defined in an application-neutral way, yet the Genre is named for a specific application. The specific metadata elements are indeed application-neutral; in an extensible metadata system, structure and values should be application independent. For example, two distinct elements shouldn't be combined irretrievably for the convenience of one application when they require separation for another; broader limitations on attributes or values shouldn't be imposed by local constraints. However, the potential metadata associated with an entity is infinite; a Genre specifies which elements are mandatory (must be declared) in order to carry out a specific function. To fulfill a particular function (like discovery, linking or rights management) certain values will be required, or useful, or not relevant. For example, for journal article searching all authors may be required, whereas for reference linking only a minimal set (e.g., first named) may be required.
The wider the net is cast in terms of covering multiple applications, the larger the Genre set will become, with a practical balance between being comprehensive and being too burdensome in terms of requirements. We might view the ultimate aim as to have a broader "all function" Genre kernel for each creation type, to which end the "functional" kernels are realistic stepping stones. Figure 1 shows schematically how genres might build by such practical applications being developed, rather than trying to achieve a full set by design ab initio.
Figure 1: Schematic illustrating application-specific Genres and metadata elements
"Well-formed" metadata (following basic principles) for a creation type X may include the elements
A B C D E F G H I J K L M
For an "X Discovery" Genre (for the application facilitating discovery or "look-up") the elements may be
A B D G H L
For an "X Rights Management" Genre the elements may be
A B C E F I J K M
An "X (All Functions)" Genre would be
A B C D E F G H I J K L M
The latter might be added to as time goes by and other specific Genres are added. In each case, the elements A,B, etc. may be additions or qualifications to the minimal kernel.
The INDECS web site contains much more detail on this approach [INDECS]. Key points of importance for DOI include:
- A fully comprehensive approach integrating descriptive metadata with rights and transactions (essential for DOI’s role in transactions of entities having rights);
- A thorough analysis and guiding principles for content typology, providing solutions to content management issues;
- The possibility of transformation processes to express the same metadata at different levels of complexity for different requirements;
- Plans for practical implementation frameworks such as RDF schema expressions and standardisation;
- A tool not a tyranny: INDECS does not prescribe a new data model to replace what may already be in use, but allows existing systems to be used and extended within a common interoperable framework.
6. Business model
The availability of a DOI as a public identifier with managed validation processes, infrastructure, common standards, and managed associated metadata provides added value but also incurs some necessary costs. The costs are broadly in three categories:
- Registration: prefix issuance, validation, maintenance, information dissemination, and guidance on content management issues; recording of appropriate metadata and /or guidance in enabling registrants to declare their own metadata in interoperable form. This function is "content-facing", and likely to be carried out by one or more organisations familiar with specific content sectors.
- Infrastructure: resolution services, development, scaling, and guidance on technical issues. This function is "technology-facing" and appropriate for organisations with skills in similar functions elsewhere.
- Governance: overall management and further development of the DOI system; this is the remit of the International DOI Foundation.
It is intended that the DOI system operate on a cost-recovery basis, with the costs born by the parties assigning DOI numbers; that is, DOI prefix holders. At the outset, a very simple model was introduced whereby a prefix assignment is purchased for a one-off fee (currently $1,000); the possibility of annual fees of some form has been noted but not yet introduced.
Ideally, the IDF would like to see the appointment of third parties to carry out the processes of registration and, potentially, all or part of the infrastructure activity, with cost recovery by these organisations on the basis of efficiencies of scale, competition, and market forces. The governance aspects would be retained by the IDF (who would appoint the third parties on agreed terms) on the basis of either cost recovery from the DOI System operators by royalty (or similar) payments, or separate funding by interested parties. Registration agencies would devise flexible pricing structures for prefixes dependent on volume, etc.
At present, the business model is far from this ideal, and all three areas are being financed essentially by what should be only the final governance structure portion: the IDF is financed largely by membership fees with a relatively small proportion of cost recovery from operational activities. This was expected and planned for during the start up period, but we do not wish to continue on this basis indefinitely; it is questionable whether this would be either feasible or sensible. Migration to a cost-recovery basis requires the appointment of one or more registration agencies. Although there are several lines of discussion open on this issue, the IDF has not so far been able to sign any agreements for the appointment of registration agencies and is carrying out this function itself. One of the issues causing delay is the need for any registration agency to commit development costs to an operation which is, by definition, a new development, and where the numbers and rate of take up are difficult to forecast. Registration agencies also need to consider the business aspects of supporting specific applications by defining Genres and ensuring that the appropriate metadata is declared, which is a new way of thinking about the development of interoperable applications: the concept is laudable, but can appropriate incentives and rewards be offered for this effort?
There are some existing services which demonstrate that such a business model (costs born by registrants) is achievable, including the ISBN model (cost recovery by building additional value-added services) and the UPC/EAN bar code model (cost recovery by sale of prefixes, annual fees, and added value services). The latter, in particular, shows the feasibility of a very large scale and ubiquitous system built on simple principles; the UPC/EAN system is used by over 800,000 organisations [Barcode]. Both of these models are being explored by the IDF. However, neither are exactly comparable with the DOI's twin requirements for identification and resolution processes.
Adding to the complexity is the largely undeveloped but potentially significant role of registration activities in the World Wide Web environment. As ICANN (Internet Corporation for Assigned Names and Numbers) opens the previous monopoly structure of domain-naming registration, additional schemes need to be put into place for identifier namespaces, resolver system registration, metadata schema registries, etc. Will these also prove to be amenable to commercial competition and exploitation, and if so, what are the implications for the DOI?
7. The future
The development path of the DOI can be described as three parallel tracks:
- Current implementations of the existing DOI mechanism. There are already several hundred thousand DOIs assigned, e.g., Academic Press has recently announced the assignment of DOIs to every article carried in its IDEAL electronic journal environment. As part of this strand of development we recognise the need to further improve the availability of information about the DOI, and to develop a DOI manual and other supporting documentation.
- Further functionality. This includes building further features compatible with the existing implementation, such as metadata registration and devising mechanisms for local resolution, defined services, etc. The majority of these tasks will be to provide a basis for open application development.
- Standards tracking. An essential component of DOI development is working with existing and emerging standards activities. This includes ensuring that DOI conforms to appropriate standards, such as registration of namespaces, but also that emerging standards activities are aware of the DOI's aims. Standards activities which we are following include official standards (ISO, NISO), Internet standards (IETF, W3C), and emerging standards in areas such as metadata (Dublin Core, INDECS).
Over the next few months, targets for the DOI Foundation include the following key issues. Where appropriate, we will produce practical prototypes and a document setting out the issues and solutions for each area [DOI Resources], as has already been the case for the topics of scope and metadata [DOIpaper2]:
- Metadata (revised summary)
- Local resolution
- Standards conformance and activities
- Registration agency requirements
- User guidelines (update and expansion of existing document)
- Future financial and operational basis (business model)
- Continued expansion of participation from all sectors of the information community
Issues and challenges
It is clear from the description above that there are several issues to be dealt with if the DOI is to succeed:
- Supporting the development process (financially and in terms of commitment by the community) whilst striving to produce demonstrable, short-term applications. Similar issues have faced initiatives such as the Dublin Core metadata activity (1995 to present) and URN definition and implementation (1991 to present). What is clear is that if the concepts of persistent identification and copyright management are accepted, then the DOI (or something like it) is needed; and that if the DOI is not supported, the likely solution may be less acceptable to the content creators and users, as it may be driven by technology alone.
- Continuing explanation and education on the principles and possibilities of the DOI, thereby encouraging the development of applications, including the creative use of the new tools available through resolution, content typology and metadata design. Included under this heading is the development of guidance materials and the overall aim of allowing DOI registrants the maximum flexibility of use and minimum hindrance in registration overheads.
- Obtaining wider support. Currently the Foundation has the financial backing of over 30 organisations, many of them significant players, but it needs further support to succeed in a realistic timeframe. The recent addition of representation from the library community when OCLC joined the foundation is a welcome step in widening the base of support for the initiative.
- Not leaving the job half-done. To stop development of the DOI at the point when it provides single point resolution to a web site, and a possible tool for reference linking of scientific journal articles, would be to miss the potential of a wider application. Internet solutions are unlikely to succeed unless they are globally applicable and show convincing power over alternatives.
- Ensuring that the DOI fits into the emerging web infrastructure: registration of URN namespaces, resolution mechanisms, browser support of resolution services, etc., and emerging standards for metadata structure and expression such as RDF.
- Creating business models, and migration strategies from the current situation to those models, which facilitate cost recovery with acceptable levels of costs to users.
The vision of what the DOI might enable is an exciting one; undoubtedly it requires effort and goodwill to ensure its success, but the effort involved in gaining consensus and making practical progress pales into insignificance beside the costs to the intellectual property community of not trying.
[Note 1] Resolution is here used to mean the act of submitting an identifier to a network service and receiving in return one or more pieces of current information related to the identifier. In the case of the Domain Name System (DNS), as an example, the resolution is from domain name, e.g., www.doi.org, to a single IP address, e.g., 220.127.116.11, which is then used to communicate with that Internet host. In the case of the Handle System, the resolution is from a handle, e.g., 10.1000/140, to one or more pieces of typed data, e.g., three URLs representing three copies of the object.
[Note 2] DOIs without associated metadata are by definition restricted to the simple model of being a routing to a single (publisher's) URL, without the possibility of independent transactions. DOIs with associated metadata, but where that metadata is not made available in a “white pages” directory for look-up, are akin to unlisted telephone numbers and may be considered usable for full commerce only within closed user groups determined by who is given access to the associated metadata. Neither represents a true open public commerce tool.
[Note 3] A transformation from the INDECS data model to a simpler representation useful for DOI purposes.
Uniform Code Council
Hill, Keith (1997).
CIS - A Collective Solution for Copyright Management in the Digital Age. Copyright World, issue 76, 1997, pp. 18-25.
Corporation for National Research Initiatives (CNRI) home page.
The Siren Song of Internet Micropayments, Information Impacts, April 1999.
International DOI Foundation web site.
The Digital Object Identifier initiative: current position and view forward; DOI discussion paper number 1 (1998). DOI 10.1000/92.
The Digital Object Identifier initiative: metadata implications, DOI discussion paper number 2 (version 3 February 1999). DOI 10.1000/131.
Paskin, Norman. The DOI: Technology meets content management.
DOI Resources page.
Erickson, John. The role of metadata supply chains in DOI-based, value-added services.
Handle System Introduction.
ICANN Internet Corporation for Assigned Names and Numbers.
IFLA Study Group on the Functional Requirements for Bibliographic Records (1998) K.G. Saur Munchen. Functional Requirements for Bibliographic Records. Final report(1998). Also available at:
Interoperability of Data in E Commerce Systems. European Commission, DG XIII, Info 2000 programme.
Ehlers, Hans-Jurgen (1994). Identification Numbering in the Book, Library, and Information World. ISBN Review 15, 214.
Lyons, Patrice A. Managing Access to Digital Information: Some basic terminology issues.
Paskin, Norman (1997). Information Identifiers. Learned Publishing, Vol. 10 No. 2, pp 135-156.
Paskin, Norman (1999). "Towards Unique Identifiers". Proceedings of the IEEE, Special Issue on "Identification and Protection of Multimedia Information" (in press).
Persistent Uniform Resource Locators home page.
DOIs used for reference linking: version 1.0.
Leibowitz, E. (1999). Bar Codes: Reading Between the Lines. Smithsonian Magazine, Feb 1999, pp. 130-146.
Sollins, K. & Masinter, L. (1994). Informational Request for Comments: 1737. Functional Requirements for Uniform Resource Names.
van der Werf
Van der Werf-Davelaar, T. (1999). Identification, location and versioning of web resources.
Copyright © 1999 Norman Paskin
Footnotes were lost in conversion to HTML. The Notes section in this paper was added, May 19, 1999, at the request of the author.
Top | Contents
Search | Author Index | Title Index | Monthly Issues
Letters | Next Story
Home | E-mail the Editor
D-Lib Magazine Access Terms and Conditions