A History of Directory Standards

Early Electronic Directories

In the early 1980s research was carried out into distributed computing systems at the Xerox Palo Alto Research Center and led to the development of Grapevine [5]. Later, the Xerox Clearinghouse was built on the results of this research. It was the first distributed directory to share user account information. The Grapevine network spanned 1500 computers in more than 50 local networks. Thanks to Clearinghouse a user could log in both at his usual desk, and at any of the Xerox overseas offices.

Special Purpose Directories

Along with the Internet came another distributed directory--the Domain Name System (DNS). DNS was designed to fulfil a specific task. The IP protocol used in all Internet transmissions employs numeric addresses. Since these are quite hard to remember for humans, a level of indirection was added: a symbolic name was associated with each IP address. At first these mappings were kept in plain text files that were centrally maintained for the whole Internet. But as the Internet grew, this soon proved unfeasible. Therefore the hierarchical DNS was introduced in which the various branches of the DNS were delegated to organizations, who are responsible for their domain.

The name-to-address mapping tables are kept by so-called name servers. If a client requests an IP address for a non local hostname, the local name server will then retrieve the address of the name server for the target's domain and in turn get the requested IP address from this server on behalf of its client. [16]

General Purpose Directories - X.500

X.500 was developed jointly by the International Telegraph and Telephone Consultative Committee (CCITT), which is now known as the International Telecommunications Union (ITU), and the International Organization for Standardization (ISO) [6]. Their aim was to define a general-purpose directory standard, which would cater for different needs. The ITU-T members, mostly national telecommunications operators, looked for a system, which would provide a white pages service for phone numbers and X.400 [37] email addresses, whereas the ISO was interested mainly in providing a name service for Open Systems Interconnect (OSI) applications. The idea was to build a globally distributed system that would offer homogenous access to the information [6]. That is why in X.500 terms directories are often referred to in the singular, i.e. there exisits one and only one directory--called ``THE Directory'' .^3.1

The first joint meeting was held in April 1986 and the standard documents--the so-called '88 edition--were published in January 1990 by CCITT as ``X.500 Blue Book Recommendations'' and in January 1991 by ISO as ``ISO/IEC 9594 - The Directory''. X.500 consists of multiple parts:

X.500 Overview of concepts, models and services [38]

X.501 Models [39]

X.509 Authentication framework [40]

X.511 Abstract service definition [41]

X.518 Procedures for distributed operation [42]

X.519 Protocol specifications [43]

X.520 Selected attribute types [44]

X.521 Selected object classes [45]

X.525 Replication [46]

X.530 Use of systems management for administration of the Directory [47]

The X.500 standard was updated in 1993 and 1997. Currently the 4th edition is being drafted. While the core functionality was defined in the 1988 edition, some areas (e.g. replication and access control) were left to be addressed later. The 1993 edition then brought the addition of an access control model and a replication mechanism called shadowing. Furthermore the orginial information model specification was significantly revised [98]. Object classes were divided into abstract, structural and auxiliary types; attributes into operational and user types. Name forms, DIT Structure and Content Rules were added; as were Matching Rules. A definition of the directory schema itself was now stored in the directory. This allowed applications to retrieve information about supported schema items.

Figure: Components of an X.500 directory service [30]

[IMAGE png]

As shown in Figure several different protocols are used: A client (DUA) will contact a server (DSA) using the Directory Access Protocol (DAP). If a DUA requests information that is not held by the DSA itself but the DSA knows where in the directory it may be stored, the DSA can retrieve this information on behalf of its clients. This chaining is carried out via the Directory System Protocol (DSP). The replication mechanism introduced in the second edition defined yet another protocol: the Directory Information Shadowing Protocol (DISP).

X.500 is defined in terms of the Abstract Syntax Notation (ASN.1) [36]. This notation is used to specify the protocol data units (PDU) of the X.500 protocols as well as the X.500 information model.

Lightweight X.500 - LDAP

Before this new service could be sucessfully implemented an important barrier needed to be overcome: its dependency on the OSI protocol stack. Due to being a protocol residing at the OSI application level, DAP required a lot of resscources. But the desktop systems of the time did not have the software or the necessary computing power to implement the full OSI protocol stack. Since these were the systems that should provide access to the directory, research was carried out into finding other means of access. These development efforts resulted in two protocols, which used the readily available TCP/IP stack: the Directory Assistance Service (DAS) [82] and the Directory Interface to X.500 Implemented Efficiently (DIXIE) [29]. DIXIE had been designed at the University of Michigan (UMich) and quickly became the protocol of choice. The disadvantage of these two protocols was that they were built to interact with one particular X.500 implementation^3.2 of the original 1988 standard. To overcome this restriction, members of the Open Systems Interconnection - Directory Services (OSI-DS) working group of the Internet Engineering Task Force (IETF) met together and developed the X.500 Lightweight Directory Access Protocol. In July 1993 their first RFCs were published [104,25]. This identified the Proposed Standard for LDAPv2, which was later to be revised by the Access and Searching of Internet Directories (ASID) working group which eventually become a Draft Standard. The resulting RFCs [105,26,49] were published in March 1995. LDAPv1 was never published as RFC.

To summarize, the key aspects of LDAP in comparison with X.500 are:

LDAP is run directly over TCP bypassing session and presentation layers.

While the actual messages in the LDAP protocol are still ASN.1 structures, most protocol data elements are carried as ordinary strings.

A lightweight version of BER^3.3 is used.

LDAP does not support chaining. Instead, clients are directly referred to a server, which has more information about the requested entry.

The first implementation of LDAP was carried out at University of Michigan and was released in mid 1992. It contained the LDAP daemon (ldapd), i.e. a gateway that translated between LDAP and DAP (see Figure ), a client library and also several programs, which were built upon this library.

Figure: LDAP as gateway to X.500

[IMAGE png]

After noticing that the overwhelming majority of queries^3.4 to X.500 DSAs came via LDAP, a new server software was written. It incorporated the actual data store mechanism into the LDAP daemon and was named ``standalone ldap daemon'' (slapd, see Figure ). In addition to a performance boost this had the convenient side effect of eliminating the need to set up and maintain a cumbersome X.500 DSA in order to provide directory services.

Figure: Directory architecture with standalone LDAP servers [30]

[IMAGE png]

The source code for the University of Michigan software has always been freely available. The release of the client library especially added to the success of the software package and of LDAP in general. This development kit made it very easy for programmers add LDAP into their applications. (see cha:sdk)

LDAPv3

The next step in LDAP development was LDAPv3, an outcome of the Access, Searching and Indexing of Directories (ASID) working group within the IETF. The wide use of LDAPv2 revealed some short comings of the protocol which were addressed by the following new features:

Schema as of X.500 (93)--LDAPv3 incorporated most of the X.500 (93) schema.

Feature and schema discovery--A special entry that lists which optional features are supported by a server was defined. This entry is called rootDSE (DSA Specific Entry). Furthermore a description of the schema used by the DSA was made available to clients in another special entry--the subschema subentry.

Internationalisation--LDAPv2 used T.61 as its character set. However, in many deployments this requirement has often been disregarded, and data was stored in the local character set.^3.5 To allow all characters to be stored unambiguously, LDAPv3 uses the UTF-8 encoding scheme of Unicode.

Referrals--While they were a proprietary extension to LDAPv2 in the University of Michigan code, referrals are now a standardized element of the protocol.

SASL--The LDAP bind operation was modified to support the Simple Authentication and Security Layer. It offers advanced authentication mechanisms as well as integrity and privacy protection. (see Section )

Extension mechanisms--by means of extended operations and controls. Extended operations allow for new features to be implemented without the need to modify the protocol itself. Existing operations may now be augmented with controls. They allow for altering the semantics of an operation, for example, to instruct the server to sort the result set from of a search request before sending it to the client.

LDAPv3 maintains downward compatibility with LDAPv2, i.e. an LDAPv2 client can still access an LDAPv3 server. The RFCs defining LDAPv3 were published in December 1997:

RFC 2251: ``Lightweight Directory Access Protocol (v3)'' includes the directory information model and describes the format of the LDAP PDUs. [96]

RFC 2252: ``Attribute Syntax Definitions'' defines how attribute types and object classes are to be constructed. [94]

RFC 2253: ``UTF-8 String Representation of Distinguished Names''. [97]

RFC 2254: ``A String Representation of LDAP Search Filters''. [24]

RFC 2255: ``The LDAP URL Format'' specifies how to represent references to directory data as a Universal Resource Locater (URL). [28]

RFC 2256: ``A Summary of the X.500(96) User Schema for use with LDAPv3'' lists which common X.500 schema elements should be supported by LDAP implementations. [92]

The development effort of LDAP did not stop with the publication of these RFCs. Two new working groups were formed in the IETF: LDAP Duplication / Replication / Update Protocols (LDUP^3.6) and LDAP Extension (LDAPext^3.7). LDUP concerns itself with designing non-proprietary replication mechanisms. Progress in this workgroup has been slow. Chartered in 1998, no document from this working group has reached RFC status yet. Arguments centre on how to retain LDAP semantics in multi-master environments. No unilateral agreement has been reached to date as to how this could be achieved.

An important area of LDAPext's work is concerned with security: Although LDAPv3 includes support for SASL, no mechanisms which avoid plain text passwords were mandated by the 1997 RFCs. They thus carry a note of the IESG^3.8, which says that the update operation should not be used until appropriate authentication mechanisms have been defined. These concerns have been addressed recently. [93] discusses which kind of protection would be needed for a specific operation and [21] gives the details on the implementation of data protection on the network by means of TLS (see Section ). These additional RFCs are now considered to be part of the core standard [20]. Other work items of LDAPext include:

Access control--To define a model by which access control information can be stored in the directory.

Language tags--The definition of attribute subtypes, which identify the language of a value.

Server-side sorting and paged results--The definition of controls, which instruct a server to sort the results of a query, or to return the results in small parts at a time.

Referral management--How to store knowledge information about data held by other servers.

APIs-- A definition of a Java API and an update of the LDAPv2 C API specification to support the new LDAPv3 features.

CLDAP--A specification of connection-less LDAP using the UDP transport layer.

In December 2000 another LDAP working group has been formed in the IETF: LDAP (v3) Revision (LDAPbis^3.9). The purpose of this working group is to revise the current documents so that they become suitable for being considered as ``Draft Standard''--the next step from ``Proposed Standard'' in the Internet Standards Process

Text-based Standards

There were some other standards that attempted to become the ``lingua franca'' for directory access--namely RWhois [103] and Whois++ [7]-- but they never gained the support and wide spread use that LDAP has. These are text-based protocols, i.e. the data is carried over a network in human-readable form. RWhois and Whois++ are derived from Whois [18] which itself is a descendant of the Finger [108] protocol. [16]

One area in which the Whois protocol is still used, is the registry service for the DNS system. But there are ongoing experiments to store this information in LDAP servers. The company NSI maintains the registry for the .com, .org and .net domains. Since the autumn of 2000 the Applied Research Department of VeriSign, NSI's parent company, offers an LDAP access^3.10 to this information.