This is the third article in the article mini-series on Python LDAP applications by Matt Butcher. The first part deals with the installation and configuration of Python-LDAP library, and the binding-unbinding operations, and changing of the LDAP password. The second article takes a look at some of LDAP operations.

In this article we will see some more LDAP operations such as add operation, delete operation etc. Then we will take a look at LDAP URL Library.

The ModRDN Operation

Another simple write operation that can be done through the Python-LDAP API is the ModRDN operation. This operation is used to change the relative DN (RDN) of a record.

We can change an RDN using the modrdn() or modrdn_s() method. These two methods take three parameters:

• The full DN
• The new RDN
• An optional flag indicating whether the attribute corresponding to the RDN should be deleted from the record

For example, if we want to change the UID attribute for uid=manny,ou=users,dc=example,dc=com, we will need to use a ModRDN operation, since this attribute is used in the DN. Here's an example for changing the UID from manny to immanuel.

>>> l.modrdn_s('uid=manny,ou=users,dc=example,dc=com',
...   'uid=immanuel', False)
(109, [])
>>> l.compare_s('uid=immanuel,ou=users,dc=example,dc=com','uid',
...   'immanuel')
1
>>>

In this example, we first use modrdn_s() to change the DN of a record from uid=manny,ou=users,dc=example,dc=com to uid=immanuel,ou=users,dc=example,dc=com. The False flag at the end of the modrdn_s() method indicates that the old UID (uid=manny) should be left in the record. The LDIF for uid=immanuel's record now, after the ModRDN operation, looks something like this:

dn: uid=immanuel,ou=Users,dc=example,dc=com
cn: Manny Kant
givenName: Manny
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
ou: Users
sn: Kant
uid: immanuel
uid: manny

If we had set the last flag to True instead of False, the manny attribute value of uid would have been deleted.

More sophisticated DN modifications can be made with the rename() and rename_s() methods. But your OpenLDAP server will need to be running the HDB backend for all of the renaming features to work.

The Add Operation

The LDAP add operation is used to add new (complete) records to the directory information tree.

Here, we will look at adding records through the add() and add_s() methods of the LDAPObject class. Both of these methods take only two parameters:

• The string DN of the new record
• A list of attribute tuples

While the first parameter is straightforward, we've looked at dozens of DNs already; the second attribute is a little trickier.

The addition list looks something like this:

add_record = [
 ('objectclass', ['person','organizationalperson','inetorgperson']),
 ('uid', ['francis']),
 ('cn', ['Francis Bacon'] ),
 ('sn', ['Bacon'] ),
 ('userpassword', ['secret']),
 ('ou', ['users'])
]

If there is only one value in the attribute value list, the value can be just a string – it need not be a list. Example: ('ou', 'user') is an acceptable alternative to ('ou', ['user']).

The list of attributes is made up of two-value tuples, where the first item of each tuple is the attribute name, and the second value is a list of attribute values. All of the values are expected to be strings.

If you have values in a dictionary, where the attribute name is the key and the attribute values are stored in a list in the dictionary value, you can use the ldap.modlist module's addModList() function to create an attributes list in the form specified above.

Once you have a list in the correct format, writing it to the directory is just a matter of executing the add() or add_s() method.

>>> l.add_s('uid=francis,ou=users,dc=example,dc=com', add_record)
(105, [])
>>>

This line performs an LDAP add operation, sending this new data to the server. The server ensures that the new record adheres to the appropriate schemas (e.g. the schemas for the person, organizationalPerson, and inetOrgPerson object classes), and then writes the entry to the directory.

As might be expected, the add() method functions the same way that the add_s() method does, except that it returns an ID number. The result must be retrieved using the result() method.

We can dump the new entry from the server (using the dump_record.py program developed earlier in the series) to verify that the record is as we expect it to be:

$ ./dump_record.py 'uid=matt,ou=users,dc=example,dc=com' 'uid=francis,
    ou=users,dc=example,dc=com'
Password for uid=matt,ou=users,dc=example,dc=com:

dn: uid=francis,ou=users,dc=example,dc=com
cn: Francis Bacon
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
ou: users
sn: Bacon
uid: francis
userPassword: secret

We can tell by comparing this record with the add_record list above that the record is correct.

The main error encountered when adding is violating the schema, either by adding attributes that are not supported, or by failing to add required attributes. When one of these conditions is met, an exception will be raised.

For example, if no structural object class is specified in the attributes, an OTHER exception will be raised. If a record does not contain the attributes used in the UID, a NAMING_VIOLATION will be raised. If a record is missing an attribute required by a structural object class, an OBJECT_CLASS_VIOLATION will be raised, and so on.

Of course, since all of these are subclasses of LDAPError, these numerous exceptions can all be caught in a try/except clause like this:

>>> try:
...     l.add_s('uid=william,ou=users,dc=example,dc=com', attrs )
... except ldap.LDAPError, e:
...     print e.message['info']
... 

This will catch any of the LDAP exceptions, and display some of the error text, rather than showing the stack trace.

Now we are ready to move on to the most complicated of writing operations: the LDAP modify operation.

Mastering OpenLDAP: Configuring, Securing and Integrating Directory Services

You can continue to learn more about OpenLDAP and its applications in the book Mastering OpenLDAP: Configuring, Securing and Integrating Directory Services.

The Modify Operation

Here we will look at the LDAP modify operation, which is used for modifying attributes – adding, replacing, or removing them from already-existing records.

The OpenLDAP command line tool ldapmodify provides one way of performing this operation. In the Python-LDAP library, the modify() and modify_s() methods provide asynchronous and synchronous methods for performing modifications to the directory information tree.

The signature of these methods is same as that of the add methods. There are two parameters: the DN and a list of modification tuples. The main difference is that the form of the tuples in this modification list is different than those in the add methods.

A tuple in a modification list has three items:

• The modification type
• The attribute name
• A list of attribute values

Modification type is one of three different constants defined in the ldap module:

• MOD_ADD: This is used to add an attribute value. If the attribute already exists (and the schema permits multiple values), the new value will be added, and the old value will remain.
• MOD_DELETE: The attribute value will be removed, if it exists.
• MOD_REPLACE: The given attribute values will replace all other values for that attribute name. In other words, all old values for the attribute will be deleted, and then this value will be added.

For example, a simple list for adding a new givenName to an existing entry might look like this:

mod_attrs = [( ldap.MOD_ADD, 'givenName', 'Francis' )]

This list contains only one attribute to be modified. It will (if successful) add a new givenName attribute to the specified record. The modification can then be done with code like this:

>>> mod_attrs = [( ldap.MOD_ADD, 'givenName', 'Francis' )]
>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)
(103, [])
>>> 

This will add the specified attribute value to the uid=francis record that we created above. As a result, dumping the LDIF record will show the newly added attribute:

dn: uid=francis,ou=users,dc=example,dc=com
cn: Francis Bacon
givenName: Francis
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
ou: users
sn: Bacon
uid: francis
userPassword: secret

The highlighted line above shows the newly added attribute value.

The modifyModList() function in the ldap.modlist module can help convert modification lists stored in dictionaries to the appropriate tuple-based format.

What if Francis decided that he preferred to go by Frank? We could perform a slightly more sophisticated modification, changing his givenName to Frank, and adding a second CN value:

>>> mod_attrs = [
...   ( ldap.MOD_REPLACE, 'givenName', 'Frank' ),
...   ( ldap.MOD_ADD, 'cn', 'Frank Bacon' )
... ]
>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)
(103, [])
>>> 

Notice that our modification list now has two different modifications. First, it will replace givenName. Second, it will add a new cn attribute value. The result will be something like this:

dn: uid=francis,ou=users,dc=example,dc=com
cn: Francis Bacon
cn: Frank Bacon
givenName: Frank
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
ou: users
sn: Bacon
uid: francis
userPassword: secret

If we wanted to change the UID attribute, we would have to use the modrdn() or modrdn_s() method, since uid is used in the DN. If we try to change it with modify_s() or modify(), we will get a NAMING_VIOLATION exception.

Finally, we can use the modify methods to remove attribute values:

>>> mod_attrs = [ (ldap.MOD_DELETE, 'cn','Francis Bacon') ]
>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)
(103, [])
>>>

This will remove only the attribute value Francis Bacon from the cn attribute. If no such value exists, a NO_SUCH_ATTRIBUTE exception will be raised. Otherwise, the value will be discarded.

Note that some attributes are required by the record's object classes to be present in an entry. Attempting to delete the last value for such an attribute will result in an OBJECT_CLASS_EXCEPTION being raised.

Removing All Attribute Values

Sometimes it is necessary to remove all of the values for an attribute in a record, instead of just one specific value, as we did above. Let's look at an example.

First, we add a few attribute values – two descriptions:

>>> mod_attrs = [ 
...   (ldap.MOD_ADD, 'description', 'Author of New Organon'),
...   (ldap.MOD_ADD, 'description', 'British empiricist') 
... ]
>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)
(103, [])

Now we have a record with two new descriptions. We can perform a very specific search to verify this.

>>> l.search_s('uid=francis,ou=users,dc=example,dc=com', 
...   ldap.SCOPE_BASE, '(uid=francis)',['description'])
[('uid=francis,ou=users,dc=example,dc=com', {'description': ['Author of
      New Organon', 'British empiricist']})]

This search looks at just the uid=francis record, and shows just the description attributes. Now, how can we delete both of these attribute values without having to supply the exact attribute values for each?

We can do this removal by creating a modification entry that uses None instead of a string for the final item in the attribute tuple:

>>> mod_attrs = [( ldap.MOD_DELETE, 'description', None )]
>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)
(103, [])
>>>

A simple search will verify that both description attribute values have been deleted:

>>> l.search_s('uid=francis,ou=users,dc=example,dc=com', 
...   ldap.SCOPE_BASE, '(uid=francis)',['description'])
[('uid=francis,ou=users,dc=example,dc=com', {})]
>>> 

The server returned one entry – one with the DN for uid=francis – but since there were no description attribute values, the dictionary is empty.

Mastering OpenLDAP: Configuring, Securing and Integrating Directory Services

You can continue to learn more about OpenLDAP and its applications in the book Mastering OpenLDAP: Configuring, Securing and Integrating Directory Services.

The Delete Operation

The last LDAP operation we will look at is the delete operation. This operation removes an entire entry from the directory information tree.

In the Python-LDAP library, it is implemented with the delete() (asynchronous) and delete_s() (synchronous) methods. These delete methods take only one parameter: the DN of the record to delete.

We can, for example, delete the record created earlier.

>>> rid = l.delete('uid=francis,ou=users,dc=example,dc=com')
>>> l.result(rid)
(107, [])

Here, the asynchronous delete() method is used to delete uid=francis from the directory information tree. Now, if we were to search of the record, a NO_SUCH_OBJECT exception would be raised.

At this point, we have looked through all of the basic operations. Next, we will take a quick look at some of the other functions in the Python-LDAP library.

The LDAP URL Library

As we saw when discussing the search method in the earlier article, performing an LDAP search requires specifying several different pieces of information: the base DN, the scope, the filter, the list of attributes returned, and so on.

All of this can be represented in a compact form, as an LDAP URL. For example, we can build an LDAP URL that contains the information necessary to perform a subtree search for any entry in the ou=users branch with a user ID, and then return mail addresses for all such records:

ldap://localhost/ou=users,dc=example,dc=com?mail?sub?(uid=*)

This filter indicates that the connection should be made (over the standard LDAP protocol) to localhost. The Base DN should be set to ou=users,dc=example,dc=com. And the mail attribute should be returned for any user in that branch of the tree (sub) who has a user id (uid=*).

The Python-LDAP library has a module designed to convert LDAP URLs to a set of components, or vice versa.

For example, building on work we have done earlier, we can create a very simple script that takes an LDAP URL, binds to the directory as anonymous, and performs a search, returning the results in LDIF form.

#!/usr/bin/env python
import ldapurl
import ldap, ldaphelper, sys

usage="""Usage: %s ldap_url

Perform an anonymous search based on the LDAP URL.n""" % 
    sys.argv[0]

if len(sys.argv) != 2:
    sys.stderr.write("Error: expected one URL or --help.nn")
    sys.stdout.write(usage)
    sys.exit(1)

url = sys.argv[1]

if url == '-h' or url == '--help':
    sys.stdout.write(usage)
    sys.exit()

if not ldapurl.isLDAPUrl( url ):
    sys.stderr.write("%s is not a valid LDAP URL.n" % url)
    sys.exit(1)

url_parts = ldapurl.LDAPUrl( url )

con_string = "%s://%s" % (url_parts.urlscheme, url_parts.hostport)
try:
    l = ldap.initialize(con_string)
    try:
        #l.start_tls_s()
        l.bind_s('', '') # anonymous bind

        raw_res = l.search_s(
            url_parts.dn,
            url_parts.scope,
            url_parts.filterstr,
            url_parts.attrs
        )

        res = ldaphelper.get_search_results( raw_res )

        for record in res:
            sys.stdout.write(record.to_ldif())

    except ldap.LDAPError, e:
        if type(e.message) == dict:
            for (k, v) in e.message.iteritems():
                sys.stderr.write("%s: %sn" % (k, v) )
        else:
            sys.stderr.write(e)
        sys.exit(1)

finally:
    try:
        l.unbind()
    except Error:
        pass

This script is in many ways similar to the earlier scripts we have created. We begin by importing the ldapurl module, which is used to parse the LDAP URL, then we import the main ldap module, and the ldaphelper module that we created earlier, as well as sys.

The next sixteen lines are boilerplate Python. We create a usage message and make sure that a URL has been passed in from the command line. This program takes one parameter, the LDAP URL. This will be stored in sys.argv[1] (sys.argv[0] will contain the name of the script). This gets stored in the variable url.

Now we begin using the ldapurl module; we use it to check the syntax of the user-supplied URL:

if not ldapurl.isLDAPUrl( url ):
    sys.stderr.write("%s is not a valid LDAP URL.n" % url)
    sys.exit(1)

The isLDAPUrl() function checks to see if the passed-in string is in the correct format. This test is very basic, but it will catch any obviously non-URL strings.

Once we know the URL is in roughly the correct format, we can parse the URL into its parts:

url_parts = ldapurl.LDAPUrl( url )

Now, url_parts is an LDAPUrl object containing information about the URL. But we need to do a little bit of re-constructive work. The ldap.initialize() function takes a string in the form of a basic LDAP URL (one that contains just the protocol, host, and port information). We can create that URL using information from the url_parts object:

con_string = "%s://%s" % (url_parts.urlscheme, url_parts.hostport)

Using Python's string formatting, we have now created a URL for the form urlscheme://host:port. This will look something like ldap://localhost or ldaps://example.com:636.

With this information at hand, we are ready to connect to the server and run the search. As usual, it is a good idea to wrap as much of the LDAP code inside of a try/finally block (to make sure that we unbind correctly) and a try/except block to make sure that we catch any exceptions that are raised:

try:
    l = ldap.initialize(con_string)
    try:

        # Some lines omitted

    except ldap.LDAPError, e:
        if type(e.message) == dict:
            for (k, v) in e.message.iteritems():
                sys.stderr.write("%s: %sn" % (k, v) )
        else:
            sys.stderr.write(e)
        sys.exit(1)

finally:
    try:
        l.unbind()
    except Error:
        pass

This should handle catching any exceptions that arise during our LDAP transactions. Now we are ready to look at the lines omitted from the example above:

l.bind_s('', '') # anonymous bind

raw_res = l.search_s(
    url_parts.dn,
    url_parts.scope,
    url_parts.filterstr,
    url_parts.attrs
)

res = ldaphelper.get_search_results( raw_res )

for record in res:
    sys.stdout.write(record.to_ldif())

First, we bind as anonymous, using simple_bind_s() with two empty strings. A server that does not allow anonymous binds might raise an exception at this point.

Next, we construct a new search, based on the contents of the URL:

raw_res = l.search_s(
    url_parts.dn,
    url_parts.scope,
    url_parts.filterstr,
    url_parts.attrs
)

All the components we need to search are stored in the LDAPUrl object:

• Base DN is in dn.
• The search scope identifier is in scope.
• The search filter is in filterstr.
• And the list of attributes to return is in attrs.

Note that there is not a way to specify timeout information in the LDPA URL, so if you want to use search_st(), you will need to specify the timeout in some other way.

From this point, the rest of the script is similar to previous example.

res = ldaphelper.get_search_results( raw_res )

for record in res:
    sys.stdout.write(record.to_ldif())

The results are converted to a list of LDAPSearchResult objects, which are then printed, in LDIF form, to standard output.

This script exemplifies how ldapurl can be used to simplify that task of parsing LDAP URLs. It can also be used to generate LDAP URLs, using a longer form of the constructor, which takes up to nine parameters:

• urlscheme: The protocol scheme, such as ldap or ldaps.
• hostport: The host and port, separated by a colon (:).
• dn: The base DN.
• attrs: A list of attribute names.
• scope: A scope constant, like ldap.SCOPE_BASE.
• filterstr: A search filter, like (objectclass=person).
• extensions: An LDAPUrlExtensions object used to provide information about an LDAP extension.
• who: A DN used for binding.
• cred: A password used for binding.

Because Python doesn't allow overriding of constructors, and because the first element of the constructor is an LDAP URL, this constructor must be called either like this:

>>> new_url = ldapurl.LDAPUrl(None, 'ldap', 'localhost', 
...    'dc=example,dc=com')

Where None is the first parameter, or, you can use named parameters:

>>> new_url = ldapurl.LDAPUrl( 
...    urlscheme='ldap', 
...    hostport='localhost', 
...    dn='dc=example,dc=com')

Once the object has been initialized, you can get the value by calling the unparse() method:

>>> new_url.unparse()
'ldap://localhost/dc=example,dc=com???' 

This will convert the object to the closest possible LDAP URL that it can. In other words, it will not fill in values (like scope) that are not specified explicitly.

The fourth and last article in this series will cover LDAP schemas.

Mastering OpenLDAP: Configuring, Securing and Integrating Directory Services

You can continue to learn more about OpenLDAP and its applications in the book Mastering OpenLDAP: Configuring, Securing and Integrating Directory Services.

About The Author