python-ldap: Retrieve only a few entries from LDAP search

user_3821298 0
Viewed 6380

I wish to mimic the ldapsearch -z flag behavior of retrieving only a specific amount of entries from LDAP using python-ldap.

However, it keeps failing with the exception SIZELIMIT_EXCEEDED.

There are multiple links where the problem is reported, but the suggested solution doesn't seem to work

Python-ldap search: Size Limit Exceeded

LDAP: ldap.SIZELIMIT_EXCEEDED

I am using search_ext_s() with sizelimit parameter set to 1, which I am sure is not more than the server limit

On Wireshark, I see that 1 entry is returned and the server raises SIZELIMIT_EXCEEDED. This is the same as ldapsearch -z behavior

But the following line raises an exception and I don't know how to retrieve the returned entry

conn.search_ext_s(<base>,ldap.SCOPE_SUBTREE,'(cn=demo_user*)',['dn'],sizelimit=1)
python-ldap ldap
3 Answers
user_100881780

The accepted answer works if you are searching for less users than specified by the server's sizelimit, but will fail if you wish to gather more than that (the default for AD is 1000 users).

Here's a Python3 implementation that I came up with after heavily editing what I found here and in the official documentation. At the time of writing this it works with the pip3 package python-ldap version 3.2.0.

def get_list_of_ldap_users():
    hostname = "google.com"
    username = "username_here"
    password = "password_here"
    base = "dc=google,dc=com"

    print(f"Connecting to the LDAP server at '{hostname}'...")
    connect = ldap.initialize(f"ldap://{hostname}")
    connect.set_option(ldap.OPT_REFERRALS, 0)
    connect.simple_bind_s(username, password)
    connect=ldap_server
    search_flt = "(cn=demo_user*)" # get all users with a specific cn
    page_size = 1 # how many users to search for in each page, this depends on the server maximum setting (default is 1000)
    searchreq_attrlist=["cn", "sn", "name", "userPrincipalName"] # change these to the attributes you care about
    req_ctrl = SimplePagedResultsControl(criticality=True, size=page_size, cookie='')
    msgid = connect.search_ext_s(base=base, scope=ldap.SCOPE_SUBTREE, filterstr=search_flt, attrlist=searchreq_attrlist, serverctrls=[req_ctrl])

    total_results = []
    pages = 0
    while True: # loop over all of the pages using the same cookie, otherwise the search will fail
        pages += 1
        rtype, rdata, rmsgid, serverctrls = connect.result3(msgid)
        for user in rdata:
            total_results.append(user)

        pctrls = [c for c in serverctrls if c.controlType == SimplePagedResultsControl.controlType]
        if pctrls:
            if pctrls[0].cookie: # Copy cookie from response control to request control
                req_ctrl.cookie = pctrls[0].cookie
                msgid = connect.search_ext_s(base=base, scope=ldap.SCOPE_SUBTREE, filterstr=search_flt, attrlist=searchreq_attrlist, serverctrls=[req_ctrl])
            else:
                break
        else:
            break
    return total_results
user_74873350

Based upon the discussion in the comments, this is how I achieved it:

import ldap

# These are not mandatory, I just have a habit
# of setting against Microsoft Active Directory

ldap.set_option(ldap.OPT_REFERRALS, 0)
ldap.set_option(ldap.OPT_PROTOCOL_VERSION, 3)

conn = ldap.initialize('ldap://<SERVER-IP>')
conn.simple_bind(<username>, <password>)

# Using async search version
ldap_result_id = conn.search_ext(<base-dn>, ldap.SCOPE_SUBTREE,
                                 <filter>, [desired-attrs],
                                 sizelimit=<your-desired-sizelimit>)
result_set = []
try:
  while 1:
    result_type, result_data = conn.result(ldap_result_id, 0)
    if (result_data == []):
      break
    else:
      # Handle the singular entry anyway you wish.
      # I am appending here
      if result_type == ldap.RES_SEARCH_ENTRY:
        result_set.append(result_data)
except ldap.SIZELIMIT_EXCEEDED:
  print 'Hitting sizelimit'

print result_set

Sample Output:

# My server has about 500 entries for 'demo_user' - 1,2,3 etc.
# My filter is '(cn=demo_user*)', attrs = ['cn'] with sizelimit of 5

$ python ldap_sizelimit.py

Hitting sizelimit
[[('CN=demo_user0,OU=DemoUsers,DC=ad,DC=local', {'cn': ['demo_user0']})], 
[('CN=demo_user1,OU=DemoUsers,DC=ad,DC=local', {'cn': ['demo_user1']})], 
[('CN=demo_user10,OU=DemoUsers,DC=ad,DC=local', {'cn': ['demo_user10']})], 
[('CN=demo_user100,OU=DemoUsers,DC=ad,DC=local', {'cn': ['demo_user100']})], 
[('CN=demo_user101,OU=DemoUsers,DC=ad,DC=local', {'cn': ['demo_user101']})]]

You may use play around with more srv controls to sort these etc. but I think the basic idea is conveyed ;)

user_38212980
Related