Provisioning Users and Groups Using Active Directory Kit

April 23, 2014

Egnyte’s enterprise file sharing is in high demand as the world seeks to exchange data via the cloud. User and group management remains very central to every step of the file-sharing feature, so I thought it would be interesting to highlight the process we’ve taken over the last few years to simplify provisioning users and groups into the Egnyte cloud.

I’ve been working in the engineering team for four years, and I still remember how time consuming it used to be to manually create numerous users and groups in Egnyte. We quickly realized that creating a tool that could mass provision users and groups from their Active Directory (AD) would make life much easier for IT admins.

Version OneThe first version of the tool - the ADKIT, as we called it, was out in few days. The purpose of the tool was to enable bulk provisioning for users and groups from a customer’s AD into their Egnyte cloud account. We decided to keep it simple: a command line tool that ran on a Microsoft Windows DOS prompt, read a configuration INI file, extracted users in a TSV (tab separated file), and then added or updated users to an Egnyte cloud account.

Design and choice of technologies were simple too: LDAP protocol to fetch users from the AD and a private REST API to add/update users on the Egnyte cloud. Being in love with Python, there wasn’t a second thought for the language of choice. There is a nice Python-LDAP module that sufficed the LDAP operations we needed, and we used PycURL for the REST API calls to the Egnyte cloud.Basic Actions of the ADKITWhen we initially launched this feature, we only supported “add” and “update” for groups. We then added in syncing to eliminate the need for customers to explicitly call for “add,” “update,” and “delete” separately. Here’s what sync means for these objects:

Add or update a user from AD to the Egnyte cloud if that user was not provisioned in the cloud.
Add or update a group from the AD to the Egnyte cloud if that group was not provisioned in the cloud. Updating a group means adding new users to the Egnyte cloud first and then associating the users to the group.
Delete a user or a group from the Egnyte cloud if the user/group was not present in AD but was present in the cloud. Deleting of a group from the cloud does not mean the users themselves get deleted.

Flexibility of Extracting Users and GroupsBefore I jump into extracting users and groups, I want to note that the ADKIT fetches the results from AD using paged results control. This is a scalable way of fetching thousands of objects from AD. We set the page size to 500. For your reference, here’s how it’s done using Python-LDAP. Look for SimplePagedResultsControl.Having helped many customers with AD integration, we knew each customer’s AD structure was different, and each customer wanted the kit to integrate into different parts of their AD and sync specific users and groups into the Egnyte cloud. We gave this flexibility by having different key-value parameters in the INI file.The basics of an LDAP search is that you need a base Distinguished Name (DN) from where to search, a search filter to filter results, and specific attributes of the objects. We provided a way to enter the base DN from where the search began, a default search filter, like below:base_dn=dc=egnyteads,dc=comuser_search_filter=(&(objectclass=user)(objectclass=organizationalPerson)(objectclass=person))group_search_filter=(&(objectclass=group)(|(groupType=-2147483640)(groupType=-2147483644)(groupType=-2147483646)))Now, the base DN above is the root DN in the AD. But, what if you want to look for users from certain organizational units independently without looking at groups from other organizational units. For this, we have two other parameters to look for users and groups separately:user_inclusion_by_ou_filter=ou=moutain view,ou=engineering;ou=poznan,ou=qa

group_inclusion_by_ou_filter=ou=mountain view groups;ou=poznan groups

The values for the above OU filters are relative DNs (RDNs) separated by a semi-colon, and these get individually prefixed to the base DN to search for users and groups respectively for that filter - example below:User base DN’sou=moutain view,ou=engineering,dc=egnyteads,dc=com

ou=poznan,ou=qa,dc=egnyteads,dc=com

Group base DN’sou=mountain view groups,dc=egnyteads,dc=com

ou=poznan groups,dc=egnyteads,dc=com

ADKIT also gave a way to sync users from a list of security groups using the following parameter:

user_inclusion_by_group_filter=mountain view engineering,poznan engineeringThe value for user_inclusion_by_group_filter above becomes a list of two groups to sync users from mountain view engineering and and poznan engineering, where each of these names are assumed as the sAMAccountName values of their respective group objects in the AD. Users that are common to these groups and who also belong to one of the OUs specified in the user_inclusion_by_ou_filter are synced to the Egnyte cloud.The ADKIT supports both security and distribution group types and so do the group scopes like universal, global and domain local, which the groupType attribute represents in the group search filter above.One of the challenging aspects of hierarchically group searching for users is the presence of cycles, and we decided not to support the cyclic membership groups. For example, groupA can contain groupB as its member, and if groupB contains groupA as its member, then the ADKIT does not support such group memberships. This required us to do a DFS (Depth First Search) for each top-level group.Mapping AD group membership to the Egnyte cloudI want to briefly mention how we sync’d AD groups to the Egnyte cloud. The Egnyte cloud does not support hierarchical groups, so the problem at hand was how to map the group hierarchy in AD to the Egnyte cloud. First off, even if a group includes groups as its members, the kit would create flat groups in the Egnyte cloud at the same level. However, we still had to preserve the hierarchy for users. For example, in the AD, if a groupA has userA1 and groupB as its members and groupB has userB1 as its member, userB1 should also be part of groupA because groupB is part of groupA. So, we came up with 3 options to sync this hierarchy for associating users into groups:

Inherit - The kit will create both groupA and groupB in the cloud, and as in my example above, in the Egnyte cloud, we associate all the users up to their parent groups in the AD. So, in the Egnyte cloud, groupA will have userA1, userB1 and groupB will have userB1.
No inherit - This will keep the users at the same level where they are. So with the above example, groupA and groupB will be created, and groupA will only have userA1 and groupB will only have userB1.
Flatten - This option creates only top level groups in the Egnyte cloud but associates all the users up to the leaf group members in hierarchy in the AD. So with the above example, only groupA will be created, and it will have both userA1 and userB1 users.

Setting user authentication type on the Egnyte CloudAmong the eight attributes of a user, I want to highlight the setting of user authentication type and role while syncing the user from the AD to the Egnyte cloud. A user can get synced with the option of AD, SSO and Egnyte authentication types.

AD - When a user enters the username and password on the Egnyte login page, a successful LDAP bind to the customer’s Active Directory authenticates the user.
SSO - Egnyte supports federated single sign-on authentication using SAML 2.0. If an Egnyte account is setup with an Identity Provider, like Okta, PingIndentity, or Onelogin, an added user can authenticate via SAML 2.0 on the cloud.
Egnyte - With Egnyte’s authentication, when a user is first created as part of ADKIT’s bulk provisioning action, a very strong, secure, but random, password is initially given to the user. The user is then guided through a password reset link from a sent email to reset its password.

A user can be synced with the following roles:

Admin - Admin of the Egnyte cloud account
Power - A user with superuser privileges
Guest - A user with minimal privileges

Over the last few years, we’ve made some great enhancements to simplify provisioning users and groups, and we will continue to add and enhance the functionality to streamline processes for our customers. For additional information, check out our helpdesk article on our Active Directory Integration.*Manish Marathe is a senior developer at Egnyte and is currently responsible for optimizing sync performance.