ForgeRock introduced the data migration service in Version 7 for migrating different IDM objects, including the managed/user from an earlier version. This is an interesting and simple tool for importing various types of IDM data. The result is less configuration and much better performance.
The service is essentially a reconciliation service from the new IDM instance using the original as the source. It is now possible to migrate data from older versions going back as far as Version 4 with the important feature of operating at the repo level, not at the managed object level. This means that data is migrated directly between the IDM backends instead of going through IDM. The advantage of this feature is that the migration is faster compared to the service being at the managed object level.
Using the migration service, we successfully migrated over 7 million users from Version 6.0.0.7. Some of the ways we optimized the migration were managing link prefetching, paging and policy enforcement. Tuning the available threads and page size was also required to improve the migration performance for the large dataset.
A limitation of the migration service is inherent in any large user dataset. When migrating millions of users, the migration service can take many hours, sometimes days. For a live system, this means that there will most likely be changes to the data during the migration. To capture these changes, we adapted the migration service, which we referred to as δ-migration. One cool feature of this δ-migration is that it can be used multiple times, if needed, and each subsequent δ-migration will need less time. Another limitation is inherent in the technology used. Since the service works at the repo level, it does not trigger any implicit synchronization present from IDM to any external directory or database.
The migration service is an important addition offered by ForgeRock, helping clients to efficiently migrate large amounts of data between different versions of IDM with much more ease than before.
