Dynamics 365 Data Export Service: a Solution for CRM Online direct data access
Each customer I’ve ever worked for on switching from On-Premise to Cloud Dynamics CRM deployment asked questions like:
Finally, I can now answer: YES! … And it works great.
Additionally, each of us, on some projects, had to come up with solutions on how to replicate Dynamics 365 (online) CRM data without the added complexity of managing an additional ETL tool; or how to keep synced data up-to-date where all updates in CRM Online are quickly synced without running frequent scheduled jobs.
But now we have the new Dynamics 365 – Data Export Service, a new Dynamics 365 App … and it’s free to use with Dynamics 365 Online.
Data Export Service allows us to replicate Dynamics 365 Online data to a Microsoft Azure SQL Database store in a customer-owned Microsoft Azure subscription. Data Export App allows us to synchronize the entire Dynamics 365 schema and data initially and then continues to synchronize data updates from Dynamics 365 Online organisation. The supported target destinations are Microsoft Azure SQL Database and Microsoft Azure SQL Server on Microsoft Azure virtual machines. Furthermore, Data Export service provides an interface for managing configuration and ongoing administration from within Dynamics 365.
Having your data in Azure SQL database enables several possibilities, since it can be used for custom reporting, integration with legacy systems or for building machine learning models.
Prerequisites to use Data Export Service are:
There are quite a few capabilities that Data Export service includes:
Furthermore, the service also exposes a REST-based API that is divided into two groups:
The API documentation can be found here: https://discovery.crmreplication.azure.net/swagger/ui/index#
As mentioned above, Data Export service can be installed from AppSource. Once installed, login to Dynamics 365 as administrator and from the Settings menu:
Next step will be to define data export profiles. Creating a new Data Export Profile is a wizard driven process and it requires us to select the entities and relationships with a few other additional profile specific settings such as name, table prefix, or enable delete log.
Quick reminder: The only entities shown in the list are the ones for which change tracking is enabled.
Export profiles, once activated, start the data synchronization – first with metadata sync, followed by initial data sync, and then with continuous delta data sync.
Metadata sync automatically creates and updates tables in the destination database.
During initial sync, the existing data in Dynamics 365 for added entities is synchronized with the destination database. After initial sync, delta sync starts, and any create, modify or delete record change for entities added to export profile is automatically synchronized.
Progress on all of the above can be followed with status and notification counters from Dynamics 365 UI.
Metrics for data synchronization during initial sync and delta sync are shown in terms of notification counters. A notification encapsulates a record change – it maps to create, modify, or delete change of a record. There are 3 main metrics that are captured:
Notification counters at the end of initial sync show a count for all records synchronized from Dynamics 365. For any errors that occurred during synchronization, you can click Download Failed Records to obtain a URL to connect with Azure Storage Explorer to see the log of failures and address the reason behind those failures such as database is down or out of space.
More information and detailed steps to setup Data Export service can be found here: https://blogs.msdn.microsoft.com/crm/2017/04/11/introduction-to-dynamics-365-data-export-service/
All in all, it’s a great new feature that helps us with customers where we need to move the on-premise Dynamics CRM to the cloud and we need to save time and cost for all custom reporting or integrations with legacy systems.
Our customers are very excited after we introduced them to Data Export service, how about yours?