CDX extensibility – Synchronizing data from channel to HQ via P-job



By default, there are very few tables that get synchronized from the store to the HQ. These are mostly tables related to the transactions being generated. However, there are scenarios where customers may want to create other data in their own tables in the channel and bring it back to HQ. In many cases, that may not need to happen in real-time and this task can be accomplished by using the pull job architecture. Below are the steps how to set this up.

Dynamics 365 for Operations table creation

Since the records will trickle in from multiple channels, stores, terminals, the table’s index must include the data that differentiate the records. A good example is the RetailTransactionTable to look at how it’s done.
Create the table with the AOT and follow these guidelines:

  • A column called “REPLICATIONCOUNTERFROMORIGIN” of type RetailReplicationCounter needs to be added.
  • Add the other data columns that you would like to synchronize (in this example, FieldStr1 and FieldInt1)
  • Set “Created Date Time” and “Modfied Date Time” to Yes.
  • Create an index that includes the fields that are needed to differentiate the records well. For retail stores, that could be Channel, Store, Staff, Terminal and the id of the record as called ChannelOriginatedDataId in this example.


Note: How to put this file into VSTS so the change is reproducible and deployable into other environments is omitted here.

Create the table schema for the Channel database

Create a table with the same name but schema [ax] in SSMS. Here are some specific requirements to follow:

  • Include columns called [REPLICATIONCOUNTERFROMORIGIN], [ROWVERSION], [CREATEDDATETIME] and [MODIFIEDDATETIME] (details below)
  • If the table in HQ is per company (SaveDataPerCompany = yes), include a column [DataAreaId]
  • Create a clustered unique index for column [REPLICATIONCOUNTERFROMORIGIN]
  • Create a nonclustered primary key index for columns that uniquely differenciate the records, including [DATAAREAID] if data is saved per company
IF (SELECT OBJECT_ID('ax.CHANNELORIGINATEDDATATABLE')) IS NULL 
BEGIN
	CREATE TABLE [ax].[CHANNELORIGINATEDDATATABLE](
		[CHANNELORIGINATEDDATAID] [nvarchar](44) NOT NULL,
		[FIELDSTR1] [nvarchar](200) NOT NULL DEFAULT (''),
		[FIELDINT1] [int] NOT NULL DEFAULT (0),
		[CHANNEL] [bigint] NOT NULL,
		[STORE] [nvarchar](10) NOT NULL,
		[TERMINAL] [nvarchar](10) NOT NULL,
		[STAFF] [nvarchar](25) NOT NULL,
		[CREATEDDATETIME] [datetime] NOT NULL,
		[MODIFIEDDATETIME] [datetime] NOT NULL,
		[REPLICATIONCOUNTERFROMORIGIN] [int] IDENTITY(1,1) NOT NULL,
		[DATAAREAID] [nvarchar](4) NOT NULL,
		[ROWVERSION] [timestamp] NOT NULL,
	 CONSTRAINT [I_AX_CHANNELORIGINATEDDATATABLE_PK] PRIMARY KEY NONCLUSTERED 
	(
		[CHANNEL] ASC,
		[STORE] ASC,
		[TERMINAL] ASC,
		[CHANNELORIGINATEDDATAID] ASC,
		[DATAAREAID] ASC
	)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
	 CONSTRAINT [I_AX_CHANNELORIGINATEDDATATABLE_REPLICATIONCOUNTERFROMORIGIN] UNIQUE CLUSTERED 
	(
		[REPLICATIONCOUNTERFROMORIGIN] ASC
	)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
	) ON [PRIMARY]
END
GO

GRANT SELECT,INSERT,UPDATE,DELETE ON [ax].CHANNELORIGINATEDDATATABLE TO [DataSyncUsersRole]
GRANT INSERT ON [ax].[CHANNELORIGINATEDDATATABLE] TO [UsersRole]

Note: How to put this file into VSTS and registered in the Retail Sdk’s Customization.settings file so the change is reproducible and deployable into other environments is omitted here.

Add the new table to the list of Channel tables

CDX needs to be aware of the schema of the channel table in order to map and sync it with the HQ table. In Retail and commerce > Headquarters setup > Retail Scheduler > Retail channel schema, click Channel tables and add a new table with the new fields:

Add a new scheduler subjob

In Retail and commerce > Headquarters setup > Retail Scheduler > Scheduler subjobs create a new subjob that maps both HQ and channel table. Make sure that:

  • the special column [REPLICATIONCOUNTERFROMORIGIN] is both mapped in the field mapping and used in the replication counter field, and
  • the slider for “Pull Data” is set to Yes.

Add the subjob to the job

In Retail and commerce > Headquarters setup > Retail Scheduler > Scheduler jobs add the subjob to a job. You could reuse the existing P-0001 job as in this example:

Finally, in Retail and commerce > Headquarters setup > Retail Scheduler you must regenerate the sync helper classes with “Generate queries”.
Note: The previous steps in the Dynamics 365 for Operations user interface to configure CDX are shown here for better illustration. Ideally, these should be coded in X++ to get reliable and automatic deployments into other environments.

Verification

Add some data into the table in the channel side ([ax] schema) and run the pull job. Then inspect the data in the HQ side: