1. To-Increase Dynamics 365 for Finance & Operations
  2. Connectivity studio
  3. Operation

Run integration

You can run an integration on these levels:

  • Project: All related tasks are run. And for each task, all related messages are run.
  • Task: All related messages are run.
  • Message: Only used if an individual message must be run.
  • Data synchronization: The data synchronization setup of the message defines which records are processed.


Application Consultant Application Consultant Start Start How is the  integration run? How is the  integration run? Run project Run project You can use a project to exchange data. To be able to run a project, tasks must be set up for the project. If you run the project, all related tasks are run.Depending on the tasks that are set up for the project, you can use a project to run:An integration or data migration.Outbound web services.Batch classes.Master data management.Test cases.Usually, you run a project in batch. Based on the defined sub-projects and tasks dependencies, tasks are run in parallel. If you do not run the project in batch, all sub-projects and tasks are run sequentially. Procedure 1. Click Connectivity studio Integration Design. 2. Click Run project. 3. In the Project field, enter or select a value. Note: By default, the currently active project, as shown on the Connectivity studio Integration design workspace, is selected. 4. In the Company group field, enter or select a value. 5. Sometimes, you want to import only the delta, which are the changes since the last import. Example: During a migration project, often, some time elapses between the migration of the data and the go-live moment. Therefore, just before the go-live moment, you want to import the latest open transactions. However, you do not want to run the full migration again (see picture). In Connectivity studio, you can do a delta run. To do a delta run: - For the project tasks, add the applicable messages with the Run for delta field set to Run. - Set the current Delta run field to Yes. As a result, if the project is run, for each project task, only the messages are run that have the Run for delta field set to Run. When a message is run in a delta run, only the record inserts are done. So, no update or delete of records is done. Select Yes in the Delta run field. Note: For outbound web service actions, you can have set up data synchronization of type 'Date range'. To apply this data synchronization setup when you run web service actions from a project, select Yes in the Delta run field. 6. Select Yes in the Process outbound queue field. 7. Sub-task: Set up batch processing. 8. Expand the Run in the background section. 9. In the Batch processing field, select an option. If Yes, also fill in the other fields. 10. Click Recurrence and fill in the fields as desired. 11. Click OK. 12. Click OK. Notes You can also run a project from the project form. Run task Run task You can use a task to run the execution of:An integration or data migration. In this case, use a task with the desired messages defined.Outbound web services. In this case, use a task with the desired outbound web service actions defined.Batch classes. In this case, use a task with the desired batch classes defined.Master data management. In this case, use a task with the desired master data entities defined.Test cases. In this case, use a task with the desired messages with test cases defined.If you run a task, all defined messages, outbound web service actions, batch classes, and master data entities are run.Usually, you run a task by running the related project. In this case, all tasks of the project are run, taking into account the defined task dependencies. However, you can run a single task, for example, for testing purposes. Procedure 1. Click Connectivity studio Integration Design. 2. Click Tasks. 3. In the list, find and select the desired task. 4. On the Action Pane, click Operation. 5. Click Run task. 6. Select Yes in the Process outbound queue field. 7. Sub-task: Set up batch processing. 8. Expand the Run in the background section. 9. In the Batch processing field, select an option. If Yes, also fill in the other fields. 10. Click Recurrence and fill in the fields as desired. 11. Click OK. 12. Click OK. Run message  from a button? Run message  from a button? Run message Run message You can run a message directly. You can do so, for example, for testing purposes or to run a specific message in batch. Procedure 1. Click Connectivity studio Integration Design. 2. On the Message tab, in the list, find and select the desired message. 3. Click Run. 4. On a message with an internal document as source document, you can set up the data synchronization type. When you run the message, you can indicate if the set up data synchronization must be used: - None: The data synchronization setup of the message is not considered. - All: If the data synchronization type of the message is 'Table events' or 'Date range', these are not considered. The message is run as if the data synchronization type is 'All'. - Table events: The data synchronization of type 'Table events' is applied. Only select this option if the data synchronization type of the message also is 'Table events'. - Date range: The data synchronization of type 'Date range' is applied. Only select this option if the data synchronization type of the message also is 'Date range'. In the Synchronization type field, select an option. Note: This field is only available if the source document of the message is an internal document. 5. Sub-task: Set up batch processing. 6. Expand the Run in the background section. 7. In the Batch processing field, select an option. If Yes, also fill in the other fields. 8. Click Recurrence and fill in the fields as desired. 9. Click OK. 10. Click OK. Run message from a button

Run message from a button

You can use button on a form to run a message. You can add a button to a form in these ways:

  • Dynamic button
    Use a dynamic button to manually start a message from a form. To add a dynamic button, no coding is required.
  • Action menu item
    Use a custom action menu item to manually start a message from a form. To add an action menu item, custom coding is required.

 Receive data from Service Bus queue or topic subscription Receive data from Service Bus queue or topic subscription You can use the 'Service Bus queue' connector to import data from a Service Bus queue.To import data, first receive the data from a Service Bus queue or topic subscription. The received data is added to the 'Received data from queue' table in Connectivity studio.On receiving data from the Service Bus:Based on the Service Bus search definitions and settings on the received data, import messages are automatically assigned to the received data records.You can use 'peek lock'. If you use 'peek lock', the received Service Bus messages are temporarily locked in the Service Bus queue or topic subscription. Procedure 1. Go to Connectivity studio > Inquiries > Service Bus queue > Received data. 2. On the Action Pane, click Operation. 3. Click Receive data 4. In the Source connector field, enter or select a value. 5. Select Yes in the Peek lock field. 6. In the Number of messages field, enter a number. 7. In the Reschedule message (hrs) field, enter a number. 8. Sub-task: Set recurrence. 9. Expand the Run in the background section. 10. Select Yes in the Batch processing field and fill in the other fields as desired. 11. Click Recurrence. 12. Fill in the recurrence settings as desired and click OK. 13. Click OK. Notes You can also use the 'Receive and process data' process. This process: Receives the data from the Service Bus queue or topic subscription. Runs the messages that are automatically assigned to the received Service Bus data records. You can indicate which data you want to receive: Queue or topic subscription data. If 'Yes', you can also use the peek lock function. Dead letter data. You cannot use the peek lock function for dead letter data. Import received Service Bus data Import received Service Bus data You can import data from a Service Bus queue or topic subscription. First, you must receive the data from the Service Bus queue or topic subscription. On receiving data from the Service Bus, based on the Service Bus search definitions and settings on the received data, import messages are automatically assigned to the received data records. When the data is received, you can import the received data from the 'Received data from queue' table into D365 FO. To import the received data, run the messages as assigned to the received data records. You can run the messages in several ways, as desired. You can run a message by running a: Project Task Message If an import message run finishes: Successfully, the related received data record gets the status 'Finished'. If 'peek lock' is used, the Service Bus message is removed from the queue or topic subscription. With errors, the related received data record gets the status 'Error'. If 'peek lock' is used, the Service Bus message is moved from the queue or topic subscription to the related dead letter queue. Note: The assigned import messages must have the 'Service Bus queue' connector as source connector. Notes You can also use the 'Receive and process data' process. This process: Receives the data from the Service Bus queue or topic subscription. Runs the messages that are automatically assigned to the received Service Bus data records. You can indicate which data you want to receive: Queue or topic subscription data. If 'Yes', you can also use the peek lock function. Dead letter data. You cannot use the peek lock function for dead letter data. Process data synchronization log Process data synchronization log For messages and web service actions, you can use table events to track data changes. You can define, for each table, which table events are logged. The table events are logged in the Data synchronization log.To fully process the logged events, process the: Data synchronization log: The logged events are processed and for each applicable message or web service action, a record is added to the Outbound queue. Outbound queue: For each record, the related message or web service action is run to export the applicable data. This topic explains how to process the data synchronization log.   @videoref:CS5 Note: Events are logged only for tables and event types that are defined in the data synchronization setup of any message or web service action. If on the data synchronization setup, the Update check box and the Check used fields check box are selected, event logging is restricted to changes to the fields as defined for the record in the applicable source document. Only the event is logged and used for further processing. So, in case of an update, an event is logged of type Update. However, the changed data is not marked as changed and not used for further processing. For example, if you change a customer address, an event is logged that the customer is updated. However, the address change as such is not logged. When, for the logged event, a record is processed in the outbound queue, the then current data of the customer is used. If a logged event is processed, it is deleted from the data synchronization log. Example: This table gives examples of the results for several data synchronization log processing scenarios: Number of pages Page size Limit records Number of records to be processed Result 5 10,000 No 40,000 4 pages of 10.000 records are processed 5 10,000 Yes 40,000 4 pages of 10.000 records are processed 5 10,000 No 80,000 5 pages of 16.000 records are processed 5 10,000 Yes 80,000 5 pages of 10.000 records are processed   Procedure 1. Click Connectivity studio Integration operations. 2. Click Process data synchronization log. 3. To improve the performance when processing a lot of logged events, you can use paging. For paging, the logged events are split over several pages that are run in parallel batch tasks. It can be that you can run only a limited number of batch jobs in parallel or that you want to use a limited number of batch jobs. If so, define the maximum allowed number of pages. In the Number of pages field, enter a number. Note: If the number of records to be processed is less than the the defined 'Number of pages' x 'Page size', less than the defined number of pages are processed. 4. You can split the records to be processed over several pages (batch tasks). Define the number of events to be processed by one batch task (page). In the Page size field, enter a number. Note: If the number of records to be processed exceeds the defined 'Number of pages' x 'Page size', and 'Limit records' is set to: No, the number of records per page is increased to get all records processed. Yes, the maximum number of records per page is fixed. So, not all records are processed. 5. Select Yes in the Limit records field. 6. Usually, all records in the data synchronization log are processed. However, you can also process a number of oldest records only. For example, if you fill in '100', only the 100 oldest records are processed. This can be helpful to investigate if issues occur when processing older records. In the Oldest set of field, enter a number. Note: If you enter a number in the 'Oldest set of' field, these fields are not considered: Number of pages Page size Limit records 7. Sub-task: Set up batch processing. 8. Expand the Run in the background section. 9. Select Yes in the Batch processing field and fill in the other fields as desired. 10. Click Recurrence and fill in the fields as desired. 11. Click OK. 12. Click OK. Notes You can also process specific data synchronization log records. To do so: Go to Connectivity studio > Inquiries > Data synchronization log. Select the records to be processed. Click Process selection. Process outbound queue Process outbound queue For messages and web service actions, you can use table events to track data changes. You can define, for each table, which table events are logged. The table events are logged in the Data synchronization log. To fully process the logged events, process the: Data synchronization log: The logged events are processed and for each applicable message or web service action, a record is added to the Outbound queue. Outbound queue: For each record, the related message or web service action is run to export the applicable data. This topic explains how to process the outbound queue.   @videoref:CS5   Outbound queue The outbound queue only has unique entries for each combination of: Record on which the event was logged Message or web service action Event type Status Note that the record and event type are known, but the (changed) data of the records is not known. When an outbound queue entry is processed, the current data of the record is used. For example, if a customer address is changed, an event is logged that the customer is updated. However, the address change as such is not logged. When, for the logged event, a record is processed in the outbound queue, the current data of the customer is used. Bundling In the outbound queue, several entries can exist for a unique message or webservice action. How these entries are processed, depends on whether bundling is active. You can activate bundling on the data synchronization setup of an outbound: Message: On processing the outbound queue, by default, all entries for a unique message are bundled in one message run, which results in one file.  Web service action: On processing the outbound queue, by default, all entries for a unique web service action are bundled in one web service action run, which results in one file. Note: You cannot apply bundling to an outbound web service action that uses a stream. EDI document flow: On processing the outbound queue, the entries are bundled in one message run or web service action run, which results in one file. In this case, the outbound queue entries are bundled by the applicable EDI party. If the bundling field is set to: Yes, all outbound queue entries for the message are bundled and processed in one message run, which results in one file. No, each outbound queue entry is processed in a separate message run. Processing the outbound queue You can process the outbound queue in several ways. To process the outbound queue for: All records, do not define a message or web service action. A specific message, define the message. A specific web service action, define the web service action. You can also run the outbound queue for a specific project. To do so, when you run a project, set the Process outbound queue field to Yes. As a result, the outbound queue is processed for the messages or web service actions that are linked to the project tasks. If an outbound queue record is processed, it gets the status: Processed, if it is processed successfully. Error, if errors occurred during processing. Procedure 1. Click Connectivity studio Integration Design. 2. Click Process outbound message queue. 3. You can process the outbound queue for a specific message. In the Message field, enter or select a value. Note: - If you want to process the outbound queue for a specific message, do not define a web service action. If you define a web service action as well, the outbound queue is only processed for the web service action. - If you do not define a message or web service action, the full outbound queue is processed. 4. You can process the outbound queue for a specific web service action. In the Web service action field, enter or select a value. Note: If you do not define a message or web service action, the full outbound queue is processed. 5. In the Company field, enter or select a value. 6. Sub-task: Set up batch processing. 7. Expand the Run in the background section. 8. Select Yes in the Batch processing field and fill in the other fields as desired. 9. Click Recurrence and fill in the fields as desired. 10. Click OK. 11. Click OK. Notes If an outbound queue record for a web service action has run with errors, the maximum number of retries is checked. If the maximum number of retries is: Not reached, the outbound queue record gets the status New. The next time the outbound queue is processed, the record is processed again. Reached, the outbound queue gets the status Error. The maximum number of retries is shown in the Max. retries field of the outbound queue record. The value of this field is defined in the Max. automatic retries of web services field in the Connectivity studio parameters. The number of retries that are already done, is shown in the Times tried field of the outbound queue record. For each retry, the value of this field is increased with 1. End End Project Task Message Import data from  Azure Service Bus Table events No Yes

Activities

Name Responsible Description

Run project

Application Consultant

 		You can use a project to exchange data. To be able to run a project, tasks must be set up for the project. If you run the project, all related tasks are run.
Depending on the tasks that are set up for the project, you can use a project to run:
  • An integration or data migration.
  • Outbound web services.
  • Batch classes.
  • Master data management.
  • Test cases.
Usually, you run a project in batch. Based on the defined sub-projects and tasks dependencies, tasks are run in parallel. If you do not run the project in batch, all sub-projects and tasks are run sequentially.

Run task

Application Consultant
You can use a task to run the execution of:
  • An integration or data migration. In this case, use a task with the desired messages defined.
  • Outbound web services. In this case, use a task with the desired outbound web service actions defined.
  • Batch classes. In this case, use a task with the desired batch classes defined.
  • Master data management. In this case, use a task with the desired master data entities defined.
  • Test cases. In this case, use a task with the desired messages with test cases defined.
If you run a task, all defined messages, outbound web service actions, batch classes, and master data entities are run.
Usually, you run a task by running the related project. In this case, all tasks of the project are run, taking into account the defined task dependencies. However, you can run a single task, for example, for testing purposes.

Run message

Application Consultant
You can run a message directly. You can do so, for example, for testing purposes or to run a specific message in batch.

Run message from a button

Application Consultant

You can use button on a form to run a message. You can add a button to a form in these ways:

  • Dynamic button
    Use a dynamic button to manually start a message from a form. To add a dynamic button, no coding is required.
  • Action menu item
    Use a custom action menu item to manually start a message from a form. To add an action menu item, custom coding is required.

Receive data from Service Bus queue or topic subscription

Application Consultant

 		You can use the 'Service Bus queue' connector to import data from a Service Bus queue.
To import data, first receive the data from a Service Bus queue or topic subscription. The received data is added to the 'Received data from queue' table in Connectivity studio.
On receiving data from the Service Bus:
  • Based on the Service Bus search definitions and settings on the received data, import messages are automatically assigned to the received data records.
  • You can use 'peek lock'. If you use 'peek lock', the received Service Bus messages are temporarily locked in the Service Bus queue or topic subscription.

Import received Service Bus data

Application Consultant

You can import data from a Service Bus queue or topic subscription.

First, you must receive the data from the Service Bus queue or topic subscription. On receiving data from the Service Bus, based on the Service Bus search definitions and settings on the received data, import messages are automatically assigned to the received data records.
When the data is received, you can import the received data from the 'Received data from queue' table into D365 FO. To import the received data, run the messages as assigned to the received data records. You can run the messages in several ways, as desired. You can run a message by running a:
  • Project
  • Task
  • Message
If an import message run finishes:
  • Successfully, the related received data record gets the status 'Finished'. If 'peek lock' is used, the Service Bus message is removed from the queue or topic subscription.
  • With errors, the related received data record gets the status 'Error'. If 'peek lock' is used, the Service Bus message is moved from the queue or topic subscription to the related dead letter queue.
Note: The assigned import messages must have the 'Service Bus queue' connector as source connector.

Process data synchronization log

Application Consultant

For messages and web service actions, you can use table events to track data changes. You can define, for each table, which table events are logged. The table events are logged in the Data synchronization log.To fully process the logged events, process the:

  1. Data synchronization log: The logged events are processed and for each applicable message or web service action, a record is added to the Outbound queue.
  2. Outbound queue: For each record, the related message or web service action is run to export the applicable data.

This topic explains how to process the data synchronization log.

 


Note:

  • Events are logged only for tables and event types that are defined in the data synchronization setup of any message or web service action.
  • If on the data synchronization setup, the Update check box and the Check used fields check box are selected, event logging is restricted to changes to the fields as defined for the record in the applicable source document.
  • Only the event is logged and used for further processing. So, in case of an update, an event is logged of type Update. However, the changed data is not marked as changed and not used for further processing. For example, if you change a customer address, an event is logged that the customer is updated. However, the address change as such is not logged. When, for the logged event, a record is processed in the outbound queue, the then current data of the customer is used.
  • If a logged event is processed, it is deleted from the data synchronization log.

Example:

This table gives examples of the results for several data synchronization log processing scenarios:

Number of pages Page size Limit records Number of records to be processed Result
5 10,000 No 40,000 4 pages of 10.000 records are processed
5 10,000 Yes 40,000 4 pages of 10.000 records are processed
5 10,000 No 80,000 5 pages of 16.000 records are processed
5 10,000 Yes 80,000 5 pages of 10.000 records are processed

 

Process outbound queue

Application Consultant
For messages and web service actions, you can use table events to track data changes. You can define, for each table, which table events are logged. The table events are logged in the Data synchronization log.
To fully process the logged events, process the:
  1. Data synchronization log: The logged events are processed and for each applicable message or web service action, a record is added to the Outbound queue.
  2. Outbound queue: For each record, the related message or web service action is run to export the applicable data.

This topic explains how to process the outbound queue.

 

 

Outbound queue
The outbound queue only has unique entries for each combination of:
  • Record on which the event was logged
  • Message or web service action
  • Event type
  • Status
Note that the record and event type are known, but the (changed) data of the records is not known. When an outbound queue entry is processed, the current data of the record is used. For example, if a customer address is changed, an event is logged that the customer is updated. However, the address change as such is not logged. When, for the logged event, a record is processed in the outbound queue, the current data of the customer is used.
Bundling

In the outbound queue, several entries can exist for a unique message or webservice action. How these entries are processed, depends on whether bundling is active. You can activate bundling on the data synchronization setup of an outbound:

  • Message: On processing the outbound queue, by default, all entries for a unique message are bundled in one message run, which results in one file. 
  • Web service action: On processing the outbound queue, by default, all entries for a unique web service action are bundled in one web service action run, which results in one file. Note: You cannot apply bundling to an outbound web service action that uses a stream.
  • EDI document flow: On processing the outbound queue, the entries are bundled in one message run or web service action run, which results in one file. In this case, the outbound queue entries are bundled by the applicable EDI party.

If the bundling field is set to:

  • Yes, all outbound queue entries for the message are bundled and processed in one message run, which results in one file.
  • No, each outbound queue entry is processed in a separate message run.
Processing the outbound queue
You can process the outbound queue in several ways. To process the outbound queue for:
  • All records, do not define a message or web service action.
  • A specific message, define the message.
  • A specific web service action, define the web service action.
You can also run the outbound queue for a specific project. To do so, when you run a project, set the Process outbound queue field to Yes. As a result, the outbound queue is processed for the messages or web service actions that are linked to the project tasks.
If an outbound queue record is processed, it gets the status:
  • Processed, if it is processed successfully.
  • Error, if errors occurred during processing.

Run message from a button

Provide feedback