To improve the performance when processing a lot of logged events, you can use paging. For paging, the logged events are split over several pages that are run in parallel batch tasks.
You define the paging setup in these fields of the Process data synchronization log dialog:
- Number of pages: It can be that you can run only a limited number of batch jobs in parallel or that you want to use a limited number of batch jobs. If so, define the maximum allowed number of pages.
- Page size: You can split the to-be-processed events over several batch task. Define the number of events to be processed by one batch task.
On the pages, the events are grouped by RecId, table, and event type. So, all events of a specific event type that are logged for a unique record, are grouped on the same page. For example, if five change events are logged for customer X, these five events are grouped on one page.
Example:
The Number of pages is 4 and the Page size is 1000.
- If 1.500 logged events are processed, these are split over 2 pages.
- If 10.000 logged events are processes, these are initially split over 10 pages. However, only 4 pages are allowed. Therefore, a recalculation is done splitting the 10.000 events over the 4 allowed pages. As a result, 4 pages with each 2.500 events are processed.
Notes:- If you do not use paging, all to-be-processed events are added to one page and processed by one batch task.
- When selecting logged events to be processed, the synchronization parameters are considered:
- Last run: Only the events are selected with a created date and time after the latest run date and time.
- Synchronization delay (minutes): Only the events are selected that are logged before the current processing date/time minus the defined number of minutes.