Activities

Activity Area Description

Activate data quality policy version

Data quality

When the full setup of a data quality policy version is finished, to apply the defined rules, you must make the version active.

Note: You cannot edit an active data quality policy version.

Copy data quality policy

Data quality

Use data quality policies to ensure that your data is of the desired quality. You can organize data quality policies, for example, by functional area, business process owner, or organization assignment.

To create a data quality policy, you can copy an existing one. When you copy a data quality policy, you must choose which versions you want to copy. In the new data quality policy, the chosen versions are added in the version sequence of the original data quality policy. Example: You copy versions 4 and 6 of the original data quality policy. In the new data quality policy, version 4 is added as version 1, and version 6 is added as version 2.

Create data quality policy

Data quality
Use data quality policies to ensure that your data is of the desired quality. You can organize data quality policies, for example, by functional area, business process owner, or organization assignment.
If you create a data quality policy, its header and first version are created.

Create data quality policy version

Data quality
You can create a new version for a data quality policy. If you create a new version, you can choose to:
  • Use an existing version as basis. As a result, all elements of the selected version are copied to the new version.
  • Create a blank version.
Note: If you create a data quality policy, automatically its first version is created.
A data quality policy version contains all elements of a data quality policy, except for the data quality policy header.
These elements are version-specific:
  • Validation rules
  • Duplicate check rules
  • Action rules
  • Organization association

Create dynamic query with the Wizard

Data quality
Use a dynamic query to find one or more records in the database. The found records are the input for further processing.

You can use a dynamic query on:

  • An action rule of type Data query.
  • A condition that is applied to a validation rule, duplicate check rule, or action rule.
This topic explains how to set up a dynamic query with the Query wizard.
You can also start the dynamic query wizard from an existing query to edit it.

Delete data quality policy

Data quality You can delete or retire a data quality policy.
If you try to delete a data quality policy:
  • With an active version, it is not deleted.
  • With only draft versions, and at least one of these versions has been active in the past, it is retired. To view the retired data quality policies, on the Data quality management workspace, click the Retired data quality policies tile.
  • With only draft versions, and none of these versions have been active in the past, it is deleted.

Develop custom action class

Data quality You can use a custom action rule to set a field value using a custom class.
To set up a custom action rule, first develop the custom class to be applied.
The custom class must extend the DQSCustomValidationBase base class.
When you extend the DQSCustomValidationBase base class:
  • Enter a unique and recognizable description() for the class. The description is shown in the lookup of the Class name field of a custom action rule.
  • Use these methods to indicate to which data the custom class is applicable:
    • getTablesList(): You can define the tables to which the custom class applies. As a result, you can select the custom class for a custom action rule with one of the applicable tables defined.
    • getExtendedDataTypesList(): You can assign extended data types to which the custom class applies, for example, 'ExternalItemId'. As a result, you can select the custom class for a custom action rule with a target field defined that uses the same extended data type.
    • getBaseTypesList(): You can define base data types to which the custom class applies, for example, String or Date. As a result, you can select the custom class for a custom action rule with a target field defined with the same data type.
  • Use the processRecord() method to define the x++ code that defines the return value that is used as input value for the target field of the action rule.
The base structure of an extended DQSCustomValidationBase class is:

Develop custom parameter value class

Data quality To define a web service parameter value, you can use a custom class to return a parameter value.
You can use a custom parameter value class to return web service parameters for validation rules and action rules of type Web service. 
The custom parameter value class must extend the DQSWebServiceParametersBase base class. This custom class must have at least one return method that returns a value.

Develop custom response or request handler class

Data quality

 

For a web service, you can use a custom class to define the:
  • Request that you send to the web service. You can, for example, use a custom class to add some binary code or other logic to the web service request. Use the PostData method to define web service requests.
  • Response that you expect from the web service. You can, for example, use a custom class to add some binary code or other logic to the web service response. Use the parseResponse method to define web service response.
The custom class must extend the DQSWebServiceCustomFormatBase base class.

Develop custom validation class

Data quality
You can use a custom validation rule to validate a field value using a custom validation class.
To set up a custom validation rule, first develop the custom validation class to be applied.
The custom validation class must extend the DQSCustomValidationBase base class.
When you extend the DQSCustomValidationBase base class:
  • Enter a unique and recognizable description() for the class. The description is shown in the lookup of the Class name field of a custom validation rule.
  • Use these methods to indicate to which data the custom class is applicable:
    • getTablesList(): You can define the tables to which the custom class applies. As a result, you can select the custom class for a custom validation rule with one of the applicable tables defined.
    • getExtendedDataTypesList(): You can assign extended data types to which the custom class applies, for example, 'ExternalItemId'. As a result, you can select the custom class for a custom validation rule with a field defined that uses the same extended data type.
    • getBaseTypesList(): You can define base data types to which the custom class applies, for example, String or Date. As a result, you can select the custom class for a custom validation rule with a field defined with the same data type.
  • Use the validateRecord() method to define the x++ code that runs the validation. This code must return one of these values:
    • True: The value in the validation rule field is valid.
    • False: The value in the validation rule field is invalid.
The base structure of an extended DQSCustomValidationBase class is:

Develop function class

Data quality You can use a function to fill the value of a condition on a query range.
For a function, a class is required. To develop this class, extend the DQFFunctionRun class.
Prerequisite: Develop the extension of the DQFFunctionRun class in the AOT before you start the steps.
Note: The steps are to have the developed function class uploaded as function.

Process quality assessment results

Data quality

As a data owner, you can review and process data quality assessment results. A data owner can be a:

  • General data owner who can review and process quality assessment results for all companies.
  • Company-specific data owner who can review and process quality assessment results only for an assigned company.

You review the data quality assessment results:

  1. By table by company.
  2. For each table-company combination, by record.
  3. For each record, by error or warning.

For each:

  • Record with errors or warnings, you can open the source record to make the desired changes.
  • Error or warning, you can set the review status.

Run data quality assessment

Data quality

You can run a data quality assessment to examine the quality of existing data. To run a data quality assessment, you use data quality assessment projects.

You can do a data quality assessment, for example, if you:

  • Have implemented a change in the Data quality studio configuration and want to make sure existing data is in line with this change.
  • Have imported data without applying data quality policies.
  • Want to periodically check the quality level of your data.

Set up action rule conditions

Data quality For each action rule, you can set up conditions. The action rule is only applied if the conditions are met.
To define an action rule condition, you use a query. The action rule is applied only to the records that are found by the query.
You can use these types of queries:
  • Inquiry: Use the standard D365 FO Inquiry (SysQueryForm) page to define the records to which the action rule applies. The inquiry only queries the table as defined for the action rule.
  • Dynamic query: Use a dynamic query to define the records to which the action rule applies. Before you can set up a condition with a dynamic query, set up the dynamic query to be applied. You can use a dynamic query, for example, to query other tables than the action rule table, to define table relations, and to have only one record returned.
You can set up several conditions, using different query types.

Set up action rule of type Custom

Data quality
Use action rules to set field values in D365 FO.
To set a field value, you can use several action types. This topic explains how to set up an action rule of type Custom.
Use a custom action rule to set a field value using a custom class.
Before you can set up a custom action rule, ensure that the custom class is developed.

Set up action rule of type Data query

Data quality

Use action rules to set field values in D365 FO.
To set a field value, you can use several action types. This topic explains how to set up an action rule of type Data query.

Use a data query action rule to find a value using a query and set the found value in the target field of the action rule.
You can use these types of queries:
  • Inquiry: Use the standard D365 FO Inquiry (SysQueryForm) page to define the query that is used to find the value to be set in the target field. The inquiry only queries the table as defined for the action rule.
  • Dynamic query: Use a dynamic query to define the query that is used to find the value to be set in the target field. Before you can set up an action rule with a dynamic query, set up the dynamic query to be applied. You can use a dynamic query, for example, to query other tables than the action rule table, to define table relations, and to have only one record returned.

Set up action rule of type Fixed value

Data quality
Use action rules to set field values in D365 FO.
To set a field value, you can use several action types. This topic explains how to set up an action rule of type Fixed value.
Use a Fixed value action rule to set a pre-defined fixed value in the target field of the action rule.

Set up action rule of type Number sequence

Data quality

Use action rules to set field values in D365 FO.
To set a field value, you can use several action types. This topic explains how to set up an action rule of type Number sequence.

Use a number sequence action rule to set the value of the target field of the action rule based on a number sequence.

Set up action rule of type Transformation list

Data quality

Use action rules to set field values in D365 FO.

To set a field value, you can use several action types. This topic explains how to set up an action rule of type Transformation list.
Use a transformation list action rule to find a value using a transformation list and set the found value in the target field of the action rule.
Before you can set up a transformation list action rule, set up the transformation list to be used.
Example:
You can set up a transformation list to transform country codes into the relevant currencies for these countries. In this case, the source field is a country field, and the target field is a currency field. If the country is US, and the transformation list action rule is applied, the currency is set to USD.

Set up action rule of type Web service

Data quality

Use action rules to set field values in D365 FO.

To set a field value, you can use several action types. This topic explains how to set up an action rule of type Web service.
Use a web service action rule to get a field value from another internal or external source, using a web service, and set this value in the target field of the action rule.
Before you can set up a web service action rule, set up the web service to be used.
You can set up a web service action rule in several ways. For a parameter of type:
  • Request, you can use the default parameter values as defined for the used web service. No specific parameter setup is required.
  • Request, you can override a default parameter value for a request parameter. To do so, you can use the type Field, Code, or Fixed value. Each type requires specific settings. Explained in subtasks 13-14.
  • Response, you can use the values, as returned by the web service, to set several values in D365 FO. Explained in subtask 15.
  • Response, you can process a response value, as returned by the web service, before you use it to set a value in D365 FO. Explained in subtask 16.

Set up condition table mapping

Data quality

On data quality policy rules, you can define conditions. If you define a condition and the applicable table for the rule is a date-effective or inheritance table, additional setup is required. The applicable table for the rule must be mapped to a related staging table or temporary table. 

For example, if the applicable table for the rule is the:

  • LogisticsPostalAddress (inheritance) table, set up a mapping to the CustomerPostalAddressStaging table.
  • DirPartyTable (date-effective) table, set up a mapping to the DirPartyStaging table.

Set up configurable lookup - Dynamic query

Data quality

You can set up a configurable lookup of type 'Dynamic query' to define the possible options in a field with a dynamic query.

Set up configurable lookup - User defined list

Data quality

You can set up a configurable lookup of type 'User defined list' to manually define the possible options in a field.

Set up data pattern

Data quality
You can use a data pattern validation rule to validate if a field value matches a defined pattern.
To set up a data pattern validation rule, first set up the data pattern to be applied. Define the data pattern, using a regular expression.
You can, for example, set up data patterns to validate:
  • Email addresses
  • Website URLs
  • VAT numbers
For more information on regular expressions, refer to Regular expression.

Set up data quality assessment project

Data quality

You can use a data quality assessment project to examine the quality of existing data.

You can do a data quality assessment, for example, if you:

  • Have implemented a change in the Data quality studio configuration and want to make sure existing data is in line with this change.
  • Have imported data without applying data quality policies.
  • Want to periodically check the quality level of your data.

In a data quality assessment project, you can apply:

  • All validation rules and duplicate check rules of a data quality policy.
  • Specific validation rules or duplicate check rules of a data quality policy.

Set up 'Data quality' organization hierarchy purpose

Data quality You can apply a data quality policy version to (a part of) an organization hierarchy. Before you can do so, assign the desired organization hierarchy to the 'Data quality' organization hierarchy purpose.

Set up duplicate check

Data quality To set up a duplicate check rule, a duplicate check is required. If the desired duplicate check does not exist, set up a new duplicate check.
For each duplicate check, define which combination of table fields is checked on duplicate values. So, the combination of field values must be unique in the table.

Set up duplicate check rule

Data quality

Use a duplicate check rule to check if the data doesn't already exist in the table.

Before you can set up a duplicate check rule, set up the duplicate check to be applied. A duplicate check defines which combination of table fields is checked on duplicate values. So, the combination of field values must be unique in the table.

Set up duplicate check rule conditions

Data quality For each duplicate check rule, you can set up conditions. The duplicate check rule is only applied if the conditions are met.
To define a duplicate check rule condition, you use a query. The duplicate check rule is applied only to the records that are found by the query.
You can use these types of queries:
  • Inquiry: Use the standard D365 FO Inquiry (SysQueryForm) page to define the records to which the duplicate check rule applies. The inquiry only queries the table as defined for the duplicate check rule.
  • Dynamic query: Use a dynamic query to define the records to which the duplicate check rule applies. Before you can set up a condition with a dynamic query, set up the dynamic query to be applied. You can use a dynamic query, for example, to query other tables than the duplicate check rule table, to define table relations, and to have only one record returned.
You can set up several conditions, using different query types.

Set up organization assignment for Data quality policy

Data quality By default, a data quality policy version is applied to all organizations as defined in the current D365 FO environment. If a data quality policy version must be applied only to specific organizations, set up the organization assignment.
To set up the organization assignment, you can use legal entities or organization hierarchies. Before you can use an organization hierarchy, assign the desired organization hierarchy to the 'Data quality' organization hierarchy purpose.

Set up reason codes

Data quality

If you do data quality assessments, for an error or warning, you can set the status to 'Accept deviation'. In this case, it is important to explain why you accept the deviation. You can do so, by selecting a reason code.

This topic explains how to set up the reason codes.

Set up secured values

Data quality

You can set up secured values to store secrets at a central place in Data quality studio. You can , for example, set up a secured value for a license key or access token.

Benefits of using secured values are:

  • One place to maintain secrets. For example, you can use a secret in several web services. If the secret expires, you only update the secured value instead of updating the secret separately for each applicable web service.
  • Secrets are not shown or visible where these are applied using secured values.

In Data quality studio, you can use secured values as a parameter for web services.

Set up transformation list

Data quality
To set up a transformation list action rule, a transformation list is required. If the desired value transformation does not exist, set up a new transformation list or add the transformation to an existing transformation list.
For each transformation list, set up the value transformations. For each value transformation, define the source value and the target value.
Example:
You can set up a transformation list to transform country codes into the relevant currencies for these countries. In this case, the source field is a country field, and the target field is a currency field. If the country is US, and the transformation list action rule is applied, the currency is set to USD.

Set up validation rule conditions

Data quality
For each validation rule, you can set up conditions. The validation rule is only applied if the conditions are met.
To define a validation rule condition, you use a query. The validation rule is applied only to the records that are found by the query.
You can use these types of queries:
  • Inquiry: Use the standard D365 FO Inquiry (SysQueryForm) page to define the records to which the validation rule applies. The inquiry only queries the table as defined for the validation rule.
  • Dynamic query: Use a dynamic query to define the records to which the validation rule applies. Before you can set up a condition with a dynamic query, set up the dynamic query to be applied. You can use a dynamic query, for example, to query other tables than the validation rule table, to define table relations, and to have only one record returned.
You can set up several conditions, using different query types.

Set up validation rule message

Data quality You can define a message that is shown if the validation rule is not met.
If no validation rule message is defined, the default message is shown if the validation is not met.
In case of a data pattern validation, the applicable data pattern can also have a message defined. In this case, the message priority is:
  1. If a validation rule message is defined, this message is shown.
  2. If no validation rule message is defined, the data pattern message is shown.
  3. If no data pattern message is defined, the default message is shown.
You can also enter a description of the validation rule.
You can translate both the validation message and the validation rule description.

Set up validation rule of type Configurable lookup

Data quality

Use validation rules to check if the data is in line with the defined standards.
To validate data, you can use several validation types. This topic explains how to set up the Configurable lookup validation type.

Use a configurable lookup validation to add a custom lookup to a field and to validate if the field value matches a value as defined by the configurable lookup.

Note: If the field, to which the configurable lookup is applied, already has a lookup, the configurable lookup overwrites the existing lookup.

Before you can set up a configurable lookup validation rule, set up the configurable lookup to be used.

Set up validation rule of type Custom

Data quality

Use validation rules to check if the data is in line with the defined standards.
To validate data, you can use several validation types. This topic explains how to set up a validation rule of type Custom.

Use a custom validation rule to validate a field value using a custom validation class.
Before you can set up a custom validation rule, ensure that the custom validation class is developed.

Set up validation rule of type Data pattern

Data quality

Use validation rules to check if the data is in line with the defined standards.
To validate data, you can use several validation types. This topic explains how to set up a validation rule of type Data pattern.

Use a data pattern validation rule to validate if a field value matches a defined pattern.
Before you can set up a data pattern validation rule, set up the data pattern to be applied.

Set up validation rule of type Mandatory or Blank

Data quality

Use validation rules to check if the data is in line with the defined standards.

To validate data, you can use several validation types. This topic explains how to set up validation rules of these types:
  • Mandatory: Makes it mandatory to fill the defined field.
  • Blank: Validates if no value is entered in a field.

Set up validation rule of type Range expression

Data quality
Use validation rules to check if the data is in line with the defined standards.
To validate data, you can use several validation types. This topic explains how to set up a validation rule of type Range expression.
Use a range expression validation rule to validate if a field value is within a defined range. To define a range expression, you can use a:
  • Fixed value or a range of values using the advanced query syntax.
  • Static method that defines the range. For example, you can use the SysQueryRangeUtil class to apply advanced date queries.
For more information on how to define ranges in the Range expression field, refer to Advanced filtering and query syntax.

Set up validation rule of type Web service

Data quality

Use validation rules to check if the data is in line with the defined standards.
To validate data, you can use several validation types. This topic explains how to set up a validation rule of type Web service validation type.

Use a web service validation rule to validate if a field value matches a value in another internal or external source, using a web service.
You can, for example, use a web service validation rule to validate an address, email, or VAT number.
Before you can set up a web service validation rule, set up the web service to be used.
You can set up a web service validation rule in several ways. For a parameter of type:
  • Request, you can use the default parameter values as defined for the used web service. No specific parameter setup is required.
  • Request, you can override a default parameter value for a request parameter. To do so, you can use the type Field, Code, or Fixed value. Each type requires specific settings. Explained in subtasks 10-12.
  • Response, you can use the values, as returned by the web service, to validate a value in D365 FO. Explained in subtask 13.
  • Response, you can process a response value, as returned by the web service, before you use it to validate a value in D365 FO. Explained in subtask 14
 

Set up web service for Data quality studio

Data quality You can use a web service in:
  • A validation rule to validate if a field value matches a value in another internal or external source.
  • An action rule to fill a field value with a value from another internal or external source.
To set up a web service validation rule or action rule, first set up the web service to be used.
You can find most of the required values to be set for the web service in the technical documentation as shared by the web service provider.

View quality assessment exceptions log

Data quality

As a data owner, you can review and process data quality assessment results. A data owner can be a:

  • General data owner who can review and process quality assessment results for all companies.
  • Company-specific data owner who can review and process quality assessment results only for an assigned company.

You can review the full unfiltered list of data quality assessment results.

Using this list, you can:

  • For each error or warning, open the source record to make the desired changes.
  • Set the review status for one or several errors and warnings at once.

Provide feedback