The core:RecordsFilter Add-On Module.

This module allows you to apply advanced filtering to the source records before they are uploaded to the Target. ** **The module is based on the Core SFDMU Add-On Api.


The example of module setup.

In order to modify records by using the RecordsFilter Core Add-On Module, you should declare the module call in your export.json.

Let's see the example of the configuration below:

{
     "objects": [
     {
         "operation": "Upsert",
         "externalId": "LastName",
         "query": "SELECT LastName, Description FROM Contact",         
         "filterRecordsAddons": [
            {
                "module": "core:RecordsFilter",
                "args" : {
                    "filterType": "BadWords",
                    "settings" : {
                        "badwordsFile": "./badwords.json",
                        "detectFields": [
                            "Description"
                        ],
                        "highlightWords" : true,
                        "outputMatches": true
                    }
                }
            }
         ]
     }
 ]
}    

In this example the RecordsFilter module executes the filter of the type defined by the filterType property. You can see, that the name of the executed filter is BadWords.

The filter is fired by the onTargetDataFiltering event. This is the only allowed event for this type of Core Add-On because the purpose of this filter is exactly to filter the records. Binding RecordsFilter Add-On to another event will throw runtime error.


Each filter has its own settings definition.

You can mix between filters and even execute multiple filters of the same type. Put all filters you want to run in the filterRecordsAddons array.

We plan to release new filters from time to time.

Find below the full list of the currently available filters.


The BadWords Filter.

The BadWords filter scans the object's fields listed in the detectFields for the unwanted words, then can output the matched words to the console or even modify the source field value to highlight these words, then upload the modified value to the Target (or write to the CSV file).

The BadWords filter can be very useful for example for GDPR requirements.

The filter is implemented thanks to the contribution of the author of the amazing tool SFDX Hardis

The full specification of the args properties for BadWords filter.

Property name Is mandatory Property type Description
filterType Yes string Always must be BadWords (the type of the filter)
settings.badwordsFile No string The path to the json file where you can list bad words you want to detect.
The default value is "badwords.json". In this case the file should be placed at the same directory with the export.json.
Field.detectFields Yes string[] Sets the list of object's fields to scan for the bad words.
Field.highlightWords No boolean Set to true to modify the input record by highlighting the matched bad words with stars, for example, if you defined the bad word list as ["bad", "word"] and the input record value is "good word bad word" will be replaced with "good word "good word ***bad*** ***word***".
Field.outputMatches No boolean Set to true to output matched bad words to the Console.

The additional information and considerations:

  • The json file allowed to be placed in any location on the disk. You can name it differently, but the default name when you omit this property is badwords.json. Populate this property with the absolute path (or the path relative to the export.json).

  • If badwords.json does not exist or has incorrect format the module will throw an error.

  • The records are getting filtered only if the desired field is included in the object's query string and the field is of updateable type.

  • Below is the required format of the badwords.json file:

    {
        "badwords" : [
            "bad",
            "word"
        ]
    }

    List all bad words int the badwords array.


See also:

Introduction to the SFDMU Add-On API Engine

The ExportFiles Core Add-On module

Full export.json format

Last updated on Sa Oct 2022