Multiple object sets.

This feature allows to process multiple sub-jobs under the same main job.

Feature overview.

If you have to perform multiple operations with the same sobject it's impossible using only the standard "objects" array, because each object can be specified only once.

To remove this limitation we introduced a new "objectSets" property. It allows you to create multiple subsets of objects an process them sequentially in separated sub-jobs under the single main job.

In the example below we have 2 object sub-sets:

• The first sub-set deletes all Accounts and Opportunities.

• The second sub-set inserts one Account record.

You can see that the Account object will be handled twice with different operations.

  {
      "objectSets": [
        {
          "objects": [
            {
              "query": "SELECT Id FROM Account",
              "operation": "DeleteHierarchy"
            },
            {
              "query": "SELECT Id FROM Opportunity",
              "operation": "DeleteHierarchy"
            }
          ]
        },
        {
          "objects": [
            {
              "query": "SELECT Name FROM Account LIMIT 1",
              "operation": "Insert"
            }
          ]
        }
      ]
}

You can use both main "objects" set and subset of "objectSets" in the same configuration.

In this case the set of "objects" will run before the objectSets.

The below configuration does the same thing as the above:

 {
     "objects": [
         {
             "query": "SELECT Id FROM Account",
             "operation": "DeleteHierarchy"
         },
         {
             "query": "SELECT Id FROM Opportunity",
             "operation": "DeleteHierarchy"
         }
      ],
      "objectSets": [
        {
          "objects": [
            {
              "query": "SELECT Name FROM Account LIMIT 1",
              "operation": "Insert"
            }
          ]
        }
    ]
}

Notes.

• If you use CSV files as the data source, all Object Sets use by default the same source csv files located in the working directory.

• If you set useSeparatedCSVFiles = true, you should provide different csv source files for each Object Set.

  E.g. for the first Object Set (i.e. Object Set #1 or for the objects from the root "objects" array of the script) - all the files should be placed in the working directory.

  For the rest of Object Sets you must place them in the sub-directory ./objectset_source/object-set-<ObjectSet index>

  E.g.:

  • ./objectset_source/object-set-2
  • ./objectset_source/object-set-3

  See also Full export.json format