I have warnings in CSVIssuesReport.csv file, how can I understand them?

Question.

I am using csv-to-org migration.

I get "COLUMN DEFINED IN THE SCRIPT IS MISSING IN THE CSV FILE" and "CSV FILE DOES NOT EXIST" warnings in the CSVIssuesReport.csv file, but I don't know how to solve that.

I am using "SELECT all ..." multiselect keyword in the export.json file for my objects as I want all the fields.

I still don't understand why it added those fields automatically and then complaining about it.

Why do I have these warnings? Do I simply ignore it?

This is my configuration:

{
    objects: [
       {
         "query": "SELECT all FROM Account",
         "operation": "Upsert",
         "externalId": "Name"
       },
    ]
}

and the csv file:

Name,Phone,TEST__c,TEST2__c,  .......etc
"Account1","+123","TEST1","TEST2",  ....etc
"Account2","+124","TEST21","TEST22",  ....etc
"Account3","+125","TEST31","TEST32",  .....etc

Answer.

Some of these warning can be ignored (as explained in details below).

It is important to understand how does the Plugin processes multiselect keywords and how it validates the source CSV file structure.

Why do these errors appear ?

• If you are using "all" multieselect keyword in your query string, it's meaning as "please, select all sobject fields which are at least readable for my user". So, the first step would be, that the Plugin actually replaces "all" with all object's fields extracted from the object's metadata even you have no source data for some of them.
• The second step is, that the Plugin scans your source csv file and actually didn't find some corresponding columns to those fields that were previously extracted by "all" select, even you do not want to migrate these fields. This is the meaning of the error: "COLUMN DEFINED IN THE SCRIPT IS MISSING IN THE CSV FILE".
• This one: "CSV FILE DOES NOT EXIST" says, that you are actually missing CSV source files for the listed objects. Why ? Can be a situation in which your Account object contains custom lookup field to some additional objects that have no corresponding source CSV files for them. The important point here, that the Plugin is automatically adding all objects that are referenced via the lookup fields into the processed object set with "Readonly" operation, despite of missing source CSV files.
  

Can you ignore these warnings?

• You can ignore COLUMN DEFINED IN THE SCRIPT IS MISSING IN THE CSV FILE warnings for columns, that you don't need their data. 

• CSV FILE DOES NOT EXIST warnings are more critical, because you are missing source data for these objects, which can cause problems in locating of parent lookup records. You must include csv files for all objects used in your configuration even they were included implicitely.

  
  

And finally, what is the recommended solution ?

• For missing fields, the good approach would be to limit "all" field selection with "excludedFields" property in order to exclude useless or suspicious fields.

• For sObjects which have missing CSV source files and you don't need to process them, the good solution is to manually exclude them from the migration. You have two options to do this:

  • The simplest way is to add these objects to the array of the excludedObjects.

    {
        "objects": [
            {
                "query": "SELECT Id, Name, Phone FROM Account",
                "externalId": "Name",
                "operation": "Upsert"
            },
            
            // ... other objects in the configuration
            
        ],
        "excludedObjects": ["Account"] // This will exclude Account object from the migration, since we have no Account.csv uin our CSV source files.
    }
    

  • The alternative way is:

  • first to include configurations for all these unnesessary objects with Readonly operation (in this case it's enough to add only Record Id to the query string),

  • then to set excluded=true:

  {
      "objects": [
          {
              "query": "SELECT Id FROM Account",
              "excluded": true // This will exclude Account object from the migration, since we have no Account.csv uin our CSV source files.
          },
          
          // ... other objects in the configuration
          
      ]
  }