Easily load data into your Scratch Org (or Sandbox): Part 1

By

4 minutes de lecture
Easily load data into your Scratch Org (or Sandbox)

This is a two parts blog series :

Every time I begin a new project for a customer and that we start talking about using Scratch Orgs, one of the question that keeps coming up is “How do I load data ?”.

What’s already available

The Salesforce CLI comes with a nice set of commands to export data, sfdx force:data:tree:export and its counterpart to importsfdx force:data:tree:import. These commands are pretty simple, you just have to pass an SOQL query for the data you want to export, but comes with 2 main issues :

  • as you need an SOQL query to use as an input, this means that the data you can export is tied to the SOQL limitations
  • some data are org-specific because of Ids, like User Ids or Record Type Ids. This doesn’t work well with Scratch Orgs where either users don’t exist or Ids will change for every new Scratch Org

At Texeï, our whole development workflow is based on Scratch Orgs, so we quickly ran into this second issue.

For each new Scratch Org, Record Types were recreated, which means different Ids were generated. At first we were just importing data without Record Type, and manually updating records to select the correct record type.

This quickly became painful. Enters custom CLI plugins.

What we’ve built

We decided to build our own import and export tool to make our own life easier. Note that these commands are not supposed to solve all use cases and were build with some principles in mind :

  • be simple to use
  • manage record types and user ids
  • be focused on Scratch Org, or at least development environment
  • be easy to export the same data again and “refresh” the dataset

There are already out there lots of tools to handle very specific and complicated use cases with data. We wanted an easy one to manage all the data needed in a development environment.

The flow is the following :

  • Create a Scratch Org
  • Create the data you need, once
  • Export all the data

That’s it.

So how do I export the data you’ll ask ? Just create a folder, let’s call it data and run a command like this :

sfdx texei:data:export --objects Account,Contact,MyCustomObject__c --outputdir ./data --targetusername MyOrg

Then, you’ll just have to run an import command to insert the exact same data in your next Scratch Org :

sfdx texei:data:import --inputdir ./data --targetusername MyOtherScratchOrg

As you can see, ease of use is the main focus on these commands. Exporting is just a matter of listing the sObjects you want, the only requirement is to list them in the correct order, which is if a Contact references an Account, accounts must be exported first. The plugin will handle Record Types for you, and use the Scratch Org user for lookups referencing users.

No filters on the records, no fields selection. And this is on purpose. Do the job once and you’re good to go.

Note: With data privacy you’ll likely have to work on your test data or create some dummy ones anyway, so filtering records was not something we wanted to implement. Also, if you don’t have a way to add data to a development environment yet, you’ve likely created test data in your sandboxes manually, so it’s just the matter of doing it one last time.

How do I install it ?

If you haven’t used Texeï’s plugin yet, installing it is damn simple :

  • install Salesforce CLI if you haven’t already
  • install Texeï’s plugin: sfdx plugins:install texei-sfdx-plugin

Note that the plugin is open source, and available on GitHub. Pull Requests welcomed 😊

Life isn’t always that simple

This is super easy to use when starting a new project or doing a pre-sales (because yes, we’re doing our pre-sales demos on Scratch Orgs 🤩), but all data models are not that easy to populate.

We’ll see how to handle more complex use cases in the second part of this blog post serie (coming soon).

Read more posts

Enforce code standards with PMD

Developers working on a project usually set coding rules to have a standardized codebase. It is an important piece of the code maintainability, and it can be very easy …
March 2023
Advices
Scratch orgs

Uncovering Salesforce Settings: A Step-by-Step Guide for Scratch Orgs

Today, it’s pretty easy to build your Scratch Org definition file when you know what Settings you want to activate, as they are mapped with the same setting names …
February 2023
Advices
Business Analyst

Core qualities of a Business Analyst?

A common definition we are used to hear is that being a Business Analyst means to have a combination of both hard skills and soft skills. What does a …
June 2022
Advices
Image d'illustration d'une employée travaillant sur un ordinateur portable

Process builder and workflow make way to Flows (½)

Overview “If you can do it with a Workflow, then do it with a Process Builder, because everything a Workflow does, a Process Builder does it better”. If you …
March 2022
Advices

Day 22 : Salesforce new “Migrate To Flow tool” in Spring 22

As most of you already know, the workflow rules and process builders are planned to be retired in 2023 (no precise date defined so far). Today, I’m going to …
December 2021
Advices

Day 18 : Fake callout responses for test classes !

Hello everybody ! Today let’s talk about Apex tests classes in Salesforce. Everyone loves a good test class, and Salesforce makes it official by requiring to have a minimum …
December 2021
Advices