snowfakery progress!

To keep up with the latest project news and updates, please visit our Github repos here, here, and here. We also have a Chatter group in the Trailblazer Community here. To get started, I suggest completing these two Trailhead trails (Install Developer Tools and Build Applications with CumulusCI).

About a year ago, I published a blog post detailing triumphs and tribulations with using the open source tool Snowfakery to generate complex datasets of completely made-up values. Well, what a year it has been! Last year, I had to speeddial more than one friend to accomplish the basic tasks of installing/accessing/using even the most rudimentary features.

Today, my friend reached out to me because she was not quite able to interpret the instructions and get Snowfakery up and running. Guess what? I knew enough to be able to hop on a Zoom and help in real time, with zero prep! I’m still not using Snowfakery on a regular basis, and I still get stuck quite a bit, but today was a red-letter-day because I was able to use my hard-fought skills to help another community member.

Sometimes, I feel like my knowledge of github, YAML, cumulusci, and Snowfakery specifically, is as fleeting as a snowflake. As soon as I catch it, it melts away. Then again, as time passes and I maintain my effort to learn these tools, also like snow, I see the accumulation beginning to stick. Forecast? 1-3 inches!

getting stuck and unstuck

Let’s talk about some places where my friend got stuck, and how we worked together to get her unstuck! These are *also* places where I’ve been stuck, so this was a welcome opportunity to practice explaining the concepts to someone who does not identify as a developer (therefore, neither of us find this stuff in our comfort zones).

• While this may be obvious to many developers, it’s not obvious to us! The place to start (when it comes to Snowfakery) is VS Code or an equivalent developer environment. From this application, you can see/edit your code files, run code (“arguments”) in the Terminal, and see results. This is a screenshot of my VS code that shows Snowfakery files from the Recipe Repository and a specific Accounts recipe file that is open.

• We knew that we wanted to use Snowfakery to generate random Accounts and deposit them in a Salesforce Sandbox instance. The Snowfakery documentation assumes that you already have (one or more) org(s) connected to CCI. We needed to check that the org was set up properly, so we used the commands, cci org list and cci org browser in the VS Code terminal to double and triple check that the org was accessible. The second one, if configured correctly, will open a new browser tab and automatically log into your Salesforce org. Like magic!! Learn more about these features in the documentation here.
  
• Snowfakery must have a recipe file in order to generate a dataset according to specific requirements. The Snowfakery documentation assumes that you already have a file that you are ready to run. We needed to create a new file in VS Code with a file extension of .yml (and save it!). Then, we needed to put a few lines of Snowfakery recipe code in there. Here is a snippet of code that is similar to ours, and also demonstrates a few of Snowfakery’s cool features for generating fake data. Can you imagine what this file will do?

- object: Account
  fields:
    Name:
      fake: company
    Description:
      fake: catch_phrase
    NumberOfEmployees:
      random_number:
        min: 0
        max: 200000

• Now that we have a file that Snowfakery can read, we need to run it to make sure that the data come out looking reasonable (and that we don’t get any errors). Before inserting the data in the Sandbox, we want to run it locally. Snowfakery has a feature for that, which allows you to create data and model it in a JSON file. This is the code for that, which you would run in your VS Code terminal (or equivalent). Below the code is a screenshot of my JSON results. It is slightly different than the argument (code) I am giving you, because I knew in advance that I wanted my JSON file to be in my “temp” folder, and you might not be working with a temp folder in your own directory.

$ snowfakery --output-format JSON --output-file account.json account.yml

• --output-format JSON means that the tool will make a file in the format of json when it finishes running (two dashes are important)
• --output-file account.json means that the file produced will have that name, and it will “live” in the same folder (aka directory) as the rest of your code for the project (unless you specify otherwise)
• account.yml is the name of the recipe file. Yours will probably have a different name.

Now that we got this working in JSON, we were ready to run it in Salesforce. Here’s the argument for that:

$ cci task run generate_and_load_from_yaml -o generator_yaml account.yml --org dev

• cci task run is basically the preamble for running many CCI things
• generate_and_load_from_yaml … I couldn’t find any documentation on this. I think you just have to know it. I think this is the command that tells CCI to use Snowfakery.
• -o generator_yaml account.yml … this is how you reference your file. Remember, your file name might be different than mine.
• --org dev (two dashes are important) the name of your org (dev) might be different. This is how CCI knows which org to put records into! (Good idea to run cci org browser dev before you try this to make sure your org is connected properly) (Your org might not be called dev)

• Now that we have a working recipe and a connected sandbox (with data, woo!), we can begin to work on refining the recipe so that it can get more and more nuanced. Luckily, the Snowfakery community has been creating and maintaining an open source repository of tried and true recipes, which work well as building blocks for creating your own complex recipes! You can access the repo online here, or you can locally download it to Github on your personal computer and use these recipes to run in your own Salesforce orgs (not in production, please!). Additionally, we are always looking for recipe contributions and testers! Get in touch with me if you would like to get involved. We also accept PRs!

scaling up

[Tangent alert] A long time ago, I remember sitting on the floor in a university conference room, scribbling on flipchart paper. I was working with a crew of intergenerational activists to plan a sequence of actions, targeting a major bank for investing in mountaintop removal coal mining. At the same time, in a living room on the other side of town, a different group of co-conspirators were painting props, or phonebanking, or some such useful endeavor. Now I can’t remember what it was! And I remember thinking, “we will never get burned out when we have so many people willing to take turns doing the work.” Now, this didn’t turn out to be 100% accurate. People did get burned out, and we tried to limit this as much as possible. But I felt then (and still do), that rotating leadership is the best way to build skills while making a social justice project sustainable for volunteers. Even better is braiding/overlapping those spurts of volunteers so that no one feels overly responsible for everything that needs to happen. (Want to learn more about this model? Check out this extremely useful article from my dear friend, Ryan).

So perhaps you can imagine my excitement as the Data Generation Toolkit team expanded our leadership circle, and welcomed new people to take on responsibility just last month!

Moreover, I had a taste of that feeling today, when I zoomed out to appreciate the plethora of creative and useful tasks that Snowfakery community members are contributing. Just off the top of my head…

• Two people are planning an upcoming conference presentation
• Two people are recording demo videos
• Four people are planning activities for our next mini-sprint event
• Several people are reviewing contributions in Github for improvements we want to make to our code
• Several people are preparing for a new release of Snowfakery planned for later this month
• And there are people around the world (some known and many anonymous) who are using Snowfakery in their data management efforts at work or in a volunteer capacity
• I’m sure there’s more going on than I can even think of!

In reflecting on all of this juicy goodness, the image of a snowflake once again came to mind. The engineer(s) who built Snowfakery came up with the name because each dataset that it produces is unique “like a snowflake.” What I can see now is that the community of people using and supporting Snowfakery is also like a snowflake, branching out in tendrils that are impossible to predict, beautiful to behold, and intricately unique. By learning Snowfakery, I was able to help another person quickly clear the obstacles that we both encountered in our path. In her learning, she will also be able to advocate and empower more people to use it. While snowflakes don’t branch out in infinite size, perhaps this metaphor can! It’s exciting and energizing to see so much engagement and creativity. I can’t wait to experience what’s next!