Up to 100 Times Faster with Pollination Cloud

How Pollination Cloud scales your local workflows for unbeatable performance

One of Pollination’s main goals is to "Deliver Better Performing Buildings, Faster". It’s simple to say we can run really fast simulations in the cloud, but the reality is that the speed of your simulation will depend on the scale and resolution of your environmental study. You may find there are instances when you don’t need to utilize cloud resources, because it may run faster on your Personal Computer (PC). So how can you determine when to use cloud computing resources over using your PC? In this post we are going to illustrate the steps required to execute an environmental simulation in the cloud versus a PC, and the difference in speed when executing a single run versus a series of parametric runs. 

Is it possible to go 100 times faster on Pollination Cloud compared to a PC? Of course it is! Read on to learn how.

Definitions

  • Job: a Job is a collection of one or more Runs.
  • Run: a Run executes the Tasks in a Recipe using the selection of inputs 
  • Recipe: a workflow template that dictates a set of tasks to be executed in order to calculate design outcomes
  • Task: a Task is a section of code to be executed. It can be compared to a node in Grasshopper or Dynamo
  • Loop: a Loop is a parallel execution of a Task, for example, executing the run-simulation task for each building in the urban masterplan
  • CPU: a Central Processing Unit is the hardware section of a computer that “computes” things. A standard Laptop has 4 to 8 CPUs. Usually 1 Task is executed using 1 CPU.
  • Docker Image: a mini virtual machine containing code to execute a specific Task
  • Bucket Storage: highly scalable data/file storage for the Cloud; think “Google Drive”.

Pollination Recipes

“A Pollination Recipe is a workflow definition that dictates a set of tasks to be executed in order to calculate design outcomes.”

Readers who have used visual scripting tools such as Grasshopper or Dynamo can think of a Recipe as analogous to a Grasshopper file where each node corresponds to a task.

We will use the ladybug-tools/dragonfly-annual-energy-use example Recipe throughout this article as a point of reference because it is relatively simple and contains a “loop” task which enables parallel execution of tasks. This recipe takes an analytical model of an urban area or a building and computes the Energy Use Intensity (EUI) of the model.

The Recipe above has a list of inputs on the left and outputs on the right. The 4 nodes or tasks in the center represent a set of tasks to be executed. Here is a list of the tasks and what each does:

  • create-sim-par: generates energy simulation parameters based on the Recipe inputs
  • convert-to-honeybee: generates a list of Honeybee models from an analytical model
  • run-simulation: runs an EnergyPlus simulation for each Honeybee model generated by the “convert-to-honeybee” task
  • compute-eui: computes the Energy Use Intensity of the entire analytical model by aggregating the outputs of each energy simulation

Jobs and Runs

To compare Pollination Recipe execution speed on a PC vs Pollination Cloud we must understand how they are executed.


A Job uses a recipe and input arguments to generates runs

If we consider a Recipe is a Workflow Template with inputs, outputs and tasks, then a Job is a Workflow Command where the Recipe is joined with a set of Input Arguments. A Pollination Job can contain multiple combinations of inputs in order to execute a Parametric Study using the same Recipe. Each selection of Input Arguments is then combined with the Job’s selected Recipe into what we call a Run. A Run executes the Tasks in a Recipe’s Workflow Template using the selection of inputs provided. 

The image below shows a Run of the ladybug-tools/dragonfly-energy-use Recipe. Notice how the “run-simulation” Task is executed 4 times because the input model was split into 4.


A Run is a Recipe executed with one set of inputs

PC vs Pollination Cloud

We will compare the speed of Jobs executed on a PC vs Pollination Cloud by looking at: 

  • Task execution: “How long does the run-simulation task take?”
  • Run and Job execution: “How long does it take to calculate the EUI of one (or more) model(s)?”

Task Execution

Generally a Task takes a bit longer to run on the Cloud because Files have to be moved from Bucket Storage to the machine executing it. This is slower than moving files inside the same PC machine. Furthermore, we may need to add machines to our compute cluster if there are many Runs being executed at the same time. Adding a machine can take between 30s and 2mins. The animation below illustrates a side-by-side comparison of a Task Execution on a PC versus on Pollination Cloud.

Watch the PC win on a small task execution.

PC Steps:

  • Copy files from Job folder to Temp folder
  • Execute the task
  • Copy the output files from the Temp folder back to the Job folder

Cloud Steps:

  • (optional) Wait for machines to be added to the compute cluster if there is no CPU capacity left
  • (optional) Pull docker images to machine if they don’t exist
  • Download input files from the Pollination Project Folder to the machine where the task is being executed
  • Execute the task
  • Upload output files from the machine to the Bucket Storage

Job and Run Execution

The duration of a Job is the time between its creation and the completion of all of its Runs. Similarly the duration of a Run is the time between its creation and the completion of all of its Tasks. Therefore a Job will be executed faster the more Tasks and Runs can be executed in parallel. Parallel execution of Tasks is limited by the computing power of the machine or machines that they are being executed in. The animation below illustrates a side-by-side comparison of a large Job Execution on a PC versus on Pollination Cloud.

Watch Pollination win when executing a large model or parametric study.

Cloud Steps

  • Push input files to Bucket Storage
  • Create Job
  • Transfer input files to Job folder in Bucket Storage
  • Schedule Runs in parallel on the Kubernetes cluster
  • Each task is executed in parallel where possible
  • Zip the Recipe output files/folders to make them readily available for download

PC Steps

  • Create Job
  • Copy input files to Job folder on PC
  • Execute Runs sequentially on PC
  • Each task is executed in parallel where possible, parallelism is limited by the number of CPUs on the PC

Jobs will execute much faster on Pollination Cloud in two settings:

  • Parametric Jobs: a Job that contains many Runs will finish faster on Pollination Cloud because they can all be executed in parallel
  • Large Models: a large Model that requires many CPUs will compute faster on Pollination because we can run machines with up to 96 CPUs and specialist hardware

100 Times Faster With Pollination Cloud

Based on the example above you could run a Parametric Study with 100 design options 100 times faster because each Run can be executed in parallel. The slightly slower execution of each Task on Pollination Cloud is compensated by the infinite amount that can be executed in parallel.

Think of how much faster you could go if your Parametric Study has 1,000 design options or a run that can be distributed to 1000s of computers? Pollination brings the power of cloud computing at your fingertips.