Run loop iterations in parallel

By default, loop iterations are executed sequentially. This means an iteration on an item runs to completion before another starts. 

When loop iterations are not dependent on one another, running the loop in Parallel mode is a good idea. Running loops in parallel significantly speeds up loop execution time.

Note that loop iterations have a limit of 50,000 (fifty thousand). This is split into 10,000 (ten thousand) per loop in parallel loops. 

When to use parallel mode

Loops in parallel mode run faster and more efficiently. Here are two common use cases for running loops in parallel.

• Query multiple threat intel vendors and process the queries simultaneously
• Send question messages to multiple users while investigating an incident and process the input simultaneously

Important notes

• Running a loop in Parallel mode will result in simultaneous calls to the same vendor (from the steps in the body of the loop). This means the vendor's rate limit might be reached and cause steps to fail. Check the vendor's documentation for rate limits and implement a retry mechanism to work around this situation.
• If there's a loop that will do more than 50 iterations at once, it will split into batches of 50 iterations at a time (ex: 100 iterations splits into two batches of 50, 80 interactions splits into a batch of 50 and a batch of 30). This will not impact the speed the steps are run.
• Having a loop inside of a parallel loop will multiply the number of iterations run, and you may hit the limit (5000) faster. If you are looping through data, use Chunk array to split the array into smaller batches and avoid hitting the limit of iterations run. 
• If a step run by a private runner inside the body of a loop fails, check the runner host's available resources and make sure they are sufficient to support the parallel loops.
• Exit and Break operators cannot be inside a parallel loop. 

When to use sequential mode

Run loops in Sequential mode when the order in which the loop iterations are executed is important. Here are a few examples:

• Pagination requests
• Retry on failure
• Escalation flow

How to use parallel mode

The execution order of the loop iterations is unknown. In the example below, the loop input is a list of IP addresses to investigate. 

Multiple IP queries are submitted to VirusTotal simultaneously so you can collect results as they are returned. You don't have to wait for a specific query's result before submitting the next one.

1. Drag the Loop operator onto the designer and switch the Mode toggle to Parallel.
   
2. Select the Type and define the data to loop over.
   1. In: the array to loop over
   2. Range: Define the Start, End, and Step size.
3.  (Optional) Use the Collect operator to collect the loop results into a single array.
   have a limit