Maestro Module: To Regenerate or Not To Regenerate...

The regenerate functionality goes back many years in the history of the Maestro engine.  I wrote the very first instance of the "Maestro" engine back in 2004/2005 where it was nowhere near as flexible and nowhere near as extensible as it is today as Maestro.  That being said, the engine's internals have been updated and upgraded over the years which has culminated in what you see today.  The concept of regenerate and regenerate all live tasks comes from the very beginning of the engine's life and lives today in Maestro 1.0's engine. 

During the inception of the engine, we were automating a rather simple workflow (only about 20 steps in total) -- however this specific workflow had many instances where it had to loop back to earlier executed tasks based on IF tasks determining if something was accepted or rejected.  Seems easy!  Just point the true or false branch of an IF task to any other task "higher" up in the chain of tasks and you're done!
Well, not quite.  That approach works... and doesn't work all at the same time.  The problem I ran in to was a simple one.  The engine has what I would call a self regulating mechanism in it to not re-create the same task twice during the execution of a flow.  The reason why the engine has to have a self regulation mechanism to prevent duplicate tasks being created is to ensure that the engine is not stuck in a deadlock loop due to a misbehaving task (especially a batch task or AND task).  I had to have full human control over the engine to specifically tell it to either recreate the task it is looping back over and that it was OK to do so.  More specifically, the engine is built this way to only ever create one instance of an AND task.  You can have many AND tasks in a flow, but there can only be one instance of each AND task created.  Enter the concept of Regeneration.
 
To Regenerate or not to Regenerate.... that is usually the question.

When in doubt, you probably want to regenerate a task.   So what does regenerate mean?  Simply put, when a flow loops back over itself due to a logic condition or path branch tells it to, the task that has a regenerate flag turned on tells the engine that it is OK to re-create that task.  This effectively bypasses the self regulating mechanism of the engine that resists creation of duplicate tasks in a flow.
A by-product of a regenerated flow means that the current process is suspended and given a status of "Regenerated".  The engine creates a new process instance of the workflow and starts it off at the regenerated task.  All data with the execution of the original flow up to the point of regeneration is all there.  Untouched.  However with only the regenerate flag on, ONLY that task will be active once the new process executes.  All other currently-in-execution or currently assigned tasks disappear.

Great!  Problem solved!!!  Not quite.... 
Remember that simple 20 step flow I mentioned above?  That flow became a 40 step flow which included the magnificent AND task.  Yes, the AND task... the task that is able to wait for all incoming branches of a flow to execute up to the AND and only continue execution when all branches have reached the AND.  Well, picture the following scenario:

As you see, there are 2 branches with an IF in only one of the branches.  The flow says to only loop back inside of the current left branch... not back up to the top of the parallel branch parent (Step 1).  With just a Regenerate flag set to "ON" on Step 2, the task Step 3 and the AND task would have been trapped in the original process which would have been put in to regeneration status.  A new process is created and as a result the ONLY active task in the new process is Step 2.  All other assigned tasks that existed disappear as they do not exist in the new process.

Sounds fine right?  No!!
The user who is assigned Step 2 would get that task in their task console.  The user assigned to Step 3 would see the task disappear from their console.
The user completes Step 2, gets assigned Step 4.  User completes Step 4, and this time passes the IF condition and as a result the AND task is created.  The AND task will wait for all incoming branches to finish..... see the problem yet?  Issue is that before regeneration, Step 3 existed.  Once completed Step 3 would flow in to the AND and as such, the AND could complete.
Not after regeneration.  The engine did not re-create Step 3 and as a result only ONE of the branches in to the AND completed.  The engine will hold the AND task as active forever.

Enter Regenerate All In Production Tasks flag.
The solution to this was to have a second flag on a task to tell the engine to not only regenerate THAT task, but also to regenerate all tasks that were not completed at that time. 
While this may seem to be what the default option should be, that is not the case.  Consider this flow:

The flow in this case will redirect back up to the top of the parallel branch chain.  In this case you do NOT want to regenerate all tasks that are in production.  Simply put, you will only regenerate the single parent-of-the-parallel-branches task.  As a result, only using the Regenerate flag is sufficient.

In either case, the regeneration options are there for you as a workflow administrator to accommodate the creation of tasks as you see fit. 
At Nextide, we've got a concept engine in the works that removes the need for both regenerate and regenerate all in production flags.  The engine's logic is altered to make some relatively safe assumptions about your flow and do the appropriate regeneration for you.