In the diagram above there are 3 state machines SM1, SM2, SM3
SM1 has 3 states (S11, S12, S13)
SM2 has 5 states (S21,S22,S23,S24,S25)
SM3 has 2 states (S31, S32)
SM1 connects to SM2 when it encounters the S12 state, it saves its current state ae_state/ae_state_retries/ae_state_started and invokes the new state machine SM2 which starts at the top and runs thru it states. SM2 connects to SM3 at state S22, so again we save the salient state attributes and navigate down to SM3 via some other object in between. When SM3 is finished processing the previous state information for SM2 is restored and we start that state machine at state S23, we then process SM2 completely and come back into processing SM1, restore its salient state attributes and resume at S13.
This above description was a normal flow the the state machine where no errors or retries were encountered.
Lets look at en error scenario, say state S31 under SM3 throws an error, this error would be percolated to the top and all the state machines would stop. This is the default behavior, the user can override the error condition with the ‘continue’ feature discussed earlier and continue on with the state machine processing. There isn’t a rollback feature yet in the state machine processing.
Lets look at a retry scenario, say state S31 throws a retry, this would stop all the state machines (SM3, SM2 and SM1) and the process will end with a retry error queueing the next automate task, we would save all the state information for SM1,SM2,SM3 along with the other key attributes that started this automation task. When the timer expires the worker would dispatch the Automate job we would read the previous state information and resume processing and start at state machine SM1, since SM1 was last at Step S12 when the retry happened S11 will be skipped, S12 will get us to SM2 which will skip S21 and move on to processing S22 which will connect to SM3, in SM3 we will resume at S31. The state attributes ae_state_retries/ae_state_started keep track of the number of the times the particular state was invoked and enforce the time and retry limits if specified for the state.
All the state machine states are run synchronously like any other automate state machine. The states machines today get asynchronous behavior by using a check method with a retry which is pretty heavily used during provisioning, that behavior is not going to change.
The state machine debugging is currently limited to the Automate logs, we log as we change states and run the on_entry/on_exit/on_error calls. If there are other requirements or ideas around the debugging we can discuss them here.