--------------------------------------------------
lines 6-79 of file: at_cascade/continue_cascade.py
--------------------------------------------------

{xrst_begin continue_cascade}

Continue Cascade From a Fit Node
################################

Prototype
*********
{xrst_literal
   # BEGIN_DEF
   # END_DEF
}

Purpose
*******
Sometimes when running the cascade, the fit or statistics for a node fails.
This may be because of something that happened on the system,
or because of some of the settings in the :ref:`glossary@root_database`.
This routine enables you to continue the cascade from such a node.

all_node_database
*****************
is a python string specifying the location of the
:ref:`all_node_db-name`
relative to the current working directory.
This argument can't be ``None``.

fit_database
************
is a python string specifying the location of a dismod_at database
relative to the current working directory.
This is a :ref:`glossary@fit_database` with the
final state after a run of
:ref:`cascade_root_node-name` or :ref:`continue_cascade-name`
that includes fitting this database.
The *fit_database* is not changed, it is only used
to identify which child jobs to fit.

fit_goal_set
************
This is a ``set`` with elements of type ``int`` (``str``)
specifying the node_id (node_name) for each element of the
:ref:`glossary@fit_goal_set` .
Each such node must be the root node, or a descendant of the root node.
In addition, it must be in the :ref:`fit_goal_table-name` ,
or an ancestor of a node in the fit goal table.

fit_type_list
*************
This is a list with one or two elements
and its possible elements are ``both`` and ``fixed``.
For each job, the first type of fit is attempted.
If it fails, and there is a second type of fit, it is attempted.
If it also fails, the corresponding job fails.

shared_unique
*************
Under normal circumstances, you should use the empty string (default value)
for this parameter.

#. Parallel runs of continue_cascade must use different values of
   *shared_unique* so the corresponding shared memory names are different;
   see :ref:`fit_parallel@shared_unique` .

#. The *shared_unique* special case is where you are running (in parallel)
   continue_cascade with the same *fit_database*, and disjoint
   *fit_goal_set* . (The intersection of disjoint sets is empty.)

#. In the special case above, the suggested value for *shared_unique*
   is ``_`` *node_name* , where *node_name* is the name of one of the nodes
   in the *fit_goal_set*. It may be necessary to include the value
   of the splitting covariate in *shared_unique*  .
   (The splitting covariate is sex in the :ref:`csv.fit-name` case.)

{xrst_end   continue_cascade}