graphcat.dynamic module

Implements computational graphs using dynamic dependency analysis.

class graphcat.dynamic.DynamicGraph

Bases: graphcat.graph.Graph

Manages a dynamic computational graph.

The graph is a collection of named tasks, connected by links that define dependencies between tasks. Updating a task implicitly updates all of its transitive dependencies. When an unfinished task is updated, it executes a user-supplied function and stores the function return value as the task output. Outputs of upstream tasks are automatically passed as inputs to downstream tasks.

add_links(source, targets)

Add links between source and targets.

Note that calling add_links() multiple times will create multiple, parallel links between tasks.

See also

set_links(), which()

Parameters:

• source (hashable object, required) – Name of the task that will act as a data source.
• targets (tuple, or list of tuples, required) – Each (task, input) tuple specifies the target of a link.

Raises:

ValueError – If source or target don’t exist.

add_task(name, fn=None)

Add a task to the graph.

This function will raise an exception if the task already exists.

See also

set_task()

idempotent alternative to add_task().

Parameters:

• name (hashable object, required) – Unique label that will identify the task.
• fn (callable, optional) – The fn object will be called whenever the task is executed. It must take two keyword arguments as parameters, label and inputs. name will contain the unique task name. inputs will be a dict mapping named inputs to a sequence of outputs returned from upstream tasks. If None (the default), graphcat.common.null() will be used.

Raises:

ValueError – If label already exists.

clear_links(source, target)

Remove links from the graph.

This method will remove all links from source to target.

Parameters:

• source (hashable object, required) – Source task name.
• target (hashable object, required) – Target task name.

Raises:

ValueError – If source or task don’t exist.

clear_tasks(names=None)

Remove tasks from the graph, along with all related links.

Note that downstream tasks will become unfinished.

Parameters:names (None, hashable object, or list|set of hashable objects, optional) – Names identifying the tasks to remove. If None (the default), all tasks are removed, emptying the graph.

is_dynamic

Returns True.

is_streaming

Returns False.

links(names=None)

Return every link originating with the given names.

Parameters:names (None, hashable object, or list|set of hashable objects, optional) – Names identifying the source tasks from which to retrieve links. Returns:links – (source, target, input) tuple for every link in the graph. Return type:list

mark_unfinished(names=None)

Set the unfinished state for tasks and all downstream dependents.

Normally, the unfinished state is set automatically when changes are made to the graph. This method is provided for callers who need to set the unfinished state in response to some outside event that the graph isn’t aware of; this should only happen in extremely rare situations.

Parameters:names (None, hashable object, or list|set of hashable objects, required) – Task names to be marked as unfinished. If None (the default), the entire graph is marked unfinished.

on_changed

Signal emitted whenever a part of the graph becomes unfinished.

Functions invoked by this signal must have the signature fn(graph), where graph is this object.

Returns:signal Return type:blinker.base.Signal

on_cycle

Signal emitted if a cycle is detected during updating.

Functions invoked by this signal must have the signature fn(graph, name), where graph is this object.

Returns:signal Return type:blinker.base.Signal

on_execute

Signal emitted before a task is executed.

Functions invoked by this signal must have the signature fn(graph, name, input), where graph is this object, name is the name of the task to be executed, and input is a dict containing the task inputs.

Returns:signal Return type:blinker.base.Signal

on_failed

Signal emitted when a task fails during execution.

Functions invoked by this signal must have the signature fn(graph, name, exception), where graph is this object, name is the name of the task that failed, and exception is the exception raised by the task.

Returns:signal Return type:blinker.base.Signal

on_finished

Signal emitted when a task executes successfully.

Functions invoked by this signal must have the signature fn(graph, name, output), where graph is this object, name is the name of the task that executed successfully, and output is the return value from the task function.

Returns:signal Return type:blinker.base.Signal

on_task_renamed

Signal emitted when a task is renamed.

Functions invoked by this signal must have the signature fn(graph, oldname, newname), where graph is this object, oldname is the original name of the task, and newname is its current name.

Returns:signal Return type:blinker.base.Signal

on_update

Signal emitted when a task is updated.

Functions invoked by this signal must have the signature fn(graph, name), where graph is this object and name is the name of the task to be updated.

Returns:signal Return type:blinker.base.Signal

output(name)

Retrieve the output from a task.

This implicitly updates the graph, so the returned value is guaranteed to be up-to-date.

Parameters:

name (hashable object, required) – Unique task name.

Returns:

output – The value returned when the task function was last executed, or None.

Return type:

any object

Raises:

• ValueError – If name doesn’t exist.
• Exception – Any exception raised by a task function will be re-raised by output().

rename_task(oldname, newname)

Change an existing task’s name.

This modifies an existing task’s name and modifies any related links as-necessary. In addition, the task and any downstream dependents will become unfinished.

Parameters:

• oldname (hashable object, required) – Existing original task name.
• newname (hashable object, required) – Unique new task name.

Raises:

ValueError – If the task with oldname doesn’t exist, or a task with newname already exists.

set_expression(name, expression, symbols=None)

Create a task that evaluates a Python expression, returning its value.

The task will automatically track implicit dependencies that arise from executing the expression.

Parameters:

• name (hashable object, required) – Unique name for the new expression task.
• expression (string, required) – Python expression that will be evaluated whenever the task is executed.
• symbols (callable, optional) – Function that returns a Python dict containing symbols that will be available to the expression when it’s executed. If None (the default), the graphcat.common.builtins() function will be used, which gives the expression access to graph, name, inputs, and extent objects that match the arguments to a normal task function.

set_links(source, targets)

Set links between source and targets.

Note

This function overrides all links from source.

Parameters:

• source (hashable object, required) – Name of the task that will act as a data source.
• targets (tuple, or list of tuples, required) – Each (task, input) tuple specifies the target of a link.

Raises:

ValueError – If source or target don’t exist.

set_parameter(target, input, source, value)

Create and link a ‘parameter’ task in one step.

Because they’re so ubiquitous, this method simplifies the creation of “parameter” tasks - tasks that return a value for use as a parameter in some other task. It consolidates creating the parameter task and linking it with an existing computational task into one step.

Parameters:

• target (hashable object, required) – Name of the task that will use the parameter.
• input (hashable object, required) – Named input that will receive the parameter.
• source (hashable object, required) – Name of the task that will store the parameter.
• value (any Python object, required) – Parameter value.

Raises:

ValueError – If target doesn’t exist.

set_task(name, fn)

Add a task to the graph if it doesn’t exist, and set its task function.

Note that this will mark downstream tasks as unfinished.

Parameters:

• name (hashable object, required) – Unique name that will identify the task.
• fn (callable, required) – The fn object will be called whenever the task is executed. It must take two keyword arguments as parameters, name and inputs. name will contain the unique task name. inputs will be a dict mapping named inputs to sequences of outputs returned from upstream tasks.

state(name)

Return the current state of a task.

Parameters:name (hashable object, required) – Unique name that identifies the task. Returns:state – Enumeration describing the current task state. Return type:graphcat.common.TaskState Raises:ValueError – If name doesn’t exist.

tasks()

Return the name of every task in the graph.

Returns:tasks – Names for every task in the graph. Return type:set

update(name)

Update a task and all of its transitive dependencies.

Parameters:

name (hashable object, required) – Name identifying the task to be updated.

Raises:

• ValueError – If the task with name doesn’t exist.
• Exception – Any exception raised by a task function will be re-raised by update().

class graphcat.dynamic.NamedInputs(graph, name)

Bases: object

Access named inputs for a graph task.

Parameters:

• graph (DynamicGraph, required) – Graph containing a task.
• name (hashable object, required) – Existing task unique name.

get(name, default=None)

Return a single input value.

Use this method to return a value when you expect to have either zero or one input that matches name.

Parameters:

• name (hashable object, required) – Name of the input value to return.
• default (any Python value, optional) – If an input matching name doesn’t exist, this value will be returned instead. Defaults to None.

Returns:

value – The value of input name, or default.

Return type:

any Python value

Raises:

KeyError: if more than one input matches name.

getall(name)

Return multiple input values.

Use this method to return every input value that matches name.

Parameters:name (hashable object, required) – Name of the input value to return. Returns:values – Values from every input that matches name. Returns an empty list if there are none. Return type:list of Python values

getone(name)

Return a single input value.

Use this method to return a value when you expect to have exactly one input that matches name.

Parameters:name (hashable object, required) – Name of the input value to return. Returns:value – The value of input name. Return type:any Python value Raises:KeyError: if more or less than one input matches name.

items()

Return names and values for every input attached to this task.

Note

For each (name, value) pair returned by this method, the value is a callable that returns the actual value from the upstream task.

Returns:values – The name and value of every input attached to this task. Return type:sequence of (hashable object, callable) tuples

keys()

Return names for every input attached to this task.

Returns:names – The name of every input attached to this task. Note that the same name may appear more than once in the sequence. Return type:sequence of hashable objects

values()

Return values for every input attached to this task.

Note

Each value returned by this method is a callable that returns the actual value from the upstream task.

Returns:values – The value of every input attached to this task, in the same order as keys(). Return type:sequence of callables