Performance Monitoring

As your computational graphs become more complex, it can be helpful to see which tasks are being executed, and how long it takes. To demonstrate how to do this, let’s start with a simple graph that simulates some long-running tasks:

import graphcat

graph = graphcat.StaticGraph()
graph.set_task("A", graphcat.delay(2))
graph.set_task("B", graphcat.delay(1))
graph.set_task("C", graphcat.delay(0.1))
graph.set_task("D", graphcat.null)
graph.add_links("A", "B")
graph.add_links("B", ["C", "D"])

Note that we’ve used graphcat.delay - which simply sleeps for the given time in seconds - as the task functions. Now, before we update the graph, we’ll create an instance of graphcat.PerformanceMonitor, which will handle gathering information about each task as it executes:

monitor = graphcat.PerformanceMonitor(graph)

Now, we’re ready to execute the graph and visualize the results:

graph.update("C")

import graphcat.diagram
import graphcat.notebook

agraph = graphcat.diagram.draw(graph, rankdir="TB")
agraph = graphcat.diagram.performance(agraph, monitor)
graphcat.notebook.display(agraph)

Note that graphcat.diagram.performance() operates by supplementing the data in the original diagram. See Visualizing Graphs for more details.

As you can see, when we merge the performance data into our graph diagram, every task that executes displays the time it took, with a color coded dot highlighting which tasks were the slowest.

You can also access the performance data manually, as a mapping from tasks to execution times:

monitor.tasks

{'A': [2.0007550716400146],
 'B': [1.0006828308105469],
 'C': [0.10363316535949707]}

Note that the execution times are stored in a list, so you can track multiple times, not just the most recent. For example, if we force task “C” to execute again:

graph.mark_unfinished("C")
graph.update("C")
monitor.tasks

{'A': [2.0007550716400146],
 'B': [1.0006828308105469],
 'C': [0.10363316535949707, 0.10011696815490723]}

Finally, the monitor can be reset to begin recording times from scratch:

monitor.reset()
graph.mark_unfinished("C")
graph.update("C")
monitor.tasks

{'C': [0.1026008129119873]}