Inside TensorFlow - TF Debugging

Explore TensorFlow debugging techniques for both TF 2 and TF 1 in this 38-minute video presentation by Software Engineer Shanqing Cai. Learn about printing Eager Tensor values, accessing graph-internal tensor values, finding device placement in pure eager execution and tf.function, visualizing function graphs, dumping Grappler outputs, step debugging, and debugging Keras models with TensorBoard callback. Gain insights into advanced debugging methods such as using tf.print() on composite tensors, fetching tensors from while loops, and accessing tf.keras Layer Activations. Discover how to optimize your TensorFlow debugging workflow and improve your machine learning development process.

Intro
Scope of this talk: "debugging" is an overloaded term in ML
Printing Eager Tensor values
Printing the value of graph-internal tensors
Homework: tf.print() on composite tensors
Programmatically access graph-internal tensor values
Programmatically fetching graph-internal tensors: While loop?
Finding device placement: Pure eager execution
Finding out device placement: tf.function
Getting and plotting the graph of a function: Colab (google3 only)
Dumping Grappler outputs: The graph that actually (almost) gets executed at runtime (bazel builds)
t.print: may change runtime graph optimization
t.config.experimental_run_functions_eagerly
Step debugging: Using tf.config.experimental_run_functions_eagerly
Step debugging: What happens inside a non-eagerly-executing function?
tf.config.experimental_run_functions eagerly does not work on tf.data.Dataset.mapo
Getting Access to tf.keras Layer Activations
Debugging Keras Models with TensorBoard callback
Parting notes