Coursera

Week 3

Website: tensorboard.dev

Callbacks

model = create_model()

model.compile(optimizer="adam",
			  loss="sparse_categorical_crossentropy",
			  metrics=["accuracy"])

log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")

tensorboard_callback = tf.keras.callbacks.TensorBoard(
	log_dir=log_dir,
	histogram_freq=1
)

model.fit(X_train,
		  y_train,
		  epochs=5,
		  validation_data=(X_test, y_test),
		  callbacks=[tensorboard_callback])

Histograms

histogram_freq=1 means one chart per epoch

Publishing model details

!tensorboard dev upload --logdir ./logs

Local TensorBoard

Let’s start with a simple network for handwritten digits

mnist = tf.keras.datasets.mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

def create_model():
	return tf.keras.models.Sequential([
		tf.keras.layers.Flatten(input_shape=(28, 28)),
		tf.keras.layers.Dense(512, activation="relu"),
		tf.keras.layers.Dropout(.2),
		tf.keras.layyers.Dense(10, activation="softmax")
	])

The fit and callback procedure:

log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")

tensorboard_callback = tf.keras.callbacks.TensorBoard(
	log_dir=log_dir,
	histogram_freq=1
)

model.fit(X_train,
		  y_train,
		  epochs=5,
		  validation_data=(X_test, y_test)
		  callbacks=[tensorboard_callback])

Calling tensorboard:

%tensorboard --logdir logs/fit

Looking at graphics in a dataset

fashion_mnist = keras.datasets.fashion_mnist

(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

img = np.reshape(train_images[0], (-1, 28, 28, 1))

# Sets up a timestamped log directory

logdir = "logs/train_data/" + datetime.now().strftime("%Y%m%d-%H%M%S")

# Creates a file writer for the log directory

file_writer = tf.summary.create_file_writer(logdir)

# Using the filewriter, log the reshaped image

with file_writer.as_default():
	tf.summary.image("Training data", img, step=0)

With more than one image:

with file_writer.as_default():
	# Don't forget to reshape

	images = np.reshape(train_images[0:25], (-1, 28, 28, 1))

	tf.summary.image("Training data", images, max_outputs=25, step=0)

%tensorboard --logdir logs/train_data

Confusion matrix

def plot_to_image(figure):
	buf = io.BytesIO()

	plt.savefig(buf, format="png")

	plt.close(figure)

	buf.seek(0)

	image = tf.image.decode_png(buf.getvalue(), channels=4)

	image = tf.expand_dims(image, 0)

	return image

Multiple callbacks

# Train the classifier

model.fit(
	train_images,
	train_labels,
	epochs=5,
	verbose=0, # Suppress chatty output
	callbacks=[tensorboard_callback, cm_callback],
	validation_data=(test_images, test_labels)
)

The cm_callback can be defined as

def log_confusion_matrix(epoch, logs):
	test_pred_raw = model.predict(test_images)
	test_pred = np.argmax(test_pred_raw, axis=1)

	cm = sklearn.metrics.confusion_matrix(test_labels, test_pred)

	figure = plot_confusion_matrix(cm, class_names=class_names)

	cm_image = plot_to_image(figure)

	with file_writer_cm.as_default():
		tf.summary.image("Confusion Matrix:", cm_image, step=epoch)

# Define the per-epoch callback
cm_callback = tf.keras.callbacks.LambdaCallback(on_epoch_end=log_confusion_matrix)

Quiz

Question	Answer
1. If you want to track training data, such as accuracy and loss and visualize it on TensorBoard, what programming pattern would you use?	Store them in a log file and use epoch-by-epoch callbacks to update it. Have Tensorboard read that log file
2. If you are using a callback to manage logging of data for TensorBoard, where would you specify it?	During the model.fit process
3. What’s the name of the class in TensorFlow that handles and manages logging to TensorBoard?	tf.keras.callbacks.TensorBoard
4. How do you get a histogram plotting each epoch in your training?	Set histogram_freq=1
5. If you want to upload your logs to tensorboard.dev to share with others, what command do you use?	tensorboard dev upload --logdir ./logs
6. When you share your logs on tensorboard.dev, what is the URL signature that you share with others?	tensorboard.dev/experiment/[code]
7. If you want to write an image to logs for TensorBoard to represent it, what API call should you use?	tf.summary.image() – with image as a parameter
8. If you have written logs (including images and more) to logs/train_data, and want to inspect them with TensorBoard, what’s the command to use?	%tensorboard --logdir logs/train_data
9. If you want to create a simple, custom, callback on-the-fly, what API would you use?	tf.keras.callbacks.LambdaCallback
10. Which module in TensorFlow provides APIs to store data that TensorBoard and other tools can use?	tf.summary