Coursera

Week 4

Intro to federated learning

Data is born at the edge

• Billions of phones and IoT devices constantly generate data
• Data enables better products and smarter models

Can data live at the edge? On-device inference offers:

• Improved latency
• Works offline
• Better battery life
• Privacy advantages

Examples of model on-device:

• GBoard: mobile keyboard (machine learning)
  • Tap typing
  • Gesture typing
  • Autocorrections
  • Predictions
  • Voice2text

How it work:

1. Select a subset of available devices (and they’re not currently being used) from a population of users
2. These devices will receive a training model which can then be retrained on those devices.
3. The training result will then be sent to the server, not the training data.
4. Server will use the model to retrain the master model.
5. Original models will still be kept in local machines. The model should be tested before deploying to the end-user.

Maintaining user privacy

There’s potential that the uploaded model can be reverse engineered to get the data on the local machine.

Two concept to maintaining data privacy:

• Federated learning only works on aggregates.
• Data can be encrypted on the server (secure aggregation).

Federated learning APIs

Google’s federated system: tensorflow.org/federated

TensorFlow Federated

• Federated Learning (FL) API
  • Implementations of federated training/evaluation
  • Can be applied to existing TF models/data
• Federated Core (FC) API
  • Allows for expressing new federated algorithms
• Local runtime for simulation

Type: {float32}@CLIENTS - lots of values @ where data live. Send to: float32@SERVER. Federated operation can be interpreted as a functiion even though its inputs and outputs are in different places.

Example

READINGS_TYPE = tff.FederatedType(tf.float32, tff.CLIENTS)

# An abstract specification of a simple distributed system
@tff.federated_computation(READINGS_TYPE)
def get_average_temperature(sensor_readings):
	return tff.federated_mean(sensor_readings)

THRESHOLD_TYPE = tff.FederatedType(
	tf.float32,
	tff.SERVER,
	all_equals=TRUE)

@tff.federated_computation(READINGS_TYPE, THRESHOLD_TYPE)
def get_fraction_over_threshold(readings, threshold):
	@tff.tf_computation(tf.float32, tf.float32)
	def _is_over_as_float(val, threshold):
		return tf.to_float(val > threshold)

	return tff.federated_mean(
		tff.federated_map(_is_over_as_float, [
			readings,
			tff.federated_broadcast(threshold)
		])
	)

Quiz

Question	Answer
1. What advantage does Federated Learning give you?	All of the above
2. What is the privacy principle of focused collection?	Devices report only the data needed for a specific computation
3. What is secure aggregation?	Devices in a network pair up, and create obfuscation keys that get cancelled out when aggregated on the server
4. TensorFlow Federated includes a Federated Learning API, a Federated Core API and a runtime for simulations. What’s the role of the Federated Learning API?	It contains implementations of federated training that can be applied to existing tensorflow models and data
5. If you want to declare a federated type, where a numeric item of data is available across all your devices, how do you do it?	You declare the type as {float32}@clients
6. If you want to do a federated computation on the server, what do you need to do to your computation function?	Attribute the function with @tff.federated_computation
7. You want to return a mean value of client values, calculated on the server, back to the clients. How do you do this?	You have to use a tff.federated_mean to calculate the value and return its results
8. If you want to try the tensorflow federated APIs, how do you install them for Python?	Pip install tensorflow-federated