This lab will walk you through how to use object detection models available in Tensorflow Hub. In the following sections, you will:
Let’s get started!
import tensorflow as tf
import tensorflow_hub as hub
from PIL import Image
from PIL import ImageOps
import tempfile
from six.moves.urllib.request import urlopen
from six import BytesIO
Tensorflow Hub is a repository of trained machine learning models which you can reuse in your own projects.
# you can switch the commented lines here to pick the other model
# inception resnet version 2
module_handle = "https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1"
# You can choose ssd mobilenet version 2 instead and compare the results
#module_handle = "https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1"
Next, you’ll load the model specified by the module_handle.
model = hub.load(module_handle)
Some models in the Tensorflow hub can be used for different tasks. So each model’s documentation should show what signature to use when running the model.
print(hub.load(module_handle).signatures.keys()). In your case, the models you will be using only have the default signature so you don’t have to worry about other types.# take a look at the available signatures for this particular model
model.signatures.keys()
KeysView(_SignatureMap({'default': <ConcreteFunction pruned(images) at 0x7F8C45BBF160>}))
Please choose the ‘default’ signature for your object detector.
detector = model.signatures['default']
This function downloads an image specified by a given “url”, pre-processes it, and then saves it to disk.
def download_and_resize_image(url, new_width=256, new_height=256):
'''
Fetches an image online, resizes it and saves it locally.
Args:
url (string) -- link to the image
new_width (int) -- size in pixels used for resizing the width of the image
new_height (int) -- size in pixels used for resizing the length of the image
Returns:
(string) -- path to the saved image
'''
# create a temporary file ending with ".jpg"
_, filename = tempfile.mkstemp(suffix=".jpg")
# opens the given URL
response = urlopen(url)
# reads the image fetched from the URL
image_data = response.read()
# puts the image data in memory buffer
image_data = BytesIO(image_data)
# opens the image
pil_image = Image.open(image_data)
# resizes the image. will crop if aspect ratio is different.
pil_image = ImageOps.fit(pil_image, (new_width, new_height), Image.ANTIALIAS)
# converts to the RGB colorspace
pil_image_rgb = pil_image.convert("RGB")
# saves the image to the temporary file created earlier
pil_image_rgb.save(filename, format="JPEG", quality=90)
print("Image downloaded to %s." % filename)
return filename
Now, using download_and_resize_image you can get a sample image online and save it locally.
# You can choose a different URL that points to an image of your choice
image_url = "https://upload.wikimedia.org/wikipedia/commons/f/fb/20130807_dublin014.JPG"
# download the image and use the original height and width
downloaded_image_path = download_and_resize_image(image_url, 3872, 2592)
Image downloaded to /tmp/tmprh27x48u.jpg.
<ipython-input-6-4e9bff65542d>:31: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use LANCZOS or Resampling.LANCZOS instead.
pil_image = ImageOps.fit(pil_image, (new_width, new_height), Image.ANTIALIAS)
This function will take in the object detection model detector and the path to a sample image, then use this model to detect objects and display its predicted class categories and detection boxes.
load_image to convert the image into a tensor.def load_img(path):
'''
Loads a JPEG image and converts it to a tensor.
Args:
path (string) -- path to a locally saved JPEG image
Returns:
(tensor) -- an image tensor
'''
# read the file
img = tf.io.read_file(path)
# convert to a tensor
img = tf.image.decode_jpeg(img, channels=3)
return img
def run_detector(detector, path):
'''
Runs inference on a local file using an object detection model.
Args:
detector (model) -- an object detection model loaded from TF Hub
path (string) -- path to an image saved locally
'''
# load an image tensor from a local file path
img = load_img(path)
# add a batch dimension in front of the tensor
converted_img = tf.image.convert_image_dtype(img, tf.float32)[tf.newaxis, ...]
# run inference using the model
result = detector(converted_img)
# save the results in a dictionary
result = {key:value.numpy() for key,value in result.items()}
# print results
print("Found %d objects." % len(result["detection_scores"]))
print(result["detection_scores"])
print(result["detection_class_entities"])
print(result["detection_boxes"])
You can run your detector by calling the run_detector function. This will print the number of objects found followed by three lists:
You will see how to overlay this information on the original image in the next sections and in this week’s assignment!
# runs the object detection model and prints information about the objects found
run_detector(detector, downloaded_image_path)
Found 100 objects.
[0.6544877 0.61146665 0.60422647 0.5926298 0.59218925 0.5804908
0.55140567 0.49466667 0.4751578 0.4734219 0.43995926 0.41484997
0.4062958 0.39828598 0.39765334 0.3762126 0.3727947 0.3657494
0.3526071 0.33274773 0.30428562 0.27276587 0.26864463 0.2577658
0.25290856 0.2461159 0.23403971 0.20343061 0.18229099 0.18045811
0.17571175 0.1643496 0.15850548 0.15665185 0.15470862 0.15452752
0.14924932 0.13340679 0.12948076 0.1264973 0.1204429 0.11767519
0.11355798 0.11114862 0.11100134 0.10914899 0.10604019 0.08940561
0.08598386 0.08280157 0.0810426 0.07806069 0.07759769 0.07628638
0.07546826 0.07444111 0.07427185 0.07205128 0.07177501 0.07102278
0.0703265 0.06809752 0.06304514 0.06285969 0.06271047 0.06224045
0.05881866 0.05815066 0.05795667 0.05787634 0.05462411 0.05274345
0.0513373 0.04826394 0.0470848 0.04682882 0.04495558 0.04405202
0.04360476 0.04113363 0.04109994 0.03968302 0.03935016 0.03912614
0.0387967 0.03878462 0.03739682 0.03606895 0.03367105 0.03366907
0.03260141 0.03253593 0.03201524 0.02983092 0.02878013 0.0286751
0.02804289 0.02783225 0.02734366 0.02668286]
[b'Person' b'Person' b'Person' b'Person' b'Footwear' b'Person' b'Building'
b'Bicycle' b'Building' b'Window' b'Person' b'Bicycle' b'Wheel'
b'Building' b'Building' b'Building' b'Person' b'Wheel' b'Window'
b'Window' b'Building' b'Person' b'Van' b'Person' b'Bicycle wheel'
b'Person' b'Window' b'Window' b'Building' b'Window' b'Window' b'Man'
b'Person' b'Woman' b'Person' b'Clothing' b'Bicycle wheel' b'Window'
b'Person' b'Window' b'Land vehicle' b'Land vehicle' b'Clothing' b'Window'
b'Bicycle' b'Land vehicle' b'House' b'House' b'Man' b'Window' b'Clothing'
b'Window' b'Footwear' b'Person' b'Man' b'Man' b'House' b'Building'
b'Person' b'Clothing' b'Window' b'Person' b'Man' b'Person' b'Furniture'
b'Jeans' b'Person' b'Person' b'Person' b'Land vehicle' b'Window' b'House'
b'Woman' b'Man' b'Window' b'Person' b'Person' b'Clothing' b'Man' b'Man'
b'Window' b'Car' b'Person' b'Man' b'Chair' b'Car' b'House' b'Window'
b'Tire' b'Clothing' b'Window' b'Clothing' b'Land vehicle' b'Window'
b'Window' b'Man' b'Van' b'Bus' b'Clothing' b'Car']
[[5.12794375e-01 5.29271007e-01 6.01662278e-01 5.52094579e-01]
[5.19746125e-01 6.01507187e-01 6.46124303e-01 6.34682953e-01]
[5.05745888e-01 5.00440776e-01 6.01349115e-01 5.23089767e-01]
[4.86308753e-01 4.12762225e-01 6.78550065e-01 4.59905535e-01]
[8.15190852e-01 9.56118405e-01 8.42701614e-01 9.87144709e-01]
[4.95466232e-01 9.23534214e-01 8.35635006e-01 9.99056816e-01]
[1.10986838e-02 1.19120395e-02 7.39750683e-01 4.24907058e-01]
[5.77825963e-01 3.66453201e-01 7.12805569e-01 4.83338177e-01]
[7.74934888e-02 4.13053840e-01 5.79458773e-01 5.60309291e-01]
[0.00000000e+00 1.19292557e-01 2.23897204e-01 1.83949053e-01]
[5.14069498e-01 7.48097956e-01 5.91962337e-01 7.66569078e-01]
[5.70778012e-01 3.61820400e-01 7.07328379e-01 4.29667264e-01]
[6.32094145e-01 3.59869927e-01 7.03841686e-01 4.11815494e-01]
[1.59081779e-02 6.84961855e-01 5.59389472e-01 8.11147094e-01]
[0.00000000e+00 7.97109246e-01 6.73735797e-01 1.00000000e+00]
[0.00000000e+00 2.17026800e-01 6.50972903e-01 4.32000995e-01]
[5.00372708e-01 3.77004564e-01 6.33350730e-01 4.14514363e-01]
[6.40339911e-01 4.45023447e-01 7.03034759e-01 4.83457565e-01]
[1.94405240e-03 0.00000000e+00 1.39331952e-01 2.62884162e-02]
[2.55179731e-03 9.66625512e-01 1.53752670e-01 1.00000000e+00]
[1.41558331e-03 1.41049293e-03 7.64848351e-01 2.69351959e-01]
[5.04901230e-01 3.60784829e-01 6.37663186e-01 3.85480016e-01]
[4.83383536e-01 6.19484186e-01 5.62658072e-01 6.61571980e-01]
[4.98201400e-01 3.64614308e-01 6.61157548e-01 4.04896617e-01]
[6.31229401e-01 3.60322893e-01 7.04146981e-01 4.11499321e-01]
[5.21806836e-01 5.77694893e-01 5.87613046e-01 6.00717723e-01]
[2.19603792e-01 3.48738879e-01 3.38255674e-01 3.77067596e-01]
[1.24826752e-01 2.50923932e-01 2.79914707e-01 2.81625867e-01]
[2.57318437e-01 5.67493677e-01 5.30910254e-01 6.87876523e-01]
[4.21753451e-02 8.74765217e-01 2.52863348e-01 9.13046122e-01]
[1.56401649e-01 4.43365514e-01 2.22233847e-01 4.75784540e-01]
[5.01994312e-01 9.21467483e-01 8.36361825e-01 1.00000000e+00]
[5.23673594e-01 5.70347011e-01 5.84506154e-01 5.91607094e-01]
[5.19169092e-01 5.99966168e-01 6.46330297e-01 6.34094715e-01]
[5.13154805e-01 6.79228485e-01 5.50981283e-01 6.92548156e-01]
[5.24344623e-01 9.24945474e-01 8.10528398e-01 9.97979462e-01]
[6.38063431e-01 4.42797363e-01 7.01729000e-01 4.84131992e-01]
[3.41055244e-02 3.55657637e-01 1.62304819e-01 3.74908775e-01]
[4.88090128e-01 4.53366995e-01 6.22257411e-01 4.79664892e-01]
[9.66472668e-04 3.07707310e-01 1.06515922e-01 3.32070351e-01]
[4.82969970e-01 6.19791746e-01 5.64779043e-01 6.60652578e-01]
[5.82391262e-01 3.64923328e-01 7.13891506e-01 4.84685272e-01]
[5.23790002e-01 7.49292910e-01 5.85470378e-01 7.65311599e-01]
[3.51464361e-01 9.74868774e-01 5.53043604e-01 9.98887062e-01]
[6.09077036e-01 4.26833600e-01 7.05196321e-01 4.87107456e-01]
[5.69254756e-01 3.59783024e-01 7.08566308e-01 4.28439111e-01]
[0.00000000e+00 8.11187387e-01 6.93582594e-01 9.93253589e-01]
[1.04297642e-02 2.29470227e-02 7.27312624e-01 4.22287345e-01]
[4.84632283e-01 4.10697728e-01 6.94742978e-01 4.63139921e-01]
[8.11543763e-02 3.84775847e-01 2.07952142e-01 4.11755383e-01]
[5.38567603e-01 6.03585064e-01 6.34740949e-01 6.34476602e-01]
[0.00000000e+00 1.24075953e-02 1.40296444e-01 2.47341208e-02]
[6.29779756e-01 6.14883065e-01 6.44907951e-01 6.25334740e-01]
[5.02842903e-01 3.82420719e-01 5.96017063e-01 4.12718743e-01]
[5.14681220e-01 7.47871101e-01 5.91947973e-01 7.66782463e-01]
[5.06433070e-01 5.00402749e-01 6.00716949e-01 5.23319721e-01]
[0.00000000e+00 2.11128518e-01 6.50825799e-01 4.34384227e-01]
[0.00000000e+00 7.06319869e-01 6.17160559e-01 8.65938902e-01]
[4.89298046e-01 4.54274952e-01 5.72619915e-01 4.76397544e-01]
[5.09207249e-01 4.16264892e-01 6.69016659e-01 4.59577113e-01]
[4.67800163e-03 8.03107023e-01 1.59582153e-01 8.40365171e-01]
[5.26175678e-01 5.68375766e-01 5.79436421e-01 5.82803071e-01]
[5.02847493e-01 3.73985976e-01 6.47125959e-01 4.12972540e-01]
[4.85917628e-01 4.44437265e-01 6.24689877e-01 4.73519832e-01]
[5.74168563e-01 2.67251253e-01 6.57761574e-01 3.20313990e-01]
[6.71982229e-01 9.40317810e-01 8.21177304e-01 9.89213943e-01]
[5.24104953e-01 5.61555922e-01 5.78347266e-01 5.80502510e-01]
[5.17590046e-01 7.57221103e-01 5.88313997e-01 7.71546006e-01]
[5.23328602e-01 5.57813644e-01 5.79029083e-01 5.73553443e-01]
[6.12360120e-01 4.27401602e-01 7.06096351e-01 4.88300323e-01]
[0.00000000e+00 2.44237095e-01 6.08887561e-02 2.93773830e-01]
[1.54844159e-02 1.94193644e-03 7.45163262e-01 2.59336591e-01]
[4.93266523e-01 9.23959553e-01 8.36913347e-01 9.97706771e-01]
[5.05293071e-01 3.60166341e-01 6.43362164e-01 3.91438335e-01]
[8.43417831e-03 2.42121428e-01 4.97449599e-02 2.83145607e-01]
[5.22109330e-01 5.36088169e-01 5.97674727e-01 5.53133249e-01]
[5.13125777e-01 5.23810029e-01 6.00540221e-01 5.42964995e-01]
[5.18315673e-01 5.03453374e-01 5.97545385e-01 5.22752941e-01]
[5.20455718e-01 6.00931644e-01 6.45991147e-01 6.34363949e-01]
[5.13168275e-01 6.79253757e-01 5.50486088e-01 6.92442954e-01]
[4.29723322e-01 8.28743696e-01 5.90048730e-01 8.64375412e-01]
[5.26593089e-01 6.27190828e-01 5.63289881e-01 6.53785110e-01]
[5.04781067e-01 3.89410704e-01 6.15231574e-01 4.19951528e-01]
[5.01324832e-01 3.64236444e-01 6.59752846e-01 4.03720200e-01]
[5.73110223e-01 2.66732633e-01 6.66223645e-01 3.18649918e-01]
[5.15102446e-01 6.24091566e-01 5.63832402e-01 6.58031881e-01]
[8.32033306e-02 4.07567859e-01 5.84343910e-01 5.58310807e-01]
[2.88201898e-01 4.62593802e-04 4.14279848e-01 3.67076769e-02]
[6.27132595e-01 3.60995114e-01 7.05960631e-01 4.09780324e-01]
[4.97159332e-01 4.55211043e-01 5.84271252e-01 4.77872044e-01]
[1.17193609e-02 3.08072507e-01 9.73201171e-02 3.25075448e-01]
[5.15894115e-01 3.80090594e-01 5.96972644e-01 4.11767155e-01]
[5.12428045e-01 6.23649001e-01 5.62436700e-01 6.57682359e-01]
[4.00773644e-01 8.84974241e-01 5.81656635e-01 9.39130306e-01]
[0.00000000e+00 9.94758308e-03 1.36253998e-01 3.15974355e-02]
[5.13905585e-01 5.29502392e-01 6.02055907e-01 5.52376151e-01]
[5.10690629e-01 6.24039352e-01 5.63410223e-01 6.58180058e-01]
[4.80379760e-01 6.20327830e-01 5.65284133e-01 6.60123467e-01]
[5.38407445e-01 9.28024352e-01 7.13617206e-01 9.99452770e-01]
[4.86337692e-01 6.20247483e-01 5.63528657e-01 6.60217881e-01]]