In this lab, you use utilize the following tools and services to deploy and run a TFX pipeline on Google Cloud that automates the development and deployment of a TensorFlow 2.3 WideDeep Classifer to predict forest cover from cartographic data:
You will then create and monitor pipeline runs using the TFX CLI as well as the KFP UI.
import yaml
# Set `PATH` to include the directory containing TFX CLI and skaffold.
PATH=%env PATH
%env PATH=/home/jupyter/.local/bin:{PATH}
env: PATH=/home/jupyter/.local/bin:/opt/conda/bin:/opt/conda/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
!python -c "import tfx; print('TFX version: {}'.format(tfx.__version__))"
!python -c "import kfp; print('KFP version: {}'.format(kfp.__version__))"
TFX version: 0.25.0
KFP version: 1.0.4
Note: this lab was built and tested with the following package versions:
TFX version: 0.25.0
KFP version: 1.0.4
(Optional) If running the above command results in different package versions or you receive an import error, upgrade to the correct versions by running the cell below:
%pip install --upgrade --user tfx==0.25.0
%pip install --upgrade --user kfp==1.0.4
Requirement already satisfied: tfx==0.25.0 in /home/jupyter/.local/lib/python3.7/site-packages (0.25.0)
Requirement already satisfied: pyyaml<6,>=3.12 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (5.4.1)
Requirement already satisfied: grpcio<2,>=1.28.1 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (1.62.1)
Requirement already satisfied: tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.3.4)
Requirement already satisfied: tensorflow-cloud<0.2,>=0.1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (0.1.13)
Requirement already satisfied: kubernetes<12,>=10.0.1 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (11.0.0)
Requirement already satisfied: ml-metadata<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.1)
Requirement already satisfied: apache-beam[gcp]<3,>=2.25 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.28.0)
Requirement already satisfied: google-api-python-client<2,>=1.7.8 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (1.12.11)
Requirement already satisfied: pyarrow<0.18,>=0.17 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.17.1)
Requirement already satisfied: tensorflow-serving-api!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.3.0)
Requirement already satisfied: six<2,>=1.10 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (1.16.0)
Requirement already satisfied: tensorflow-data-validation<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: attrs<21,>=19.3.0 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (20.3.0)
Requirement already satisfied: tensorflow-hub<0.10,>=0.9.0 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (0.9.0)
Requirement already satisfied: tensorflow-model-analysis<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: tensorflow-transform<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: tfx-bsl<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: click<8,>=7 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (7.1.2)
Requirement already satisfied: docker<5,>=4.1 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (4.4.4)
Requirement already satisfied: keras-tuner<2,>=1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (1.0.1)
Requirement already satisfied: absl-py<0.11,>=0.9 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (0.10.0)
Requirement already satisfied: protobuf<4,>=3.12.2 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (3.20.3)
Requirement already satisfied: jinja2<3,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.11.3)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.3.1.1)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.6.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.4.7)
Requirement already satisfied: crcmod<2.0,>=1.7 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (4.1.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.26.0)
Requirement already satisfied: httplib2<0.18.0,>=0.8 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.17.4)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.8.2)
Requirement already satisfied: typing-extensions<3.8.0,>=3.7.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.7.4.3)
Requirement already satisfied: future<1.0.0,>=0.18.2 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.18.2)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.0.0)
Requirement already satisfied: pytz>=2018.3 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2021.3)
Requirement already satisfied: avro-python3!=1.9.2,<1.10.0,>=1.8.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.9.2.1)
Requirement already satisfied: pydot<2,>=1.2.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.4.2)
Requirement already satisfied: numpy<1.20.0,>=1.14.3 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.18.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.12.2)
Requirement already satisfied: google-cloud-dlp<2,>=0.12.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.0.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.3.2)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.16.3)
Requirement already satisfied: google-cloud-build<3,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.0.0)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.2.2)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.19.3)
Requirement already satisfied: cachetools<5,>=3.1.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (4.2.4)
Requirement already satisfied: google-cloud-bigquery<2,>=1.6.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.28.3)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7.3)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.0.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7.3)
Requirement already satisfied: google-auth<2,>=1.18.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.35.0)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7.0)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.5.31)
Requirement already satisfied: google-cloud-datastore<2,>=1.7.1 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.15.5)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker<5,>=4.1->tfx==0.25.0) (1.2.1)
Requirement already satisfied: google-auth-httplib2>=0.0.3 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0) (0.1.0)
Requirement already satisfied: google-api-core<3dev,>=1.21.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0) (1.34.1)
Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0) (3.0.1)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from jinja2<3,>=2.7.3->tfx==0.25.0) (1.1.1)
Requirement already satisfied: colorama in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (0.4.4)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (4.62.3)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (0.8.9)
Requirement already satisfied: scipy in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (1.7.3)
Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (1.0.1)
Requirement already satisfied: terminaltables in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (3.1.0)
Requirement already satisfied: urllib3>=1.24.2 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (1.26.7)
Requirement already satisfied: requests-oauthlib in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (1.3.0)
Requirement already satisfied: certifi>=14.05.14 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (2021.10.8)
Requirement already satisfied: setuptools>=21.0.0 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (59.4.0)
Requirement already satisfied: gast==0.3.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.3.3)
Requirement already satisfied: google-pasta>=0.1.8 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.2.0)
Requirement already satisfied: keras-preprocessing<1.2,>=1.1.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.1.2)
Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.10.0)
Requirement already satisfied: tensorboard<3,>=2.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.3.0)
Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.37.0)
Requirement already satisfied: tensorflow-estimator<2.4.0,>=2.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.3.0)
Requirement already satisfied: opt-einsum>=2.3.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (3.3.0)
Requirement already satisfied: termcolor>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.1.0)
Requirement already satisfied: wrapt>=1.11.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.13.3)
Requirement already satisfied: astunparse==1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.6.3)
Requirement already satisfied: tensorflow-datasets<3.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow-cloud<0.2,>=0.1->tfx==0.25.0) (3.0.0)
Requirement already satisfied: google-cloud-storage in /opt/conda/lib/python3.7/site-packages (from tensorflow-cloud<0.2,>=0.1->tfx==0.25.0) (1.43.0)
Requirement already satisfied: pandas<2,>=1.0 in /home/jupyter/.local/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0) (1.3.5)
Requirement already satisfied: tensorflow-metadata<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0) (0.25.0)
Requirement already satisfied: joblib<0.15,>=0.12 in /home/jupyter/.local/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0) (0.14.1)
Requirement already satisfied: ipywidgets<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (7.6.5)
Requirement already satisfied: ipython<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (7.30.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /home/jupyter/.local/lib/python3.7/site-packages (from google-api-core<3dev,>=1.21.0->google-api-python-client<2,>=1.7.8->tfx==0.25.0) (1.63.0)
Requirement already satisfied: fasteners>=0.14 in /opt/conda/lib/python3.7/site-packages (from google-apitools<0.5.32,>=0.5.31->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.18.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.18.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.2.7)
Requirement already satisfied: google-resumable-media<2.0dev,>=0.6.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.3.3)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigtable<2,>=0.31.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.12.3)
Requirement already satisfied: libcst>=0.2.5 in /opt/conda/lib/python3.7/site-packages (from google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.3.23)
Requirement already satisfied: proto-plus>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.19.8)
Requirement already satisfied: docopt in /opt/conda/lib/python3.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.6.2)
Requirement already satisfied: traitlets>=4.2 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.1.1)
Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.1.3)
Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.8.0)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (3.0.22)
Requirement already satisfied: pickleshare in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.7.5)
Requirement already satisfied: backcall in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.2.0)
Requirement already satisfied: decorator in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.1.0)
Requirement already satisfied: pygments in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (2.10.0)
Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.18.1)
Requirement already satisfied: nbformat>=4.2.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.1.3)
Requirement already satisfied: ipykernel>=4.5.1 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.5.1)
Requirement already satisfied: ipython-genutils~=0.2.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.2.0)
Requirement already satisfied: widgetsnbextension~=3.5.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (3.5.2)
Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.0.2)
Requirement already satisfied: pbr>=0.11 in /opt/conda/lib/python3.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in /opt/conda/lib/python3.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.4.8)
Requirement already satisfied: pyparsing>=2.1.4 in /opt/conda/lib/python3.7/site-packages (from pydot<2,>=1.2.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.0.6)
Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.0.8)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.1)
Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.0.2)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.8.0)
Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (3.3.6)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.4.6)
Requirement already satisfied: promise in /opt/conda/lib/python3.7/site-packages (from tensorflow-datasets<3.1.0->tensorflow-cloud<0.2,>=0.1->tfx==0.25.0) (2.3)
Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib->kubernetes<12,>=10.0.1->tfx==0.25.0) (3.1.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->keras-tuner<2,>=1->tfx==0.25.0) (3.0.0)
Requirement already satisfied: grpcio-status<2.0dev,>=1.33.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core<3dev,>=1.21.0->google-api-python-client<2,>=1.7.8->tfx==0.25.0) (1.42.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.7/site-packages (from google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.1.2)
Requirement already satisfied: debugpy<2.0,>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.5.1)
Requirement already satisfied: jupyter-client<8.0 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (7.1.0)
Requirement already satisfied: argcomplete>=1.12.3 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.12.3)
Requirement already satisfied: tornado<7.0,>=4.2 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.1)
Requirement already satisfied: importlib-metadata<5 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.8.2)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /opt/conda/lib/python3.7/site-packages (from jedi>=0.16->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.8.3)
Requirement already satisfied: typing-inspect>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from libcst>=0.2.5->google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.7.1)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /opt/conda/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.2.1)
Requirement already satisfied: jupyter-core in /opt/conda/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.9.1)
Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.7/site-packages (from pexpect>4.3->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.7.0)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.2.5)
Requirement already satisfied: notebook>=4.4.1 in /opt/conda/lib/python3.7/site-packages (from widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.4.6)
Requirement already satisfied: cffi>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from google-crc32c<2.0dev,>=1.0->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.15.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata<5->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (3.6.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.18.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.4.0)
Requirement already satisfied: nest-asyncio>=1.5 in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.5.4)
Requirement already satisfied: entrypoints in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.3)
Requirement already satisfied: pyzmq>=13 in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (22.3.0)
Requirement already satisfied: terminado>=0.8.3 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.12.1)
Requirement already satisfied: Send2Trash>=1.8.0 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.8.0)
Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.12.0)
Requirement already satisfied: nbconvert in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.3.0)
Requirement already satisfied: argon2-cffi in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (21.1.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.4.3)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.7/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.21)
Requirement already satisfied: testpath in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.5.0)
Requirement already satisfied: pandocfilters>=1.4.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.5.0)
Requirement already satisfied: mistune<2,>=0.8.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.8.4)
Requirement already satisfied: bleach in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.1.0)
Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.5.9)
Requirement already satisfied: jupyterlab-pygments in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.1.2)
Requirement already satisfied: defusedxml in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.7.1)
Requirement already satisfied: webencodings in /opt/conda/lib/python3.7/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.5.1)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (21.3)
Note: you may need to restart the kernel to use updated packages.
Requirement already satisfied: kfp==1.0.4 in /home/jupyter/.local/lib/python3.7/site-packages (1.0.4)
Requirement already satisfied: PyYAML in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (5.4.1)
Requirement already satisfied: click in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (7.1.2)
Requirement already satisfied: google-auth>=1.6.1 in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (1.35.0)
Requirement already satisfied: jsonschema>=3.0.1 in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4) (4.2.1)
Requirement already satisfied: kubernetes<12.0.0,>=8.0.0 in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (11.0.0)
Requirement already satisfied: requests-toolbelt>=0.8.0 in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (0.10.1)
Requirement already satisfied: strip-hints in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (0.1.10)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4) (0.8.9)
Requirement already satisfied: Deprecated in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (1.2.14)
Requirement already satisfied: cloudpickle in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4) (2.0.0)
Requirement already satisfied: kfp-server-api<2.0.0,>=0.2.5 in /home/jupyter/.local/lib/python3.7/site-packages (from kfp==1.0.4) (1.8.5)
Requirement already satisfied: google-cloud-storage>=1.13.0 in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4) (1.43.0)
Requirement already satisfied: six>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4) (1.16.0)
Requirement already satisfied: setuptools>=40.3.0 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4) (59.4.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4) (4.2.4)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4) (0.2.7)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4) (4.8)
Requirement already satisfied: google-api-core<3.0dev,>=1.29.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-storage>=1.13.0->kfp==1.0.4) (1.34.1)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-storage>=1.13.0->kfp==1.0.4) (2.26.0)
Requirement already satisfied: google-cloud-core<3.0dev,>=1.6.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-storage>=1.13.0->kfp==1.0.4) (1.7.3)
Requirement already satisfied: protobuf in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-storage>=1.13.0->kfp==1.0.4) (3.20.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=1.3.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-storage>=1.13.0->kfp==1.0.4) (1.3.3)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4) (0.18.0)
Requirement already satisfied: importlib-metadata in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4) (4.8.2)
Requirement already satisfied: importlib-resources>=1.4.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4) (5.4.0)
Requirement already satisfied: attrs>=17.4.0 in /home/jupyter/.local/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4) (20.3.0)
Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.7/site-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.0.4) (2.8.2)
Requirement already satisfied: urllib3>=1.15 in /opt/conda/lib/python3.7/site-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.0.4) (1.26.7)
Requirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.0.4) (2021.10.8)
Requirement already satisfied: requests-oauthlib in /opt/conda/lib/python3.7/site-packages (from kubernetes<12.0.0,>=8.0.0->kfp==1.0.4) (1.3.0)
Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12.0.0,>=8.0.0->kfp==1.0.4) (1.2.1)
Requirement already satisfied: wrapt<2,>=1.10 in /opt/conda/lib/python3.7/site-packages (from Deprecated->kfp==1.0.4) (1.13.3)
Requirement already satisfied: wheel in /opt/conda/lib/python3.7/site-packages (from strip-hints->kfp==1.0.4) (0.37.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /home/jupyter/.local/lib/python3.7/site-packages (from google-api-core<3.0dev,>=1.29.0->google-cloud-storage>=1.13.0->kfp==1.0.4) (1.63.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.7/site-packages (from google-resumable-media<3.0dev,>=1.3.0->google-cloud-storage>=1.13.0->kfp==1.0.4) (1.1.2)
Requirement already satisfied: zipp>=3.1.0 in /opt/conda/lib/python3.7/site-packages (from importlib-resources>=1.4.0->jsonschema>=3.0.1->kfp==1.0.4) (3.6.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.6.1->kfp==1.0.4) (0.4.8)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage>=1.13.0->kfp==1.0.4) (3.1)
Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage>=1.13.0->kfp==1.0.4) (2.0.8)
Requirement already satisfied: typing-extensions>=3.6.4 in /home/jupyter/.local/lib/python3.7/site-packages (from importlib-metadata->jsonschema>=3.0.1->kfp==1.0.4) (3.7.4.3)
Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib->kubernetes<12.0.0,>=8.0.0->kfp==1.0.4) (3.1.1)
Requirement already satisfied: cffi>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from google-crc32c<2.0dev,>=1.0->google-resumable-media<3.0dev,>=1.3.0->google-cloud-storage>=1.13.0->kfp==1.0.4) (1.15.0)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.7/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0->google-resumable-media<3.0dev,>=1.3.0->google-cloud-storage>=1.13.0->kfp==1.0.4) (2.21)
Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to pick up the correct package versions.
Navigate to AI Platform Pipelines page in the Google Cloud Console.
Note you may have already deployed an AI Pipelines instance during the Setup for the lab series. If so, you can proceed using that instance. If not:
1. Create or select an existing Kubernetes cluster (GKE) and deploy AI Platform. Make sure to select "Allow access to the following Cloud APIs https://www.googleapis.com/auth/cloud-platform"
to allow for programmatic access to your pipeline by the Kubeflow SDK for the rest of the lab. Also, provide an App instance name
such as “tfx” or “mlops”.
Validate the deployment of your AI Platform Pipelines instance in the console before proceeding.
The pipeline source code can be found in the pipeline
folder.
%cd pipeline
/home/jupyter/mlops-on-gcp/workshops/tfx-caip-tf23/lab-02-tfx-pipeline/labs/pipeline
!ls -la
total 72
drwxr-xr-x 4 jupyter jupyter 4096 Mar 24 16:59 .
drwxr-xr-x 4 jupyter jupyter 4096 Mar 24 17:08 ..
-rw-r--r-- 1 jupyter jupyter 97 Mar 24 16:53 Dockerfile
drwxr-xr-x 2 jupyter jupyter 4096 Mar 24 16:57 __pycache__
-rw-r--r-- 1 jupyter jupyter 300 Mar 24 16:59 build.yaml
-rw-r--r-- 1 jupyter jupyter 1666 Mar 24 16:53 config.py
-rw-r--r-- 1 jupyter jupyter 1222 Mar 24 16:53 features.py
-rw-r--r-- 1 jupyter jupyter 11493 Mar 24 16:53 model.py
-rw-r--r-- 1 jupyter jupyter 11084 Mar 24 16:53 pipeline.py
-rw-r--r-- 1 jupyter jupyter 2032 Mar 24 16:53 preprocessing.py
-rw-r--r-- 1 jupyter jupyter 3282 Mar 24 16:53 runner.py
drwxr-xr-x 2 jupyter jupyter 4096 Mar 24 16:53 schema
-rw-r--r-- 1 jupyter jupyter 4573 Mar 24 17:04 tfx_covertype_continuous_training.tar.gz
The config.py
module configures the default values for the environment specific settings and the default values for the pipeline runtime parameters.
The default values can be overwritten at compile time by providing the updated values in a set of environment variables. You will set custom environment variables later on this lab.
The pipeline.py
module contains the TFX DSL defining the workflow implemented by the pipeline.
The preprocessing.py
module implements the data preprocessing logic the Transform
component.
The model.py
module implements the training, tuning, and model building logic for the Trainer
and Tuner
components.
The runner.py
module configures and executes KubeflowDagRunner
. At compile time, the KubeflowDagRunner.run()
method converts the TFX DSL into the pipeline package in the argo format for execution on your hosted AI Platform Pipelines instance.
The features.py
module contains feature definitions common across preprocessing.py
and model.py
.
You will use TFX CLI to compile and deploy the pipeline. As explained in the previous section, the environment specific settings can be provided through a set of environment variables and embedded into the pipeline package at compile time.
Update the below constants with the settings reflecting your lab environment.
GCP_REGION
- the compute region for AI Platform Training, Vizier, and Prediction.ARTIFACT_STORE
- An existing GCS bucket. You can use any bucket or use the GCS bucket created during installation of AI Platform Pipelines. The default bucket name will contain the kubeflowpipelines-
prefix. When specifying the bucket, do not use the trailing slash (/) at the end of the bucket name.# Use the following command to identify the GCS bucket for metadata and pipeline storage.
!gsutil ls
gs://qwiklabs-gcp-04-1852ebd59c9f-kubeflowpipelines-default/
CUSTOM_SERVICE_ACCOUNT
- In the gcp console Click on the Navigation Menu. Navigate to IAM & Admin
, then to Service Accounts
and use the service account starting with prefix - 'tfx-tuner-caip-service-account'
. This enables CloudTuner and the Google Cloud AI Platform extensions Tuner component to work together and allows for distributed and parallel tuning backed by AI Platform Vizier’s hyperparameter search algorithm. Please see the lab setup README
for setup instructions.ENDPOINT
- set the ENDPOINT
constant to the endpoint to your AI Platform Pipelines instance. The endpoint to the AI Platform Pipelines instance can be found on the AI Platform Pipelines page in the Google Cloud Console. Open the SETTINGS for your instance and use the value of the host
variable in the Connect to this Kubeflow Pipelines instance from a Python client via Kubeflow Pipelines SKD section of the SETTINGS window. The format is '...pipelines.googleusercontent.com'
.#TODO: Set your environment resource settings here for GCP_REGION, ARTIFACT_STORE_URI, ENDPOINT, and CUSTOM_SERVICE_ACCOUNT.
GCP_REGION = 'us-east1'
ARTIFACT_STORE_URI = 'gs://qwiklabs-gcp-04-1852ebd59c9f-kubeflowpipelines-default' #Change
ENDPOINT = '1fdd6c89bc6b1864-dot-us-east1.pipelines.googleusercontent.com' #Change
CUSTOM_SERVICE_ACCOUNT = 'tfx-tuner-caip-service-account@qwiklabs-gcp-04-1852ebd59c9f.iam.gserviceaccount.com' #Change
PROJECT_ID = !(gcloud config get-value core/project)
PROJECT_ID = PROJECT_ID[0]
# Set your resource settings as environment variables. These override the default values in pipeline/config.py.
%env GCP_REGION={GCP_REGION}
%env ARTIFACT_STORE_URI={ARTIFACT_STORE_URI}
%env CUSTOM_SERVICE_ACCOUNT={CUSTOM_SERVICE_ACCOUNT}
%env PROJECT_ID={PROJECT_ID}
env: GCP_REGION=us-east1
env: ARTIFACT_STORE_URI=gs://qwiklabs-gcp-04-1852ebd59c9f-kubeflowpipelines-default
env: CUSTOM_SERVICE_ACCOUNT=tfx-tuner-caip-service-account@qwiklabs-gcp-04-1852ebd59c9f.iam.gserviceaccount.com
env: PROJECT_ID=qwiklabs-gcp-04-1852ebd59c9f
Default pipeline runtime environment values are configured in the pipeline folder config.py
. You will set their values directly below:
PIPELINE_NAME
- the pipeline’s globally unique name. For each pipeline update, each pipeline version uploaded to KFP will be reflected on the Pipelines
tab in the Pipeline name > Version name
dropdown in the format PIPELINE_NAME_datetime.now()
.
MODEL_NAME
- the pipeline’s unique model output name for AI Platform Prediction. For multiple pipeline runs, each pushed blessed model will create a new version with the format 'v{}'.format(int(time.time()))
.
DATA_ROOT_URI
- the URI for the raw lab dataset gs://cloud-training/OCBL203/workshop-datasets
.
CUSTOM_TFX_IMAGE
- the image name of your pipeline container build by skaffold and published by Cloud Build
to Cloud Container Registry
in the format 'gcr.io/{}/{}'.format(PROJECT_ID, PIPELINE_NAME)
.
RUNTIME_VERSION
- the TensorFlow runtime version. This lab was built and tested using TensorFlow 2.3
.
PYTHON_VERSION
- the Python runtime version. This lab was built and tested using Python 3.7
.
USE_KFP_SA
- The pipeline can run using a security context of the GKE default node pool’s service account or the service account defined in the user-gcp-sa
secret of the Kubernetes namespace hosting Kubeflow Pipelines. If you want to use the user-gcp-sa
service account you change the value of USE_KFP_SA
to True
. Note that the default AI Platform Pipelines configuration does not define the user-gcp-sa
secret.
ENABLE_TUNING
- boolean value indicating whether to add the Tuner
component to the pipeline or use hyperparameter defaults. See the model.py
and pipeline.py
files for details on how this changes the pipeline topology across pipeline versions. You will create pipeline versions without and with tuning enabled in the subsequent lab exercises for comparison.
PIPELINE_NAME = 'tfx_covertype_continuous_training'
MODEL_NAME = 'tfx_covertype_classifier'
DATA_ROOT_URI = 'gs://cloud-training/OCBL203/workshop-datasets'
CUSTOM_TFX_IMAGE = 'gcr.io/{}/{}'.format(PROJECT_ID, PIPELINE_NAME)
RUNTIME_VERSION = '2.3'
PYTHON_VERSION = '3.7'
USE_KFP_SA=False
ENABLE_TUNING=False
%env PIPELINE_NAME={PIPELINE_NAME}
%env MODEL_NAME={MODEL_NAME}
%env DATA_ROOT_URI={DATA_ROOT_URI}
%env KUBEFLOW_TFX_IMAGE={CUSTOM_TFX_IMAGE}
%env RUNTIME_VERSION={RUNTIME_VERSION}
%env PYTHON_VERIONS={PYTHON_VERSION}
%env USE_KFP_SA={USE_KFP_SA}
%env ENABLE_TUNING={ENABLE_TUNING}
env: PIPELINE_NAME=tfx_covertype_continuous_training
env: MODEL_NAME=tfx_covertype_classifier
env: DATA_ROOT_URI=gs://cloud-training/OCBL203/workshop-datasets
env: KUBEFLOW_TFX_IMAGE=gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training
env: RUNTIME_VERSION=2.3
env: PYTHON_VERIONS=3.7
env: USE_KFP_SA=False
env: ENABLE_TUNING=False
You can build and upload the pipeline to the AI Platform Pipelines instance in one step, using the tfx pipeline create
command. The tfx pipeline create
goes through the following steps:
ENDPOINT
to the hosted AI Platform instance.As you debug the pipeline DSL, you may prefer to first use the tfx pipeline compile
command, which only executes the compilation step. After the DSL compiles successfully you can use tfx pipeline create
to go through all steps.
!tfx pipeline compile --engine kubeflow --pipeline_path runner.py
CLI
Compiling pipeline
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:`instance_name` is deprecated, please set node id directly using`with_id()` or `.id` setter.
WARNING:absl:`instance_name` is deprecated, please set node id directly using`with_id()` or `.id` setter.
[0mPipeline compiled successfully.
Pipeline package path: /home/jupyter/mlops-on-gcp/workshops/tfx-caip-tf23/lab-02-tfx-pipeline/labs/pipeline/tfx_covertype_continuous_training.tar.gz
Note: you should see a {PIPELINE_NAME}.tar.gz
file appear in your current pipeline directory.
After the pipeline code compiles without any errors you can use the tfx pipeline create
command to perform the full build and deploy the pipeline. You will deploy your compiled pipeline container hosted on Google Container Registry e.g. gcr.io/[PROJECT_ID]/tfx_covertype_continuous_training
to run on AI Platform Pipelines with the TFX CLI.
# TODO: Your code here to use the TFX CLI to deploy your pipeline image to AI Platform Pipelines.
!tfx pipeline create \
--pipeline_path=runner.py \
--endpoint={ENDPOINT} \
--build_target_image={CUSTOM_TFX_IMAGE}
CLI
Creating pipeline
Detected Kubeflow.
Use --engine flag if you intend to use a different orchestrator.
Reading build spec from build.yaml
Target image gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training is not used. If the build spec is provided, update the target image in the build spec file build.yaml.
[Skaffold] Generating tags...
[Skaffold] - gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training -> gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training:latest
[Skaffold] Checking cache...
[Skaffold] - gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training: Not found. Building
[Skaffold] Starting build...
[Skaffold] Building [gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training]...
[Skaffold] Sending build context to Docker daemon 59.39kB
[Skaffold] Step 1/4 : FROM tensorflow/tfx:0.25.0
[Skaffold] 0.25.0: Pulling from tensorflow/tfx
[Skaffold] bd47987755ba: Pulling fs layer
[Skaffold] 831c222b21d8: Pulling fs layer
[Skaffold] 3c2cba919283: Pulling fs layer
[Skaffold] e378d88a5f59: Pulling fs layer
[Skaffold] df37508d2f5c: Pulling fs layer
[Skaffold] c28e7cc900d1: Pulling fs layer
[Skaffold] 9019978541a7: Pulling fs layer
[Skaffold] 80dc388c898c: Pulling fs layer
[Skaffold] afebcf787e04: Pulling fs layer
[Skaffold] b32cc9704312: Pulling fs layer
[Skaffold] a0336ba74309: Pulling fs layer
[Skaffold] e378d88a5f59: Waiting
[Skaffold] df37508d2f5c: Waiting
[Skaffold] c28e7cc900d1: Waiting
[Skaffold] 9019978541a7: Waiting
[Skaffold] 80dc388c898c: Waiting
[Skaffold] afebcf787e04: Waiting
[Skaffold] b32cc9704312: Waiting
[Skaffold] a0336ba74309: Waiting
[Skaffold] 3c2cba919283: Verifying Checksum
[Skaffold] 3c2cba919283: Download complete
[Skaffold] 831c222b21d8: Verifying Checksum
[Skaffold] 831c222b21d8: Download complete
[Skaffold] bd47987755ba: Verifying Checksum
[Skaffold] bd47987755ba: Download complete
[Skaffold] e378d88a5f59: Verifying Checksum
[Skaffold] e378d88a5f59: Download complete
[Skaffold] c28e7cc900d1: Verifying Checksum
[Skaffold] c28e7cc900d1: Download complete
[Skaffold] 9019978541a7: Verifying Checksum
[Skaffold] 9019978541a7: Download complete
[Skaffold] df37508d2f5c: Verifying Checksum
[Skaffold] df37508d2f5c: Download complete
[Skaffold] afebcf787e04: Verifying Checksum
[Skaffold] afebcf787e04: Download complete
[Skaffold] a0336ba74309: Verifying Checksum
[Skaffold] a0336ba74309: Download complete
[Skaffold] b32cc9704312: Verifying Checksum
[Skaffold] b32cc9704312: Download complete
[Skaffold] 80dc388c898c: Verifying Checksum
[Skaffold] 80dc388c898c: Download complete
[Skaffold] bd47987755ba: Pull complete
[Skaffold] 831c222b21d8: Pull complete
[Skaffold] 3c2cba919283: Pull complete
[Skaffold] e378d88a5f59: Pull complete
[Skaffold] df37508d2f5c: Pull complete
[Skaffold] c28e7cc900d1: Pull complete
[Skaffold] 9019978541a7: Pull complete
[Skaffold] 80dc388c898c: Pull complete
[Skaffold] afebcf787e04: Pull complete
[Skaffold] b32cc9704312: Pull complete
[Skaffold] a0336ba74309: Pull complete
[Skaffold] Digest: sha256:0700c27c6492b8b2998e7d543ca13088db8d40ef26bd5c6eec58245ff8cdec35
[Skaffold] Status: Downloaded newer image for tensorflow/tfx:0.25.0
[Skaffold] ---> 05d9b228cf63
[Skaffold] Step 2/4 : WORKDIR ./pipeline
[Skaffold] ---> Running in 2ce9a0a42c04
[Skaffold] Removing intermediate container 2ce9a0a42c04
[Skaffold] ---> 0190ef3a29bb
[Skaffold] Step 3/4 : COPY ./ ./
[Skaffold] ---> 7d747996a857
[Skaffold] Step 4/4 : ENV PYTHONPATH="/pipeline:${PYTHONPATH}"
[Skaffold] ---> Running in 7e0217db5f15
[Skaffold] Removing intermediate container 7e0217db5f15
[Skaffold] ---> e21ea7895ffa
[Skaffold] Successfully built e21ea7895ffa
[Skaffold] Successfully tagged gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training:latest
[Skaffold] The push refers to repository [gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training]
[Skaffold] 002637b07e02: Preparing
[Skaffold] 53e03ee10ffb: Preparing
[Skaffold] 5dadc0a09248: Preparing
[Skaffold] 8fb12d3bda49: Preparing
[Skaffold] 2471eac28ba8: Preparing
[Skaffold] 674ba689ae71: Preparing
[Skaffold] 4058ae03fa32: Preparing
[Skaffold] e3437c61d457: Preparing
[Skaffold] 84ff92691f90: Preparing
[Skaffold] 54b00d861a7a: Preparing
[Skaffold] c547358928ab: Preparing
[Skaffold] 84ff92691f90: Preparing
[Skaffold] c4e66be694ce: Preparing
[Skaffold] 47cc65c6dd57: Preparing
[Skaffold] 674ba689ae71: Waiting
[Skaffold] 4058ae03fa32: Waiting
[Skaffold] e3437c61d457: Waiting
[Skaffold] 84ff92691f90: Waiting
[Skaffold] 54b00d861a7a: Waiting
[Skaffold] c547358928ab: Waiting
[Skaffold] c4e66be694ce: Waiting
[Skaffold] 47cc65c6dd57: Waiting
[Skaffold] 8fb12d3bda49: Layer already exists
[Skaffold] 5dadc0a09248: Layer already exists
[Skaffold] 2471eac28ba8: Layer already exists
[Skaffold] 4058ae03fa32: Layer already exists
[Skaffold] 674ba689ae71: Layer already exists
[Skaffold] e3437c61d457: Layer already exists
[Skaffold] 84ff92691f90: Layer already exists
[Skaffold] 54b00d861a7a: Layer already exists
[Skaffold] c547358928ab: Layer already exists
[Skaffold] c4e66be694ce: Layer already exists
[Skaffold] 47cc65c6dd57: Layer already exists
[Skaffold] 002637b07e02: Pushed
[Skaffold] 53e03ee10ffb: Pushed
[Skaffold] latest: digest: sha256:f20c76d3eed2090848ebfa41b5cc5cec8920f8431a1ebc6bd3d70bed26f591a9 size: 3267
[Skaffold] Build [gcr.io/qwiklabs-gcp-04-1852ebd59c9f/tfx_covertype_continuous_training] succeeded
[Skaffold]
[Skaffold] Help improve Skaffold with our 2-minute anonymous survey: run 'skaffold survey'
[Skaffold] To help improve the quality of this product, we collect anonymized usage data for details on what is tracked and how we use this data visit <https://skaffold.dev/docs/resources/telemetry/>. This data is handled in accordance with our privacy policy <https://policies.google.com/privacy>
[Skaffold]
[Skaffold] You may choose to opt out of this collection by running the following command:
[Skaffold] skaffold config set --global collect-metrics false
New container image is built. Target image is available in the build spec file.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
WARNING:absl:`instance_name` is deprecated, please set node id directly using`with_id()` or `.id` setter.
WARNING:absl:`instance_name` is deprecated, please set node id directly using`with_id()` or `.id` setter.
[0mPipeline compiled successfully.
Pipeline package path: /home/jupyter/mlops-on-gcp/workshops/tfx-caip-tf23/lab-02-tfx-pipeline/labs/pipeline/tfx_covertype_continuous_training.tar.gz
{'created_at': datetime.datetime(2024, 3, 24, 17, 12, 9, tzinfo=tzlocal()),
'default_version': {'code_source_url': None,
'created_at': datetime.datetime(2024, 3, 24, 17, 12, 9, tzinfo=tzlocal()),
'description': None,
'id': '23557e15-acdb-4aae-bc85-85956c77f397',
'name': 'tfx_covertype_continuous_training',
'package_url': None,
'parameters': [{'name': 'pipeline-root',
'value': 'gs://qwiklabs-gcp-04-1852ebd59c9f-kubeflowpipelines-default/tfx_covertype_continuous_training/'},
{'name': 'data-root-uri',
'value': 'gs://cloud-training/OCBL203/workshop-datasets'},
{'name': 'eval-steps', 'value': '500'},
{'name': 'train-steps', 'value': '5000'}],
'resource_references': [{'key': {'id': 'af667d38-e303-46a6-9e77-59bda43f55af',
'type': 'PIPELINE'},
'name': None,
'relationship': 'OWNER'}]},
'description': None,
'error': None,
'id': 'af667d38-e303-46a6-9e77-59bda43f55af',
'name': 'tfx_covertype_continuous_training',
'parameters': [{'name': 'pipeline-root',
'value': 'gs://qwiklabs-gcp-04-1852ebd59c9f-kubeflowpipelines-default/tfx_covertype_continuous_training/'},
{'name': 'data-root-uri',
'value': 'gs://cloud-training/OCBL203/workshop-datasets'},
{'name': 'eval-steps', 'value': '500'},
{'name': 'train-steps', 'value': '5000'}],
'resource_references': None,
'url': None}
Please access the pipeline detail page at http://1fdd6c89bc6b1864-dot-us-east1.pipelines.googleusercontent.com/#/pipelines/details/af667d38-e303-46a6-9e77-59bda43f55af
Pipeline "tfx_covertype_continuous_training" created successfully.
Hint: review the TFX CLI documentation on the “pipeline group” to create your pipeline. You will need to specify the --pipeline_path
to point at the pipeline DSL and runner defined locally in runner.py
, --endpoint
, and --build_target_image
arguments using the environment variables specified above.
Note: you should see a build.yaml
file in your pipeline folder created by skaffold. The TFX CLI compile triggers a custom container to be built with skaffold using the instructions in the Dockerfile
.
If you need to redeploy the pipeline you can first delete the previous version using tfx pipeline delete
or you can update the pipeline in-place using tfx pipeline update
.
To delete the pipeline:
tfx pipeline delete --pipeline_name {PIPELINE_NAME} --endpoint {ENDPOINT}
To update the pipeline:
tfx pipeline update --pipeline_path runner.py --endpoint {ENDPOINT}
After the pipeline has been deployed, you can trigger and monitor pipeline runs using TFX CLI.
Hint: review the TFX CLI documentation on the “run group”.
# TODO: your code here to trigger a pipeline run with the TFX CLI
!tfx run create --pipeline_name={PIPELINE_NAME} --endpoint={ENDPOINT}
CLI
Creating a run for pipeline: tfx_covertype_continuous_training
Detected Kubeflow.
Use --engine flag if you intend to use a different orchestrator.
Run created for pipeline: tfx_covertype_continuous_training
+-----------------------------------+--------------------------------------+----------+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| pipeline_name | run_id | status | created_at | link |
+===================================+======================================+==========+===========================+==========================================================================================================================+
| tfx_covertype_continuous_training | 209bf885-08fd-49af-9608-5a3f20265cdf | Pending | 2024-03-24T17:12:18+00:00 | http://1fdd6c89bc6b1864-dot-us-east1.pipelines.googleusercontent.com/#/runs/details/209bf885-08fd-49af-9608-5a3f20265cdf |
+-----------------------------------+--------------------------------------+----------+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
To view the status of existing pipeline runs:
!tfx run list --pipeline_name {PIPELINE_NAME} --endpoint {ENDPOINT}
CLI
Listing all runs of pipeline: tfx_covertype_continuous_training
Detected Kubeflow.
Use --engine flag if you intend to use a different orchestrator.
+-----------------------------------+--------------------------------------+----------+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| pipeline_name | run_id | status | created_at | link |
+===================================+======================================+==========+===========================+==========================================================================================================================+
| tfx_covertype_continuous_training | 209bf885-08fd-49af-9608-5a3f20265cdf | Running | 2024-03-24T17:12:18+00:00 | http://1fdd6c89bc6b1864-dot-us-east1.pipelines.googleusercontent.com/#/runs/details/209bf885-08fd-49af-9608-5a3f20265cdf |
+-----------------------------------+--------------------------------------+----------+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
To retrieve the status of a given run:
RUN_ID='[YOUR RUN ID]'
!tfx run status --pipeline_name {PIPELINE_NAME} --run_id {RUN_ID} --endpoint {ENDPOINT}
CLI
Usage: tfx run status [OPTIONS]
Try 'tfx run status --help' for help.
Error: Got unexpected extra arguments (RUN ID])
A full pipeline run without tuning enabled will take about 40 minutes to complete. You can view the run’s progress using the TFX CLI commands above or in the Kubeflow Pipeline Dashboard.
Incorporating automatic model hyperparameter tuning into a continuous training TFX pipeline workflow enables faster experimentation, development, and deployment of a top performing model.
The previous pipeline version read from hyperparameter default values in the search space defined in _get_hyperparameters()
in model.py
and used these values to build a TensorFlow WideDeep Classifier model.
You can choose to deploy a new pipeline version with the Tuner
component added to the pipeline that calls out to the AI Platform Vizier service for distributed and parallelized hyperparameter tuning. The Tuner
component "best_hyperparameters"
artifact will be passed directly to your Trainer
component to deploy the top performing model. Review pipeline.py
to see how this environment variable changes the pipeline topology. Also, review the tuning function in model.py
for configuring CloudTuner
.
Note that you might not want to tune the hyperparameters every time you retrain your model due to the computational cost. Once you have used Tuner
determine a good set of hyperparameters, you can remove Tuner
from your pipeline and use model hyperparameters defined in your model code or use a ImporterNode
to import the Tuner
"best_hyperparameters"
artifact from a previous Tuner
run to your model Trainer
.
A full pipeline run with tuning enabled will take about 50 minutes and can be executed in parallel while the previous pipeline run without tuning continues running.
Take the time to review the pipeline metadata artifacts created in the GCS artifact repository for each component including data splits, your Tensorflow SavedModel, model evaluation results, etc. as the pipeline executes. In the GCP console, you can also view the Dataflow jobs for pipeline data processing as well as the AI Platform Training jobs for model training and tuning.
When your pipelines runs are complete, review your model versions on Cloud AI Platform Prediction and model evaluation metrics. Did your model performance improve with hyperparameter tuning?
In this lab, you learned how to build and deploy a TFX pipeline with the TFX CLI and then update, build and deploy a new pipeline with automatic hyperparameter tuning. You practiced triggered continuous pipeline runs using the TFX CLI as well as the Kubeflow Pipelines UI.
In the next lab, you will construct a Cloud Build CI/CD workflow that further automates the building and deployment of the TensorFlow WideDeep Classifer pipeline code introduced in this lab.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.