In this lab, you will walk through authoring a Cloud Build CI/CD workflow that automatically builds and deploys the same TFX pipeline from lab-02.ipynb. You will also integrate your workflow with GitHub by setting up a trigger that starts the workflow when a new tag is applied to the GitHub repo hosting the pipeline’s code.
import yaml
# Set `PATH` to include the directory containing TFX CLI.
PATH=%env PATH
%env PATH=/home/jupyter/.local/bin:{PATH}
env: PATH=/home/jupyter/.local/bin:/home/jupyter/.local/bin:/opt/conda/bin:/opt/conda/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
!python -c "import tfx; print('TFX version: {}'.format(tfx.__version__))"
TFX version: 0.25.0
Note: this lab was built and tested with the following package versions:
TFX version: 0.25.0
(Optional) If the TFX version above does not match the lab tested defaults, run the command below:
%pip install --upgrade --user tfx==0.25.0
Requirement already satisfied: tfx==0.25.0 in /home/jupyter/.local/lib/python3.7/site-packages (0.25.0)
Requirement already satisfied: tensorflow-data-validation<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: pyyaml<6,>=3.12 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (5.4.1)
Requirement already satisfied: tensorflow-serving-api!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.3.0)
Requirement already satisfied: six<2,>=1.10 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (1.16.0)
Requirement already satisfied: tfx-bsl<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: absl-py<0.11,>=0.9 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (0.10.0)
Requirement already satisfied: click<8,>=7 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (7.1.2)
Requirement already satisfied: keras-tuner<2,>=1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (1.0.1)
Requirement already satisfied: grpcio<2,>=1.28.1 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (1.62.1)
Requirement already satisfied: apache-beam[gcp]<3,>=2.25 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.28.0)
Requirement already satisfied: docker<5,>=4.1 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (4.4.4)
Requirement already satisfied: tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.3.4)
Requirement already satisfied: tensorflow-cloud<0.2,>=0.1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (0.1.13)
Requirement already satisfied: google-api-python-client<2,>=1.7.8 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (1.12.11)
Requirement already satisfied: tensorflow-hub<0.10,>=0.9.0 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (0.9.0)
Requirement already satisfied: ml-metadata<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.1)
Requirement already satisfied: pyarrow<0.18,>=0.17 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.17.1)
Requirement already satisfied: kubernetes<12,>=10.0.1 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (11.0.0)
Requirement already satisfied: tensorflow-transform<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: attrs<21,>=19.3.0 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (20.3.0)
Requirement already satisfied: jinja2<3,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0) (2.11.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (3.20.3)
Requirement already satisfied: tensorflow-model-analysis<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tfx==0.25.0) (0.25.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.4.7)
Requirement already satisfied: avro-python3!=1.9.2,<1.10.0,>=1.8.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.9.2.1)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.12.2)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.6.0)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.3.1.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (4.1.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.0.0)
Requirement already satisfied: httplib2<0.18.0,>=0.8 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.17.4)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.26.0)
Requirement already satisfied: numpy<1.20.0,>=1.14.3 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.18.5)
Requirement already satisfied: pydot<2,>=1.2.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.4.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7)
Requirement already satisfied: future<1.0.0,>=0.18.2 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.18.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2021.3)
Requirement already satisfied: typing-extensions<3.8.0,>=3.7.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.7.4.3)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.5.31)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7.0)
Requirement already satisfied: cachetools<5,>=3.1.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (4.2.4)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.2.2)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.16.3)
Requirement already satisfied: google-cloud-build<3,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.0.0)
Requirement already satisfied: google-cloud-bigquery<2,>=1.6.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.28.3)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7.3)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.19.3)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.7.3)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.3.2)
Requirement already satisfied: google-cloud-dlp<2,>=0.12.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.0.0)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.0.2)
Requirement already satisfied: google-cloud-datastore<2,>=1.7.1 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.15.5)
Requirement already satisfied: google-auth<2,>=1.18.0 in /home/jupyter/.local/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.35.0)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker<5,>=4.1->tfx==0.25.0) (1.2.1)
Requirement already satisfied: google-api-core<3dev,>=1.21.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0) (1.34.1)
Requirement already satisfied: google-auth-httplib2>=0.0.3 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0) (0.1.0)
Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0) (3.0.1)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from jinja2<3,>=2.7.3->tfx==0.25.0) (1.1.1)
Requirement already satisfied: terminaltables in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (3.1.0)
Requirement already satisfied: colorama in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (0.4.4)
Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (1.0.1)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (0.8.9)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (4.62.3)
Requirement already satisfied: scipy in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0) (1.7.3)
Requirement already satisfied: urllib3>=1.24.2 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (1.26.7)
Requirement already satisfied: requests-oauthlib in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (1.3.0)
Requirement already satisfied: setuptools>=21.0.0 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (59.4.0)
Requirement already satisfied: certifi>=14.05.14 in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0) (2021.10.8)
Requirement already satisfied: gast==0.3.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.3.3)
Requirement already satisfied: wrapt>=1.11.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.13.3)
Requirement already satisfied: termcolor>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.1.0)
Requirement already satisfied: keras-preprocessing<1.2,>=1.1.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.1.2)
Requirement already satisfied: tensorflow-estimator<2.4.0,>=2.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.3.0)
Requirement already satisfied: astunparse==1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.6.3)
Requirement already satisfied: opt-einsum>=2.3.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (3.3.0)
Requirement already satisfied: google-pasta>=0.1.8 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.2.0)
Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.10.0)
Requirement already satisfied: tensorboard<3,>=2.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.3.0)
Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.37.0)
Requirement already satisfied: tensorflow-datasets<3.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow-cloud<0.2,>=0.1->tfx==0.25.0) (3.0.0)
Requirement already satisfied: google-cloud-storage in /opt/conda/lib/python3.7/site-packages (from tensorflow-cloud<0.2,>=0.1->tfx==0.25.0) (1.43.0)
Requirement already satisfied: pandas<2,>=1.0 in /home/jupyter/.local/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0) (1.3.5)
Requirement already satisfied: tensorflow-metadata<0.26,>=0.25 in /home/jupyter/.local/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0) (0.25.0)
Requirement already satisfied: joblib<0.15,>=0.12 in /home/jupyter/.local/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0) (0.14.1)
Requirement already satisfied: ipywidgets<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (7.6.5)
Requirement already satisfied: ipython<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (7.30.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /home/jupyter/.local/lib/python3.7/site-packages (from google-api-core<3dev,>=1.21.0->google-api-python-client<2,>=1.7.8->tfx==0.25.0) (1.63.0)
Requirement already satisfied: fasteners>=0.14 in /opt/conda/lib/python3.7/site-packages (from google-apitools<0.5.32,>=0.5.31->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.18.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.2.7)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.18.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (4.8)
Requirement already satisfied: google-resumable-media<2.0dev,>=0.6.0 in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.3.3)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigtable<2,>=0.31.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.12.3)
Requirement already satisfied: libcst>=0.2.5 in /opt/conda/lib/python3.7/site-packages (from google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.3.23)
Requirement already satisfied: proto-plus>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.19.8)
Requirement already satisfied: docopt in /opt/conda/lib/python3.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.6.2)
Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.18.1)
Requirement already satisfied: pygments in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (2.10.0)
Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.1.3)
Requirement already satisfied: backcall in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.2.0)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (3.0.22)
Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.8.0)
Requirement already satisfied: decorator in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.1.0)
Requirement already satisfied: pickleshare in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.7.5)
Requirement already satisfied: traitlets>=4.2 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.1.1)
Requirement already satisfied: nbformat>=4.2.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.1.3)
Requirement already satisfied: ipykernel>=4.5.1 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.5.1)
Requirement already satisfied: ipython-genutils~=0.2.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.2.0)
Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.0.2)
Requirement already satisfied: widgetsnbextension~=3.5.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (3.5.2)
Requirement already satisfied: pbr>=0.11 in /opt/conda/lib/python3.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in /opt/conda/lib/python3.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.4.8)
Requirement already satisfied: pyparsing>=2.1.4 in /opt/conda/lib/python3.7/site-packages (from pydot<2,>=1.2.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.0.6)
Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.0.8)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (3.1)
Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (2.0.2)
Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (3.3.6)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (1.8.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0) (0.4.6)
Requirement already satisfied: promise in /opt/conda/lib/python3.7/site-packages (from tensorflow-datasets<3.1.0->tensorflow-cloud<0.2,>=0.1->tfx==0.25.0) (2.3)
Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib->kubernetes<12,>=10.0.1->tfx==0.25.0) (3.1.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->keras-tuner<2,>=1->tfx==0.25.0) (3.0.0)
Requirement already satisfied: grpcio-status<2.0dev,>=1.33.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core<3dev,>=1.21.0->google-api-python-client<2,>=1.7.8->tfx==0.25.0) (1.42.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.7/site-packages (from google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.1.2)
Requirement already satisfied: tornado<7.0,>=4.2 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.1)
Requirement already satisfied: argcomplete>=1.12.3 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.12.3)
Requirement already satisfied: debugpy<2.0,>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.5.1)
Requirement already satisfied: jupyter-client<8.0 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (7.1.0)
Requirement already satisfied: importlib-metadata<5 in /opt/conda/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.8.2)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /opt/conda/lib/python3.7/site-packages (from jedi>=0.16->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.8.3)
Requirement already satisfied: typing-inspect>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from libcst>=0.2.5->google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.7.1)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /opt/conda/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.2.1)
Requirement already satisfied: jupyter-core in /opt/conda/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.9.1)
Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.7/site-packages (from pexpect>4.3->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.7.0)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.2.5)
Requirement already satisfied: notebook>=4.4.1 in /opt/conda/lib/python3.7/site-packages (from widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.4.6)
Requirement already satisfied: cffi>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from google-crc32c<2.0dev,>=1.0->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (1.15.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata<5->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (3.6.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.18.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (5.4.0)
Requirement already satisfied: entrypoints in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.3)
Requirement already satisfied: pyzmq>=13 in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (22.3.0)
Requirement already satisfied: nest-asyncio>=1.5 in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.5.4)
Requirement already satisfied: Send2Trash>=1.8.0 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.8.0)
Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.12.0)
Requirement already satisfied: terminado>=0.8.3 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.12.1)
Requirement already satisfied: nbconvert in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (6.3.0)
Requirement already satisfied: argon2-cffi in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (21.1.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (0.4.3)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.7/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery<2,>=1.6.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0) (2.21)
Requirement already satisfied: mistune<2,>=0.8.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.8.4)
Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.5.9)
Requirement already satisfied: bleach in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (4.1.0)
Requirement already satisfied: defusedxml in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.7.1)
Requirement already satisfied: pandocfilters>=1.4.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (1.5.0)
Requirement already satisfied: testpath in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.5.0)
Requirement already satisfied: jupyterlab-pygments in /opt/conda/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.1.2)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (21.3)
Requirement already satisfied: webencodings in /opt/conda/lib/python3.7/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0) (0.5.1)
Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to pick up the correct package versions.
Review the cloudbuild.yaml file to understand how the CI/CD workflow is implemented and how environment specific settings are abstracted using Cloud Build variables.
The Cloud Build CI/CD workflow automates the steps you walked through manually during lab-02:
The Cloud Build workflow configuration uses both standard and custom Cloud Build builders. The custom builder encapsulates TFX CLI.
Navigate to AI Platform Pipelines page in the Google Cloud Console.
Make sure to select "Allow access to the following Cloud APIs https://www.googleapis.com/auth/cloud-platform" to allow for programmatic access to your pipeline by the Kubeflow SDK for the rest of the lab. Also, provide an App instance name such as “tfx” or “mlops”. Note you may have already deployed an AI Pipelines instance during the Setup for the lab series. If so, you can proceed using that instance below in the next step.
Validate the deployment of your AI Platform Pipelines instance in the console before proceeding.
Update the below constants with the settings reflecting your lab environment.
GCP_REGION - the compute region for AI Platform Training and PredictionARTIFACT_STORE - the GCS bucket created during installation of AI Platform Pipelines. The bucket name starts with the kubeflowpipelines- prefix.# Use the following command to identify the GCS bucket for metadata and pipeline storage.
!gsutil ls
gs://qwiklabs-gcp-02-4a8af9550a2a-kubeflowpipelines-default/
CUSTOM_SERVICE_ACCOUNT - In the gcp console Click on the Navigation Menu and navigate to IAM & Admin, then to Service Accounts and use the service account starting with prefix - ‘tfx-tuner-caip-service-account’. This enables CloudTuner and the Google Cloud AI Platform extensions Tuner component to work together and allows for distributed and parallel tuning backed by AI Platform Vizier’s hyperparameter search algorithm. Please see the lab setup README for setup instructions.ENDPOINT - set the ENDPOINT constant to the endpoint to your AI Platform Pipelines instance. The endpoint to the AI Platform Pipelines instance can be found on the AI Platform Pipelines page in the Google Cloud Console. Open the SETTINGS for your instance and use the value of the host variable in the Connect to this Kubeflow Pipelines instance from a Python client via Kubeflow Pipelines SKD section of the SETTINGS window. The format is '....[region].pipelines.googleusercontent.com'.#TODO: Set your environment resource settings here for GCP_REGION, ARTIFACT_STORE_URI, ENDPOINT, and CUSTOM_SERVICE_ACCOUNT.
GCP_REGION = 'us-central1'
ARTIFACT_STORE_URI = 'gs://qwiklabs-gcp-02-4a8af9550a2a-kubeflowpipelines-default'
CUSTOM_SERVICE_ACCOUNT = 'tfx-tuner-caip-service-account@qwiklabs-gcp-02-4a8af9550a2a.iam.gserviceaccount.com'
ENDPOINT = '36d154e9fbbae949-dot-us-central1.pipelines.googleusercontent.com'
PROJECT_ID = !(gcloud config get-value core/project)
PROJECT_ID = PROJECT_ID[0]
!cat tfx-cli/Dockerfile
FROM gcr.io/deeplearning-platform-release/tf2-cpu.2-3
COPY requirements.txt .
RUN python3 -m pip install -U -r requirements.txt
ENTRYPOINT ["tfx"]
!cat tfx-cli/requirements.txt
pandas>1.0.0
tfx==0.25.0
kfp==1.0.4
requests-toolbelt==0.10.1
Hint: Review the Cloud Build gcloud command line reference for builds submit. Your image should follow the format gcr.io/[PROJECT_ID]/[IMAGE_NAME]:latest. Note the source code for the tfx-cli is in the directory ./tfx-cli.
IMAGE_NAME='tfx-cli'
TAG='latest'
IMAGE_URI='gcr.io/{}/{}:{}'.format(PROJECT_ID, IMAGE_NAME, TAG)
# TODO: Your gcloud command here to build tfx-cli and submit to Container Registry.
!gcloud builds submit --timeout=15m --tag {IMAGE_URI} tfx-cli
Creating temporary tarball archive of 2 file(s) totalling 210 bytes before compression.
Uploading tarball of [tfx-cli] to [gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302632.785731-8a10013a829e483a94fc130d320b3d0e.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/qwiklabs-gcp-02-4a8af9550a2a/locations/global/builds/75100203-9a68-4337-98a8-6ae34f10df7c].
Logs are available at [https://console.cloud.google.com/cloud-build/builds/75100203-9a68-4337-98a8-6ae34f10df7c?project=1092329185547].
----------------------------- REMOTE BUILD OUTPUT ------------------------------
starting build "75100203-9a68-4337-98a8-6ae34f10df7c"
FETCHSOURCE
Fetching storage object: gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302632.785731-8a10013a829e483a94fc130d320b3d0e.tgz#1711302634123502
Copying gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302632.785731-8a10013a829e483a94fc130d320b3d0e.tgz#1711302634123502...
/ [1 files][ 316.0 B/ 316.0 B]
Operation completed over 1 objects/316.0 B.
BUILD
Already have image (with digest): gcr.io/cloud-builders/docker
Sending build context to Docker daemon 3.072kB
Step 1/4 : FROM gcr.io/deeplearning-platform-release/tf2-cpu.2-3
latest: Pulling from deeplearning-platform-release/tf2-cpu.2-3
edaedc954fb5: Already exists
30ec18464f16: Pulling fs layer
b1210a962261: Pulling fs layer
4f4fb700ef54: Pulling fs layer
b372f52c3359: Pulling fs layer
46820f8f141d: Pulling fs layer
7e106c42709e: Pulling fs layer
06336d59a562: Pulling fs layer
41427056d78b: Pulling fs layer
e48eb86068f2: Pulling fs layer
56e204eacca1: Pulling fs layer
1c8205fce1bf: Pulling fs layer
9f79d1d1728e: Pulling fs layer
dca84d2f2059: Pulling fs layer
6ce62f5256c2: Pulling fs layer
3642ea3e80d4: Pulling fs layer
e16a032d433f: Pulling fs layer
2478c9cd2ebf: Pulling fs layer
f629568b334f: Pulling fs layer
44cb10575b68: Pulling fs layer
ed4ddcc30bc5: Pulling fs layer
571fcf7c9d32: Pulling fs layer
8d49c0dfcba1: Pulling fs layer
ec88b0069f5b: Pulling fs layer
ba047d14b151: Pulling fs layer
46820f8f141d: Waiting
7e106c42709e: Waiting
06336d59a562: Waiting
41427056d78b: Waiting
b372f52c3359: Waiting
e48eb86068f2: Waiting
56e204eacca1: Waiting
1c8205fce1bf: Waiting
9f79d1d1728e: Waiting
dca84d2f2059: Waiting
6ce62f5256c2: Waiting
3642ea3e80d4: Waiting
e16a032d433f: Waiting
2478c9cd2ebf: Waiting
f629568b334f: Waiting
44cb10575b68: Waiting
ed4ddcc30bc5: Waiting
571fcf7c9d32: Waiting
8d49c0dfcba1: Waiting
ec88b0069f5b: Waiting
ba047d14b151: Waiting
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
b1210a962261: Verifying Checksum
b1210a962261: Download complete
30ec18464f16: Verifying Checksum
30ec18464f16: Download complete
7e106c42709e: Verifying Checksum
7e106c42709e: Download complete
06336d59a562: Verifying Checksum
06336d59a562: Download complete
41427056d78b: Verifying Checksum
41427056d78b: Download complete
e48eb86068f2: Verifying Checksum
e48eb86068f2: Download complete
46820f8f141d: Verifying Checksum
46820f8f141d: Download complete
1c8205fce1bf: Verifying Checksum
1c8205fce1bf: Download complete
9f79d1d1728e: Verifying Checksum
9f79d1d1728e: Download complete
dca84d2f2059: Verifying Checksum
dca84d2f2059: Download complete
6ce62f5256c2: Verifying Checksum
6ce62f5256c2: Download complete
b372f52c3359: Verifying Checksum
b372f52c3359: Download complete
30ec18464f16: Pull complete
3642ea3e80d4: Verifying Checksum
3642ea3e80d4: Download complete
e16a032d433f: Verifying Checksum
e16a032d433f: Download complete
b1210a962261: Pull complete
2478c9cd2ebf: Verifying Checksum
2478c9cd2ebf: Download complete
f629568b334f: Verifying Checksum
f629568b334f: Download complete
44cb10575b68: Verifying Checksum
44cb10575b68: Download complete
4f4fb700ef54: Pull complete
ed4ddcc30bc5: Verifying Checksum
ed4ddcc30bc5: Download complete
56e204eacca1: Verifying Checksum
56e204eacca1: Download complete
8d49c0dfcba1: Verifying Checksum
8d49c0dfcba1: Download complete
ec88b0069f5b: Verifying Checksum
ec88b0069f5b: Download complete
ba047d14b151: Verifying Checksum
ba047d14b151: Download complete
571fcf7c9d32: Verifying Checksum
571fcf7c9d32: Download complete
b372f52c3359: Pull complete
46820f8f141d: Pull complete
7e106c42709e: Pull complete
06336d59a562: Pull complete
41427056d78b: Pull complete
e48eb86068f2: Pull complete
56e204eacca1: Pull complete
1c8205fce1bf: Pull complete
9f79d1d1728e: Pull complete
dca84d2f2059: Pull complete
6ce62f5256c2: Pull complete
3642ea3e80d4: Pull complete
e16a032d433f: Pull complete
2478c9cd2ebf: Pull complete
f629568b334f: Pull complete
44cb10575b68: Pull complete
ed4ddcc30bc5: Pull complete
571fcf7c9d32: Pull complete
8d49c0dfcba1: Pull complete
ec88b0069f5b: Pull complete
ba047d14b151: Pull complete
Digest: sha256:6a991e0d3d0f88b9905d69ac769783cecb301b7ebf6aec38708e15cff9c67e10
Status: Downloaded newer image for gcr.io/deeplearning-platform-release/tf2-cpu.2-3:latest
---> 1fe9cc789976
Step 2/4 : COPY requirements.txt .
---> 5d4ba14a06ff
Step 3/4 : RUN python3 -m pip install -U -r requirements.txt
---> Running in 454ab777e8d7
Requirement already satisfied: pandas>1.0.0 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (1.3.5)
Collecting tfx==0.25.0 (from -r requirements.txt (line 2))
Obtaining dependency information for tfx==0.25.0 from https://files.pythonhosted.org/packages/5a/eb/6c4b43670d2c23343541a1fedc9fa4def2ebc5e2f0c8153464d6201c53c8/tfx-0.25.0-py3-none-any.whl.metadata
Downloading tfx-0.25.0-py3-none-any.whl.metadata (11 kB)
Collecting kfp==1.0.4 (from -r requirements.txt (line 3))
Downloading kfp-1.0.4.tar.gz (116 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.6/116.6 kB 5.6 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting requests-toolbelt==0.10.1 (from -r requirements.txt (line 4))
Obtaining dependency information for requests-toolbelt==0.10.1 from https://files.pythonhosted.org/packages/05/d3/bf87a36bff1cb88fd30a509fd366c70ec30676517ee791b2f77e0e29817a/requests_toolbelt-0.10.1-py2.py3-none-any.whl.metadata
Downloading requests_toolbelt-0.10.1-py2.py3-none-any.whl.metadata (14 kB)
Requirement already satisfied: absl-py<0.11,>=0.9 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (0.10.0)
Requirement already satisfied: apache-beam[gcp]<3,>=2.25 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (2.28.0)
Requirement already satisfied: attrs<21,>=19.3.0 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (20.3.0)
Requirement already satisfied: click<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (7.1.2)
Requirement already satisfied: docker<5,>=4.1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (4.4.4)
Requirement already satisfied: google-api-python-client<2,>=1.7.8 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (1.8.0)
Requirement already satisfied: grpcio<2,>=1.28.1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (1.48.1)
Requirement already satisfied: jinja2<3,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (2.11.3)
Requirement already satisfied: keras-tuner<2,>=1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: kubernetes<12,>=10.0.1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (11.0.0)
Collecting ml-metadata<0.26,>=0.25 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for ml-metadata<0.26,>=0.25 from https://files.pythonhosted.org/packages/a2/aa/2437cbac9b4f3455065532a108a40934dc3741817485d2a93eb45340c0b8/ml_metadata-0.25.1-cp37-cp37m-manylinux2010_x86_64.whl.metadata
Downloading ml_metadata-0.25.1-cp37-cp37m-manylinux2010_x86_64.whl.metadata (4.9 kB)
Requirement already satisfied: protobuf<4,>=3.12.2 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (3.20.1)
Collecting pyarrow<0.18,>=0.17 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for pyarrow<0.18,>=0.17 from https://files.pythonhosted.org/packages/14/78/dcd7f290cd018581b5c73f6c87e2b004f1161cdf6f55c7b2c87d78174592/pyarrow-0.17.1-cp37-cp37m-manylinux2014_x86_64.whl.metadata
Downloading pyarrow-0.17.1-cp37-cp37m-manylinux2014_x86_64.whl.metadata (2.8 kB)
Requirement already satisfied: pyyaml<6,>=3.12 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (5.4.1)
Requirement already satisfied: six<2,>=1.10 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (1.16.0)
Requirement already satisfied: tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (2.3.4)
Requirement already satisfied: tensorflow-hub<0.10,>=0.9.0 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (0.9.0)
Requirement already satisfied: tensorflow-cloud<0.2,>=0.1 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (0.1.16)
Collecting tensorflow-data-validation<0.26,>=0.25 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for tensorflow-data-validation<0.26,>=0.25 from https://files.pythonhosted.org/packages/0a/a4/3ed3ad02916bb19b1c0a747e8f6bf3917205b8f3d4e18b6d61ea6fd4ce7e/tensorflow_data_validation-0.25.0-cp37-cp37m-manylinux2010_x86_64.whl.metadata
Downloading tensorflow_data_validation-0.25.0-cp37-cp37m-manylinux2010_x86_64.whl.metadata (13 kB)
Collecting tensorflow-model-analysis<0.26,>=0.25 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for tensorflow-model-analysis<0.26,>=0.25 from https://files.pythonhosted.org/packages/a2/95/13f2d8db3ba957ba16f51f585385260402c3a6f23c77d34d15a16e2d847f/tensorflow_model_analysis-0.25.0-py3-none-any.whl.metadata
Downloading tensorflow_model_analysis-0.25.0-py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: tensorflow-serving-api!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15 in /opt/conda/lib/python3.7/site-packages (from tfx==0.25.0->-r requirements.txt (line 2)) (2.3.0)
Collecting tensorflow-transform<0.26,>=0.25 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for tensorflow-transform<0.26,>=0.25 from https://files.pythonhosted.org/packages/4e/37/5a8224e6462c1d0b9e88e57191145bcb07d9928d93279f7df727a80d24c7/tensorflow_transform-0.25.0-py3-none-any.whl.metadata
Downloading tensorflow_transform-0.25.0-py3-none-any.whl.metadata (8.5 kB)
Collecting tfx-bsl<0.26,>=0.25 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for tfx-bsl<0.26,>=0.25 from https://files.pythonhosted.org/packages/5e/16/42a834cdfd7f0ff426d90440952548d8b108753824397f6cc17f0b9c3c86/tfx_bsl-0.25.0-cp37-cp37m-manylinux2010_x86_64.whl.metadata
Downloading tfx_bsl-0.25.0-cp37-cp37m-manylinux2010_x86_64.whl.metadata (6.1 kB)
Requirement already satisfied: google-cloud-storage>=1.13.0 in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4->-r requirements.txt (line 3)) (2.1.0)
Requirement already satisfied: google-auth>=1.6.1 in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4->-r requirements.txt (line 3)) (1.35.0)
Requirement already satisfied: cloudpickle in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4->-r requirements.txt (line 3)) (1.3.0)
Collecting kfp-server-api<2.0.0,>=0.2.5 (from kfp==1.0.4->-r requirements.txt (line 3))
Downloading kfp-server-api-1.8.5.tar.gz (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.1/58.1 kB 9.0 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: jsonschema>=3.0.1 in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4->-r requirements.txt (line 3)) (4.17.3)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from kfp==1.0.4->-r requirements.txt (line 3)) (0.9.0)
Collecting Deprecated (from kfp==1.0.4->-r requirements.txt (line 3))
Obtaining dependency information for Deprecated from https://files.pythonhosted.org/packages/20/8d/778b7d51b981a96554f29136cd59ca7880bf58094338085bcf2a979a0e6a/Deprecated-1.2.14-py2.py3-none-any.whl.metadata
Downloading Deprecated-1.2.14-py2.py3-none-any.whl.metadata (5.4 kB)
Collecting strip-hints (from kfp==1.0.4->-r requirements.txt (line 3))
Downloading strip-hints-0.1.10.tar.gz (29 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: requests<3.0.0,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from requests-toolbelt==0.10.1->-r requirements.txt (line 4)) (2.31.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from pandas>1.0.0->-r requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.7/site-packages (from pandas>1.0.0->-r requirements.txt (line 1)) (2023.3.post1)
Requirement already satisfied: numpy>=1.17.3 in /opt/conda/lib/python3.7/site-packages (from pandas>1.0.0->-r requirements.txt (line 1)) (1.21.6)
Requirement already satisfied: avro-python3!=1.9.2,<1.10.0,>=1.8.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.9.2.1)
Requirement already satisfied: crcmod<2.0,>=1.7 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.7)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.8.0)
Requirement already satisfied: future<1.0.0,>=0.18.2 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.18.3)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.7.2)
Requirement already satisfied: httplib2<0.18.0,>=0.8 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.17.4)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.0.0)
Collecting numpy>=1.17.3 (from pandas>1.0.0->-r requirements.txt (line 1))
Obtaining dependency information for numpy>=1.17.3 from https://files.pythonhosted.org/packages/08/d6/a6aaa29fea945bc6c61d11f6e0697b325ff7446de5ffd62c2fa02f627048/numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl.metadata
Downloading numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl.metadata (2.0 kB)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (3.13.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (4.1.3)
Requirement already satisfied: pydot<2,>=1.2.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.4.2)
Requirement already satisfied: typing-extensions<3.8.0,>=3.7.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (3.7.4.3)
Requirement already satisfied: cachetools<5,>=3.1.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.5.31)
Requirement already satisfied: google-cloud-datastore<2,>=1.7.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.15.5)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.7.2)
Requirement already satisfied: google-cloud-bigquery<2,>=1.6.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.28.3)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.7.3)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.7.3)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.19.3)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.2)
Requirement already satisfied: google-cloud-dlp<2,>=0.12.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.0.2)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.3.2)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.16.3)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.0.2)
Requirement already satisfied: google-cloud-build<3,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.0.2)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker<5,>=4.1->tfx==0.25.0->-r requirements.txt (line 2)) (1.6.1)
Requirement already satisfied: google-auth-httplib2>=0.0.3 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0->-r requirements.txt (line 2)) (0.1.0)
Requirement already satisfied: google-api-core<2dev,>=1.13.0 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0->-r requirements.txt (line 2)) (1.34.0)
Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from google-api-python-client<2,>=1.7.8->tfx==0.25.0->-r requirements.txt (line 2)) (3.0.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4->-r requirements.txt (line 3)) (0.3.0)
Requirement already satisfied: setuptools>=40.3.0 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4->-r requirements.txt (line 3)) (68.2.2)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth>=1.6.1->kfp==1.0.4->-r requirements.txt (line 3)) (4.9)
Requirement already satisfied: google-resumable-media>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-storage>=1.13.0->kfp==1.0.4->-r requirements.txt (line 3)) (1.3.3)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from jinja2<3,>=2.7.3->tfx==0.25.0->-r requirements.txt (line 2)) (2.0.1)
Requirement already satisfied: importlib-metadata in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4->-r requirements.txt (line 3)) (4.11.4)
Requirement already satisfied: importlib-resources>=1.4.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4->-r requirements.txt (line 3)) (5.12.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4->-r requirements.txt (line 3)) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=3.0.1->kfp==1.0.4->-r requirements.txt (line 3)) (0.18.1)
Requirement already satisfied: terminaltables in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0->-r requirements.txt (line 2)) (3.1.10)
Requirement already satisfied: colorama in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0->-r requirements.txt (line 2)) (0.4.4)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0->-r requirements.txt (line 2)) (4.63.0)
Requirement already satisfied: scipy in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0->-r requirements.txt (line 2)) (1.7.3)
Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.7/site-packages (from keras-tuner<2,>=1->tfx==0.25.0->-r requirements.txt (line 2)) (1.0.2)
Requirement already satisfied: urllib3>=1.15 in /opt/conda/lib/python3.7/site-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.0.4->-r requirements.txt (line 3)) (1.26.16)
Requirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.0.4->-r requirements.txt (line 3)) (2023.7.22)
Requirement already satisfied: requests-oauthlib in /opt/conda/lib/python3.7/site-packages (from kubernetes<12,>=10.0.1->tfx==0.25.0->-r requirements.txt (line 2)) (1.3.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.0.1->requests-toolbelt==0.10.1->-r requirements.txt (line 4)) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.0.1->requests-toolbelt==0.10.1->-r requirements.txt (line 4)) (3.4)
Requirement already satisfied: astunparse==1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (1.6.3)
Requirement already satisfied: gast==0.3.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (0.3.3)
Requirement already satisfied: google-pasta>=0.1.8 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.0)
Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (2.10.0)
Requirement already satisfied: keras-preprocessing<1.2,>=1.1.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (1.1.2)
Obtaining dependency information for numpy>=1.17.3 from https://files.pythonhosted.org/packages/d6/c6/58e517e8b1fb192725cfa23c01c2e60e4e6699314ee9684a1c5f5c9b27e1/numpy-1.18.5-cp37-cp37m-manylinux1_x86_64.whl.metadata
Downloading numpy-1.18.5-cp37-cp37m-manylinux1_x86_64.whl.metadata (2.1 kB)
Requirement already satisfied: opt-einsum>=2.3.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (3.3.0)
Requirement already satisfied: tensorboard<3,>=2.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (2.3.0)
Requirement already satisfied: tensorflow-estimator<2.4.0,>=2.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (2.3.0)
Requirement already satisfied: termcolor>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (2.3.0)
Requirement already satisfied: wrapt>=1.11.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (1.15.0)
Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.7/site-packages (from tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (0.41.2)
Requirement already satisfied: tensorflow-datasets in /opt/conda/lib/python3.7/site-packages (from tensorflow-cloud<0.2,>=0.1->tfx==0.25.0->-r requirements.txt (line 2)) (3.2.1)
Requirement already satisfied: joblib<0.15,>=0.12 in /opt/conda/lib/python3.7/site-packages (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.14.1)
Collecting tensorflow-metadata<0.26,>=0.25 (from tensorflow-data-validation<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for tensorflow-metadata<0.26,>=0.25 from https://files.pythonhosted.org/packages/57/95/daf3259ed1ca4f262bde2c483356174c86b6a6a39e5dbf814801c52b7b8d/tensorflow_metadata-0.25.0-py3-none-any.whl.metadata
Downloading tensorflow_metadata-0.25.0-py3-none-any.whl.metadata (2.2 kB)
Requirement already satisfied: ipython<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (7.33.0)
Requirement already satisfied: ipywidgets<8,>=7 in /opt/conda/lib/python3.7/site-packages (from tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (7.8.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client<2,>=1.7.8->tfx==0.25.0->-r requirements.txt (line 2)) (1.56.4)
Collecting protobuf<4,>=3.12.2 (from tfx==0.25.0->-r requirements.txt (line 2))
Obtaining dependency information for protobuf<4,>=3.12.2 from https://files.pythonhosted.org/packages/4c/12/62e1d5505c172e1a7f803d83b0b1693f7952c3c271eb2f155703012ae67a/protobuf-3.20.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata
Downloading protobuf-3.20.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (679 bytes)
Requirement already satisfied: fasteners>=0.14 in /opt/conda/lib/python3.7/site-packages (from google-apitools<0.5.32,>=0.5.31->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.19)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigtable<2,>=0.31.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.12.6)
Requirement already satisfied: proto-plus<2.0.0dev,>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.22.3)
Requirement already satisfied: libcst<1.0.0dev,>=0.2.5 in /opt/conda/lib/python3.7/site-packages (from google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.4.10)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.7/site-packages (from google-resumable-media>=1.3.0->google-cloud-storage>=1.13.0->kfp==1.0.4->-r requirements.txt (line 3)) (1.5.0)
Requirement already satisfied: docopt in /opt/conda/lib/python3.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.6.2)
Requirement already satisfied: zipp>=3.1.0 in /opt/conda/lib/python3.7/site-packages (from importlib-resources>=1.4.0->jsonschema>=3.0.1->kfp==1.0.4->-r requirements.txt (line 3)) (3.15.0)
Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.19.0)
Requirement already satisfied: decorator in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (5.1.1)
Requirement already satisfied: pickleshare in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.7.5)
Requirement already satisfied: traitlets>=4.2 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (5.9.0)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (3.0.39)
Requirement already satisfied: pygments in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.16.1)
Requirement already satisfied: backcall in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.0)
Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.1.6)
Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.7/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (4.8.0)
Requirement already satisfied: comm>=0.1.3 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.1.4)
Requirement already satisfied: ipython-genutils~=0.2.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.0)
Requirement already satisfied: widgetsnbextension~=3.6.6 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (3.6.6)
Requirement already satisfied: jupyterlab-widgets<3,>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.1.7)
Requirement already satisfied: pbr>=0.11 in /opt/conda/lib/python3.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (5.11.1)
Requirement already satisfied: pyasn1>=0.1.7 in /opt/conda/lib/python3.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.5.0)
Requirement already satisfied: pyparsing>=2.1.4 in /opt/conda/lib/python3.7/site-packages (from pydot<2,>=1.2.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (3.1.1)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (3.4.4)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (1.8.1)
Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->tensorflow!=2.0.*,!=2.1.*,!=2.2.*,<3,>=1.15.2->tfx==0.25.0->-r requirements.txt (line 2)) (2.1.2)
Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib->kubernetes<12,>=10.0.1->tfx==0.25.0->-r requirements.txt (line 2)) (3.2.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->keras-tuner<2,>=1->tfx==0.25.0->-r requirements.txt (line 2)) (3.1.0)
Requirement already satisfied: promise in /opt/conda/lib/python3.7/site-packages (from tensorflow-datasets->tensorflow-cloud<0.2,>=0.1->tfx==0.25.0->-r requirements.txt (line 2)) (2.3)
Requirement already satisfied: grpcio-status<2.0dev,>=1.33.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client<2,>=1.7.8->tfx==0.25.0->-r requirements.txt (line 2)) (1.48.1)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/conda/lib/python3.7/site-packages (from jedi>=0.16->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.8.3)
Requirement already satisfied: typing-inspect>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from libcst<1.0.0dev,>=0.2.5->google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.9.0)
Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.7/site-packages (from pexpect>4.3->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.7.0)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.6)
Requirement already satisfied: notebook>=4.4.1 in /opt/conda/lib/python3.7/site-packages (from widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (6.5.6)
Requirement already satisfied: tornado>=6.1 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (6.2)
Requirement already satisfied: pyzmq<25,>=17 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (24.0.1)
Requirement already satisfied: argon2-cffi in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (23.1.0)
Requirement already satisfied: jupyter-core>=4.6.1 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (4.11.1)
Requirement already satisfied: jupyter-client<8,>=5.3.4 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (7.4.9)
Requirement already satisfied: nbformat in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (5.8.0)
Requirement already satisfied: nbconvert>=5 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (6.4.5)
Requirement already satisfied: nest-asyncio>=1.5 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.5.6)
Requirement already satisfied: ipykernel in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (6.16.2)
Requirement already satisfied: Send2Trash>=1.8.0 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.8.2)
Requirement already satisfied: terminado>=0.8.3 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.17.1)
Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.17.1)
Requirement already satisfied: nbclassic>=0.4.7 in /opt/conda/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.0.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from typing-inspect>=0.4.0->libcst<1.0.0dev,>=0.2.5->google-cloud-build<3,>=2.0.0->apache-beam[gcp]<3,>=2.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.0.0)
Requirement already satisfied: entrypoints in /opt/conda/lib/python3.7/site-packages (from jupyter-client<8,>=5.3.4->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.4)
Requirement already satisfied: jupyter-server>=1.8 in /opt/conda/lib/python3.7/site-packages (from nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: notebook-shim>=0.2.3 in /opt/conda/lib/python3.7/site-packages (from nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.3)
Requirement already satisfied: mistune<2,>=0.8.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.8.4)
Requirement already satisfied: jupyterlab-pygments in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.2.2)
Requirement already satisfied: bleach in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (6.0.0)
Requirement already satisfied: pandocfilters>=1.4.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.5.0)
Requirement already satisfied: testpath in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.6.0)
Requirement already satisfied: defusedxml in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.7.1)
Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (4.12.2)
Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in /opt/conda/lib/python3.7/site-packages (from nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.5.13)
Requirement already satisfied: fastjsonschema in /opt/conda/lib/python3.7/site-packages (from nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.18.0)
Requirement already satisfied: argon2-cffi-bindings in /opt/conda/lib/python3.7/site-packages (from argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (21.2.0)
Requirement already satisfied: debugpy>=1.0 in /opt/conda/lib/python3.7/site-packages (from ipykernel->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.6.3)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from ipykernel->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (23.1)
Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from ipykernel->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (5.9.3)
Requirement already satisfied: anyio<4,>=3.1.0 in /opt/conda/lib/python3.7/site-packages (from jupyter-server>=1.8->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (3.7.1)
Requirement already satisfied: cffi>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.15.1)
Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.3.2.post1)
Requirement already satisfied: webencodings in /opt/conda/lib/python3.7/site-packages (from bleach->nbconvert>=5->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (0.5.1)
Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.7/site-packages (from anyio<4,>=3.1.0->jupyter-server>=1.8->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.3.0)
Requirement already satisfied: exceptiongroup in /opt/conda/lib/python3.7/site-packages (from anyio<4,>=3.1.0->jupyter-server>=1.8->nbclassic>=0.4.7->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (1.1.3)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.7/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.6->ipywidgets<8,>=7->tensorflow-model-analysis<0.26,>=0.25->tfx==0.25.0->-r requirements.txt (line 2)) (2.21)
Downloading tfx-0.25.0-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 47.0 MB/s eta 0:00:00
Downloading requests_toolbelt-0.10.1-py2.py3-none-any.whl (54 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.5/54.5 kB 7.8 MB/s eta 0:00:00
Downloading ml_metadata-0.25.1-cp37-cp37m-manylinux2010_x86_64.whl (2.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 65.0 MB/s eta 0:00:00
Downloading pyarrow-0.17.1-cp37-cp37m-manylinux2014_x86_64.whl (63.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.8/63.8 MB 20.5 MB/s eta 0:00:00
Downloading numpy-1.18.5-cp37-cp37m-manylinux1_x86_64.whl (20.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.1/20.1 MB 60.8 MB/s eta 0:00:00
Downloading tensorflow_data_validation-0.25.0-cp37-cp37m-manylinux2010_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 59.2 MB/s eta 0:00:00
Downloading tensorflow_model_analysis-0.25.0-py3-none-any.whl (1.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 66.0 MB/s eta 0:00:00
Downloading tensorflow_transform-0.25.0-py3-none-any.whl (378 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 378.9/378.9 kB 35.2 MB/s eta 0:00:00
Downloading tfx_bsl-0.25.0-cp37-cp37m-manylinux2010_x86_64.whl (2.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 65.2 MB/s eta 0:00:00
Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
Downloading protobuf-3.20.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 56.5 MB/s eta 0:00:00
Downloading tensorflow_metadata-0.25.0-py3-none-any.whl (44 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.7/44.7 kB 6.0 MB/s eta 0:00:00
Building wheels for collected packages: kfp, kfp-server-api, strip-hints
Building wheel for kfp (setup.py): started
Building wheel for kfp (setup.py): finished with status 'done'
Created wheel for kfp: filename=kfp-1.0.4-py3-none-any.whl size=159829 sha256=ce65d47097d90fa05294e279cee68610711f106986cc8ca31975eff0b10fb15c
Stored in directory: /root/.cache/pip/wheels/65/1c/be/3d7366d2288bf1587e4fe6cd0c1ebdce5e3bada21b70a29e66
Building wheel for kfp-server-api (setup.py): started
Building wheel for kfp-server-api (setup.py): finished with status 'done'
Created wheel for kfp-server-api: filename=kfp_server_api-1.8.5-py3-none-any.whl size=99696 sha256=87612cd79c64d77d2b41c1463d805c984fe8bc28c417f1f5d8431743b5fabcee
Stored in directory: /root/.cache/pip/wheels/77/0e/7b/ed385d69453b7b754834c01d83fa9f5708ba66b4f6ed5d6a35
Building wheel for strip-hints (setup.py): started
Building wheel for strip-hints (setup.py): finished with status 'done'
Created wheel for strip-hints: filename=strip_hints-0.1.10-py2.py3-none-any.whl size=22283 sha256=56ef98080e53961d6c954c1cd260d79806a46403e1781d16037a56ccdfdcec9f
Stored in directory: /root/.cache/pip/wheels/5e/14/c3/6e44e9b2545f2d570b03f5b6d38c00b7534aa8abb376978363
Successfully built kfp kfp-server-api strip-hints
[91mDEPRECATION: tensorflow-serving-api 2.3.0 has a non-standard dependency specifier grpcio>=1.0<2. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of tensorflow-serving-api or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
[0mInstalling collected packages: strip-hints, protobuf, numpy, Deprecated, requests-toolbelt, pyarrow, ml-metadata, kfp-server-api, tensorflow-metadata, kfp, tfx-bsl, tensorflow-transform, tensorflow-data-validation, tensorflow-model-analysis, tfx
Attempting uninstall: protobuf
Found existing installation: protobuf 3.20.1
Uninstalling protobuf-3.20.1:
Successfully uninstalled protobuf-3.20.1
Attempting uninstall: numpy
Found existing installation: numpy 1.21.6
Uninstalling numpy-1.21.6:
Successfully uninstalled numpy-1.21.6
Attempting uninstall: pyarrow
Found existing installation: pyarrow 12.0.1
Uninstalling pyarrow-12.0.1:
Successfully uninstalled pyarrow-12.0.1
Attempting uninstall: ml-metadata
Found existing installation: ml-metadata 0.26.0
Uninstalling ml-metadata-0.26.0:
Successfully uninstalled ml-metadata-0.26.0
Attempting uninstall: tensorflow-metadata
Found existing installation: tensorflow-metadata 0.26.0
Uninstalling tensorflow-metadata-0.26.0:
Successfully uninstalled tensorflow-metadata-0.26.0
Attempting uninstall: tfx-bsl
Found existing installation: tfx-bsl 0.26.1
Uninstalling tfx-bsl-0.26.1:
Successfully uninstalled tfx-bsl-0.26.1
Attempting uninstall: tensorflow-transform
Found existing installation: tensorflow-transform 0.26.0
Uninstalling tensorflow-transform-0.26.0:
Successfully uninstalled tensorflow-transform-0.26.0
Attempting uninstall: tensorflow-data-validation
Found existing installation: tensorflow-data-validation 0.26.1
Uninstalling tensorflow-data-validation-0.26.1:
Successfully uninstalled tensorflow-data-validation-0.26.1
Attempting uninstall: tensorflow-model-analysis
Found existing installation: tensorflow-model-analysis 0.26.0
Uninstalling tensorflow-model-analysis-0.26.0:
Successfully uninstalled tensorflow-model-analysis-0.26.0
Attempting uninstall: tfx
Found existing installation: tfx 0.26.4
Uninstalling tfx-0.26.4:
Successfully uninstalled tfx-0.26.4
[91mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
fairness-indicators 0.26.0 requires tensorflow-data-validation<0.27,>=0.26, but you have tensorflow-data-validation 0.25.0 which is incompatible.
fairness-indicators 0.26.0 requires tensorflow-model-analysis<0.27,>=0.26, but you have tensorflow-model-analysis 0.25.0 which is incompatible.
ml-pipelines-sdk 0.26.4 requires ml-metadata<0.27,>=0.26, but you have ml-metadata 0.25.1 which is incompatible.
[0mSuccessfully installed Deprecated-1.2.14 kfp-1.0.4 kfp-server-api-1.8.5 ml-metadata-0.25.1 numpy-1.18.5 protobuf-3.20.3 pyarrow-0.17.1 requests-toolbelt-0.10.1 strip-hints-0.1.10 tensorflow-data-validation-0.25.0 tensorflow-metadata-0.25.0 tensorflow-model-analysis-0.25.0 tensorflow-transform-0.25.0 tfx-0.25.0 tfx-bsl-0.25.0
[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[0mRemoving intermediate container 454ab777e8d7
---> ba54d425dfe9
Step 4/4 : ENTRYPOINT ["tfx"]
---> Running in a9ad7efc467e
Removing intermediate container a9ad7efc467e
---> 9fd45affea89
Successfully built 9fd45affea89
Successfully tagged gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli:latest
PUSH
Pushing gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli:latest
The push refers to repository [gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli]
e1049b3e5dd7: Preparing
8e85fb39102a: Preparing
c13327243e9e: Preparing
4b5e7443c6ac: Preparing
e9773b32694d: Preparing
cab5005f4efc: Preparing
f6face19c9dd: Preparing
92affaab67e8: Preparing
8c672b5f8a22: Preparing
73066cf8673e: Preparing
2ca62cf21221: Preparing
b9a424722d3f: Preparing
8a11b9d4bbb9: Preparing
f2e60533bd85: Preparing
f8eee1652f88: Preparing
ddac17ce4d28: Preparing
731f70dfb97d: Preparing
dc8463ac1279: Preparing
5520158f7ef1: Preparing
152607a648ba: Preparing
62bb75d94c35: Preparing
5f70bf18a086: Preparing
1a1ffc16734e: Preparing
8449535b89d9: Preparing
5f70bf18a086: Preparing
9a2ad63887b6: Preparing
bca2d3c0b431: Preparing
954c82bdeb5f: Preparing
cab5005f4efc: Waiting
f6face19c9dd: Waiting
92affaab67e8: Waiting
8c672b5f8a22: Waiting
73066cf8673e: Waiting
2ca62cf21221: Waiting
b9a424722d3f: Waiting
8a11b9d4bbb9: Waiting
f2e60533bd85: Waiting
f8eee1652f88: Waiting
ddac17ce4d28: Waiting
731f70dfb97d: Waiting
dc8463ac1279: Waiting
5520158f7ef1: Waiting
152607a648ba: Waiting
62bb75d94c35: Waiting
5f70bf18a086: Waiting
1a1ffc16734e: Waiting
8449535b89d9: Waiting
9a2ad63887b6: Waiting
bca2d3c0b431: Waiting
954c82bdeb5f: Waiting
e9773b32694d: Mounted from deeplearning-platform-release/tf2-cpu.2-3
4b5e7443c6ac: Mounted from deeplearning-platform-release/tf2-cpu.2-3
c13327243e9e: Mounted from deeplearning-platform-release/tf2-cpu.2-3
f6face19c9dd: Layer already exists
92affaab67e8: Layer already exists
cab5005f4efc: Mounted from deeplearning-platform-release/tf2-cpu.2-3
8c672b5f8a22: Layer already exists
8e85fb39102a: Pushed
73066cf8673e: Layer already exists
2ca62cf21221: Layer already exists
b9a424722d3f: Layer already exists
8a11b9d4bbb9: Layer already exists
f2e60533bd85: Layer already exists
f8eee1652f88: Layer already exists
ddac17ce4d28: Layer already exists
731f70dfb97d: Layer already exists
5520158f7ef1: Layer already exists
dc8463ac1279: Layer already exists
152607a648ba: Layer already exists
62bb75d94c35: Layer already exists
5f70bf18a086: Layer already exists
1a1ffc16734e: Layer already exists
9a2ad63887b6: Layer already exists
8449535b89d9: Layer already exists
bca2d3c0b431: Layer already exists
954c82bdeb5f: Layer already exists
e1049b3e5dd7: Pushed
latest: digest: sha256:91ae5c3067724ff5c20759d9c439c0b78f2b3e57329232daf560fc79d7361e6f size: 6183
DONE
--------------------------------------------------------------------------------
ID CREATE_TIME DURATION SOURCE IMAGES STATUS
75100203-9a68-4337-98a8-6ae34f10df7c 2024-03-24T17:50:34+00:00 4M57S gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302632.785731-8a10013a829e483a94fc130d320b3d0e.tgz gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli (+1 more) SUCCESS
You can manually trigger Cloud Build runs using the gcloud builds submit command.
PIPELINE_NAME='tfx_covertype_continuous_training'
MODEL_NAME='tfx_covertype_classifier'
DATA_ROOT_URI='gs://cloud-training/OCBL203/workshop-datasets'
TAG_NAME='test'
TFX_IMAGE_NAME='lab-03-tfx-image'
PIPELINE_FOLDER='pipeline'
PIPELINE_DSL='runner.py'
RUNTIME_VERSION='2.3'
PYTHON_VERSION='3.7'
USE_KFP_SA='False'
ENABLE_TUNING='True'
SUBSTITUTIONS="""
_GCP_REGION={},\
_ARTIFACT_STORE_URI={},\
_CUSTOM_SERVICE_ACCOUNT={},\
_ENDPOINT={},\
_PIPELINE_NAME={},\
_MODEL_NAME={},\
_DATA_ROOT_URI={},\
_TFX_IMAGE_NAME={},\
TAG_NAME={},\
_PIPELINE_FOLDER={},\
_PIPELINE_DSL={},\
_RUNTIME_VERSION={},\
_PYTHON_VERSION={},\
_USE_KFP_SA={},\
_ENABLE_TUNING={},
""".format(GCP_REGION,
ARTIFACT_STORE_URI,
CUSTOM_SERVICE_ACCOUNT,
ENDPOINT,
PIPELINE_NAME,
MODEL_NAME,
DATA_ROOT_URI,
TFX_IMAGE_NAME,
TAG_NAME,
PIPELINE_FOLDER,
PIPELINE_DSL,
RUNTIME_VERSION,
PYTHON_VERSION,
USE_KFP_SA,
ENABLE_TUNING
).strip()
Hint: you can manually trigger Cloud Build runs using the gcloud builds submit command. See the documentation for pass the cloudbuild.yaml file and SUBSTITIONS as arguments.
# TODO: write gcloud builds submit command to trigger manual pipeline run.
!gcloud builds submit . --timeout=15m --config cloudbuild.yaml --substitutions {SUBSTITUTIONS}
Creating temporary tarball archive of 13 file(s) totalling 102.1 KiB before compression.
Uploading tarball of [.] to [gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302950.727818-7d3f79f1ca114aa0b7177b4c59f25254.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/qwiklabs-gcp-02-4a8af9550a2a/locations/global/builds/fa6ca464-457b-4913-abde-8ba14fc29cef].
Logs are available at [https://console.cloud.google.com/cloud-build/builds/fa6ca464-457b-4913-abde-8ba14fc29cef?project=1092329185547].
----------------------------- REMOTE BUILD OUTPUT ------------------------------
starting build "fa6ca464-457b-4913-abde-8ba14fc29cef"
FETCHSOURCE
Fetching storage object: gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302950.727818-7d3f79f1ca114aa0b7177b4c59f25254.tgz#1711302950923393
Copying gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302950.727818-7d3f79f1ca114aa0b7177b4c59f25254.tgz#1711302950923393...
/ [1 files][ 22.4 KiB/ 22.4 KiB]
Operation completed over 1 objects/22.4 KiB.
BUILD
Starting Step #0
Step #0: Already have image (with digest): gcr.io/cloud-builders/docker
Step #0: Sending build context to Docker daemon 41.47kB
Step #0: Step 1/4 : FROM tensorflow/tfx:0.25.0
Step #0: 0.25.0: Pulling from tensorflow/tfx
Step #0: bd47987755ba: Pulling fs layer
Step #0: 831c222b21d8: Pulling fs layer
Step #0: 3c2cba919283: Pulling fs layer
Step #0: e378d88a5f59: Pulling fs layer
Step #0: df37508d2f5c: Pulling fs layer
Step #0: c28e7cc900d1: Pulling fs layer
Step #0: 9019978541a7: Pulling fs layer
Step #0: 80dc388c898c: Pulling fs layer
Step #0: afebcf787e04: Pulling fs layer
Step #0: b32cc9704312: Pulling fs layer
Step #0: a0336ba74309: Pulling fs layer
Step #0: e378d88a5f59: Waiting
Step #0: df37508d2f5c: Waiting
Step #0: c28e7cc900d1: Waiting
Step #0: 9019978541a7: Waiting
Step #0: 80dc388c898c: Waiting
Step #0: afebcf787e04: Waiting
Step #0: b32cc9704312: Waiting
Step #0: a0336ba74309: Waiting
Step #0: 3c2cba919283: Verifying Checksum
Step #0: 3c2cba919283: Download complete
Step #0: 831c222b21d8: Verifying Checksum
Step #0: 831c222b21d8: Download complete
Step #0: bd47987755ba: Verifying Checksum
Step #0: bd47987755ba: Download complete
Step #0: e378d88a5f59: Verifying Checksum
Step #0: e378d88a5f59: Download complete
Step #0: 9019978541a7: Verifying Checksum
Step #0: 9019978541a7: Download complete
Step #0: c28e7cc900d1: Verifying Checksum
Step #0: c28e7cc900d1: Download complete
Step #0: df37508d2f5c: Verifying Checksum
Step #0: df37508d2f5c: Download complete
Step #0: bd47987755ba: Pull complete
Step #0: afebcf787e04: Verifying Checksum
Step #0: afebcf787e04: Download complete
Step #0: a0336ba74309: Verifying Checksum
Step #0: a0336ba74309: Download complete
Step #0: 831c222b21d8: Pull complete
Step #0: 3c2cba919283: Pull complete
Step #0: 80dc388c898c: Verifying Checksum
Step #0: 80dc388c898c: Download complete
Step #0: b32cc9704312: Verifying Checksum
Step #0: b32cc9704312: Download complete
Step #0: e378d88a5f59: Pull complete
Step #0: df37508d2f5c: Pull complete
Step #0: c28e7cc900d1: Pull complete
Step #0: 9019978541a7: Pull complete
Step #0: 80dc388c898c: Pull complete
Step #0: afebcf787e04: Pull complete
Step #0: b32cc9704312: Pull complete
Step #0: a0336ba74309: Pull complete
Step #0: Digest: sha256:0700c27c6492b8b2998e7d543ca13088db8d40ef26bd5c6eec58245ff8cdec35
Step #0: Status: Downloaded newer image for tensorflow/tfx:0.25.0
Step #0: ---> 05d9b228cf63
Step #0: Step 2/4 : WORKDIR ./pipeline
Step #0: ---> Running in 14625eb34afd
Step #0: Removing intermediate container 14625eb34afd
Step #0: ---> 3231742940d7
Step #0: Step 3/4 : COPY ./ ./
Step #0: ---> 17a60d442362
Step #0: Step 4/4 : ENV PYTHONPATH="/pipeline:${PYTHONPATH}"
Step #0: ---> Running in 9a0c0c77cd78
Step #0: Removing intermediate container 9a0c0c77cd78
Step #0: ---> 2747c7690c7a
Step #0: Successfully built 2747c7690c7a
Step #0: Successfully tagged gcr.io/qwiklabs-gcp-02-4a8af9550a2a/lab-03-tfx-image:test
Finished Step #0
Starting Step #1
Step #1: Pulling image: gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli
Step #1: Using default tag: latest
Step #1: latest: Pulling from qwiklabs-gcp-02-4a8af9550a2a/tfx-cli
Step #1: edaedc954fb5: Already exists
Step #1: 30ec18464f16: Pulling fs layer
Step #1: b1210a962261: Pulling fs layer
Step #1: 4f4fb700ef54: Pulling fs layer
Step #1: b372f52c3359: Pulling fs layer
Step #1: 46820f8f141d: Pulling fs layer
Step #1: 7e106c42709e: Pulling fs layer
Step #1: 06336d59a562: Pulling fs layer
Step #1: 41427056d78b: Pulling fs layer
Step #1: e48eb86068f2: Pulling fs layer
Step #1: 56e204eacca1: Pulling fs layer
Step #1: 1c8205fce1bf: Pulling fs layer
Step #1: 9f79d1d1728e: Pulling fs layer
Step #1: dca84d2f2059: Pulling fs layer
Step #1: 6ce62f5256c2: Pulling fs layer
Step #1: 3642ea3e80d4: Pulling fs layer
Step #1: e16a032d433f: Pulling fs layer
Step #1: 2478c9cd2ebf: Pulling fs layer
Step #1: f629568b334f: Pulling fs layer
Step #1: 44cb10575b68: Pulling fs layer
Step #1: ed4ddcc30bc5: Pulling fs layer
Step #1: 571fcf7c9d32: Pulling fs layer
Step #1: 8d49c0dfcba1: Pulling fs layer
Step #1: ec88b0069f5b: Pulling fs layer
Step #1: ba047d14b151: Pulling fs layer
Step #1: 71c92cbd2e5f: Pulling fs layer
Step #1: 32464a57442d: Pulling fs layer
Step #1: b372f52c3359: Waiting
Step #1: 46820f8f141d: Waiting
Step #1: 7e106c42709e: Waiting
Step #1: 06336d59a562: Waiting
Step #1: 41427056d78b: Waiting
Step #1: e48eb86068f2: Waiting
Step #1: 56e204eacca1: Waiting
Step #1: 1c8205fce1bf: Waiting
Step #1: 9f79d1d1728e: Waiting
Step #1: dca84d2f2059: Waiting
Step #1: 6ce62f5256c2: Waiting
Step #1: 3642ea3e80d4: Waiting
Step #1: e16a032d433f: Waiting
Step #1: 2478c9cd2ebf: Waiting
Step #1: f629568b334f: Waiting
Step #1: 44cb10575b68: Waiting
Step #1: ed4ddcc30bc5: Waiting
Step #1: 571fcf7c9d32: Waiting
Step #1: 8d49c0dfcba1: Waiting
Step #1: ec88b0069f5b: Waiting
Step #1: ba047d14b151: Waiting
Step #1: 71c92cbd2e5f: Waiting
Step #1: 32464a57442d: Waiting
Step #1: 4f4fb700ef54: Verifying Checksum
Step #1: 4f4fb700ef54: Download complete
Step #1: b1210a962261: Verifying Checksum
Step #1: b1210a962261: Download complete
Step #1: 30ec18464f16: Verifying Checksum
Step #1: 30ec18464f16: Download complete
Step #1: 7e106c42709e: Verifying Checksum
Step #1: 7e106c42709e: Download complete
Step #1: 06336d59a562: Verifying Checksum
Step #1: 06336d59a562: Download complete
Step #1: 41427056d78b: Verifying Checksum
Step #1: 41427056d78b: Download complete
Step #1: e48eb86068f2: Verifying Checksum
Step #1: e48eb86068f2: Download complete
Step #1: 46820f8f141d: Verifying Checksum
Step #1: 46820f8f141d: Download complete
Step #1: 1c8205fce1bf: Verifying Checksum
Step #1: 1c8205fce1bf: Download complete
Step #1: 9f79d1d1728e: Verifying Checksum
Step #1: 9f79d1d1728e: Download complete
Step #1: dca84d2f2059: Verifying Checksum
Step #1: dca84d2f2059: Download complete
Step #1: 6ce62f5256c2: Verifying Checksum
Step #1: 6ce62f5256c2: Download complete
Step #1: 3642ea3e80d4: Verifying Checksum
Step #1: 3642ea3e80d4: Download complete
Step #1: e16a032d433f: Download complete
Step #1: 2478c9cd2ebf: Verifying Checksum
Step #1: 2478c9cd2ebf: Download complete
Step #1: b372f52c3359: Verifying Checksum
Step #1: b372f52c3359: Download complete
Step #1: 44cb10575b68: Verifying Checksum
Step #1: 44cb10575b68: Download complete
Step #1: 30ec18464f16: Pull complete
Step #1: f629568b334f: Download complete
Step #1: 56e204eacca1: Verifying Checksum
Step #1: 56e204eacca1: Download complete
Step #1: b1210a962261: Pull complete
Step #1: ed4ddcc30bc5: Verifying Checksum
Step #1: ed4ddcc30bc5: Download complete
Step #1: 4f4fb700ef54: Pull complete
Step #1: ec88b0069f5b: Verifying Checksum
Step #1: ec88b0069f5b: Download complete
Step #1: 8d49c0dfcba1: Verifying Checksum
Step #1: 8d49c0dfcba1: Download complete
Step #1: 71c92cbd2e5f: Verifying Checksum
Step #1: 71c92cbd2e5f: Download complete
Step #1: ba047d14b151: Verifying Checksum
Step #1: ba047d14b151: Download complete
Step #1: 32464a57442d: Verifying Checksum
Step #1: 32464a57442d: Download complete
Step #1: 571fcf7c9d32: Verifying Checksum
Step #1: 571fcf7c9d32: Download complete
Step #1: b372f52c3359: Pull complete
Step #1: 46820f8f141d: Pull complete
Step #1: 7e106c42709e: Pull complete
Step #1: 06336d59a562: Pull complete
Step #1: 41427056d78b: Pull complete
Step #1: e48eb86068f2: Pull complete
Step #1: 56e204eacca1: Pull complete
Step #1: 1c8205fce1bf: Pull complete
Step #1: 9f79d1d1728e: Pull complete
Step #1: dca84d2f2059: Pull complete
Step #1: 6ce62f5256c2: Pull complete
Step #1: 3642ea3e80d4: Pull complete
Step #1: e16a032d433f: Pull complete
Step #1: 2478c9cd2ebf: Pull complete
Step #1: f629568b334f: Pull complete
Step #1: 44cb10575b68: Pull complete
Step #1: ed4ddcc30bc5: Pull complete
Step #1: 571fcf7c9d32: Pull complete
Step #1: 8d49c0dfcba1: Pull complete
Step #1: ec88b0069f5b: Pull complete
Step #1: ba047d14b151: Pull complete
Step #1: 71c92cbd2e5f: Pull complete
Step #1: 32464a57442d: Pull complete
Step #1: Digest: sha256:91ae5c3067724ff5c20759d9c439c0b78f2b3e57329232daf560fc79d7361e6f
Step #1: Status: Downloaded newer image for gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli:latest
Step #1: gcr.io/qwiklabs-gcp-02-4a8af9550a2a/tfx-cli:latest
Step #1: CLI
Step #1: Creating pipeline
Step #1: New container image is built. Target image is available in the build spec file.
Step #1: WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
Step #1: WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
Step #1: WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
Step #1: WARNING:absl:RuntimeParameter is only supported on Cloud-based DAG runner currently.
Step #1: WARNING:absl:`instance_name` is deprecated, please set node id directly using`with_id()` or `.id` setter.
Step #1: WARNING:absl:`instance_name` is deprecated, please set node id directly using`with_id()` or `.id` setter.
Step #1: Pipeline compiled successfully.
Step #1: Pipeline package path: /workspace/pipeline/tfx_covertype_continuous_training-test.tar.gz
Step #1: {'created_at': datetime.datetime(2024, 3, 24, 18, 1, 20, tzinfo=tzlocal()),
Step #1: 'default_version': {'code_source_url': None,
Step #1: 'created_at': datetime.datetime(2024, 3, 24, 18, 1, 20, tzinfo=tzlocal()),
Step #1: 'description': None,
Step #1: 'id': '5e7940e3-da63-4c9a-9cc4-6649a74aed3e',
Step #1: 'name': 'tfx_covertype_continuous_training-test',
Step #1: 'package_url': None,
Step #1: 'parameters': [{'name': 'pipeline-root',
Step #1: 'value': 'gs://qwiklabs-gcp-02-4a8af9550a2a-kubeflowpipelines-default/tfx_covertype_continuous_training-test/'},
Step #1: {'name': 'data-root-uri',
Step #1: 'value': 'gs://cloud-training/OCBL203/workshop-datasets'},
Step #1: {'name': 'eval-steps', 'value': '500'},
Step #1: {'name': 'train-steps', 'value': '5000'}],
Step #1: 'resource_references': [{'key': {'id': '533f8266-1acb-4747-b64a-dc55f206abd8',
Step #1: 'type': 'PIPELINE'},
Step #1: 'name': None,
Step #1: 'relationship': 'OWNER'}]},
Step #1: 'description': None,
Step #1: 'error': None,
Step #1: 'id': '533f8266-1acb-4747-b64a-dc55f206abd8',
Step #1: 'name': 'tfx_covertype_continuous_training-test',
Step #1: 'parameters': [{'name': 'pipeline-root',
Step #1: 'value': 'gs://qwiklabs-gcp-02-4a8af9550a2a-kubeflowpipelines-default/tfx_covertype_continuous_training-test/'},
Step #1: {'name': 'data-root-uri',
Step #1: 'value': 'gs://cloud-training/OCBL203/workshop-datasets'},
Step #1: {'name': 'eval-steps', 'value': '500'},
Step #1: {'name': 'train-steps', 'value': '5000'}],
Step #1: 'resource_references': None,
Step #1: 'url': None}
Step #1: Please access the pipeline detail page at http://36d154e9fbbae949-dot-us-central1.pipelines.googleusercontent.com/#/pipelines/details/533f8266-1acb-4747-b64a-dc55f206abd8
Step #1: Pipeline "tfx_covertype_continuous_training-test" created successfully.
Finished Step #1
PUSH
Pushing gcr.io/qwiklabs-gcp-02-4a8af9550a2a/lab-03-tfx-image:test
The push refers to repository [gcr.io/qwiklabs-gcp-02-4a8af9550a2a/lab-03-tfx-image]
337589714630: Preparing
7c268e72d79b: Preparing
5dadc0a09248: Preparing
8fb12d3bda49: Preparing
2471eac28ba8: Preparing
674ba689ae71: Preparing
4058ae03fa32: Preparing
e3437c61d457: Preparing
84ff92691f90: Preparing
54b00d861a7a: Preparing
c547358928ab: Preparing
84ff92691f90: Preparing
c4e66be694ce: Preparing
47cc65c6dd57: Preparing
674ba689ae71: Waiting
4058ae03fa32: Waiting
e3437c61d457: Waiting
84ff92691f90: Waiting
54b00d861a7a: Waiting
c547358928ab: Waiting
c4e66be694ce: Waiting
47cc65c6dd57: Waiting
5dadc0a09248: Layer already exists
8fb12d3bda49: Layer already exists
2471eac28ba8: Layer already exists
674ba689ae71: Layer already exists
4058ae03fa32: Layer already exists
e3437c61d457: Layer already exists
84ff92691f90: Layer already exists
54b00d861a7a: Layer already exists
c547358928ab: Layer already exists
47cc65c6dd57: Layer already exists
c4e66be694ce: Layer already exists
7c268e72d79b: Pushed
337589714630: Pushed
test: digest: sha256:ca0808dacf76048b140558f643c8b51639389896d57d3bcd0d05ebb3a4cd3d23 size: 3266
DONE
--------------------------------------------------------------------------------
ID CREATE_TIME DURATION SOURCE IMAGES STATUS
fa6ca464-457b-4913-abde-8ba14fc29cef 2024-03-24T17:55:51+00:00 5M38S gs://qwiklabs-gcp-02-4a8af9550a2a_cloudbuild/source/1711302950.727818-7d3f79f1ca114aa0b7177b4c59f25254.tgz gcr.io/qwiklabs-gcp-02-4a8af9550a2a/lab-03-tfx-image:test SUCCESS
In this exercise you integrate your CI/CD workflow with GitHub, using Cloud Build GitHub App. You will set up a trigger that starts the CI/CD workflow when a new tag is applied to the GitHub repo managing the pipeline source code. You will use a fork of this repo as your source GitHub repository.
Connect the fork you created in the previous step to your Google Cloud project and create a trigger following the steps in the Creating GitHub app trigger article. Use the following values on the Edit trigger form:
| Field | Value |
|---|---|
| Name | [YOUR TRIGGER NAME] |
| Description | [YOUR TRIGGER DESCRIPTION] |
| Event | Tag |
| Source | [YOUR FORK] |
| Tag (regex) | .* |
| Build Configuration | Cloud Build configuration file (yaml or json) |
| Cloud Build configuration file location | / workshops/tfx-caip-tf23/lab-03-tfx-cicd/labs/cloudbuild.yaml |
Use the following values for the substitution variables:
| Variable | Value |
|---|---|
| _GCP_REGION | [YOUR GCP_REGION] |
| _CUSTOM_SERVICE_ACCOUNT | [YOUR CUSTOM_SERVICE_ACCOUNT] |
| _ENDPOINT | [Your inverting proxy host pipeline ENDPOINT] |
| _TFX_IMAGE_NAME | lab-03-tfx-image |
| _PIPELINE_NAME | tfx_covertype_continuous_training |
| _MODEL_NAME | tfx_covertype_classifier |
| _DATA_ROOT_URI | gs://cloud-training/OCBL203/workshop-datasets |
| _PIPELINE_FOLDER | workshops/tfx-caip-tf23/lab-03-tfx-cicd/labs/pipeline |
| _PIPELINE_DSL | runner.py |
| _PYTHON_VERSION | 3.7 |
| _RUNTIME_VERSION | 2.3 |
| _USE_KFP_SA | False |
| _ENABLE_TUNING | True |
To start an automated build create a new release of the repo in GitHub. Alternatively, you can start the build by applying a tag using git.
git tag [TAG NAME]
git push origin --tags
After you see the pipeline finish building on the Cloud Build dashboard, return to AI Platform Pipelines in the console. Click OPEN PIPELINES DASHBOARD and view the newly deployed pipeline. Creating a release tag on GitHub will create a pipeline with the name tfx_covertype_continuous_training-[TAG NAME] while doing so from GitHub will create a pipeline with the name tfx_covertype_continuous_training_github-[TAG NAME].
In this lab, you walked through authoring a Cloud Build CI/CD workflow that automatically builds and deploys a TFX pipeline. You also integrated your TFX workflow with GitHub by setting up a Cloud Build trigger. In the next lab, you will walk through inspection of TFX metadata and pipeline artifacts created during TFX pipeline runs.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.