ML Community Day is November 9! Join us for updates from TensorFlow, JAX, and more Learn more

tfc.run

Runs your Tensorflow code in Google Cloud Platform.

Used in the notebooks

Used in the guide

entry_point Optional string. File path to the python file or iPython notebook that contains the TensorFlow code. Note this path must be in the current working directory tree. Example - 'train.py', 'training/mnist.py', 'mnist.ipynb' If entry_point is not provided, then

  • If you are in an iPython notebook environment, then the current notebook is taken as the entry_point.
  • Otherwise, the current python script is taken as the entry_point.
requirements_txt Optional string. File path to requirements.txt file containing additional pip dependencies if any. ie. a file with a list of pip dependency package names. Note this path must be in the current working directory tree. Example - 'requirements.txt', 'deps/reqs.txt'
docker_config Optional DockerConfig. Represents Docker related configuration for the run API.
  • image: Optional Docker image URI for the Docker image being built.
  • parent_image: Optional parent Docker image to use.
  • cache_from: Optional Docker image URI to be used as a cache when building the new Docker image.
  • image_build_bucket: Optional GCS bucket name to be used for building a Docker image via Google Cloud Build. Defaults to 'auto'. 'auto' maps to a default tfc.DockerConfig instance.
  • distribution_strategy 'auto' or None. Defaults to 'auto'. 'auto' means we will take care of creating a Tensorflow distribution strategy instance based on the machine configurations you have provided using the chief_config, worker_config and worker_count params.
  • If the number of workers > 0, we will use tf.distribute.experimental.MultiWorkerMirroredStrategy or tf.distribute.experimental.TPUStrategy based on the accelerator type.
  • If number of GPUs > 0, we will use tf.distribute.MirroredStrategy
  • Otherwise, we will use tf.distribute.OneDeviceStrategy If you have created a distribution strategy instance in your script already, please set distribution_strategy as None here. For example, if you are using tf.keras custom training loops, you will need to create a strategy in the script for distributing the dataset.
  • chief_config Optional MachineConfig that represents the configuration for the chief worker in a distribution cluster. Defaults to 'auto'. 'auto' maps to a standard gpu config such as COMMON_MACHINE_CONFIGS.T4_1X (8 cpu cores, 30GB memory, 1 Nvidia Tesla T4). For TPU strategy, chief_config refers to the config of the host that controls the TPU workers.
    worker_config Optional MachineConfig that represents the configuration for the general workers in a distribution cluster. Defaults to 'auto'. 'auto' maps to a standard gpu config such as COMMON_MACHINE_CONFIGS.T4_1X (8 cpu cores, 30GB memory, 1 Nvidia Tesla T4). For TPU strategy, worker_config should be a TPU config with 8 TPU cores (eg. COMMON_MACHINE_CONFIGS.TPU).
    worker_count Optional integer that represents the number of general workers in a distribution cluster. Defaults to 0. This count does not include the chief worker. For TPU strategy, worker_count should be set to 1.
    entry_point_args Optional list of strings. Defaults to None. Command line arguments to pass to the entry_point program.
    stream_logs Boolean flag which when enabled streams logs back from the cloud job.
    job_labels Dict of str: str. Labels to organize jobs. You can specify up to 64 key-value pairs in lowercase letters and numbers, where the first character must be lowercase letter. For more details see resource-labels
    service_account The email address of a user-managed service account to be used for training instead of the service account that AI Platform Training uses by default. see custom-service-account
    **kwargs Additional keyword arguments.

    A dictionary with two keys.'job_id' - the training job id and 'docker_image'- Docker image generated for the training job.