Skip to content

Commit e617a54

Browse files
laurenyulaurenyu
authored
doc: update image-building instructions (#368)
1 parent 2a99e1d commit e617a54

File tree

1 file changed

+26
-110
lines changed

1 file changed

+26
-110
lines changed

‎README.rst‎

Lines changed: 26 additions & 110 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,17 @@
1-
===============================
2-
SageMaker TensorFlow Containers
3-
===============================
1+
=====================================
2+
SageMaker TensorFlow Training Toolkit
3+
=====================================
44

5-
SageMaker TensorFlow Containers is an open source library for making the
5+
The SageMaker TensorFlow Training Toolkit is an open source library for making the
66
TensorFlow framework run on `Amazon SageMaker <https://aws.amazon.com/documentation/sagemaker/>`__.
77

88
This repository also contains Dockerfiles which install this library, TensorFlow, and dependencies
99
for building SageMaker TensorFlow images.
1010

11-
For information on running TensorFlow jobs on SageMaker: `Python
12-
SDK <https://github.com/aws/sagemaker-python-sdk>`__.
11+
For information on running TensorFlow jobs on SageMaker:
1312

14-
For notebook examples: `SageMaker Notebook
15-
Examples <https://github.com/awslabs/amazon-sagemaker-examples>`__.
13+
- `SageMaker Python SDK documentation <https://sagemaker.readthedocs.io/en/stable/using_tf.html>`__
14+
- `SageMaker Notebook Examples <https://github.com/awslabs/amazon-sagemaker-examples>`__
1615

1716
Table of Contents
1817
-----------------
@@ -51,131 +50,48 @@ Building your Image
5150
utilizes Docker containers to run all training jobs & inference endpoints.
5251

5352
The Docker images are built from the Dockerfiles specified in
54-
`Docker/ <https://github.com/aws/sagemaker-tensorflow-containers/tree/master/docker>`__.
53+
`docker/ <https://github.com/aws/sagemaker-tensorflow-containers/tree/master/docker>`__.
5554

56-
The Docker files are grouped based on TensorFlow version and separated
55+
The Dockerfiles are grouped based on TensorFlow version and separated
5756
based on Python version and processor type.
5857

59-
The Docker files for TensorFlow 2.0 are available in the
60-
`tf-2 <https://github.com/aws/sagemaker-tensorflow-container/tree/tf-2>`__ branch, in
61-
`docker/2.0.0/ <https://github.com/aws/sagemaker-tensorflow-container/tree/tf-2/docker/2.0.0>`__.
58+
The Dockerfiles for TensorFlow 2.0+ are available in the
59+
`tf-2 <https://github.com/aws/sagemaker-tensorflow-container/tree/tf-2>`__ branch.
6260

63-
The Docker images, used to run training & inference jobs, are built from
64-
both corresponding "base" and "final" Dockerfiles.
65-
66-
Base Images
67-
~~~~~~~~~~~
68-
69-
The "base" Dockerfile encompass the installation of the framework and all of the dependencies
70-
needed. It is needed before building image for TensorFlow 1.8.0 and before.
71-
Building a base image is not required for images for TensorFlow 1.9.0 and onwards.
72-
73-
Tagging scheme is based on <tensorflow_version>-<processor>-<python_version>. (e.g. 1.4
74-
.1-cpu-py2)
75-
76-
All "final" Dockerfiles build images using base images that use the tagging scheme
77-
above.
78-
79-
If you want to build your "base" Docker image, then use:
80-
81-
::
82-
83-
# All build instructions assume you're building from the same directory as the Dockerfile.
84-
85-
# CPU
86-
docker build -t tensorflow-base:<tensorflow_version>-cpu-<python_version> -f Dockerfile.cpu .
87-
88-
# GPU
89-
docker build -t tensorflow-base:<tensorflow_version>-gpu-<python_version> -f Dockerfile.gpu .
90-
91-
::
92-
93-
# Example
94-
95-
# CPU
96-
docker build -t tensorflow-base:1.4.1-cpu-py2 -f Dockerfile.cpu .
97-
98-
# GPU
99-
docker build -t tensorflow-base:1.4.1-gpu-py2 -f Dockerfile.gpu .
100-
101-
Final Images
102-
~~~~~~~~~~~~
103-
104-
The "final" Dockerfiles encompass the installation of the SageMaker specific support code.
105-
106-
For images of TensorFlow 1.8.0 and before, all "final" Dockerfiles use `base images for building <https://github
107-
.com/aws/sagemaker-tensorflow-containers/blob/master/docker/1.4.1/final/py2/Dockerfile.cpu#L2>`__.
108-
109-
These "base" images are specified with the naming convention of
110-
tensorflow-base:<tensorflow_version>-<processor>-<python_version>.
111-
112-
Before building "final" images:
113-
114-
Build your "base" image. Make sure it is named and tagged in accordance with your "final"
115-
Dockerfile. Skip this step if you want to build image of Tensorflow Version 1.9.0 and above.
116-
117-
If you want to build "final" Docker images, for versions 1.6 and above, you will first need to download the appropriate tensorflow pip wheel, then pass in its location as a build argument. These can be obtained from pypi. For example, the files for 1.6.0 are here:
118-
119-
https://pypi.org/project/tensorflow/1.6.0/#files
120-
https://pypi.org/project/tensorflow-gpu/1.6.0/#files
121-
122-
Note that you need to use the tensorflow-gpu wheel when building the GPU image.
123-
124-
Then run:
125-
126-
::
127-
128-
# All build instructions assumes you're building from the same directory as the Dockerfile.
129-
130-
# CPU
131-
docker build -t <image_name>:<tag> --build-arg py_version=<py_version> --build-arg framework_installable=<path to tensorflow binary> -f Dockerfile.cpu .
132-
133-
# GPU
134-
docker build -t <image_name>:<tag> --build-arg py_version=<py_version> --build-arg framework_installable=<path to tensorflow binary> -f Dockerfile.gpu .
135-
136-
::
137-
138-
# Example
139-
docker build -t preprod-tensorflow:1.6.0-cpu-py2 --build-arg py_version=2
140-
--build-arg framework_installable=tensorflow-1.6.0-cp27-cp27mu-manylinux1_x86_64.whl -f Dockerfile.cpu .
141-
142-
The dockerfiles for 1.4 and 1.5 build from source instead, so when building those, you don't need to download the wheel beforehand:
61+
To build the images, first copy the files under
62+
`docker/build_artifacts/ <https://github.com/aws/sagemaker-tensorflow-container/tree/tf-2/docker/build_artifacts>`__
63+
to the folder container the Dockerfile you wish to build.
14364

14465
::
14566

146-
# All build instructions assumes you're building from the same directory as the Dockerfile.
147-
148-
# CPU
149-
docker build -t <image_name>:<tag> -f Dockerfile.cpu .
67+
# Example for building a TF 2.1 image with Python 3
68+
cp docker/build_artifacts/* docker/2.1.0/py3/.
15069

151-
# GPU
152-
docker build -t <image_name>:<tag> -f Dockerfile.gpu .
70+
After that, go to the directory containing the Dockerfile you wish to build,
71+
and run ``docker build`` to build the image.
15372

15473
::
15574

156-
# Example
157-
158-
# CPU
159-
docker build -t preprod-tensorflow:1.4.1-cpu-py2 -f Dockerfile.cpu .
160-
161-
# GPU
162-
docker build -t preprod-tensorflow:1.4.1-gpu-py2 -f Dockerfile.gpu .
75+
# Example for building a TF 2.1 image for CPU with Python 3
76+
cd docker/2.1.0/py3
77+
docker build -t tensorflow-training:2.1.0-cpu-py3 -f Dockerfile.cpu .
16378

79+
Don't forget the period at the end of the ``docker build`` command!
16480

16581
Running the tests
16682
-----------------
16783

168-
Running the tests requires installation of the SageMaker TensorFlow Container code and its test
84+
Running the tests requires installation of the SageMaker TensorFlow Training Toolkit code and its test
16985
dependencies.
17086

17187
::
17288

173-
git clone https://github.com/aws/sagemaker-tensorflow-containers.git
174-
cd sagemaker-tensorflow-containers
89+
git clone https://github.com/aws/sagemaker-tensorflow-container.git
90+
cd sagemaker-tensorflow-container
17591
pip install -e .[test]
17692

17793
Tests are defined in
178-
`test/ <https://github.com/aws/sagemaker-tensorflow-containers/tree/master/test>`__
94+
`test/ <https://github.com/aws/sagemaker-tensorflow-container/tree/master/test>`__
17995
and include unit, integration and functional tests.
18096

18197
Unit Tests

0 commit comments

Comments
 (0)