Skip to content

Commit 6aef9e3

Browse files
nadiayanadiaya
authored
doc: remove confusing information from the Readme. (#387)
1 parent a22e3df commit 6aef9e3

File tree

2 files changed

+71
-237
lines changed

2 files changed

+71
-237
lines changed

‎CONTRIBUTING.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Contributing Guidelines
2+
3+
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
4+
documentation, we greatly value feedback and contributions from our community.
5+
6+
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
7+
information to effectively respond to your bug report or contribution.
8+
9+
10+
## Reporting Bugs/Feature Requests
11+
12+
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
13+
14+
When filing an issue, please check [existing open](https://github.com/aws/sagemaker-tensorflow-training-toolkit/issues), or [recently closed](https://github.com/aws/sagemaker-tensorflow-training-toolkit/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
15+
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
16+
17+
* A reproducible test case or series of steps
18+
* The version of our code being used
19+
* Any modifications you've made relevant to the bug
20+
* Anything unusual about your environment or deployment
21+
22+
23+
## Contributing via Pull Requests
24+
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
25+
26+
1. You are working against the latest source on the *master* branch.
27+
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
28+
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
29+
30+
To send us a pull request, please:
31+
32+
1. Fork the repository.
33+
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
34+
3. Ensure local tests pass.
35+
4. Commit to your fork using clear commit messages.
36+
5. Send us a pull request, answering any default questions in the pull request interface.
37+
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
38+
39+
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
40+
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
41+
42+
43+
## Finding contributions to work on
44+
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws/sagemaker-tensorflow-training-toolkit/labels/help%20wanted) issues is a great place to start.
45+
46+
47+
## Code of Conduct
48+
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
49+
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
50+
[email protected] with any additional questions or comments.
51+
52+
53+
## Security issue notifications
54+
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
55+
56+
57+
## Licensing
58+
59+
See the [LICENSE](https://github.com/aws/sagemaker-tensorflow-training-toolkit//blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
60+
61+
We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

‎README.rst‎

Lines changed: 10 additions & 237 deletions
Original file line numberDiff line numberDiff line change
@@ -1,256 +1,29 @@
1-
===============================
2-
SageMaker TensorFlow Containers
3-
===============================
1+
=====================================
2+
SageMaker TensorFlow Training Toolkit
3+
=====================================
44

5-
SageMaker TensorFlow Containers is an open source library for making the
6-
TensorFlow framework run on `Amazon SageMaker <https://aws.amazon.com/documentation/sagemaker/>`__.
5+
SageMaker TensorFlow Training Toolkit is an open-source library for using TensorFlow to train models on Amazon SageMaker.
76

8-
This repository also contains Dockerfiles which install this library, TensorFlow, and dependencies
9-
for building SageMaker TensorFlow images.
7+
For inference, see `SageMaker TensorFlow Inference Toolkit <https://github.com/aws/sagemaker-tensorflow-serving-container>`__.
108

11-
For information on running TensorFlow jobs on SageMaker: `Python
12-
SDK <https://github.com/aws/sagemaker-python-sdk>`__.
9+
For the Dockerfiles used for building SageMaker TensorFlow Containers, see `AWS Deep Learning Containers <https://github.com/aws/deep-learning-containers>`__.
10+
11+
For information on running TensorFlow jobs on Amazon SageMaker, please refer to the `SageMaker Python SDK documentation <https://github.com/aws/sagemaker-python-sdk>`__.
1312

1413
For notebook examples: `SageMaker Notebook
1514
Examples <https://github.com/awslabs/amazon-sagemaker-examples>`__.
1615

17-
Table of Contents
18-
-----------------
19-
20-
#. `Getting Started <#getting-started>`__
21-
#. `Building your Image <#building-your-image>`__
22-
#. `Running the tests <#running-the-tests>`__
23-
24-
Getting Started
25-
---------------
26-
27-
Prerequisites
28-
~~~~~~~~~~~~~
29-
30-
Make sure you have installed all of the following prerequisites on your
31-
development machine:
32-
33-
- `Docker <https://www.docker.com/>`__
34-
35-
For Testing on GPU
36-
^^^^^^^^^^^^^^^^^^
37-
38-
- `Nvidia-Docker <https://github.com/NVIDIA/nvidia-docker>`__
39-
40-
Recommended
41-
^^^^^^^^^^^
42-
43-
- A Python environment management tool. (e.g.
44-
`PyEnv <https://github.com/pyenv/pyenv>`__,
45-
`VirtualEnv <https://virtualenv.pypa.io/en/stable/>`__)
46-
47-
Building your Image
48-
-------------------
49-
50-
`Amazon SageMaker <https://aws.amazon.com/documentation/sagemaker/>`__
51-
utilizes Docker containers to run all training jobs & inference endpoints.
52-
53-
The Docker images are built from the Dockerfiles specified in
54-
`Docker/ <https://github.com/aws/sagemaker-tensorflow-containers/tree/master/docker>`__.
55-
56-
The Docker files are grouped based on TensorFlow version and separated
57-
based on Python version and processor type.
58-
59-
The Docker files for TensorFlow 2.0 are available in the
60-
`tf-2 <https://github.com/aws/sagemaker-tensorflow-container/tree/tf-2>`__ branch, in
61-
`docker/2.0.0/ <https://github.com/aws/sagemaker-tensorflow-container/tree/tf-2/docker/2.0.0>`__.
62-
63-
The Docker images, used to run training & inference jobs, are built from
64-
both corresponding "base" and "final" Dockerfiles.
65-
66-
Base Images
67-
~~~~~~~~~~~
68-
69-
The "base" Dockerfile encompass the installation of the framework and all of the dependencies
70-
needed. It is needed before building image for TensorFlow 1.8.0 and before.
71-
Building a base image is not required for images for TensorFlow 1.9.0 and onwards.
72-
73-
Tagging scheme is based on <tensorflow_version>-<processor>-<python_version>. (e.g. 1.4
74-
.1-cpu-py2)
75-
76-
All "final" Dockerfiles build images using base images that use the tagging scheme
77-
above.
78-
79-
Before building these images, you need to have a pip-installable binary of this repository saved locally. To create the SageMaker Tensorflow Container Python package:
80-
81-
::
82-
# Create the binary
83-
git clone https://github.com/aws/sagemaker-tensorflow-container.git
84-
cd sagemaker-tensorflow-container
85-
python setup.py sdist
86-
cp dist/sagemaker_tensorflow_training*.tar.gz docker/<tensorflow_version>/sagemaker_tensorflow_training.tar.gz
87-
88-
Once you have copied the tensorflow_training.tar.gz to the desired location [same directory as the Dockerfile], you can then build the image.
89-
90-
If you want to build your "base" Docker image, then use:
91-
92-
::
93-
94-
# All build instructions assume you're building from the same directory as the Dockerfile.
95-
96-
# CPU
97-
docker build -t tensorflow-base:<tensorflow_version>-cpu-<python_version> -f Dockerfile.cpu .
98-
99-
# GPU
100-
docker build -t tensorflow-base:<tensorflow_version>-gpu-<python_version> -f Dockerfile.gpu .
101-
102-
::
103-
104-
# Example
105-
106-
# CPU
107-
docker build -t tensorflow-base:1.4.1-cpu-py2 -f Dockerfile.cpu .
108-
109-
# GPU
110-
docker build -t tensorflow-base:1.4.1-gpu-py2 -f Dockerfile.gpu .
111-
112-
Final Images
113-
~~~~~~~~~~~~
114-
115-
The "final" Dockerfiles encompass the installation of the SageMaker specific support code.
116-
117-
For images of TensorFlow 1.8.0 and before, all "final" Dockerfiles use `base images for building <https://github
118-
.com/aws/sagemaker-tensorflow-containers/blob/master/docker/1.4.1/final/py2/Dockerfile.cpu#L2>`__.
119-
120-
These "base" images are specified with the naming convention of
121-
tensorflow-base:<tensorflow_version>-<processor>-<python_version>.
122-
123-
Before building "final" images:
124-
125-
Build your "base" image. Make sure it is named and tagged in accordance with your "final"
126-
Dockerfile. Skip this step if you want to build image of Tensorflow Version 1.9.0 and above.
127-
128-
If you want to build "final" Docker images, for versions 1.6 and above, you will first need to download the appropriate tensorflow pip wheel, then pass in its location as a build argument. These can be obtained from pypi. For example, the files for 1.6.0 are here:
129-
130-
https://pypi.org/project/tensorflow/1.6.0/#files
131-
https://pypi.org/project/tensorflow-gpu/1.6.0/#files
132-
133-
Note that you need to use the tensorflow-gpu wheel when building the GPU image.
134-
135-
Then run:
136-
137-
::
138-
139-
# All build instructions assumes you're building from the same directory as the Dockerfile.
140-
141-
# CPU
142-
docker build -t <image_name>:<tag> --build-arg py_version=<py_version> --build-arg framework_installable=<path to tensorflow binary> -f Dockerfile.cpu .
143-
144-
# GPU
145-
docker build -t <image_name>:<tag> --build-arg py_version=<py_version> --build-arg framework_installable=<path to tensorflow binary> -f Dockerfile.gpu .
146-
147-
::
148-
149-
# Example
150-
docker build -t preprod-tensorflow:1.6.0-cpu-py2 --build-arg py_version=2
151-
--build-arg framework_installable=tensorflow-1.6.0-cp27-cp27mu-manylinux1_x86_64.whl -f Dockerfile.cpu .
152-
153-
The dockerfiles for 1.4 and 1.5 build from source instead, so when building those, you don't need to download the wheel beforehand:
154-
155-
::
156-
157-
# All build instructions assumes you're building from the same directory as the Dockerfile.
158-
159-
# CPU
160-
docker build -t <image_name>:<tag> -f Dockerfile.cpu .
161-
162-
# GPU
163-
docker build -t <image_name>:<tag> -f Dockerfile.gpu .
164-
165-
::
166-
167-
# Example
168-
169-
# CPU
170-
docker build -t preprod-tensorflow:1.4.1-cpu-py2 -f Dockerfile.cpu .
171-
172-
# GPU
173-
docker build -t preprod-tensorflow:1.4.1-gpu-py2 -f Dockerfile.gpu .
174-
175-
176-
Running the tests
177-
-----------------
178-
179-
Running the tests requires installation of the SageMaker TensorFlow Container code and its test
180-
dependencies.
181-
182-
::
183-
184-
git clone https://github.com/aws/sagemaker-tensorflow-containers.git
185-
cd sagemaker-tensorflow-containers
186-
pip install -e .[test]
187-
188-
Tests are defined in
189-
`test/ <https://github.com/aws/sagemaker-tensorflow-containers/tree/master/test>`__
190-
and include unit, integration and functional tests.
191-
192-
Unit Tests
193-
~~~~~~~~~~
194-
195-
If you want to run unit tests, then use:
196-
197-
::
198-
199-
# All test instructions should be run from the top level directory
200-
201-
pytest test/unit
202-
203-
Integration Tests
204-
~~~~~~~~~~~~~~~~~
205-
206-
Running integration tests require `Docker <https://www.docker.com/>`__ and `AWS
207-
credentials <https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html>`__,
208-
as the integration tests make calls to a couple AWS services. The integration and functional
209-
tests require configurations specified within their respective
210-
`conftest.py <https://github.com/aws/sagemaker-tensorflow-containers/blob/master/test/integration/conftest.py>`__.Make sure to update the account-id and region at a minimum.
211-
212-
Integration tests on GPU require `Nvidia-Docker <https://github.com/NVIDIA/nvidia-docker>`__.
213-
214-
Before running integration tests:
215-
216-
#. Build your Docker image.
217-
#. Pass in the correct pytest arguments to run tests against your Docker image.
218-
219-
If you want to run local integration tests, then use:
220-
221-
::
222-
223-
# Required arguments for integration tests are found in test/integ/conftest.py
224-
225-
pytest test/integration --docker-base-name <your_docker_image> \
226-
--tag <your_docker_image_tag> \
227-
--framework-version <tensorflow_version> \
228-
--processor <cpu_or_gpu>
229-
230-
::
231-
232-
# Example
233-
pytest test/integration --docker-base-name preprod-tensorflow \
234-
--tag 1.0 \
235-
--framework-version 1.4.1 \
236-
--processor cpu
237-
238-
Functional Tests
239-
~~~~~~~~~~~~~~~~
240-
241-
Functional tests are removed from the current branch, please see them in older branch `r1.0 <https://github.com/aws/sagemaker-tensorflow-container/tree/r1.0#functional-tests>`__.
242-
24316
Contributing
24417
------------
24518

24619
Please read
247-
`CONTRIBUTING.md <https://github.com/aws/sagemaker-tensorflow-containers/blob/master/CONTRIBUTING.md>`__
20+
`CONTRIBUTING.md <https://github.com/aws/sagemaker-tensorflow-training-toolkit/blob/master/CONTRIBUTING.md>`__
24821
for details on our code of conduct, and the process for submitting pull
24922
requests to us.
25023

25124
License
25225
-------
25326

254-
SageMaker TensorFlow Containers is licensed under the Apache 2.0 License. It is copyright 2018
27+
SageMaker TensorFlow Training Toolkit is licensed under the Apache 2.0 License. It is copyright 2018
25528
Amazon.com, Inc. or its affiliates. All Rights Reserved. The license is available at:
25629
http://aws.amazon.com/apache2.0/

0 commit comments

Comments
 (0)