Release preparation

Author: igorborgest

MANIFEST.in

@@ -4,8 +4,8 @@ exclude *.egg-info
 exclude data_samples*
 exclude dist*
 exclude docs*
+exclude dev*
 exclude htmlcov*
 exclude testing*
 exclude building*
 exclude tutorials*
-exclude tutorials*

README.md

@@ -42,35 +42,35 @@ df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)
 df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")
 ```
 
-## [Read The Docs](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/index.html)
+## [Read The Docs](https://aws-data-wrangler.readthedocs.io/)
 
-- [**What is AWS Data Wrangler?**](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/what.html)
-- [**Install**](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html)
-  - [PyPi (pip)](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#pypi-pip)
-  - [Conda](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#conda)
-  - [AWS Lambda Layer](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#aws-lambda-layer)
-  - [AWS Glue Wheel](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#aws-glue-wheel)
-  - [Amazon SageMaker Notebook](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#amazon-sagemaker-notebook)
-  - [Amazon SageMaker Notebook Lifecycle](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#amazon-sagemaker-notebook-lifecycle)
-  - [EMR](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#emr)
-  - [From source](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/install.html#from-source)
-- [**Tutorials**](https://github.com/awslabs/aws-data-wrangler/tree/dev-1.0.0/tutorials)
-  - [01 - Introduction](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/01%20-%20Introduction.ipynb)
-  - [02 - Sessions](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/02%20-%20Sessions.ipynb)
-  - [03 - Amazon S3](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/03%20-%20Amazon%20S3.ipynb)
-  - [04 - Parquet Datasets](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/04%20-%20Parquet%20Datasets.ipynb)
-  - [05 - Glue Catalog](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/05%20-%20Glue%20Catalog.ipynb)
-  - [06 - Amazon Athena](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/06%20-%20Amazon%20Athena.ipynb)
-  - [07 - Databases (Redshift, MySQL and PostgreSQL)](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/07%20-%20Redshift%2C%20MySQL%2C%20PostgreSQL.ipynb)
-  - [08 - Redshift Copy & Unload.ipynb](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/08%20-%20Redshift%20Copy%20%26%20Unload.ipynb)
-  - [09 - Redshift - Append, Overwrite and Upsert.ipynb](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/09%20-%20Redshift%20-%20Append%2C%20Overwrite%2C%20Upsert.ipynb)
-  - [10 - Parquet Crawler.ipynb](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/tutorials/10%20-%20Parquet%20Crawler.ipynb)
-- [**API Reference**](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html)
-  - [Amazon S3](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html#amazon-s3)
-  - [AWS Glue Catalog](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html#aws-glue-catalog)
-  - [Amazon Athena](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html#amazon-athena)
-  - [Databases (Redshift, PostgreSQL, MySQL)](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html#databases-redshift-postgresql-mysql)
-  - [EMR](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html#emr)
-  - [CloudWatch Logs](https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html#cloudwatch-logs)
-- [**License**](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/LICENSE)
-- [**Contributing**](https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/CONTRIBUTING.md)
+- [**What is AWS Data Wrangler?**](https://aws-data-wrangler.readthedocs.io/en/latest/what.html)
+- [**Install**](https://aws-data-wrangler.readthedocs.io/en/latest/install.html)
+  - [PyPi (pip)](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#pypi-pip)
+  - [Conda](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#conda)
+  - [AWS Lambda Layer](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#aws-lambda-layer)
+  - [AWS Glue Wheel](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#aws-glue-wheel)
+  - [Amazon SageMaker Notebook](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#amazon-sagemaker-notebook)
+  - [Amazon SageMaker Notebook Lifecycle](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#amazon-sagemaker-notebook-lifecycle)
+  - [EMR](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#emr)
+  - [From source](https://aws-data-wrangler.readthedocs.io/en/latest/install.html#from-source)
+- [**Tutorials**](https://github.com/awslabs/aws-data-wrangler/tree/master/tutorials)
+  - [01 - Introduction](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/01%20-%20Introduction.ipynb)
+  - [02 - Sessions](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/02%20-%20Sessions.ipynb)
+  - [03 - Amazon S3](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/03%20-%20Amazon%20S3.ipynb)
+  - [04 - Parquet Datasets](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/04%20-%20Parquet%20Datasets.ipynb)
+  - [05 - Glue Catalog](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/05%20-%20Glue%20Catalog.ipynb)
+  - [06 - Amazon Athena](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/06%20-%20Amazon%20Athena.ipynb)
+  - [07 - Databases (Redshift, MySQL and PostgreSQL)](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/07%20-%20Redshift%2C%20MySQL%2C%20PostgreSQL.ipynb)
+  - [08 - Redshift - Copy & Unload.ipynb](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/08%20-%20Redshift%20-%20Copy%20%26%20Unload.ipynb)
+  - [09 - Redshift - Append, Overwrite and Upsert.ipynb](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/09%20-%20Redshift%20-%20Append%2C%20Overwrite%2C%20Upsert.ipynb)
+  - [10 - Parquet Crawler.ipynb](https://github.com/awslabs/aws-data-wrangler/blob/master/tutorials/10%20-%20Parquet%20Crawler.ipynb)
+- [**API Reference**](https://aws-data-wrangler.readthedocs.io/en/latest/api.html)
+  - [Amazon S3](https://aws-data-wrangler.readthedocs.io/en/latest/api.html#amazon-s3)
+  - [AWS Glue Catalog](https://aws-data-wrangler.readthedocs.io/en/latest/api.html#aws-glue-catalog)
+  - [Amazon Athena](https://aws-data-wrangler.readthedocs.io/en/latest/api.html#amazon-athena)
+  - [Databases (Redshift, PostgreSQL, MySQL)](https://aws-data-wrangler.readthedocs.io/en/latest/api.html#databases-redshift-postgresql-mysql)
+  - [EMR](https://aws-data-wrangler.readthedocs.io/en/latest/api.html#emr)
+  - [CloudWatch Logs](https://aws-data-wrangler.readthedocs.io/en/latest/api.html#cloudwatch-logs)
+- [**License**](https://github.com/awslabs/aws-data-wrangler/blob/master/LICENSE)
+- [**Contributing**](https://github.com/awslabs/aws-data-wrangler/blob/master/CONTRIBUTING.md)

awswrangler/_utils.py

@@ -26,7 +26,7 @@ def client(service_name: str, session: Optional[boto3.Session] = None) -> boto3.
     return ensure_session(session=session).client(
         service_name=service_name,
         use_ssl=True,
-        config=botocore.config.Config(retries={"mode": "adaptive", "max_attempts": 10}),
+        config=botocore.config.Config(retries={"max_attempts": 15}),
     )
 
 
@@ -134,7 +134,7 @@ def get_fs(
         default_cache_type="none",
         default_fill_cache=False,
         default_block_size=134_217_728,  # 128 MB (50 * 2**20)
-        config_kwargs={"retries": {"mode": "adaptive", "max_attempts": 10}},
+        config_kwargs={"retries": {"max_attempts": 15}},
         session=ensure_session(session=session)._session,  # pylint: disable=protected-access
         s3_additional_kwargs=s3_additional_kwargs,
         use_listings_cache=False,

building/Dockerfile

@@ -17,6 +17,6 @@ ADD requirements-dev.txt /root/
 RUN pip3 install -r /root/requirements-dev.txt
 RUN rm -rf /root/requirements-dev.txt
 
-RUN pip3 install boto3==1.10.34 botocore==1.13.34
+RUN pip3 install boto3==1.12.22 botocore==1.15.22
 
 ENTRYPOINT ["/bin/sh"]

docs/source/conf.py

@@ -83,10 +83,10 @@
     "navbar_links": [
         ("What is Data Wrangler?", "what"),
         ("Install", "install"),
-        ("Tutorials", "https://github.com/awslabs/aws-data-wrangler/tree/dev-1.0.0/tutorials", True),
+        ("Tutorials", "https://github.com/awslabs/aws-data-wrangler/tree/latest/tutorials", True),
         ("API Reference", "api"),
-        ("License", "https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/LICENSE", True),
-        ("Contributing", "https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/CONTRIBUTING.md", True),
+        ("License", "https://github.com/awslabs/aws-data-wrangler/blob/latest/LICENSE", True),
+        ("Contributing", "https://github.com/awslabs/aws-data-wrangler/blob/latest/CONTRIBUTING.md", True),
         ("GitHub", "https://github.com/awslabs/aws-data-wrangler", True),
     ],
     # Render the next and previous page links in navbar. (Default: true)

docs/source/index.rst

@@ -33,8 +33,8 @@ Read The Docs
 
    what
    install
-   Tutorials <https://github.com/awslabs/aws-data-wrangler/tree/dev-1.0.0/tutorials>
+   Tutorials <https://github.com/awslabs/aws-data-wrangler/tree/latest/tutorials>
    api
-   License <https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/LICENSE>
-   Contributing <https://github.com/awslabs/aws-data-wrangler/blob/dev-1.0.0/CONTRIBUTING.md>
+   License <https://github.com/awslabs/aws-data-wrangler/blob/latest/LICENSE>
+   Contributing <https://github.com/awslabs/aws-data-wrangler/blob/latest/CONTRIBUTING.md>
 

docs/source/what.rst

@@ -5,4 +5,4 @@ An `open-source <https://github.com/awslabs/aws-data-wrangler>`_ Python package
 
 Built on top of other open-source projects like `Pandas <https://github.com/pandas-dev/pandas>`_, `Apache Arrow <https://github.com/apache/arrow>`_, `Boto3 <https://github.com/boto/boto3>`_, `s3fs <https://github.com/dask/s3fs>`_, `SQLAlchemy <https://github.com/sqlalchemy/sqlalchemy>`_, `Psycopg2 <https://github.com/psycopg/psycopg2>`_ and `PyMySQL <https://github.com/PyMySQL/PyMySQL>`_, it offers abstracted functions to execute usual ETL tasks like load/unload data from **Data Lakes**, **Data Warehouses** and **Databases**.
 
-Check our `tutorials <https://github.com/awslabs/aws-data-wrangler/tree/dev-1.0.0/tutorials>`_ or the `list of functionalities <https://aws-data-wrangler.readthedocs.io/en/dev-1.0.0/api.html>`_.
+Check our `tutorials <https://github.com/awslabs/aws-data-wrangler/tree/latest/tutorials>`_ or the `list of functionalities <https://aws-data-wrangler.readthedocs.io/en/latest/api.html>`_.

requirements-dev.txt

@@ -14,7 +14,7 @@ awscli~=1.18.39
 cfn-lint~=0.29.4
 twine~=3.1.1
 wheel~=0.34.2
-sphinx~=3.0.0
+sphinx~=3.0.1
 sphinx_bootstrap_theme~=0.7.1
 jupyter~=1.0.0
 jupyterlab~=2.1.0

requirements.txt

@@ -1,9 +1,10 @@
 numpy~=1.18.2
 pandas~=1.0.3
 pyarrow~=0.16.0
-boto3>=1.10.34
+boto3>=1.12.22
+botocore>=1.15.22
 s3fs~=0.4.2
 psycopg2-binary~=2.8.5
 pymysql~=0.9.3
-SQLAlchemy==1.3.15
+SQLAlchemy==1.3.13
 sqlalchemy-redshift~=0.7.7

setup.cfg

@@ -1,5 +1,2 @@
-[bdist_wheel]
-python-tag = glue
-
 [metadata]
 license_file = LICENSE