Release v1.2.0 beta.1 (#137)

* Initial collection commit for v0.0.1 (#1)

* hotfix readme url and license syntax (#4)

* Corrected relative MD url(s), zos dependency; updated galaxy version
* Updated playbook README version minor value to v.0.0.2

* added arg parser to module_utils (#3)

* Hotfix/122/update readme (#5)

* update README.md to reflect collection
* hot fix for zos_data_set existence checking and validation
* update README.md version tag links to v.0.0.3
* Updates to README.md that include fixing a malformed link
* Remove dependency list from modules
Dependencies will be stated on the collections README.md

* update type in galaxy.yml tags.

* Update and merge master versions  to dev branch (#12)

* Initial collection commit for v0.0.1 (#1) (#2)

* hotfix readme url and license syntax (#4)
* Corrected relative MD url(s), zos dependency; updated galaxy version
* Updated playbook README version minor value to v.0.0.2
* Hotfix/122/update readme (#5)
* update README.md to reflect collection
* hot fix for zos_data_set existence checking and validation
* update README.md version tag links to v.0.0.3
* Updates to README.md that include fixing a malformed link
* Remove dependency list from modules
Dependencies will be stated on the collections README.md

* Update content to reflect latest version (#10)
* Update content to reflect version 0.0.4
* Updated docs to use the master branch over the tagged version

* Update colleciton with functional test cases (#9)
* Added functional test cases and refactored existing test cases
* Updated copyright and license to functional test cases
* Test updates for validity with public repo

* refactor and patch job output checking (#6)
* refactor and patch job output checking
* updated documentation, output formatting, and added RC validation
* updated response information
* updated parsing of abend codes
* updated return example formatting

* Removing unused test directories

* Update README-ZOAU.md

* Bugfix/remove merge message artifacts (#13)

* removed artifacts
* removed artifacts
* added back .gitignore
* removed stragglers, updated gitignore

* Updated job output for improved DD handling (#14)

 Fixes the job summary output being returned when return_output=false.

* Security module updates (#15)

* patched security holes
* updated test cases
* removed unnecessary import
* updated execution bits

* Update modules and documentation corrections (#17)

* update documentation, remove unused imports
* updated requirements
* updated module imports, formatting, and spelling
* doc update and formatting update

* z/OS Ping Module Feature  Delivered (#18)

* added zos_ping, test, and test fixture update
* Description completeness
* Grammar completeness
* Pulled repetitive code into functions
* updated module docs
* Minor spelling corrections

* Repo updates to satisfy ansible-test sanity testing requirements (#19)

* update formatting
* formatting changes to appease ansible-test
* update tests
* fix string format use and quotes
* update README
* remove trailing whitespace
* update group_vars trailing spaces
* remove shebang in attempt to fix encoding issue
* add back shebang lines

* Added security updates and test cases (#20)

* add new types for parser, update arg handling zos_job_submit
* updated module argument parsing and security tests
* uncommented security tests
* updated README to reflect use of MIT license
* update hash function
* test and formatting updates
* updated documentation to pass ansible-test
* switch name back to normal.py, add appropriate ignores
* update ignore files
* added ignore for GPL
* updated README, .gitignore and removed MIT code
* remove broken tests
* increase robustness of zos_job_query
* make job query even more robust

* Webdocs content for IBM z/OS Core Collection (#21)

* Initial commit for reStructuredText content
* Updates and clean up to the template
* Updated grammar
* Grammar edit
* Grammar fixes
* Text updates
* Updates to restructured text, fixed link, renamed collection
* Fix ibm_zos_core bold
* Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters)
* ansible doc changes to modules for better formatting

* remove original_message return parameter (#23)

* remove original_message return parameter
* Fix reported linting errors from PR
* Fix reported linting errors from PR
* Fix reported linting errors from PR for zos_job_query.py

* Module document updates, Makefile changes and jinja support for defau… (#24)

* Module document updates, Makefile changes and jinja support for default:int
* Doc updates per linting rules

* Update mmodule doc and remove md files (#25)

* Removed all MD files from docs directory
* Remove README.md , its replaced by README.rst
* Fix bolding for ibm_zos_core
* Fix highlight for ibm_zos_core
* Updated index.rst to correspond to the README.md updates
* Updated README.rst to point to webdocs
* Updated README.rst webdocs URL
* Update the modules to their ansible-doc

* Documentation updates (#26)

* Updated galaxy.yml to reflect latest version 1.0
* Documentation copyright updates and formatting
* Documentation udpates

* Update z/OS data set module and add catalog functionality (#27)

* add VTOC shared utility
* zos_data_set catalog/uncatalog working for non vsam
* add VTOC shared utility
* zos_data_set catalog/uncatalog working for non vsam
* catalog for VSAM working, bugs fixed, tests added
* satisfy ansible-test requirements, minor arg validation updates
* removed unit tests
* raise min . ansible requirement due to security vulnerability
* added documentation to VTOC

* Update product name in webdocs (#28)

* Update product name in webdocs
* Fix trailing newline
* Document updates include some template bugs

* Integraded document review comments (#29)

* Integrated document review comments
* Incorrect break for bulleted line

* Bug in Jinja template where property would tab to much (#31)

* Updates to documentation and galaxy configuration (#32)

* Fix to README.rst, duplicate wording (#34)

* Remove duplicate action plugins link (#35)

* testing readme rendering (#36)

Ported the README.rst file to README.md to meet collection requirements.

* Corrected README to point to MD, and long tag removed (#39)

* Add two additional arg types to BetterArgHandler (#44)

* Add two additional arg types
* Remove os from os.path.isabs

Co-authored-by: Asif Mahmud <[email protected]>

* Data Set Utility for Common Data Set Related Tasks (#48)

* DataSetUtils initial commit
* LISTDS and LISTCAT output processing
* Raise exception when data set is open for write
* Fix LISTDS regex
* Fix indentation
* Add USS path support
* Fix linting errors
Co-authored-by: Asif Mahmud <[email protected]>

* Bugfix/832/fix non vsam non sms existence checking zos data set (#49)

* updated catalog handling for non-vsam, non-sms
* updated some method comment structures

* bugfix/822/determine-data-set-type (#50)

* Use mvscmd CLI instead of Python API
* Fix linting errors
* Remove unused variable

Co-authored-by: Asif Mahmud <[email protected]>

* Extend dynamic function support to the better arg parser (#51)

* Dynamic argument support for dynamic functions.
* improve default handling for reusability of BetterArgParser object.
* add test cases for dynamic argument support
* removed print statements from test cases

* Updates to data-set-utils  exceptions (#52)

* Avoid raising exception for uncatalogued data sets
* verify listcat output for uncatalogued datasets

Co-authored-by: Asif Mahmud <[email protected]>

* Removed return block from documentation (#53)

Removed message and changed, empty return block now

* Initial commit for module zos_operator (#45)

* Initial commit for moduel zos_operator
* Linting updates
* Linting corrections v2
* Linting corrections v3
* Fixed the collection dependency name
* adding various command test, invalid test
* fix lint errors
* fix lint err
* Corrected License, updated messages and renamed functional tests
* Update doc and return type
* Documentation update
Co-authored-by: xiaoping <[email protected]>

* Fixed PATH  (#55)

* Update/973/add new supported data set characters betterargparser (#56)

* updated valid data set types builtin type name fix

* added test cases for data set types

* Bugfix when checking if data set exists (#57)

* Return False instead of None when data set doesn't exist

* Test for membership should ne 'not in'

Co-authored-by: Asif Mahmud <[email protected]>

* Committing module  zos_operator_action_query (#43)

* Reviewd changes for feature zos_operator_action_query.py

* Fix linting errors reported in PR

* Linting corrections

* fix the syntax error

* fix lint errors

* fix lint error

* fix lint error

* fix lint error

* fix function test

* Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic

* Update author tag

* Update the collection reference

* add function test(need revise later) & doc update

* updated regex and value capturing

* Initial commit for module zos_operator (#45)

* Initial commit for moduel zos_operator
* Linting updates
* Linting corrections v2
* Linting corrections v3
* Fixed the collection dependency name
* adding various command test, invalid test
* fix lint errors
* fix lint err
* Corrected License, updated messages and renamed functional tests
* Update doc and return type
* Documentation update
Co-authored-by: xiaoping <[email protected]>

* updated docs, made environment agnostic test cases

* added support for "*" and fixed splat handling

* updated test cases

* updated regex matching method, updates tests

* Update short description

Co-authored-by: xiaoping <[email protected]>
Co-authored-by: Blake Becker <[email protected]>

* Check for data set member existence (#58)

* Check for data set member existence

* Resolve linting issue

Co-authored-by: Asif Mahmud <[email protected]>

* Bugfix for job name capture group (#60)

* Better ZOAU import missing messages (#61)

* Better ZOAU import missing messages
* added missing license and imports

* Changed behavior on failure for zosoperator_action_query (#63)

* changed behavior on failure
* update tests for new expected output

* Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64)

* updates to fix job submission as other user

* updated and optimized job utility

* removed usage of ZOAU and updated jobs utility usage

* update job_output return structure

* improved error messages

* add job log cleanup for test cases

* Initial module zos_tso_command (#46)

* Linting corrections

* Update doc so html will generate

* Lintig updates

* Lintig updates v2

* Lintig updates v3

* Lintig updates v4

* Lintig updates v5

* Lintig updates v6

* Lintig updates v7

* Lintig updates v8

* Delete test_zos_tso_command_unit.py

Remove the unit test as it only mocks things.

* Documenation updates

* Update errro message

* zos-tso-command update

* linting update

* testcase linting updates

* linting update

* linting update

* linting update

* linting update

* linting updates

* linting update

* new authorized long command handling method

* linting update

* test cases update

* linting update

* linting update

* linting update

* tso-command new version

* tso-command linting update

* linting update

* linting update

* linting update'

* linting removal

* lint removal

* linting testcases

Co-authored-by: bjmaxy <[email protected]>
Co-authored-by: “bjmaxy” <[email protected]>

* Update module_utils with an encode utility (#59)

* initial commmit for encode_utils

* support for string encoding conversion

* Support MVS data set conversion and string conversion modification

* Recursively copy to PDS fix

* Linting correction

* Linting updates

* Linting updates 01

* fix ENOENT problem after shutil.move

* updates with zos_encode

* Merged branch and commit for zos_encode (#65)

* Merged branch and commit for zos_encode

* Lint error , trailing white space

* z/OS Fetch Module (#7)

* initial commit

* begin updating examples

* zos_fetch module

* return rc, stdout, stderr as separate return parameters on failure

* untrack non-fetch related files

* add rest of the files

* Hotfix/122/update readme (#5)

* update README.md to reflect collection
* hot fix for zos_data_set existence checking and validation
* update README.md version tag links to v.0.0.3
* Updates to README.md that include fixing a malformed link
* Remove dependency list from modules
Dependencies will be stated on the collections README.md

* update type in galaxy.yml tags.

* Update and merge master versions  to dev branch (#12)

* Initial collection commit for v0.0.1 (#1) (#2)

* hotfix readme url and license syntax (#4)
* Corrected relative MD url(s), zos dependency; updated galaxy version
* Updated playbook README version minor value to v.0.0.2
* Hotfix/122/update readme (#5)
* update README.md to reflect collection
* hot fix for zos_data_set existence checking and validation
* update README.md version tag links to v.0.0.3
* Updates to README.md that include fixing a malformed link
* Remove dependency list from modules
Dependencies will be stated on the collections README.md

* Update content to reflect latest version (#10)
* Update content to reflect version 0.0.4
* Updated docs to use the master branch over the tagged version

* Update colleciton with functional test cases (#9)
* Added functional test cases and refactored existing test cases
* Updated copyright and license to functional test cases
* Test updates for validity with public repo

* refactor and patch job output checking (#6)
* refactor and patch job output checking
* updated documentation, output formatting, and added RC validation
* updated response information
* updated parsing of abend codes
* updated return example formatting

* Removing unused test directories

* Update README-ZOAU.md

* Bugfix/remove merge message artifacts (#13)

* removed artifacts
* removed artifacts
* added back .gitignore
* removed stragglers, updated gitignore

* added unit and functional tests

* check file existence when LISTDS returns invalid dataset name

* added 'wait_s' parameter

* clarified description of encoding parameter

* make use_qaulifer false by default

* remove is_uss and os_vsam parameters

* remove data_set directory

* remove is_uss parameter from examples

* message return block

* Fix YAML formatting

* return code formatting

* remove fail_on_missing from action plugin

* fail if USS file does not exist

* data sets are assumed to be cataloged

* instead of entire dict, pass individual values

* redundant code when fetching data set member

* rename transfer_from_uss to fetch_non_partitioned_data_set

* remove test cases for uncataloged datasets

* import naming conflict

* add __future__ imports

* add __future__ imports

* assuming all data sets are cataloged

* assuming all data sets are cataloged

* fix formatting issues

* fix formatting issues

* remove blank lines with whitespaces

* change example member name data to DATA for accuracy.

* continuation line unindented for visual indent

* conform to PEP8 standard

* avoid automatic field numbering for format

* specify python interpreter

* extract only data set name if member present

* Remove newline character from error message

* Update RETURN block with sample message

* Remove blank line from return block

* Update version_added to correspond to Ansible version

* Check if temp datasets are being created successfully

* Datasets.read() returns None for empty data sets

* Use BetterArgParser to parse module args

* Added parameter types

* Remove trailing whitespaces

* Avoid passing extra args from action plugin

* Remove unused imports

* Ignore lack of GPLv3 license complaint

* catch ValueError thrown ny BetterArgParser

* Blank line contains whitespace

* Reduce line lengths to 80

* Add white space around string addition

* Checking volume type no longer required

* Check if src or dest is provided first

* Fix indentation

* Indent multiline string for better readability

* Use DataSetUtils to get data set type

* Use DataSetUtils to determine data set existence

* Remove unit tests

* Fix linting

* Add test case for empty data sets

* default params for _fail_json

* Test case for missing source

* Test case for insufficient write permission

* Test case for insufficient desgt write permission

* Catch exceptions while executing module

* Use updated DataSetutils constructor

* Updated functional tests

* Fix linting errors

* Use ansible fetch module to fetch USS files

* Checking for USS type is unnecessary

* Use sha256 instead of sha1 for checksum validation

* Remove message return block

* Missing uss file test case

* Minor cleanup

* Add stdout_lines and stderr_lines to module response

* Update action plugin with stderr_lines and stdout_lines

* YAML syntax issue

* Corrected spelling errors

* Inaccurate description of flat

* No need to wrap error messages returned by DataSetUtils

* Update doc regarding HFS and ZFS support

* Handle encoding for USS files

* Add encoding to doc

* Calculate temp file size for VSAM storage

* Preserve encoding using SFTP

* Resolve PEP8 issues

* Check for data set member existence

* raw binary not supported in PY2

* Fix linting issues

* Change doc to reflect USS absolute path

* Prepend dataset hlq before parameter verification

* src not declared before being referenced

* Add test case using use_qualifier

* Updated doc link for fetch L(..) and suboptions

* Use tempfile.mkstemp() for temp data storage

* Remove default for encoding

* default encoding values are None

* Create destination dirs when flat is false

* Add test case for flat hostname

* Fix spelling

* exist_ok is not valid keyword for PY2

* Detect read-only file system

* Handle sftp errors

* Capture module_stderr in case of module failure

* Ensure dest is a directory for PDS transfer

* Ensure dest is a directory except when fetching a member

* Fix data set member destination path

* Default values for encoding is not needed

* Remove unused imports

* Update 'msg' description

* Clean up temp files in case of module failure

* Fix linting errors

Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: ddimatos <[email protected]>
Co-authored-by: Bryant Panyarachun <[email protected]>
Co-authored-by: Blake Becker <[email protected]>

* enhanced zoau import error message (#66)

* enhanced zoau import error message

* satisfy pep8 witespace requirement

* Added new playbook for latest module collection and a filter

* fixed linting issues, added license

* updated spacing to conform to formatting standards

* updated shell to command where possible

* Use encode util to convert encoding (#68)

Co-authored-by: Asif Mahmud <[email protected]>

* Update RST and MD doc files to reflect next release

* Update group vars to reflect zoau 1.0.2 new env vars

* Update encode short description to fit within the limits of doc length

* Update makefile for more generic use by others

* Update to fix RST lint error , short line

* Rename sub-dir from plugin to filter, doc error

* doc tree correction

* Update release notes to correspond to beta release tag

* Update sample playbooks with doc, configuration and new variables

* Upate docs to highlight users to review sample playbooks and JCL before running

* Update sample JCL to run on my systems with default values and notes

* Add supporting doc for behavioral inventory parameters and ansible.cfg

* Fix the md style URL in the index RST file

* Update short descriptions for modules

* Fix README.md link to IMS collection

* Fix README.md link to collection artifacts

* Linting updates

* Add an example that requires a command to be escaped (#71)

* refine backup_file & put it into encode_utils (#70)

* refine backup_file & put it into encode_utils

* correction for sanity test

* refactor for security and usability, fix test case

* move copy logic to utility

* refactor zos_encode, improve backup functionality

Co-authored-by: Blake Becker <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* Update galaxy.yml  to reflect the latest version and authors

* Fix import name change resulting from refactor

* updated usage of encode_utils to encode

* Documentation updates, corrections and alterations (#75)

* Documentation updates, corrections and alterations
* ID review-round 2
* Update the user description in the doc

Co-authored-by: RadhaV2020 <[email protected]>

* Content/81/zos tso command docreview (#73)

* ID review of zos-tso-commands
* plugins/modules/zos_fetch.py
* plugins/modules/zos_fetch.py
* plugins/modules/zos_fetch.py
* plugins/modules/zos_fetch.py
* Fix indentation

Co-authored-by: ddimatos <[email protected]>

* Fix extra space in doc

* fixed zos_ping action plugin issues (#77)

* Add  out-of-box awx support (#80)

* Added encoding supplemental doc (#78)

* Added encoding supplemental doc

* Removed references to modules not available and causing doc warnings

* Fix Inline literal start-string without end-string.

* docs/source/supplementary.rst

* docs/source/supplementary.rst

* docs/source/supplementary.rst

* docs/source/supplementary.rst

Co-authored-by: Radha Varadachari <[email protected]>

* Add binary support copying across USS and MVS

* Copy USS file in binary mode

* fixed bug and cleaned up code (#82)

* fixed bug and cleaned up code

* fixed linting issues

* Avoid specifying -B and -F for cp

* Copy MVS source to MVS dest

* Documentation web content updates (#84)

* First round of review of static docs

* docs/source/installation.rst

* Update block to rendor with syntax highlights

Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type.

* Incorporated Demetri's review comments

Co-authored-by: Radha Varadachari <[email protected]>

* Replace only the first occurance of 'rec'

* Minor updates in install,quickstart, and req pages (#86)

* Updated make file with comments and recover from failure

* Update Backup util (mvs_file_backup) (#85)

* Alternative mvs_file_backup

* fix backup logic, add encode backup tests

* add mvs backup tests, util fixes

* fix test for new logic path

Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: Blake Becker <[email protected]>

* Udpate the restructured text to reflect version release (#89)

* minor fixes and additional tests for zos_encode (#88)

* removed Jobs.submit usage due to invalid response (#92)

* Update to galaxy.yml to exclude files

* Update makefile to support rexx written modules

* Generate and push zos_ping rst doc

* Corrected clean up of a directory in makefile

* Fixes spacing in RETURN values and formatting (#97)

* Validate bk_dsn after assigning a value

* Return backup path

* Fix alloc command

* Remove trailing whitespace

* Fix job utility to work on all z/OS systems (#99)

* Shinx and sphinx-versioning support and updates (#100)

* Updated Makefile to support sphinx-versioning and renamed some of the operations

* Update configuration to support sphinx-versioning and update options for sphinx

* Update configuration with linting corrections

* Correct and update the installation of a pre-release (#102)

* Correct and update the installation of a pre-release

* minor fixes to docs

Co-authored-by: Radha Varadachari <[email protected]>

* Update data set utils (#104)

* Update data set utils

* Update function signatures

* Module parameter not needed for encode util

Co-authored-by: Asif Mahmud <[email protected]>

* Change default encoding from UTF-8 to ISO8859-1

* Fetching non-existent VSAM

* Update doc for encoding parameter

* Update encoding to handle different charsets

* Remove encoding default from argument spec

* encoding validation and doc update

* Fix typo

* Bugfix/case conversion fix zos tso command (#108)

* fixed issues with commands forced to uppercase

* add support for single or multi commands, added relevant test cases

* updated alias doc

* Added index file to support shared doc site (#109)

* Remove module parameter from EncodeUtils constructor

* fix issue when no default encoding specified (#112)

* fix issue when no default encoding specified

* fix local job submission

* Feature/26/ansible zos copy module (#105)

* initial commit

* begin updating examples

* default value 'yes/no' -> 'true/false'

* updated documentation

* copy to seq data set

* copy local to seq data set

* add data set size to module response

* functional test cases for copy

* test case for sequential data set

* clean up copy to ps function

* copy to pdse

* assuming all data sets are cataloged

* copy to PDS

* Copy to VSAM

* Copy to PDS(E) member

* Actual directory size instead of max file size

* Copy from remote source

* Update copy

* Reduce line lengths to less than 80

* Use mvscmd cli

* remote pdse copy

* IEBCOPY to copy data sets

* saving changes

* Create remote data set if it doesn't exist

* Reorganize remote copy transfer

* Update return params

* Avoid using global module

* Remove unused imports

* Copy to USS

* Clean up in case of module failure

* Back up USS file

* Copy content to remote file

* Copy to sequential data set

* Copy local dir to PDS

* Copy remote PDS to PDS

* Update backup file

* USS dest not accessible if not present

* Explicitly tag destination file

* Changed file mode, group and owner

* Change common file args for remote src

* Change file common args after copying to dest

* Preserve local file mode

* Convert encoding for remote src

* Added comments

* Use backup util to back up files and data sets

* Add validate parameter

* Copy VSAM to VSAM

* Support for symlinks

* Copy PDS/PDSE member to USS

* Fix formatting

* Module should fail for empty VSAM

* Added function signatures for all test cases

* User better_arg_parser to determine member type

* Add binary support for copying to USS target

* Copy to PS in binary mode

* Correct object instantiations

* Import copy util

* Remove write_rc

* Resolve import issues

* Move vsam copy handler to main copy handler class

* Copy local dir to USS dir

* Infer member name when it isn't provided

* Added test cases for local and remote copy

* src_member evalutes to True

* Comments for extract_dsname()

* IEBCOPY fails for incompatible recfm

* Copy PDS member to PS

* Copy data set member to non-existent PS

* Copy local symlink

* Add test case for uss file backup

* Fix mvs file backup

* Functional tests complete

* Fix linting issues

* Remove trailing whitespace

* Ignore GPLv3 license

* Remove is_vsam parameter

* Fix license information

* Fail action plugin

* Add test cases to ensure module failure

* Remove unneded parameters

* Refactor to use updated data_set util

* I/O abend when copying PDSE member to non-existent PS

* Updated doc, comments, linting

* Fix linting issues

* Remove duplicate key from dic

* Avoid hard-coding tmp path name

* Change backup_path parameter to backup_file

* Copy to data set member partial

* Error converting encoding for local dir

* Fix tmp cleanup bug

* Updated test cases for backup_file

* plugins/modules/zos_copy.py

* Reactor code for better readability

* Fix pep8 issue

* Cleanup error on module failure

* plugins/modules/zos_copy.py

* Doc update mode parameter

* Use native shell instead of bash for functional testing

* Return invocation parameter

* Remove unnecessary imports

* plugins/modules/zos_copy.py

* Return backup_file on module failure

Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: Radha Varadachari <[email protected]>

* Feature/1089/zos lineinfile (#111)

* initial version for interface and options

* add backup option

* update lineinfile

* make golden path work

* fix the bugs so that method could support none vars

* refactor based on new dsed

* updated params and return

* Initial check in for backup feature support

* support all charsets; updated backup feature doc

* clean up; used better_arg_parser

* use latest data_set module_util

* change the returned string to be loadable by json.loads (BOF/EOF)

* uss test cases

* dataset test cases; create a dataset/file with content instead of being dependent on existing source

* uss test env set bug fix

* added UTF-8 in test cases; simplified dataset names

* use new dedicated ZOAU function for lineinfile

* added assert for expected strings

* encoding parameter from absent

* Linting updates from pipeline tests

* Linting updates and corrections

* Linting udpates continued

* Update sanity exemption file and linting corrections

* Doc error, indentation off, breaking rest of doc parse

* module arg and doc updates

* Formatting updates

* Further doc updates

* regexp alias fix

* updated docs; added comments; updated parameters

* fix sanity checks

* fix RETURN key type; lint error

* code review changes

* fixed linting issues

* removed incorrectly added choices for insertafter/insertbefore

* added negative test cases fo not supported datasets

* moved choices to description; fixed test cases lint issue

* changed the choices format in description

* add escape if string was quoted

* added test cases for escaped/unescaped quotation

* used 'src' for target location; moved the params checking before taking backup

* fixed indentation

* Doc edits for lineinfile, encode, and fetch

* remove relative import method

* removed the default values for insertafter/insertbefore

* renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO

* remove jcl tmp directory if it already exists

* run all test_*

Co-authored-by: xiaoping <[email protected]>
Co-authored-by: behnam-ibm <[email protected]>
Co-authored-by: ddimatos <[email protected]>
Co-authored-by: Radha Varadachari <[email protected]>
Co-authored-by: Blake Becker <[email protected]>

* Content/1067/community doc (#113)

* Correct doc link to IMS collection to point to Galaxy

* Added testing doc to community guidlines

* Clean/delete accidental file

* Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module

* Added better arg parse to new dev addition to community guides

* Fixes broken site anchor to the YAML configuration

* Fix MD style Link

* Remove additional MD style links

* Fix mailformed RST link

* Fix more RST links

* Python syntax fixes in samples

* Update betterArgParse to BetterArgParser

* Expand pipelining=true doc

* Correct Unicode quotes to ASCII quotes

* Statement of direction on zoau and python

* doc review of community guidelines

* Doc review of Community guidelines

* Fix malformed RST table

* Update group vars requirements needed by IBM Pypthon

* Update doc with var settings and sample all.yml

* Update environ vars

* Added environ LANG doc

* Added doc to clarify shell support

* Added advanced group_vars sample and doc

* Update docs to reflect IBM Python announcement

Co-authored-by: Radha Varadachari <[email protected]>

* Remove unnecessary description of PDSE transfer

* Update version tag in doc generation conf

* Add missing link reference

* Doc review (#118)

Co-authored-by: Radha Varadachari <[email protected]>

* Update IBM Python download link

* Add system info to job query (#127)

* Add system info to job query

* Change sysname to system for job query

Co-authored-by: Asif Mahmud <[email protected]>

* Close file descriptor (#128)

Co-authored-by: Asif Mahmud <[email protected]>

* Feature/2298/zos raw (#120)

* add utilities for zos_raw

* updated DD statement to better handle concatenations

* update parameter names

* fix concat naming

* refactor of zos_data_set, add new features

* updated examples

* added steplib dd type

* fix steplib data set

* update valid concat members

* fix datasetdefinition for steplibs

* update dd statement types

* argument parsing working in zos_raw

* module argument parsing updates

* updated valid arguments for consistency

* update tests for new zos_data_set format

* zos_raw module working

* module doc updates

* updated function documentation

* backup, reuse and replace working

* updated documentation

* cleaned up wrapper classes

* updated existence checking, function doc updates

* spelling fixes, added new parameters for dd_unix

* add list of string support for StdinDefinition

* properly handle space units

* initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions

* added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options

* update example playbooks to account for new zos_data_set arguments

* added initial functional tests for unix file dd statements

* added access group support, fixed status group

* added remaining tests for dd_unix and dd_dummy

* fixed bug on dd_concat contents or backup returned

* add dd_concat test cases

* update parms to parm to match jcl naming

* added execution tests for zos_raw

* add final test cases

* add special type functions for ksds related params

* update import handling for special cases

* add VIO DD support

* uncomment test cases

* fix basic linting issues, default arguments

* add support for directory blocks

* fix some linting issues

* Revert "fix some linting issues"

This reverts commit 44d0e14.

* linting fixes

* module doc and linting updates

* additional doc fixes for linting sanity tests

* added examples

* added catch for nonexistent deletion

* updated docs for proper formatting

* update ignore files for new module

* added dd_output support, standardized dd names

* removed unnecessary import wrappers

* improved validation for encoding types.

* added new example with dd_output type

* Merge branch 'origin/dev'

* Revert "Merge branch 'origin/dev'"

This reverts commit 1befb8e.

* fixes for python 2 controller support

* modify encode utils import

* add utf-8 encoding tag to unit tests

* improve check mode support

* fix mock of AnsibleModule for encoding check cases

* fix unit test formatting for pep8

* fix test case helpers

* comment out failing test case unrelated to zos_raw

* update encoding conversion handling

* update rexx variables to capture correct environment variables

* update naming based on discussion

* rename test case files, update test cases

* update default for output data set

* update test ignore files for rename

* doc edits (#133)

* doc edits

* fix minor pep8 issues

Co-authored-by: Blake Becker <[email protected]>

* add python 2 support

* remove redundant code comments

* add temp data set support, backwards compatibility fix with old interface, update docs and tests

* update documentation for return_content

* minor doc changes for clarity

* improve documentation clarity

* fix incorrect syntax for option name

Co-authored-by: RadhaV2020 <[email protected]>

* Update configuration vars in doc and yml (#131)

* remove ZOAU Python API deps for zos_operator (#132)

* remove ZOAU deps which break in 1.0.3

* add back shebang

* remove unsafe shell usage

* upload helper wrapper and fix usage (#135)

* upload helper wrapper and fix usage

* fix whitespace issue

* fix unit tests for raw

* zos_copy Update (#134)

* pathlib not suported in py 2.7

* Doc update

* Memebers being deleted during pds backup

* Migrate some utils to module utils

* Generate a temp member name

* Doc update

* Use ZOAU API to delete members

* Fix PEP8 issues

* Delete members from existing PDS

* Check for member delete return code

* Follow symlink for tmp

* Update cleanup process

* Ignore backup request for empty PDS

* Make 'is_empty()' generic

* local dir endswith forward slash

* Avoid allocating dest during PDS copy

* Escape special characters in member name

* Escape special character for src and dest

* Add 'model_ds' parameter

* Add model_ds support for PS

* Fix allocation issues for PS

* Specify BLKSIZE when allocating model

* Check for dest existence from action plugin

* Doc update

* data set util update

* Change force default on action plugin

* Remove _fail_action() from action plugin

* Update vtoc usage

* Fix bug in action plugin

* Use DataSet API for zos_encode

* Update data set API for fetch

Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: Blake Becker <[email protected]>

* Content/2516/release doc update (#136)

* Update the version in galaxy.yml

* edits to support doc gen

* Updated for doc gen

* updated all module doc for release

* Update template to upper case boolean when no default present

* update raw module doc to fomat as the indentation intended

* updated module RST

* add details to module enhancements

* Update message around Rocket's python

* Update Rocket Python Message

* Reference Rocket Python configuration in requirements

* Updated doc to reflect the latest dependencies

* upload helper wrapper and fix usage (#135)

* upload helper wrapper and fix usage

* fix whitespace issue

* fix unit tests for raw

* zos_copy Update (#134)

* pathlib not suported in py 2.7

* Doc update

* Memebers being deleted during pds backup

* Migrate some utils to module utils

* Generate a temp member name

* Doc update

* Use ZOAU API to delete members

* Fix PEP8 issues

* Delete members from existing PDS

* Check for member delete return code

* Follow symlink for tmp

* Update cleanup process

* Ignore backup request for empty PDS

* Make 'is_empty()' generic

* local dir endswith forward slash

* Avoid allocating dest during PDS copy

* Escape special characters in member name

* Escape special character for src and dest

* Add 'model_ds' parameter

* Add model_ds support for PS

* Fix allocation issues for PS

* Specify BLKSIZE when allocating model

* Check for dest existence from action plugin

* Doc update

* data set util update

* Change force default on action plugin

* Remove _fail_action() from action plugin

* Update vtoc usage

* Fix bug in action plugin

* Use DataSet API for zos_encode

* Update data set API for fetch

Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: Blake Becker <[email protected]>

* Remove see also reference to external collections

* updated zos_copy module RST

* Sample playbook for latest version

* Updated ZOAU PTF information with the latest PTF UI70435

* Updates to raw doc to handle the error - Definition list ends without a blank line

* Update copy module docs

* Update and add clarity around Rocket Python

* Requirements order change

Co-authored-by: Blake Becker <[email protected]>
Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: Asif Mahmud <[email protected]>

* Update conf for v1.2.0-beta.1

* Fix  E231: missing whitespace

Co-authored-by: Blake Becker <[email protected]>
Co-authored-by: Bryant Panyarachun <[email protected]>
Co-authored-by: Jake Jackson <[email protected]>
Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: Asif Mahmud <[email protected]>
Co-authored-by: xiaoping <[email protected]>
Co-authored-by: bjmaxy <[email protected]>
Co-authored-by: “bjmaxy” <[email protected]>
Co-authored-by: zlbjlu <[email protected]>
Co-authored-by: RadhaV2020 <[email protected]>
Co-authored-by: Radha Varadachari <[email protected]>
Co-authored-by: balkajbaf <[email protected]>
Co-authored-by: behnam-ibm <[email protected]>

Author: 12 people

docs/Makefile

@@ -87,4 +87,3 @@ html-all:
 %: Makefile
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 	@echo "Completed HTML text generation, run 'make view-html'"
-

docs/source/community_guides_docs/zos_ansible_module_testing.rst

@@ -5,28 +5,59 @@
 Prerequisites
 =============
 
-The following prerequisites must be met before you develop and run test cases for z/OS Ansible modules.
+The following prerequisites must be met before you develop and run test cases
+for z/OS Ansible modules.
 
-Control Node
+Control node
 ------------
-* Latest Python 3.X (recommended)
-* Ansible 2.8 or later 
 
+* `Ansible version`_: 2.9 or later
+* `Python`_: 3 or later
+* `OpenSSH`_
 
-z/OS Target Host
-----------------
-* `Z Open Automation Utilities`_
-* Python 3 >= 3.6
+.. _Ansible version:
+   https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html
+.. _Python:
+   https://www.python.org/downloads/release/latest
+.. _OpenSSH:
+   https://www.openssh.com/
 
-.. _Z Open Automation Utilities:
-   https://www.ibm.com/support/knowledgecenter/en/SSKFYE_1.0.0/zoautil_overview.html
+
+Managed node
+------------
+
+* `IBM Open Enterprise Python for z/OS`_
+* z/OS `V2R3`_ or `later`_
+* `IBM Z Open Automation Utilities`_ (ZOAU)
+* The z/OS® shell
+
+   * IBM z/OS collections are dependent on specific versions of ZOAU. When
+     configuring the dependencies, refer to the collections requirements
+     and dependencies to ensure proper execution.
+* `z/OS OpenSSH`_
+
+
+.. _IBM Open Enterprise Python for z/OS:
+   http://www.ibm.com/products/open-enterprise-python-zos
+
+.. _V2R3:
+   https://www.ibm.com/support/knowledgecenter/SSLTBW_2.3.0/com.ibm.zos.v2r3/en/homepage.html
+
+.. _later:
+   https://www.ibm.com/support/knowledgecenter/SSLTBW
+
+.. _IBM Z Open Automation Utilities:
+   https://www.ibm.com/support/knowledgecenter/en/SSKFYE
+
+.. _z/OS OpenSSH:
+   https://www.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.e0za100/ch1openssh.htm
 
 
 Configuration and Dependencies
 ==============================
 
 This section explains how to configure the environment and which dependencies to
-install to run both the functional and unti test cases.
+install to run both the functional and unit test cases.
 
 Install dependencies
 --------------------
@@ -94,39 +125,22 @@ Example YAML configuration:
 
 .. code-block:: yaml
 
-   host: tivlp02.svl.ibm.com
-   user: bbecker
-   python_path: /var/python27/python27/bin/python
+   host: some.host.name
+   user: myuser
+   python_path: /usr/lpp/IBM/cyp/v3r8/pyz
    environment:
-       INSTALL_PYTHON: /var/python27
-       RELEASE_NAME: python-2017-04-12
-       RELEASE_TYPE: py27
-       RELEASE_DIR: /var/python27
-       PKGS_BASE: /var/python27/pkgs
-       PYTHON_ENV: python27
-       PYTHON_HOME: /var/python27/python27
-       PYTHON: /var/python27/python27/bin
-       LIBPATH: /var/python27/python27/lib:${LIBPATH}
-       FFI_LIB: /var/python27/python27/lib/ffi
-       TERMINFO: /var/python27/python27/share/terminfo
-       PKG_CONFIG_PATH: /var/python27/python27/share/pkgconfig:/var/python27/python27/lib/pkgconfig
-       CURL_CA_BUNDLE: /var/python27/python27/etc/ssl/cacert.pem
-       # environment variables for mvsutils/mvscmd
-       _: /hsstools/bin/env
-       _BPXK_AUTOCVT: ON
-       # * ensure quotations are provided correctly where needed!!
-       _CEE_RUNOPTS: '"FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"'
-       _TAG_REDIR_ERR: txt
-       _TAG_REDIR_IN: txt
-       _TAG_REDIR_OUT: txt
-       TOOLS_ROOT: /hsstools
-       GIT_SHELL: /hsstools/bin/bash
-       GIT_EXEC_PATH: /hsstools/git-2.14.4/libexec/git-core
-       GIT_TEMPLATE_DIR: /hsstools/git-2.14.4/share/git-core/templates
-       PATH: /hsstools/git-2.14.4/bin:/bin:/var/bin:/usr/lpp/java/J8.0/bin:/var/python27/python-2017-04-12-py27/python27/bin
-       # Java environment
-       JAVA_HOME: /usr/lpp/java/J8.0_64
-
+     _BPXK_AUTOCVT: "ON"
+     _CEE_RUNOPTS: "'FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)'"
+     _TAG_REDIR_ERR: txt
+     _TAG_REDIR_IN: txt
+     _TAG_REDIR_OUT: txt
+     LANG: C
+     ZOAU_ROOT: "/usr/lpp/IBM/zoautil"
+     ZOAU_HOME: "/usr/lpp/IBM/zoautil"
+     LIBPATH: "/usr/lpp/IBM/zoautil/lib:/usr/lpp/IBM/cyp/v3r8/pyz/lib:/lib:/usr/lib:."
+     ZOAUTIL_DIR: "/usr/lpp/IBM/zoautil"
+     PYTHONPATH: "/usr/lpp/IBM/zoautil/lib"
+     PATH: "/usr/lpp/IBM/zoautil/bin:/usr/lpp/IBM/cyp/v3r8/pyz/bin:/bin:/var/bin"
 
 Run Functional and Unit Tests
 =============================

docs/source/conf.py

@@ -147,7 +147,7 @@
 # White list which Git tags documentation will be generated and linked into the
 # version selection box. This is currently a manual selection, until more
 # versions are released, there are no regular expressions used.
-scv_whitelist_tags = ('v1.0.0', 'v1.1.0')
+scv_whitelist_tags = ('v1.0.0', 'v1.1.0', 'v1.2.0-beta.1')
 
 # Sort versions by one or more values. Valid values are semver, alpha, and time.
 # Semantic is referred to as 'semver', this would ensure our latest VRM is

docs/source/installation.rst

@@ -52,7 +52,6 @@ Here are a few examples of installing the **IBM z/OS core collection**:
    $ ansible-galaxy collection install ibm.ibm_zos_core
    $ ansible-galaxy collection install -f ibm.ibm_zos_core
    $ ansible-galaxy collection install --force ibm.ibm_zos_core
-   
 
 The collection installation progress will be output to the console. Note the
 location of the installation so that you can review other content included with