From 345d77c0e3c69dbee096aab082284ad0ceacdd1b Mon Sep 17 00:00:00 2001 From: Stefan Maerz Date: Mon, 29 Jun 2020 11:43:50 -0400 Subject: [PATCH 1/5] Adding strawman afw policy guide. --- accounts/afw_policy_guide.rst | 1080 +++++++++++++++++++++++++++++++++ 1 file changed, 1080 insertions(+) create mode 100644 accounts/afw_policy_guide.rst diff --git a/accounts/afw_policy_guide.rst b/accounts/afw_policy_guide.rst new file mode 100644 index 00000000..2ff2d261 --- /dev/null +++ b/accounts/afw_policy_guide.rst @@ -0,0 +1,1080 @@ +**************************** +OLCF Policy Guides +**************************** + +OLCF Acknowledgement +==================== + +Users should acknowledge the OLCF in all publications and presentations +that speak to work performed on OLCF resources: + + This research used resources of the Oak Ridge Leadership Computing + Facility at the Oak Ridge National Laboratory, which is supported by the + Office of Science of the U.S. Department of Energy under Contract No. + DE-AC05-00OR22725. + + +Computing Policy +================ + +.. note:: + This details an official policy of the OLCF, and must be + agreed to by the following persons as a condition of access to and use + of OLCF computational resources: + + - Principal Investigators (Non-Profit) + - Principal Investigators (Industry) + - All Users + + **Title:** Computing Policy **Version:** 12.10 + +Computer Use +------------ + +Computers, software, and communications systems provided by the OLCF are +to be used for work associated with and within the scope of the approved +project. The use of OLCF resources for personal or non-work-related +activities is prohibited. All computers, networks, E-mail, and storage +systems are property of the United States Government. Any misuse or +unauthorized access is prohibited, and is subject to criminal and civil +penalties. OLCF systems are provided to our users without any warranty. +OLCF will not be held liable in the event of any system failure or data +loss or corruption for any reason including, but not limited to: +negligence, malicious action, accidental loss, software errors, hardware +failures, network losses, or inadequate configuration of any computing +resource or ancillary system. + +Data Use +-------- + +Prohibited Data +^^^^^^^^^^^^^^^ + +The OLCF computer systems are operated as research systems and only +contain data related to scientific research and do not contain +personally identifiable information (data that falls under the Privacy +Act of 1974 5U.S.C. 552a). Use of OLCF resources to store, manipulate, +or remotely access any national security information is strictly +prohibited. This includes, but is not limited to: classified +information, unclassified controlled nuclear information (UCNI), naval +nuclear propulsion information (NNPI), the design or development of +nuclear, biological, or chemical weapons or any weapons of mass +destruction. Authors/generators/owners of information are responsible +for its correct categorization as sensitive or non-sensitive. Owners of +sensitive information are responsible for its secure handling, +transmission, processing, storage, and disposal on OLCF systems. +Principal investigators, users, or project delegates that use OLCF +resources, or are responsible for overseeing projects that use OLCF +resources, are strictly responsible for knowing whether their project +generates any of these prohibited data types or information that falls +under Export Control. For questions, contact help@nccs.gov. + +Confidentiality, Integrity, and Availability +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The OLCF systems provide protections to maintain the confidentiality, +integrity, and availability of user data. Measures include the +availability of file permissions, archival systems with access control +lists, and parity and CRC checks on data paths and files. It is the +user’s responsibility to set access controls appropriately for the data. +In the event of system failure or malicious actions, the OLCF makes no +guarantee against loss of data or that a user’s data can be accessed, +changed, or deleted by another individual. It is the user’s +responsibility to insure the appropriate level of backup and integrity +checks on critical data and programs. + +Data Modification/Destruction +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users are prohibited from taking unauthorized actions to intentionally +modify or delete information or programs. + +Data Retention +^^^^^^^^^^^^^^ + +The OLCF reserves the right to remove any data at any time and/or +transfer data to other users working on the same or similar project once +a user account is deleted or a person no longer has a business +association with the OLCF. After a sensitive project has ended or has +been terminated, all data related to the project must be purged from all +OLCF computing resources within 30 days. + +Software Use +------------ + +All software used on OLCF computers must be appropriately acquired and +used according to the appropriate software license agreement. +Possession, use, or transmission of illegally obtained software is +prohibited. Likewise, users shall not copy, store, or transfer +copyrighted software, except as permitted by the owner of the copyright. +Only export-controlled codes approved by the Export Control Office may +be run by parties with sensitive data agreements. + +Malicious Software +^^^^^^^^^^^^^^^^^^ + +Users must not intentionally introduce or use malicious software such as +computer viruses, Trojan horses, or worms. + +Reconstruction of Information or Software +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users are not allowed to reconstruct information or software for which +they are not authorized. This includes but is not limited to any reverse +engineering of copyrighted software or firmware present on OLCF +computing resources. + +User Accountability +------------------- + +Users are accountable for their actions and may be held accountable to +applicable administrative or legal sanctions. + +Monitoring and Privacy +^^^^^^^^^^^^^^^^^^^^^^ + +Users are advised that there is no expectation of privacy of your +activities on any system that is owned by, leased or operated by +UT-Battelle on behalf of the U.S. Department of Energy (DOE). The +Company retains the right to monitor all activities on these systems, to +access any computer files or electronic mail messages, and to disclose +all or part of information gained to authorized individuals or +investigative agencies, all without prior notice to, or consent from, +any user, sender, or addressee. This access to information or a system +by an authorized individual or investigative agency is in effect during +the period of your access to information on a DOE computer and for a +period of three years thereafter. OLCF personnel and users are required +to address, safeguard against, and report misuse, abuse and criminal +activities. Misuse of OLCF resources can lead to temporary or permanent +disabling of accounts, loss of DOE allocations, and administrative or +legal actions. Users who have not accessed a OLCF computing resource in +at least 6 months will be disabled. They will need to reapply to regain +access to their account. All users must reapply annually. + +Authentication and Authorization +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +All users are required to use a one-time password for authentication. +Tokens will be distributed to OLCF users. Users will be required to +create a Personal Identification Number (PIN). This is used in +conjunction with a generated token code as part of a two-factor +authentication implementation. Accounts on the OLCF machines are for the +exclusive use of the individual user named in the account application. +Users should not share accounts or tokens with anyone. If evidence is +found that more than one person is using an account, that account will +be disabled immediately. Users are not to attempt to receive unintended +messages or access information by some unauthorized means, such as +imitating another system, impersonating another user or other person, +misuse of legal user credentials (usernames, tokens, etc.), or by +causing some system component to function incorrectly. Users are +prohibited from changing or circumventing access controls to allow +themselves or others to perform actions outside their authorized +privileges. Users must notify the OLCF immediately when they become +aware that any of the accounts used to access OLCF have been +compromised. Users should inform the OLCF promptly of any changes in +their contact information (E-mail, phone, affiliation, etc.) Updates +should be sent to accounts@ccs.ornl.gov. + +Foreign National Access +^^^^^^^^^^^^^^^^^^^^^^^ + +Applicants who appear on a restricted foreign country listing in section +15 CFR 740.7 License Exceptions for Computers are denied access based on +US Foreign Policy. The countries cited are Cuba, Iran, North Korea, +Sudan, and Syria. Additionally, no work may be performed on OLCF +computers on behalf of foreign nationals from these countries. + +Denial of Service +^^^^^^^^^^^^^^^^^ + +Users may not deliberately interfere with other users accessing system +resources.   + +Data Management Policy +====================== + +.. note:: + This details an official policy of the OLCF, and must be + agreed to by the following persons as a condition of access to or use of + OLCF computational resources: + + - Principal Investigators (Non-Profit) + - Principal Investigators (Industry) + - All Users + + **Title:** Data Management Policy **Version:** 20.02 + +Introduction +------------ + +The OLCF provides a comprehensive suite of hardware and software +resources for the creation, manipulation, and retention of scientific +data. This document comprises guidelines for acceptable use of those +resources. It is an official policy of the OLCF, and as such, must be +agreed to by relevant parties as a condition of access to and use of +OLCF computational resources. + +Data Storage Resources +^^^^^^^^^^^^^^^^^^^^^^ + +The OLCF provides an array of data storage platforms, each designed with +a particular purpose in mind. Storage areas are broadly divided into two +categories: those intended for user data and those intended for project +data. Within each of the two categories, we provide different sub-areas, +each with an intended purpose: + ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Purpose | Storage Area | Path | ++==================================================================================================+===================+============================================+ +| Long-term data for routine access that is unrelated to a project | *User Home* | ``/ccs/home/[userid]`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Long-term data for archival access that is unrelated to a project | *User Archive* | ``/home/[userid]`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Long-term project data for routine access that's shared with other project members | *Project Home* | ``/ccs/proj/[projid]`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Short-term project data for fast, batch job access that you don't want to share | *Member Work* | ``/gpfs/alpine/[projid]/scratch/[userid]`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Short-term project data for fast, batch job access that's shared with other project members | *Project Work* | ``/gpfs/alpine/[projid]/proj-shared`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Short-term project data for fast, batch job access that's shared with those outside your project | *World Work* | ``/gpfs/alpine/[projid]/world-shared`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Long-term project data for archival access that you don't want to share | *Member Archive* | ``/hpss/prod/[projid]/users/$USER`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Long-term project data for archival access that's shared with other project members | *Project Archive* | ``/hpss/prod/[projid]/proj-shared`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ +| Long-term project data for archival access that's shared with those outside your project | *World Archive* | ``/hpss/prod/[projid]/world-shared`` | ++--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ + +For more information about using the data storage archiving +systems, please refer to the pages on :ref:`data-storage-and-transfers`. + +User Home +^^^^^^^^^ + +Home directories for each user are NFS-mounted on all OLCF systems and +are intended to store long-term, frequently-accessed user data. User +Home areas are backed up on a daily basis. This file system does not +generally provide the input/output (I/O) performance required by most +compute jobs, and is not available to compute jobs on most systems. See +the section :ref:`retention-policy` for more details on +applicable quotas, backups, purge, and retention timeframes. + +User Archive +^^^^^^^^^^^^ + +The High Performance Storage System (HPSS) is the tape-archive storage +system at the OLCF and is the storage technology that supports the User +Archive areas. HPSS is intended for data that do not require day-to-day +access. + +.. note:: + Use of this directory for data storage is deprecated in favor of storing + data in the User, Project, and World Archive directories. For new users, + this directory is a "link farm" with symlinks to that user's /hpss/prod + directories. Data for existing users remains in this directory but should + be moved into a User/Project/World Archive directory, at which time this + directory will automatically convert to a link farm. + +Project Home +^^^^^^^^^^^^ + +Project Home directories are NFS-mounted on selected OLCF systems and +are intended to store long-term, frequently-accessed data that is needed +by all collaborating members of a project. Project Home areas are backed +up on a daily basis. This file system does not generally provide the +input/output (I/O) performance required by most compute jobs, and is not +available to compute jobs on most systems. See the section +:ref:`retention-policy` for more details on applicable +quotas, backups, purge, and retention timeframes. + +Member Work +^^^^^^^^^^^ + +Project members get an individual Member Work directory for each associated +project; these reside in the center-wide, high-capacity Spectrum Scale file +system on large, fast disk areas intended for global (parallel) access to +temporary/scratch storage. Member Work areas are not shared with other +users of the system and are intended for project data that the user does +not want to make available to other users. Member Work directories are +provided commonly across all systems. Because of the scratch nature of the +file system, it is not backed up and files are automatically purged on a +regular basis. Files should not be retained in this file system for long, +but rather should be migrated to Project Home or Project Archive space as +soon as the files are not actively being used. If a file system associated +with your Member Work directory is nearing capacity, the OLCF may contact +you to request that you reduce the size of your Member Work directory. See +the section :ref:`retention-policy` for more details on applicable quotas, +backups, purge, and retention timeframes. + +Project Work +^^^^^^^^^^^^ + +Each project is granted a Project Work directory; these reside in the +center-wide, high-capacity Spectrum Scale file system on large, fast disk +areas intended for global (parallel) access to temporary/scratch storage. +Project Work directories can be accessed by all members of a project and +are intended for sharing data within a project. Project Work directories +are provided commonly across most systems. Because of the scratch nature of +the file system, it is not backed up and files are automatically purged on +a regular bases. Files should not be retained in this file system for long, +but rather should be migrated to Project Home or Project Archive space as +soon as the files are not actively being used. If a file system associated +with Project Work storage is nearing capacity, the OLCF may contact the PI +of the project to request that he or she reduce the size of the Project +Work directory. See the section :ref:`retention-policy` for more details on +applicable quotas, backups, purge, and retention timeframes. + +World Work +^^^^^^^^^^ + +Each project has a World Work directory that resides in the center-wide, +high-capacity Spectrum Scale file system on large, fast disk areas intended +for global (parallel) access to temporary/scratch storage. World Work areas +can be accessed by all users of the system and are intended for sharing of +data between projects. World Work directories are provided commonly across +most systems. Because of the scratch nature of the file system, it is not +backed up and files are automatically purged on a regular bases. Files +should not be retained in this file system for long, but rather should be +migrated to Project Home or Project Archive space as soon as the files are +not actively being used. If a file system associated with World Work +storage is nearing capacity, the OLCF may contact the PI of the project to +request that he or she reduce the size of the World Work directory. See the +section :ref:`retention-policy` for more details on applicable quotas, +backups, purge, and retention timeframes. + +Member Archive +^^^^^^^^^^^^^^ + +Project members get an individual Member Archive directory for each +associated project; these reside on the High Performance Storage System +(HPSS), OLCF's tape-archive storage system. Member Archive areas are not +shared with other users of the system and are intended for project data +that the user does not want to make available to other users. HPSS is +intended for data that do not require day-to-day access. Users should not +store data unrelated to OLCF projects on HPSS. Users should periodically +review files and remove unneeded ones. See the section +:ref:`retention-policy` for more details on applicable quotas, backups, +purge, and retention timeframes. + +Project Archive +^^^^^^^^^^^^^^^ + +Each project is granted a Project Archive directory; these reside on the +High Performance Storage System (HPSS), OLCF's tape-archive storage system. +Project Archive directories are shared among all members of a project and +are intended for sharing data within a project. HPSS is intended for data +that do not require day-to-day access. Users should not store data +unrelated to OLCF projects on HPSS. Project members should also +periodically review files and remove unneeded ones. See the section +:ref:`retention-policy` for more details on applicable quotas, backups, +purge, and retention timeframes. + +World Archive +^^^^^^^^^^^^^ + +Each project is granted a World Archive directory; these reside on the High +Performance Storage System (HPSS), OLCF's tape-archive storage system. +World Archive areas are shared among all users of the system and are +intended for sharing data between projects. HPSS is intended for data that +do not require day-to-day access. Users should not store data unrelated to +OLCF projects on HPSS. Users should periodically review files and remove +unneeded ones. See the section :ref:`retention-policy` for more details on +applicable quotas, backups, purge, and retention timeframes. + + +.. _retention-policy: + +Data Retention, Purge, & Quotas +------------------------------- + +Summary +^^^^^^^ + +The following table details quota, backup, purge, and retention +information for each user-centric and project-centric storage area +available at the OLCF. + +**User-Centric Storage Areas** + ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| Area | Path | Type | Permissions | Quota | Backups | Purged | Retention | On Compute Nodes | ++=====================+=============================================+================+=============+========+=========+=========+============+==================+ +| User Home | ``/ccs/home/[userid]`` | NFS | User set | 50 GB | Yes | No | 90 days | Read-only | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| User Archive [#f1]_ | ``/home/[userid]`` | HPSS | User set | 2TB | No | No | 90 days | No | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| User Archive [#f2]_ | ``/home/[userid]`` | HPSS | 700 | N/A | N/A | N/A | N/A | No | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ + +**Project-Centric Storage Areas** + ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| Area | Path | Type | Permissions | Quota | Backups | Purged | Retention | On Compute Nodes | ++=====================+=============================================+================+=============+========+=========+=========+============+==================+ +| Project Home | ``/ccs/proj/[projid]`` | NFS | 770 | 50 GB | Yes | No | 90 days | Read-only | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| Member Work | ``/gpfs/alpine/[projid]/scratch/[userid]`` | Spectrum Scale | 700 [#f3]_ | 50 TB | No | 90 days | N/A [#f4]_ | Yes | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| Project Work | ``/gpfs/alpine/[projid]/proj-shared`` | Spectrum Scale | 770 | 50 TB | No | 90 days | N/A [#f4]_ | Yes | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| World Work | ``/gpfs/alpine/[projid]/world-shared`` | Spectrum Scale | 775 | 50 TB | No | 90 days | N/A [#f4]_ | Yes | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| Member Archive | ``/hpss/prod/[projid]/users/$USER`` | HPSS | 700 | 100 TB | No | No | 90 days | No | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| Project Archive | ``/hpss/prod/[projid]/proj-shared`` | HPSS | 770 | 100 TB | No | No | 90 days | No | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ +| World Archive | ``/hpss/prod/[projid]/world-shared`` | HPSS | 775 | 100 TB | No | No | 90 days | No | ++---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ + +| *Area -* The general name of storage area. +| *Path -* The path (symlink) to the storage area's directory. +| *Type -* The underlying software technology supporting the storage area. +| *Permissions -* UNIX Permissions enforced on the storage area's top-level directory. +| *Quota -* The limits placed on total number of bytes and/or files in the storage area. +| *Backups -* States if the data is automatically duplicated for disaster recovery purposes. +| *Purged -* Period of time, post-file-access, after which a file will be marked as eligible for permanent deletion. +| *Retention -* Period of time, post-account-deactivation or post-project-end, after which data will be marked as eligible for permanent deletion. + + **Important!** Files within "Work" directories (i.e., Member Work, + Project Work, World Work) are *not* backed up and are *purged* on a + regular basis according to the timeframes listed above. + +.. rubric:: Footnotes + +.. [#f1] This entry is for legacy User Archive directories which contained user data on January 14, 2020. There is also a quota/limit of 2,000 files on this directory. + +.. [#f2] User Archive directories that were created (or had no user data) after January 14, 2020. Settings other than permissions are not applicable because directories are root-owned and contain no user files. + +.. [#f3] Permissions on Member Work directories can be controlled to an extent by project members. By default, only the project member has any accesses, but accesses can be granted to other project members by setting group permissions accordingly on the Member Work directory. The parent directory of the Member Work directory prevents accesses by "UNIX-others" and cannot be changed (security measures). + +.. [#f4] Retention is not applicable as files will follow purge cycle. + +On Summit, Rhea and the DTNs, additional paths to the various project-centric work areas are available +via the following symbolic links and/or environment variables: + +- Member Work Directory: ``/gpfs/alpine/scratch/[userid]/[projid]`` or ``$MEMBERWORK/[projid]`` +- Project Work Directory: ``/gpfs/alpine/proj-shared/[projid]`` or ``$PROJWORK/[projid]`` +- World Work Directory: ``/gpfs/alpine/world-shared/[projid]`` or ``$WORLDWORK/[projid]`` + + +Data Retention Overview +^^^^^^^^^^^^^^^^^^^^^^^ + +By default, there is no lifetime retention for any data on OLCF +resources. The OLCF specifies a limited post-deactivation timeframe +during which user and project data will be retained. When the retention +timeframe expires, the OLCF retains the right to delete data. If you +have data retention needs outside of the default policy, please notify +the OLCF. + +User Data Retention +^^^^^^^^^^^^^^^^^^^ + +The user data retention policy exists to reclaim storage space after a +user account is deactivated, e.g., after the user’s involvement on all +OLCF projects concludes. By default, the OLCF will retain data in +user-centric storage areas only for a designated amount of time after +the user’s account is deactivated. During this time, a user can request +a temporary user account extension for data access. See the section +:ref:`retention-policy` for details on retention +timeframes for each user-centric storage area. + +Project Data Retention +^^^^^^^^^^^^^^^^^^^^^^ + +The project data retention policy exists to reclaim storage space after +a project ends. By default, the OLCF will retain data in project-centric +storage areas only for a designated amount of time after the project end +date. During this time, a project member can request a temporary user +account extension for data access. See the section :ref:`retention-policy` +for details on purge and retention timeframes +for each project-centric storage area. + +Sensitive Project Data Retention +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For sensitive projects only, all data related to the project must be +purged from all OLCF computing resources within 30 days of the project’s +end or termination date. + +Transfer of Member Work and Member Archive Data +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Although the Member Work and Member Archive directories are for storage +of data a user does not want to make available to other users on the +system, files in these directories are still considered project data +and can be reassigned to another user at the PI's request. + +Data Purges +^^^^^^^^^^^ + +Data purge mechanisms are enabled on some OLCF file system directories +in order to maintain sufficient disk space availability for job +execution. Files in these scratch areas are automatically purged on a +regular purge timeframe. If a file system with an active purge policy is +nearing capacity, the OLCF may contact you to request that you reduce +the size of a directory within that file system, even if the purge +timeframe has not been exceeded. See the section :ref:`retention-policy` +for details on purge timeframes for each storage area, if applicable. + +Storage Space Quotas +^^^^^^^^^^^^^^^^^^^^ + +Each user-centric and project-centric storage area has an associated +quota, which could be a hard (systematically-enforceable) quota or a +soft (policy-enforceable) quota. Storage usage will be monitored +continually. When a user or project exceeds a soft quota for a storage +area, the user or project PI will be contacted and will be asked if at +all possible to purge data from the offending area. See the section +:ref:`retention-policy` for details on quotas for each storage area. + +Data Prohibitions & Safeguards +------------------------------ + +Prohibited Data +^^^^^^^^^^^^^^^ + +The OLCF computer systems are operated as research systems and only +contain data related to scientific research and do not contain +personally identifiable information (data that falls under the Privacy +Act of 1974 5U.S.C. 552a). Use of OLCF resources to store, manipulate, +or remotely access any national security information is strictly +prohibited. This includes, but is not limited to: classified +information, unclassified controlled nuclear information (UCNI), naval +nuclear propulsion information (NNPI), the design or development of +nuclear, biological, or chemical weapons or any weapons of mass +destruction. Authors/generators/owners of information are responsible +for its correct categorization as sensitive or non-sensitive. Owners of +sensitive information are responsible for its secure handling, +transmission, processing, storage, and disposal on OLCF systems. +Principal investigators, users, or project delegates that use OLCF +resources, or are responsible for overseeing projects that use OLCF +resources, are strictly responsible for knowing whether their project +generates any of these prohibited data types or information that falls +under Export Control. For questions, contact help@olcf.ornl.gov. + +Unauthorized Data Modification +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users are prohibited from taking unauthorized actions to intentionally +modify or delete information or programs. + +Data Confidentiality, Integrity, & Availability +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The OLCF systems provide protections to maintain the confidentiality, +integrity, and availability of user data. Measures include: the +availability of file permissions, archival systems with access control +lists, and parity/CRC checks on data paths/files. It is the user’s +responsibility to set access controls appropriately for data. In the +event of system failure or malicious actions, the OLCF makes no +guarantee against loss of data nor makes a guarantee that a user’s data +could not be potentially accessed, changed, or deleted by another +individual. It is the user’s responsibility to insure the appropriate +level of backup and integrity checks on critical data and programs. + +Administrator Access to Data +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +OLCF resources are federal computer systems, and as such, users should +have no explicit or implicit expectation of privacy. OLCF employees and +authorized vendor personnel with “root” privileges have access to all +data on OLCF systems. Such employees can also login to OLCF systems as +other users. As a general rule, OLCF employees will not discuss your +data with any unauthorized entities nor grant access to data files to +any person other than the UNIX “owner” of the data file, except in the +following situations: + +- When the owner of the data requests a change of ownership for any + reason, e.g., the owner is leaving the project and grants the PI + ownership of the data. +- In situations of suspected abuse/misuse computational resources, + criminal activity, or cyber-security violations. + +Note that the above applies even to project PIs. In general, the OLCF +will not overwrite existing UNIX permissions on data files owned by +project members for the purpose of granting access to the project PI. +Project PIs should work closely with project members throughout the +duration of the project to ensure UNIX permissions are set +appropriately. + +Software +-------- + +Software Licensing +^^^^^^^^^^^^^^^^^^ + +All software used on OLCF computers must be appropriately acquired and +used according to the appropriate software license agreement. +Possession, use, or transmission of illegally obtained software is +prohibited. Likewise, users shall not copy, store, or transfer +copyrighted software, except as permitted by the owner of the copyright. +Only export-controlled codes approved by the Export Control Office may +be run by parties with sensitive data agreements. + +Malicious Software +^^^^^^^^^^^^^^^^^^ + +Users must not intentionally introduce or use malicious software, +including but not limited to, computer viruses, Trojan horses, or +computer worms. + +Reconstruction of Information or Software +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users are not permitted to reconstruct information or software for which +they are not authorized. This includes but is not limited to any reverse +engineering of copyrighted software or firmware present on OLCF +computing resources.   + +Security Policy +=============== + +.. note:: + This details an official policy of the OLCF, and must be + agreed to by the following persons as a condition of access to or use of + OLCF computational resources: + + - Principal Investigators (Non-Profit) + - Principal Investigators (Industry) + - All Users + + **Title:**\ Security Policy **Version:** 12.10 + +The Oak Ridge Leadership Computing Facility (OLCF) computing resources +are provided to users for research purposes. All users must agree to +abide by all security measures described in this document. Failure to +comply with security procedures will result in termination of access to +OLCF computing resources and possible legal actions. + +Scope +----- + +The requirements outlined in this document apply to all individuals who +have an OLCF account. It is your responsibility to ensure that all +individuals have the proper need-to-know before allowing them access to +the information on OLCF computing resources. This document will outline +the main security concerns. + +Personal Use +------------ + +OLCF computing resources are for business use only. Installation or use +of software for personal use is not allowed. Incidents of abuse will +result in account termination. Inappropriate uses include, but are not +limited to: + +- Sexually oriented information +- Downloading, copying, or distributing copyrighted materials without + prior permission from the owner +- Downloading or storing large files or utilizing streaming media for + personal use (e.g., music files, graphic files, internet radio, video + streams, etc.) +- Advertising, soliciting, or selling + +Accessing OLCF Computational Resources +-------------------------------------- + +Access to systems is provided via Secure Shell version 2 (sshv2). You +will need to ensure that your ssh client supports keyboard-interactive +authentication. The method of setting up this authentication varies from +client to client, so you may need to contact your local administrator +for assistance. Most new implementations support this authentication +type, and many ssh clients are available on the web. Login sessions will +be automatically terminated after a period of inactivity. When you apply +for an account, you will be mailed an RSA SecurID token. You will also +be sent a request to complete identity verification. When your account is +approved, your RSA SecurID token will also be enabled. Please refer to our +:ref:`system-user-guides` for more information on host access. DO NOT share your +PIN or RSA SecurID token with anyone. Sharing of accounts will result in +termination. If your SecurID token is stolen or misplaced, contact the OLCF +immediately and report the missing token. Upon termination of your account +access, return the token to the OLCF in person or via mail. + +Data Management +--------------- + +The OLCF uses a standard file system structure to assist users with data +organization on OLCF systems. Complete details about all file systems +available to OLCF users can be found in the Data Management Policy +section. + +Sensitive Data +-------------- + +Additional file systems and file protections may be employed for +sensitive data. If you are a user on a project producing sensitive data, +further instructions will be given by the OLCF. The following guidelines +apply to sensitive data: + +- Only store sensitive data in designated locations. Do not store + sensitive data in your User Home directory. +- Never allow access to your sensitive data to anyone outside of your + group. +- Transfer of sensitive data must be through the use encrypted methods + (scp, sftp, etc). +- All sensitive data must be removed from all OLCF resources when your + project has concluded. + +Data Transfer +------------- + +The OLCF offers two dedicated data transfer nodes to users. The nodes have been +tuned specifically for wide area data transfers, and also perform well on the +local area. There are also several utilities that the OLCF recommends for data +transfer. Please refer to our :ref:`system-user-guides` for information about +the DTNs and available utilities. + +.. Titan Scheduling Policy +.. ======================= +.. +.. .. note:: +.. This details an official policy of the OLCF, and must be +.. agreed to by the following persons as a condition of access to or use of +.. OLCF computational resources: +.. +.. - Principal Investigators (Non-Profit) +.. - Principal Investigators (Industry) +.. - All Users +.. +.. **Title:** Titan Scheduling Policy **Version:** 13.02 +.. +.. In a simple batch queue system, jobs run in a first-in, first-out (FIFO) +.. order. This often does not make effective use of the system. A large job +.. may be next in line to run. If the system is using a strict FIFO queue, +.. many processors sit idle while the large job waits to run. *Backfilling* +.. would allow smaller, shorter jobs to use those otherwise idle resources, +.. and with the proper algorithm, the start time of the large job would not +.. be delayed. While this does make more effective use of the system, it +.. indirectly encourages the submission of smaller jobs. +.. +.. The DOE Leadership-Class Job Mandate +.. ------------------------------------ +.. +.. As a DOE Leadership Computing Facility, the OLCF has a mandate that a +.. large portion of Titan's usage come from large, *leadership-class* (aka +.. *capability*) jobs. To ensure the OLCF complies with DOE directives, we +.. strongly encourage users to run jobs on Titan that are as large as their +.. code will warrant. To that end, the OLCF implements queue policies that +.. enable large jobs to run in a timely fashion. +.. +.. .. note:: +.. The OLCF implements queue policies that encourage the +.. submission and timely execution of large, leadership-class jobs on +.. Titan. +.. +.. The basic priority-setting mechanism for jobs waiting in the queue is +.. the time a job has been waiting relative to other jobs in the queue. +.. However, several factors are applied by the batch system to modify the +.. *apparent* time a job has been waiting. These factors include: +.. +.. - The number of nodes requested by the job. +.. - The queue to which the job is submitted. +.. - The 8-week history of usage for the project associated with the job. +.. - The 8-week history of usage for the user associated with the job. +.. +.. If your jobs require resources outside these queue policies, please complete the +.. relevant request form on the `Special Requests +.. `__ +.. page. If you have any questions or comments on the queue policies below, please +.. direct them to the User Assistance Center. +.. +.. Job Priority by Processor Count +.. ------------------------------- +.. +.. Jobs are *aged* according to the job's requested processor count (older +.. age equals higher queue priority). Each job's requested processor count +.. places it into a specific *bin*. Each bin has a different aging +.. parameter, which all jobs in the bin receive. +.. +.. +-------+-------------+-------------+------------------------+----------------------+ +.. | Bin | Min Nodes | Max Nodes | Max Walltime (Hours) | Aging Boost (Days) | +.. +=======+=============+=============+========================+======================+ +.. | 1 | 11,250 | -- | 24.0 | 15 | +.. +-------+-------------+-------------+------------------------+----------------------+ +.. | 2 | 3,750 | 11,249 | 24.0 | 5 | +.. +-------+-------------+-------------+------------------------+----------------------+ +.. | 3 | 313 | 3,749 | 12.0 | 0 | +.. +-------+-------------+-------------+------------------------+----------------------+ +.. | 4 | 126 | 312 | 6.0 | 0 | +.. +-------+-------------+-------------+------------------------+----------------------+ +.. | 5 | 1 | 125 | 2.0 | 0 | +.. +-------+-------------+-------------+------------------------+----------------------+ +.. +.. FairShare Scheduling Policy +.. --------------------------- +.. +.. FairShare, as its name suggests, tries to push each user and project +.. towards their fair share of the system's utilization: in this case, 5% +.. of the system's utilization per user and 10% of the system's utilization +.. per project. To do this, the job scheduler adds (30) minutes priority +.. aging per user and (1) hour of priority aging per project for every (1) +.. percent the user or project is under its fair share value for the prior +.. (8) weeks. Similarly, the job scheduler subtracts priority in the same +.. way for users or projects that are over their fair share. For instance, +.. a user who has personally used 0.0% of the system's utilization over the +.. past (8) weeks who is on a project that has also used 0.0% of the +.. system's utilization will get a (12.5) hour bonus (5 \* 30 min for the +.. user + 10 \* 1 hour for the project). In contrast, a user who has +.. personally used 0.0% of the system's utilization on a project that has +.. used 12.5% of the system's utilization would get no bonus (5 \* 30 min +.. for the user - 2.5 \* 1 hour for the project). +.. +.. ``batch`` Queue Policy +.. ---------------------- +.. +.. The ``batch`` queue is the default queue for production work on Titan. +.. Most work on Titan is handled through this queue. It enforces the +.. following policies: +.. +.. - Limit of (4) *eligible-to-run* jobs per user. +.. - Jobs in excess of the per user limit above will be placed into a +.. *held* state, but will change to eligible-to-run at the appropriate +.. time. +.. - Users may have only (2) jobs in bin 5 *running* at any time. Any +.. additional jobs will be blocked until one of the running jobs +.. completes. +.. +.. .. note:: +.. The *eligible-to-run* state is not the *running* state. +.. Eligible-to-run jobs have not started and are waiting for resources. +.. Running jobs are actually executing. +.. +.. ``killable`` Queue Policy +.. ------------------------- +.. +.. At the start of a scheduled system outage, a *queue reservation* is used +.. to ensure that no jobs are running. In the ``batch`` queue, the +.. scheduler will not start a job if it expects that the job would not +.. complete (based on the job's user-specified max walltime) before the +.. reservation's start time. In constrast, the ``killable`` queue allows +.. the scheduler to start a job even if it will *not* complete before a +.. scheduled reservation. It enforces the following policies: +.. +.. - Jobs will be killed if still running when a system outage begins. +.. - The scheduler will stop scheduling jobs in the ``killable`` queue (1) +.. hour before a scheduled outage. +.. - Maximum-job-per-user limits are the same (i.e., in conjunction with) +.. the ``batch`` queue. +.. - Any killed jobs will be automatically re-queued after a system outage +.. completes. +.. +.. ``debug`` Queue Policy +.. ---------------------- +.. +.. The ``debug`` queue is intended to provide faster turnaround times for +.. the code development, testing, and debugging cycle. For example, +.. interactive parallel work is an ideal use for the debug queue. It +.. enforces the following policies: +.. +.. - Production jobs are not allowed. +.. - Maximum job walltime of (1) hour. +.. - Limit of (1) job per user *regardless of the job's state*. +.. - Jobs receive a (2)-day priority aging boost for scheduling. +.. +.. .. warning:: +.. Users who misuse the ``debug`` queue may have further +.. access to the queue denied. +.. +.. Allocation Overuse Policy +.. ------------------------- +.. +.. Projects that overrun their allocation are still allowed to run on OLCF +.. systems, although at a reduced priority. Like the adjustment for the +.. number of processors requested above, this is an adjustment to the +.. apparent submit time of the job. However, this adjustment has the effect +.. of making jobs appear much younger than jobs submitted under projects +.. that have not exceeded their allocation. In addition to the priority +.. change, these jobs are also limited in the amount of wall time that can +.. be used. For example, consider that ``job1`` is submitted at the same +.. time as ``job2``. The project associated with ``job1`` is over its +.. allocation, while the project for ``job2`` is not. The batch system will +.. consider ``job2`` to have been waiting for a longer time than ``job1``. +.. Also projects that are at 125% of their allocated time will be limited +.. to only one running job at a time. The adjustment to the apparent submit +.. time depends upon the percentage that the project is over its +.. allocation, as shown in the table below: +.. +.. +------------------------+----------------------+--------------------------+------------------+ +.. | % Of Allocation Used | Priority Reduction | Number eligible-to-run | Number running | +.. +========================+======================+==========================+==================+ +.. | < 100% | 0 days | 4 jobs | unlimited jobs | +.. +------------------------+----------------------+--------------------------+------------------+ +.. | 100% to 125% | 30 days | 4 jobs | unlimited jobs | +.. +------------------------+----------------------+--------------------------+------------------+ +.. | > 125% | 365 days | 4 jobs | 1 job | +.. +------------------------+----------------------+--------------------------+------------------+ +.. +.. System Reservation Policy +.. ------------------------- +.. +.. Projects may request to reserve a set of processors for a period of time +.. through the reservation request form, which can be found on the `Special +.. Requests `__ +.. page. If the reservation is granted, the reserved processors will be +.. blocked from general use for a given period of time. Only users that +.. have been authorized to use the reservation can utilize those resources. +.. Since no other users can access the reserved resources, it is crucial +.. that groups given reservations take care to ensure the utilization on +.. those resources remains high. To prevent reserved resources from +.. remaining idle for an extended period of time, reservations are +.. monitored for inactivity. If activity falls below 50% of the reserved +.. resources for more than (30) minutes, the reservation will be canceled +.. and the system will be returned to normal scheduling. A new reservation +.. must be requested if this occurs. Since a reservation makes resources +.. unavailable to the general user population, projects that are granted +.. reservations will be charged (regardless of their actual utilization) a +.. CPU-time equivalent to +.. ``(# of cores reserved) * (length of reservation in hours)``. + +INCITE Allocation Under-utilization Policy +========================================== + +.. note:: + This details an official policy of the OLCF, and must be + agreed to by the following persons as a condition of access to and use + of OLCF computational resources: + + - INCITE Principal Investigators + + **Title:** INCITE Allocation Under-utilization Policy **Version:** 12.10 + +The OLCF has a *pull-back* policy for under-utilization of INCITE +allocations. Under-utilized INCITE project allocations will have +core-hours removed from their outstanding core-hour project balance at +specific times during the INCITE calendar year. The following table +summarizes the current under-utilization policy: + ++-------------+---------------------+-----------------------------------+ +| Date | Utilization to-Date | Forfeited Amount | ++=============+=====================+===================================+ +| May 1 | < 10% | Up to 30% of remaining allocation | ++ +---------------------+-----------------------------------+ +| | < 15% | Up to 15% of remaining allocation | ++-------------+---------------------+-----------------------------------+ +| September 1 | < 10% | Up to 75% of remaining allocation | ++ +---------------------+-----------------------------------+ +| | < 33% | Up to 50% of remaining allocation | ++ +---------------------+-----------------------------------+ +| | < 50% | Up to 33% of remaining allocation | ++-------------+---------------------+-----------------------------------+ + +For example, a 1,000,000 core-hour INCITE project that has utilized only +50,000 core-hours (5% of the allocation) on May 1st would forfeit (0.30 +\* 950,000) = 285,000 core-hours from their remaining allocation.   + +Project Reporting Policy +======================== + +.. note:: + This details an official policy of the OLCF, and must be + agreed to by the following persons as a condition of access to and use + of OLCF computational resources: + + - Principal Investigators (Non-Profit) + - Principal Investigators (Industry) + + **Title:** Project Reporting Policy **Version:** 12.10 + +Principal Investigators of current OLCF projects must submit a quarterly +progress report. The quarterly reports are essential as the OLCF must +diligently track the use of the center's resources. In keeping with +this, the OLCF (and DOE Leadership Computing Facilities in general) +imposes the following penalties for late submission: + ++-----------------+--------------------------------------------------------------------------------------------------------------+ +| Timeframe | Penalty | ++=================+==============================================================================================================+ +| 1 Month Late | Job submissions against offending project will be suspended. | ++-----------------+--------------------------------------------------------------------------------------------------------------+ +| 3 Months Late | Login privileges will be suspended for all OLCF resources for all users associated with offending project. | ++-----------------+--------------------------------------------------------------------------------------------------------------+ + +   + +Non-proprietary Institutional User Agreement Policy +=================================================== + +.. note:: + This details an official policy of the OLCF, and must be + agreed to by the following persons as a condition of access to and use + of OLCF computational resources: + + - Principal Investigators (Non-Profit) + - All Users + + **Title:** Non-proprietary Institutional User Agreement Policy + **Version:** 12.10 + +Users of DOE-designated User Facilities must understand and agree to the +following Institutional User Agreement clause: I understand that my +institution has entered into a User Agreement with UT-Battelle, the +management and operating contractor for the U.S. Department of Energy’s +(DOE) Oak Ridge National Laboratory (ORNL), that governs my research +ORNL’s DOE-designated User Facilities. I have read and understand my +obligations under that Agreement, including the provisions summarized +below. You may check with your institution or contact +accounts@ccs.ornl.gov if you require a copy of your User Agreement. + +Access +-------- + +I understand that my access is limited to certain designated areas +and/or systems, and my access may be revoked if I pose a security, +safety, or operational risk. + +Rules and Regulations +------------------------ + +I will follow the applicable ORNL rules, regulations and requirements, +including those requirements of the ORNL User Facility. I will follow +the requirements set forth in training if assigned to me by the ORNL +User Facility. + +Safety and Health +------------------- + +I will take all reasonable precautions to protect the safety and health +of others and comply with all applicable safety and health requirements. + +Intent to Publish +------------------- + +I will use best efforts to publish the results from my use of the ORNL +User Facility in an open scientific journal or significant industry +technical journal or conference proceedings. I will `acknowledge use of +the ORNL User Facility <#olcf-acknowledgement>`__ in the publication and +notify the ORNL User Facility of any publications that result from my +use of the facility. + +Export Control +---------------- + +I will comply with all U.S. Export Control laws and regulations and be +responsible for the appropriate handling and transfer of any export +controlled information, which may require advance U.S. Government +authorization. + +Intellectual Property +------------------------ + +| I will disclose any invention conceived as a part of the work at a + ORNL User Facility and will protect the invention until a patent + application can be filed. I understand that my institution may elect + title to the invention and the U.S. Government retains rights to the + invention. + +Special Requests and Policy Exemptions +====================================== + +To request policy exemptions, please submit the appropriate webform available on +the :ref:`documents-and-forms` page. Special request forms allow a user to: + +- Request Software installations +- Request relaxed queue limits for a job +- Request a system reservation +- Request a disk quota increase +- Request a User Work area purge exemption + +Special requests are reviewed weekly by the OLCF Resource Utilization +Council. Please contact help@olcf.ornl.gov for more information. From 193f1caeafd6e019565632176beddb85d2d5be33 Mon Sep 17 00:00:00 2001 From: Stefan Maerz Date: Tue, 7 Jul 2020 14:16:25 -0400 Subject: [PATCH 2/5] adding data policy --- accounts/AFW_data_policy_guide.rst | 334 +++++++++ accounts/afw_policy_guide.rst | 1080 ---------------------------- 2 files changed, 334 insertions(+), 1080 deletions(-) create mode 100644 accounts/AFW_data_policy_guide.rst delete mode 100644 accounts/afw_policy_guide.rst diff --git a/accounts/AFW_data_policy_guide.rst b/accounts/AFW_data_policy_guide.rst new file mode 100644 index 00000000..662743ca --- /dev/null +++ b/accounts/AFW_data_policy_guide.rst @@ -0,0 +1,334 @@ +Data Use +-------- + +Prohibited Data +^^^^^^^^^^^^^^^ + +The NCCS computer systems are operated in support of USAF and only +contain data related to scientific research and do not contain +personally identifiable information (data that falls under the Privacy +Act of 1974 5U.S.C. 552a). Use of NCCS resources to store, manipulate, +or remotely access any national security information is strictly +prohibited. This includes, but is not limited to: classified +information, unclassified controlled nuclear information (UCNI), naval +nuclear propulsion information (NNPI), the design or development of +nuclear, biological, or chemical weapons or any weapons of mass +destruction. Authors/generators/owners of information are responsible +for its correct categorization as sensitive or non-sensitive. Owners of +sensitive information are responsible for its secure handling, +transmission, processing, storage, and disposal on NCCS systems. +Principal investigators, users, or project delegates that use NCCS +resources, or are responsible for overseeing projects that use NCCS +resources, are strictly responsible for knowing whether their project +generates any of these prohibited data types or information that falls +under Export Control. For questions, contact help@nccs.gov. + +Confidentiality, Integrity, and Availability +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The NCCS systems provide protections to maintain the confidentiality, +integrity, and availability of user data. Measures include the +availability of file permissions, archival systems with access control +lists, and parity and CRC checks on data paths and files. It is the +user’s responsibility to set access controls appropriately for the data. +In the event of system failure or malicious actions, the NCCS makes no +guarantee against loss of data or that a user’s data can be accessed, +changed, or deleted by another individual. It is the user’s +responsibility to insure the appropriate level of backup and integrity +checks on critical data and programs. + +Administrator Access to Data +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +NCCS resources are federal computer systems, and as such, users should +have no explicit or implicit expectation of privacy. NCCS employees and +authorized vendor personnel with “root” privileges have access to all +data on NCCS systems. Such employees can also login to NCCS systems as +other users. As a general rule, NCCS employees will not discuss your +data with any unauthorized entities nor grant access to data files to +any person other than the UNIX “owner” of the data file, except in the +following situations: + +- When the owner of the data requests a change of ownership for any + reason, e.g., the owner is leaving the project and grants the PI + ownership of the data. +- In situations of suspected abuse/misuse computational resources, + criminal activity, or cyber-security violations. + +Note that the above applies even to project PIs. In general, the NCCS +will not overwrite existing UNIX permissions on data files owned by +project members for the purpose of granting access to the project PI. +Project PIs should work closely with project members throughout the +duration of the project to ensure UNIX permissions are set +appropriately. + +Data Modification/Destruction +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users are prohibited from taking unauthorized actions to intentionally +modify or delete information or programs. + +Data Retention +^^^^^^^^^^^^^^ + +The NCCS reserves the right to remove any data at any time and/or +transfer data to other users working on the same or similar project once +a user account is deleted or a person no longer has a business +association with the NCCS. After a sensitive project has ended or has +been terminated, all data related to the project must be purged from all +NCCS computing resources within 30 days. + +Software Use +------------ + +All software used on NCCS computers must be appropriately acquired and +used according to the appropriate software license agreement. +Possession, use, or transmission of illegally obtained software is +prohibited. Likewise, users shall not copy, store, or transfer +copyrighted software, except as permitted by the owner of the copyright. +Only export-controlled codes approved by the Export Control Office may +be run by parties with sensitive data agreements. + +Malicious Software +^^^^^^^^^^^^^^^^^^ + +Users must not intentionally introduce or use malicious software such as +computer viruses, Trojan horses, or worms. + +Reconstruction of Information or Software +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users are not allowed to reconstruct information or software for which +they are not authorized. This includes but is not limited to any reverse +engineering of copyrighted software or firmware present on NCCS +computing resources. + +User Accountability +------------------- + +Users are accountable for their actions and may be held accountable to +applicable administrative or legal sanctions. + +Monitoring and Privacy +^^^^^^^^^^^^^^^^^^^^^^ + +Users are advised that there is no expectation of privacy of your +activities on any system that is owned by, leased or operated by +UT-Battelle on behalf of the U.S. Department of Energy (DOE). The +Company retains the right to monitor all activities on these systems, to +access any computer files or electronic mail messages, and to disclose +all or part of information gained to authorized individuals or +investigative agencies, all without prior notice to, or consent from, +any user, sender, or addressee. This access to information or a system +by an authorized individual or investigative agency is in effect during +the period of your access to information on a DOE computer and for a +period of three years thereafter. NCCS personnel and users are required +to address, safeguard against, and report misuse, abuse and criminal +activities. Misuse of NCCS resources can lead to temporary or permanent +disabling of accounts, loss of DOE allocations, and administrative or +legal actions. Users who have not accessed a NCCS computing resource in +at least 6 months will be disabled. They will need to reapply to regain +access to their account. All users must reapply annually. + +Authentication and Authorization +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +All users are required to use a one-time password for authentication. +Tokens will be distributed to users. Users will be required to +create a Personal Identification Number (PIN). This is used in +conjunction with a generated token code as part of a two-factor +authentication implementation. Accounts on the NCCS machines are for the +exclusive use of the individual user named in the account application. +Users should not share accounts or tokens with anyone. If evidence is +found that more than one person is using an account, that account will +be disabled immediately. Users are not to attempt to receive unintended +messages or access information by some unauthorized means, such as +imitating another system, impersonating another user or other person, +misuse of legal user credentials (usernames, tokens, etc.), or by +causing some system component to function incorrectly. Users are +prohibited from changing or circumventing access controls to allow +themselves or others to perform actions outside their authorized +privileges. Users must notify the NCCS immediately when they become +aware that any of the accounts used to access NCCS have been +compromised. Users should inform the NCCS promptly of any changes in +their contact information (E-mail, phone, affiliation, etc.) Updates +should be sent to accounts@ccs.ornl.gov. + +Foreign National Access +^^^^^^^^^^^^^^^^^^^^^^^ + +Applicants who appear on a restricted foreign country listing in section +15 CFR 740.7 License Exceptions for Computers are denied access based on +US Foreign Policy. The countries cited are Cuba, Iran, North Korea, +Sudan, and Syria. Additionally, no work may be performed on NCCS +computers on behalf of foreign nationals from these countries. + +Denial of Service +^^^^^^^^^^^^^^^^^ + +Users may not deliberately interfere with other users accessing system +resources.   + +AFW Data Management Policy +====================== + +.. note:: + This details an official policy of the NCCS, and must be + agreed to by the following persons as a condition of access to or use of + NCCS computational resources: + + - Principal Investigators + - All Users + + **Title:** AFW Data Management Policy **Version:** 1.0 + +Introduction +------------ + +The NCCS provides a comprehensive suite of hardware and software +resources for the creation, manipulation, and retention of scientific +data. This document comprises guidelines for acceptable use of those +resources. It is an official policy of the NCCS, and as such, must be +agreed to by relevant parties as a condition of access to and use of +NCCS computational resources. + +Data Storage Resources +^^^^^^^^^^^^^^^^^^^^^^ + +The NCCS provides an array of data storage platforms, each designed with +a particular purpose in mind. Storage areas are broadly divided into two +categories: those intended for user data and those intended for project +data. Within each of the two categories, we provide different sub-areas, +each with an intended purpose. + + +User Home +^^^^^^^^^ + +Home directories for each user are NFS-mounted on all NCCS systems and +are intended to store long-term, frequently-accessed user data. User +Home areas are backed up on a daily basis. This file system does not +generally provide the input/output (I/O) performance required by most +compute jobs, and is not available to compute jobs on most systems. See +the section :ref:`retention-policy` for more details on +applicable quotas, backups, purge, and retention timeframes. + +Project Home +^^^^^^^^^^^^ + +Project Home directories are NFS-mounted on selected NCCS systems and +are intended to store long-term, frequently-accessed data that is needed +by all collaborating members of a project. Project Home areas are backed +up on a daily basis. This file system does not generally provide the +input/output (I/O) performance required by most compute jobs, and is not +available to compute jobs on most systems. See the section +:ref:`retention-policy` for more details on applicable +quotas, backups, purge, and retention timeframes. + +Member Work +^^^^^^^^^^^ + +Project members get an individual Member Work directory for each associated +project; these reside in the high performance parallel file system +on large, fast disk areas intended for global (parallel) access to +temporary/scratch storage. Member Work areas are not shared with other +users of the system and are intended for project data that the user does +not want to make available to other users. Member Work directories are +provided commonly across all systems. Because of the scratch nature of the +file system, it is not backed up. If a file system associated +with your Member Work directory is nearing capacity, the NCCS may contact +you to request that you reduce the size of your Member Work directory. See +the section :ref:`retention-policy` for more details on applicable quotas, +backups, purge, and retention timeframes. + +Project Work +^^^^^^^^^^^^ + +Each project is granted a Project Work directory; these reside in the +center-wide, high-performance parallel file system on large, fast disk +areas intended for global (parallel) access to temporary/scratch storage. +Project Work directories can be accessed by all members of a project and +are intended for sharing data within a project. Project Work directories +are provided commonly across most systems. Because of the scratch nature of +the file system, it is not backed up. If a file system associated +with Project Work storage is nearing capacity, the NCCS may contact the PI +of the project to request that he or she reduce the size of the Project +Work directory. See the section :ref:`retention-policy` for more details on +applicable quotas, backups, purge, and retention timeframes. + +World Work +^^^^^^^^^^ + +Each project has a World Work directory that resides in the center-wide, +high-capacity parallel file system on large, fast disk areas intended +for global (parallel) access to temporary/scratch storage. World Work areas +can be accessed by all users of the system and are intended for sharing of +data between projects. World Work directories are provided commonly across +most systems. Because of the scratch nature of the file system, it is not +backed up. If a file system associated with World Work +storage is nearing capacity, the NCCS may contact the PI of the project to +request that he or she reduce the size of the World Work directory. See the +section :ref:`retention-policy` for more details on applicable quotas, +backups, purge, and retention timeframes. + + + +.. _retention-policy: + +Data Retention, Purge, & Quotas +------------------------------- + + +Data Retention Overview +^^^^^^^^^^^^^^^^^^^^^^^ + +By default, there is no lifetime retention for any data on NCCS +resources. The NCCS specifies a limited post-deactivation timeframe +during which user and project data will be retained. When the retention +timeframe expires, the NCCS retains the right to delete data. If you +have data retention needs outside of the default policy, please notify +the NCCS. + +User Data Retention +^^^^^^^^^^^^^^^^^^^ + +The user data retention policy exists to reclaim storage space after a +user account is deactivated, e.g., after the user’s involvement on all +NCCS projects concludes. By default, the NCCS will retain data in +user-centric storage areas only for a designated amount of time after +the user’s account is deactivated. During this time, a user can request +a temporary user account extension for data access. See the section +:ref:`retention-policy` for details on retention +timeframes for each user-centric storage area. + +Project Data Retention +^^^^^^^^^^^^^^^^^^^^^^ + +The project data retention policy exists to reclaim storage space after +a project ends. By default, the NCCS will retain data in project-centric +storage areas only for a designated amount of time after the project end +date. During this time, a project member can request a temporary user +account extension for data access. See the section :ref:`retention-policy` +for details on purge and retention timeframes +for each project-centric storage area. + + +Data Purges +^^^^^^^^^^^ + +Data purge mechanisms are enabled on some NCCS file system directories +in order to maintain sufficient disk space availability for job execution. +By default, these purge mechanisms are disabled on Air Force partnership +file systems. Should the file system exceed critical capacity thresholds; +the NCCS reserves the right to purge files to regain file system stability. + +Storage Space Quotas +^^^^^^^^^^^^^^^^^^^^ + +Each user-centric and project-centric storage area has an associated +quota, which could be a hard (systematically-enforceable) quota or a +soft (policy-enforceable) quota. Storage usage will be monitored +continually. When a user or project exceeds a soft quota for a storage +area, the user or project PI will be contacted and will be asked if at +all possible to purge data from the offending area. See the section +:ref:`retention-policy` for details on quotas for each storage area. diff --git a/accounts/afw_policy_guide.rst b/accounts/afw_policy_guide.rst deleted file mode 100644 index 2ff2d261..00000000 --- a/accounts/afw_policy_guide.rst +++ /dev/null @@ -1,1080 +0,0 @@ -**************************** -OLCF Policy Guides -**************************** - -OLCF Acknowledgement -==================== - -Users should acknowledge the OLCF in all publications and presentations -that speak to work performed on OLCF resources: - - This research used resources of the Oak Ridge Leadership Computing - Facility at the Oak Ridge National Laboratory, which is supported by the - Office of Science of the U.S. Department of Energy under Contract No. - DE-AC05-00OR22725. - - -Computing Policy -================ - -.. note:: - This details an official policy of the OLCF, and must be - agreed to by the following persons as a condition of access to and use - of OLCF computational resources: - - - Principal Investigators (Non-Profit) - - Principal Investigators (Industry) - - All Users - - **Title:** Computing Policy **Version:** 12.10 - -Computer Use ------------- - -Computers, software, and communications systems provided by the OLCF are -to be used for work associated with and within the scope of the approved -project. The use of OLCF resources for personal or non-work-related -activities is prohibited. All computers, networks, E-mail, and storage -systems are property of the United States Government. Any misuse or -unauthorized access is prohibited, and is subject to criminal and civil -penalties. OLCF systems are provided to our users without any warranty. -OLCF will not be held liable in the event of any system failure or data -loss or corruption for any reason including, but not limited to: -negligence, malicious action, accidental loss, software errors, hardware -failures, network losses, or inadequate configuration of any computing -resource or ancillary system. - -Data Use --------- - -Prohibited Data -^^^^^^^^^^^^^^^ - -The OLCF computer systems are operated as research systems and only -contain data related to scientific research and do not contain -personally identifiable information (data that falls under the Privacy -Act of 1974 5U.S.C. 552a). Use of OLCF resources to store, manipulate, -or remotely access any national security information is strictly -prohibited. This includes, but is not limited to: classified -information, unclassified controlled nuclear information (UCNI), naval -nuclear propulsion information (NNPI), the design or development of -nuclear, biological, or chemical weapons or any weapons of mass -destruction. Authors/generators/owners of information are responsible -for its correct categorization as sensitive or non-sensitive. Owners of -sensitive information are responsible for its secure handling, -transmission, processing, storage, and disposal on OLCF systems. -Principal investigators, users, or project delegates that use OLCF -resources, or are responsible for overseeing projects that use OLCF -resources, are strictly responsible for knowing whether their project -generates any of these prohibited data types or information that falls -under Export Control. For questions, contact help@nccs.gov. - -Confidentiality, Integrity, and Availability -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The OLCF systems provide protections to maintain the confidentiality, -integrity, and availability of user data. Measures include the -availability of file permissions, archival systems with access control -lists, and parity and CRC checks on data paths and files. It is the -user’s responsibility to set access controls appropriately for the data. -In the event of system failure or malicious actions, the OLCF makes no -guarantee against loss of data or that a user’s data can be accessed, -changed, or deleted by another individual. It is the user’s -responsibility to insure the appropriate level of backup and integrity -checks on critical data and programs. - -Data Modification/Destruction -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Users are prohibited from taking unauthorized actions to intentionally -modify or delete information or programs. - -Data Retention -^^^^^^^^^^^^^^ - -The OLCF reserves the right to remove any data at any time and/or -transfer data to other users working on the same or similar project once -a user account is deleted or a person no longer has a business -association with the OLCF. After a sensitive project has ended or has -been terminated, all data related to the project must be purged from all -OLCF computing resources within 30 days. - -Software Use ------------- - -All software used on OLCF computers must be appropriately acquired and -used according to the appropriate software license agreement. -Possession, use, or transmission of illegally obtained software is -prohibited. Likewise, users shall not copy, store, or transfer -copyrighted software, except as permitted by the owner of the copyright. -Only export-controlled codes approved by the Export Control Office may -be run by parties with sensitive data agreements. - -Malicious Software -^^^^^^^^^^^^^^^^^^ - -Users must not intentionally introduce or use malicious software such as -computer viruses, Trojan horses, or worms. - -Reconstruction of Information or Software -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Users are not allowed to reconstruct information or software for which -they are not authorized. This includes but is not limited to any reverse -engineering of copyrighted software or firmware present on OLCF -computing resources. - -User Accountability -------------------- - -Users are accountable for their actions and may be held accountable to -applicable administrative or legal sanctions. - -Monitoring and Privacy -^^^^^^^^^^^^^^^^^^^^^^ - -Users are advised that there is no expectation of privacy of your -activities on any system that is owned by, leased or operated by -UT-Battelle on behalf of the U.S. Department of Energy (DOE). The -Company retains the right to monitor all activities on these systems, to -access any computer files or electronic mail messages, and to disclose -all or part of information gained to authorized individuals or -investigative agencies, all without prior notice to, or consent from, -any user, sender, or addressee. This access to information or a system -by an authorized individual or investigative agency is in effect during -the period of your access to information on a DOE computer and for a -period of three years thereafter. OLCF personnel and users are required -to address, safeguard against, and report misuse, abuse and criminal -activities. Misuse of OLCF resources can lead to temporary or permanent -disabling of accounts, loss of DOE allocations, and administrative or -legal actions. Users who have not accessed a OLCF computing resource in -at least 6 months will be disabled. They will need to reapply to regain -access to their account. All users must reapply annually. - -Authentication and Authorization -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -All users are required to use a one-time password for authentication. -Tokens will be distributed to OLCF users. Users will be required to -create a Personal Identification Number (PIN). This is used in -conjunction with a generated token code as part of a two-factor -authentication implementation. Accounts on the OLCF machines are for the -exclusive use of the individual user named in the account application. -Users should not share accounts or tokens with anyone. If evidence is -found that more than one person is using an account, that account will -be disabled immediately. Users are not to attempt to receive unintended -messages or access information by some unauthorized means, such as -imitating another system, impersonating another user or other person, -misuse of legal user credentials (usernames, tokens, etc.), or by -causing some system component to function incorrectly. Users are -prohibited from changing or circumventing access controls to allow -themselves or others to perform actions outside their authorized -privileges. Users must notify the OLCF immediately when they become -aware that any of the accounts used to access OLCF have been -compromised. Users should inform the OLCF promptly of any changes in -their contact information (E-mail, phone, affiliation, etc.) Updates -should be sent to accounts@ccs.ornl.gov. - -Foreign National Access -^^^^^^^^^^^^^^^^^^^^^^^ - -Applicants who appear on a restricted foreign country listing in section -15 CFR 740.7 License Exceptions for Computers are denied access based on -US Foreign Policy. The countries cited are Cuba, Iran, North Korea, -Sudan, and Syria. Additionally, no work may be performed on OLCF -computers on behalf of foreign nationals from these countries. - -Denial of Service -^^^^^^^^^^^^^^^^^ - -Users may not deliberately interfere with other users accessing system -resources.   - -Data Management Policy -====================== - -.. note:: - This details an official policy of the OLCF, and must be - agreed to by the following persons as a condition of access to or use of - OLCF computational resources: - - - Principal Investigators (Non-Profit) - - Principal Investigators (Industry) - - All Users - - **Title:** Data Management Policy **Version:** 20.02 - -Introduction ------------- - -The OLCF provides a comprehensive suite of hardware and software -resources for the creation, manipulation, and retention of scientific -data. This document comprises guidelines for acceptable use of those -resources. It is an official policy of the OLCF, and as such, must be -agreed to by relevant parties as a condition of access to and use of -OLCF computational resources. - -Data Storage Resources -^^^^^^^^^^^^^^^^^^^^^^ - -The OLCF provides an array of data storage platforms, each designed with -a particular purpose in mind. Storage areas are broadly divided into two -categories: those intended for user data and those intended for project -data. Within each of the two categories, we provide different sub-areas, -each with an intended purpose: - -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Purpose | Storage Area | Path | -+==================================================================================================+===================+============================================+ -| Long-term data for routine access that is unrelated to a project | *User Home* | ``/ccs/home/[userid]`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Long-term data for archival access that is unrelated to a project | *User Archive* | ``/home/[userid]`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Long-term project data for routine access that's shared with other project members | *Project Home* | ``/ccs/proj/[projid]`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Short-term project data for fast, batch job access that you don't want to share | *Member Work* | ``/gpfs/alpine/[projid]/scratch/[userid]`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Short-term project data for fast, batch job access that's shared with other project members | *Project Work* | ``/gpfs/alpine/[projid]/proj-shared`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Short-term project data for fast, batch job access that's shared with those outside your project | *World Work* | ``/gpfs/alpine/[projid]/world-shared`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Long-term project data for archival access that you don't want to share | *Member Archive* | ``/hpss/prod/[projid]/users/$USER`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Long-term project data for archival access that's shared with other project members | *Project Archive* | ``/hpss/prod/[projid]/proj-shared`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ -| Long-term project data for archival access that's shared with those outside your project | *World Archive* | ``/hpss/prod/[projid]/world-shared`` | -+--------------------------------------------------------------------------------------------------+-------------------+--------------------------------------------+ - -For more information about using the data storage archiving -systems, please refer to the pages on :ref:`data-storage-and-transfers`. - -User Home -^^^^^^^^^ - -Home directories for each user are NFS-mounted on all OLCF systems and -are intended to store long-term, frequently-accessed user data. User -Home areas are backed up on a daily basis. This file system does not -generally provide the input/output (I/O) performance required by most -compute jobs, and is not available to compute jobs on most systems. See -the section :ref:`retention-policy` for more details on -applicable quotas, backups, purge, and retention timeframes. - -User Archive -^^^^^^^^^^^^ - -The High Performance Storage System (HPSS) is the tape-archive storage -system at the OLCF and is the storage technology that supports the User -Archive areas. HPSS is intended for data that do not require day-to-day -access. - -.. note:: - Use of this directory for data storage is deprecated in favor of storing - data in the User, Project, and World Archive directories. For new users, - this directory is a "link farm" with symlinks to that user's /hpss/prod - directories. Data for existing users remains in this directory but should - be moved into a User/Project/World Archive directory, at which time this - directory will automatically convert to a link farm. - -Project Home -^^^^^^^^^^^^ - -Project Home directories are NFS-mounted on selected OLCF systems and -are intended to store long-term, frequently-accessed data that is needed -by all collaborating members of a project. Project Home areas are backed -up on a daily basis. This file system does not generally provide the -input/output (I/O) performance required by most compute jobs, and is not -available to compute jobs on most systems. See the section -:ref:`retention-policy` for more details on applicable -quotas, backups, purge, and retention timeframes. - -Member Work -^^^^^^^^^^^ - -Project members get an individual Member Work directory for each associated -project; these reside in the center-wide, high-capacity Spectrum Scale file -system on large, fast disk areas intended for global (parallel) access to -temporary/scratch storage. Member Work areas are not shared with other -users of the system and are intended for project data that the user does -not want to make available to other users. Member Work directories are -provided commonly across all systems. Because of the scratch nature of the -file system, it is not backed up and files are automatically purged on a -regular basis. Files should not be retained in this file system for long, -but rather should be migrated to Project Home or Project Archive space as -soon as the files are not actively being used. If a file system associated -with your Member Work directory is nearing capacity, the OLCF may contact -you to request that you reduce the size of your Member Work directory. See -the section :ref:`retention-policy` for more details on applicable quotas, -backups, purge, and retention timeframes. - -Project Work -^^^^^^^^^^^^ - -Each project is granted a Project Work directory; these reside in the -center-wide, high-capacity Spectrum Scale file system on large, fast disk -areas intended for global (parallel) access to temporary/scratch storage. -Project Work directories can be accessed by all members of a project and -are intended for sharing data within a project. Project Work directories -are provided commonly across most systems. Because of the scratch nature of -the file system, it is not backed up and files are automatically purged on -a regular bases. Files should not be retained in this file system for long, -but rather should be migrated to Project Home or Project Archive space as -soon as the files are not actively being used. If a file system associated -with Project Work storage is nearing capacity, the OLCF may contact the PI -of the project to request that he or she reduce the size of the Project -Work directory. See the section :ref:`retention-policy` for more details on -applicable quotas, backups, purge, and retention timeframes. - -World Work -^^^^^^^^^^ - -Each project has a World Work directory that resides in the center-wide, -high-capacity Spectrum Scale file system on large, fast disk areas intended -for global (parallel) access to temporary/scratch storage. World Work areas -can be accessed by all users of the system and are intended for sharing of -data between projects. World Work directories are provided commonly across -most systems. Because of the scratch nature of the file system, it is not -backed up and files are automatically purged on a regular bases. Files -should not be retained in this file system for long, but rather should be -migrated to Project Home or Project Archive space as soon as the files are -not actively being used. If a file system associated with World Work -storage is nearing capacity, the OLCF may contact the PI of the project to -request that he or she reduce the size of the World Work directory. See the -section :ref:`retention-policy` for more details on applicable quotas, -backups, purge, and retention timeframes. - -Member Archive -^^^^^^^^^^^^^^ - -Project members get an individual Member Archive directory for each -associated project; these reside on the High Performance Storage System -(HPSS), OLCF's tape-archive storage system. Member Archive areas are not -shared with other users of the system and are intended for project data -that the user does not want to make available to other users. HPSS is -intended for data that do not require day-to-day access. Users should not -store data unrelated to OLCF projects on HPSS. Users should periodically -review files and remove unneeded ones. See the section -:ref:`retention-policy` for more details on applicable quotas, backups, -purge, and retention timeframes. - -Project Archive -^^^^^^^^^^^^^^^ - -Each project is granted a Project Archive directory; these reside on the -High Performance Storage System (HPSS), OLCF's tape-archive storage system. -Project Archive directories are shared among all members of a project and -are intended for sharing data within a project. HPSS is intended for data -that do not require day-to-day access. Users should not store data -unrelated to OLCF projects on HPSS. Project members should also -periodically review files and remove unneeded ones. See the section -:ref:`retention-policy` for more details on applicable quotas, backups, -purge, and retention timeframes. - -World Archive -^^^^^^^^^^^^^ - -Each project is granted a World Archive directory; these reside on the High -Performance Storage System (HPSS), OLCF's tape-archive storage system. -World Archive areas are shared among all users of the system and are -intended for sharing data between projects. HPSS is intended for data that -do not require day-to-day access. Users should not store data unrelated to -OLCF projects on HPSS. Users should periodically review files and remove -unneeded ones. See the section :ref:`retention-policy` for more details on -applicable quotas, backups, purge, and retention timeframes. - - -.. _retention-policy: - -Data Retention, Purge, & Quotas -------------------------------- - -Summary -^^^^^^^ - -The following table details quota, backup, purge, and retention -information for each user-centric and project-centric storage area -available at the OLCF. - -**User-Centric Storage Areas** - -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| Area | Path | Type | Permissions | Quota | Backups | Purged | Retention | On Compute Nodes | -+=====================+=============================================+================+=============+========+=========+=========+============+==================+ -| User Home | ``/ccs/home/[userid]`` | NFS | User set | 50 GB | Yes | No | 90 days | Read-only | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| User Archive [#f1]_ | ``/home/[userid]`` | HPSS | User set | 2TB | No | No | 90 days | No | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| User Archive [#f2]_ | ``/home/[userid]`` | HPSS | 700 | N/A | N/A | N/A | N/A | No | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ - -**Project-Centric Storage Areas** - -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| Area | Path | Type | Permissions | Quota | Backups | Purged | Retention | On Compute Nodes | -+=====================+=============================================+================+=============+========+=========+=========+============+==================+ -| Project Home | ``/ccs/proj/[projid]`` | NFS | 770 | 50 GB | Yes | No | 90 days | Read-only | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| Member Work | ``/gpfs/alpine/[projid]/scratch/[userid]`` | Spectrum Scale | 700 [#f3]_ | 50 TB | No | 90 days | N/A [#f4]_ | Yes | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| Project Work | ``/gpfs/alpine/[projid]/proj-shared`` | Spectrum Scale | 770 | 50 TB | No | 90 days | N/A [#f4]_ | Yes | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| World Work | ``/gpfs/alpine/[projid]/world-shared`` | Spectrum Scale | 775 | 50 TB | No | 90 days | N/A [#f4]_ | Yes | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| Member Archive | ``/hpss/prod/[projid]/users/$USER`` | HPSS | 700 | 100 TB | No | No | 90 days | No | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| Project Archive | ``/hpss/prod/[projid]/proj-shared`` | HPSS | 770 | 100 TB | No | No | 90 days | No | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ -| World Archive | ``/hpss/prod/[projid]/world-shared`` | HPSS | 775 | 100 TB | No | No | 90 days | No | -+---------------------+---------------------------------------------+----------------+-------------+--------+---------+---------+------------+------------------+ - -| *Area -* The general name of storage area. -| *Path -* The path (symlink) to the storage area's directory. -| *Type -* The underlying software technology supporting the storage area. -| *Permissions -* UNIX Permissions enforced on the storage area's top-level directory. -| *Quota -* The limits placed on total number of bytes and/or files in the storage area. -| *Backups -* States if the data is automatically duplicated for disaster recovery purposes. -| *Purged -* Period of time, post-file-access, after which a file will be marked as eligible for permanent deletion. -| *Retention -* Period of time, post-account-deactivation or post-project-end, after which data will be marked as eligible for permanent deletion. - - **Important!** Files within "Work" directories (i.e., Member Work, - Project Work, World Work) are *not* backed up and are *purged* on a - regular basis according to the timeframes listed above. - -.. rubric:: Footnotes - -.. [#f1] This entry is for legacy User Archive directories which contained user data on January 14, 2020. There is also a quota/limit of 2,000 files on this directory. - -.. [#f2] User Archive directories that were created (or had no user data) after January 14, 2020. Settings other than permissions are not applicable because directories are root-owned and contain no user files. - -.. [#f3] Permissions on Member Work directories can be controlled to an extent by project members. By default, only the project member has any accesses, but accesses can be granted to other project members by setting group permissions accordingly on the Member Work directory. The parent directory of the Member Work directory prevents accesses by "UNIX-others" and cannot be changed (security measures). - -.. [#f4] Retention is not applicable as files will follow purge cycle. - -On Summit, Rhea and the DTNs, additional paths to the various project-centric work areas are available -via the following symbolic links and/or environment variables: - -- Member Work Directory: ``/gpfs/alpine/scratch/[userid]/[projid]`` or ``$MEMBERWORK/[projid]`` -- Project Work Directory: ``/gpfs/alpine/proj-shared/[projid]`` or ``$PROJWORK/[projid]`` -- World Work Directory: ``/gpfs/alpine/world-shared/[projid]`` or ``$WORLDWORK/[projid]`` - - -Data Retention Overview -^^^^^^^^^^^^^^^^^^^^^^^ - -By default, there is no lifetime retention for any data on OLCF -resources. The OLCF specifies a limited post-deactivation timeframe -during which user and project data will be retained. When the retention -timeframe expires, the OLCF retains the right to delete data. If you -have data retention needs outside of the default policy, please notify -the OLCF. - -User Data Retention -^^^^^^^^^^^^^^^^^^^ - -The user data retention policy exists to reclaim storage space after a -user account is deactivated, e.g., after the user’s involvement on all -OLCF projects concludes. By default, the OLCF will retain data in -user-centric storage areas only for a designated amount of time after -the user’s account is deactivated. During this time, a user can request -a temporary user account extension for data access. See the section -:ref:`retention-policy` for details on retention -timeframes for each user-centric storage area. - -Project Data Retention -^^^^^^^^^^^^^^^^^^^^^^ - -The project data retention policy exists to reclaim storage space after -a project ends. By default, the OLCF will retain data in project-centric -storage areas only for a designated amount of time after the project end -date. During this time, a project member can request a temporary user -account extension for data access. See the section :ref:`retention-policy` -for details on purge and retention timeframes -for each project-centric storage area. - -Sensitive Project Data Retention -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For sensitive projects only, all data related to the project must be -purged from all OLCF computing resources within 30 days of the project’s -end or termination date. - -Transfer of Member Work and Member Archive Data -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Although the Member Work and Member Archive directories are for storage -of data a user does not want to make available to other users on the -system, files in these directories are still considered project data -and can be reassigned to another user at the PI's request. - -Data Purges -^^^^^^^^^^^ - -Data purge mechanisms are enabled on some OLCF file system directories -in order to maintain sufficient disk space availability for job -execution. Files in these scratch areas are automatically purged on a -regular purge timeframe. If a file system with an active purge policy is -nearing capacity, the OLCF may contact you to request that you reduce -the size of a directory within that file system, even if the purge -timeframe has not been exceeded. See the section :ref:`retention-policy` -for details on purge timeframes for each storage area, if applicable. - -Storage Space Quotas -^^^^^^^^^^^^^^^^^^^^ - -Each user-centric and project-centric storage area has an associated -quota, which could be a hard (systematically-enforceable) quota or a -soft (policy-enforceable) quota. Storage usage will be monitored -continually. When a user or project exceeds a soft quota for a storage -area, the user or project PI will be contacted and will be asked if at -all possible to purge data from the offending area. See the section -:ref:`retention-policy` for details on quotas for each storage area. - -Data Prohibitions & Safeguards ------------------------------- - -Prohibited Data -^^^^^^^^^^^^^^^ - -The OLCF computer systems are operated as research systems and only -contain data related to scientific research and do not contain -personally identifiable information (data that falls under the Privacy -Act of 1974 5U.S.C. 552a). Use of OLCF resources to store, manipulate, -or remotely access any national security information is strictly -prohibited. This includes, but is not limited to: classified -information, unclassified controlled nuclear information (UCNI), naval -nuclear propulsion information (NNPI), the design or development of -nuclear, biological, or chemical weapons or any weapons of mass -destruction. Authors/generators/owners of information are responsible -for its correct categorization as sensitive or non-sensitive. Owners of -sensitive information are responsible for its secure handling, -transmission, processing, storage, and disposal on OLCF systems. -Principal investigators, users, or project delegates that use OLCF -resources, or are responsible for overseeing projects that use OLCF -resources, are strictly responsible for knowing whether their project -generates any of these prohibited data types or information that falls -under Export Control. For questions, contact help@olcf.ornl.gov. - -Unauthorized Data Modification -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Users are prohibited from taking unauthorized actions to intentionally -modify or delete information or programs. - -Data Confidentiality, Integrity, & Availability -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The OLCF systems provide protections to maintain the confidentiality, -integrity, and availability of user data. Measures include: the -availability of file permissions, archival systems with access control -lists, and parity/CRC checks on data paths/files. It is the user’s -responsibility to set access controls appropriately for data. In the -event of system failure or malicious actions, the OLCF makes no -guarantee against loss of data nor makes a guarantee that a user’s data -could not be potentially accessed, changed, or deleted by another -individual. It is the user’s responsibility to insure the appropriate -level of backup and integrity checks on critical data and programs. - -Administrator Access to Data -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -OLCF resources are federal computer systems, and as such, users should -have no explicit or implicit expectation of privacy. OLCF employees and -authorized vendor personnel with “root” privileges have access to all -data on OLCF systems. Such employees can also login to OLCF systems as -other users. As a general rule, OLCF employees will not discuss your -data with any unauthorized entities nor grant access to data files to -any person other than the UNIX “owner” of the data file, except in the -following situations: - -- When the owner of the data requests a change of ownership for any - reason, e.g., the owner is leaving the project and grants the PI - ownership of the data. -- In situations of suspected abuse/misuse computational resources, - criminal activity, or cyber-security violations. - -Note that the above applies even to project PIs. In general, the OLCF -will not overwrite existing UNIX permissions on data files owned by -project members for the purpose of granting access to the project PI. -Project PIs should work closely with project members throughout the -duration of the project to ensure UNIX permissions are set -appropriately. - -Software --------- - -Software Licensing -^^^^^^^^^^^^^^^^^^ - -All software used on OLCF computers must be appropriately acquired and -used according to the appropriate software license agreement. -Possession, use, or transmission of illegally obtained software is -prohibited. Likewise, users shall not copy, store, or transfer -copyrighted software, except as permitted by the owner of the copyright. -Only export-controlled codes approved by the Export Control Office may -be run by parties with sensitive data agreements. - -Malicious Software -^^^^^^^^^^^^^^^^^^ - -Users must not intentionally introduce or use malicious software, -including but not limited to, computer viruses, Trojan horses, or -computer worms. - -Reconstruction of Information or Software -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Users are not permitted to reconstruct information or software for which -they are not authorized. This includes but is not limited to any reverse -engineering of copyrighted software or firmware present on OLCF -computing resources.   - -Security Policy -=============== - -.. note:: - This details an official policy of the OLCF, and must be - agreed to by the following persons as a condition of access to or use of - OLCF computational resources: - - - Principal Investigators (Non-Profit) - - Principal Investigators (Industry) - - All Users - - **Title:**\ Security Policy **Version:** 12.10 - -The Oak Ridge Leadership Computing Facility (OLCF) computing resources -are provided to users for research purposes. All users must agree to -abide by all security measures described in this document. Failure to -comply with security procedures will result in termination of access to -OLCF computing resources and possible legal actions. - -Scope ------ - -The requirements outlined in this document apply to all individuals who -have an OLCF account. It is your responsibility to ensure that all -individuals have the proper need-to-know before allowing them access to -the information on OLCF computing resources. This document will outline -the main security concerns. - -Personal Use ------------- - -OLCF computing resources are for business use only. Installation or use -of software for personal use is not allowed. Incidents of abuse will -result in account termination. Inappropriate uses include, but are not -limited to: - -- Sexually oriented information -- Downloading, copying, or distributing copyrighted materials without - prior permission from the owner -- Downloading or storing large files or utilizing streaming media for - personal use (e.g., music files, graphic files, internet radio, video - streams, etc.) -- Advertising, soliciting, or selling - -Accessing OLCF Computational Resources --------------------------------------- - -Access to systems is provided via Secure Shell version 2 (sshv2). You -will need to ensure that your ssh client supports keyboard-interactive -authentication. The method of setting up this authentication varies from -client to client, so you may need to contact your local administrator -for assistance. Most new implementations support this authentication -type, and many ssh clients are available on the web. Login sessions will -be automatically terminated after a period of inactivity. When you apply -for an account, you will be mailed an RSA SecurID token. You will also -be sent a request to complete identity verification. When your account is -approved, your RSA SecurID token will also be enabled. Please refer to our -:ref:`system-user-guides` for more information on host access. DO NOT share your -PIN or RSA SecurID token with anyone. Sharing of accounts will result in -termination. If your SecurID token is stolen or misplaced, contact the OLCF -immediately and report the missing token. Upon termination of your account -access, return the token to the OLCF in person or via mail. - -Data Management ---------------- - -The OLCF uses a standard file system structure to assist users with data -organization on OLCF systems. Complete details about all file systems -available to OLCF users can be found in the Data Management Policy -section. - -Sensitive Data --------------- - -Additional file systems and file protections may be employed for -sensitive data. If you are a user on a project producing sensitive data, -further instructions will be given by the OLCF. The following guidelines -apply to sensitive data: - -- Only store sensitive data in designated locations. Do not store - sensitive data in your User Home directory. -- Never allow access to your sensitive data to anyone outside of your - group. -- Transfer of sensitive data must be through the use encrypted methods - (scp, sftp, etc). -- All sensitive data must be removed from all OLCF resources when your - project has concluded. - -Data Transfer -------------- - -The OLCF offers two dedicated data transfer nodes to users. The nodes have been -tuned specifically for wide area data transfers, and also perform well on the -local area. There are also several utilities that the OLCF recommends for data -transfer. Please refer to our :ref:`system-user-guides` for information about -the DTNs and available utilities. - -.. Titan Scheduling Policy -.. ======================= -.. -.. .. note:: -.. This details an official policy of the OLCF, and must be -.. agreed to by the following persons as a condition of access to or use of -.. OLCF computational resources: -.. -.. - Principal Investigators (Non-Profit) -.. - Principal Investigators (Industry) -.. - All Users -.. -.. **Title:** Titan Scheduling Policy **Version:** 13.02 -.. -.. In a simple batch queue system, jobs run in a first-in, first-out (FIFO) -.. order. This often does not make effective use of the system. A large job -.. may be next in line to run. If the system is using a strict FIFO queue, -.. many processors sit idle while the large job waits to run. *Backfilling* -.. would allow smaller, shorter jobs to use those otherwise idle resources, -.. and with the proper algorithm, the start time of the large job would not -.. be delayed. While this does make more effective use of the system, it -.. indirectly encourages the submission of smaller jobs. -.. -.. The DOE Leadership-Class Job Mandate -.. ------------------------------------ -.. -.. As a DOE Leadership Computing Facility, the OLCF has a mandate that a -.. large portion of Titan's usage come from large, *leadership-class* (aka -.. *capability*) jobs. To ensure the OLCF complies with DOE directives, we -.. strongly encourage users to run jobs on Titan that are as large as their -.. code will warrant. To that end, the OLCF implements queue policies that -.. enable large jobs to run in a timely fashion. -.. -.. .. note:: -.. The OLCF implements queue policies that encourage the -.. submission and timely execution of large, leadership-class jobs on -.. Titan. -.. -.. The basic priority-setting mechanism for jobs waiting in the queue is -.. the time a job has been waiting relative to other jobs in the queue. -.. However, several factors are applied by the batch system to modify the -.. *apparent* time a job has been waiting. These factors include: -.. -.. - The number of nodes requested by the job. -.. - The queue to which the job is submitted. -.. - The 8-week history of usage for the project associated with the job. -.. - The 8-week history of usage for the user associated with the job. -.. -.. If your jobs require resources outside these queue policies, please complete the -.. relevant request form on the `Special Requests -.. `__ -.. page. If you have any questions or comments on the queue policies below, please -.. direct them to the User Assistance Center. -.. -.. Job Priority by Processor Count -.. ------------------------------- -.. -.. Jobs are *aged* according to the job's requested processor count (older -.. age equals higher queue priority). Each job's requested processor count -.. places it into a specific *bin*. Each bin has a different aging -.. parameter, which all jobs in the bin receive. -.. -.. +-------+-------------+-------------+------------------------+----------------------+ -.. | Bin | Min Nodes | Max Nodes | Max Walltime (Hours) | Aging Boost (Days) | -.. +=======+=============+=============+========================+======================+ -.. | 1 | 11,250 | -- | 24.0 | 15 | -.. +-------+-------------+-------------+------------------------+----------------------+ -.. | 2 | 3,750 | 11,249 | 24.0 | 5 | -.. +-------+-------------+-------------+------------------------+----------------------+ -.. | 3 | 313 | 3,749 | 12.0 | 0 | -.. +-------+-------------+-------------+------------------------+----------------------+ -.. | 4 | 126 | 312 | 6.0 | 0 | -.. +-------+-------------+-------------+------------------------+----------------------+ -.. | 5 | 1 | 125 | 2.0 | 0 | -.. +-------+-------------+-------------+------------------------+----------------------+ -.. -.. FairShare Scheduling Policy -.. --------------------------- -.. -.. FairShare, as its name suggests, tries to push each user and project -.. towards their fair share of the system's utilization: in this case, 5% -.. of the system's utilization per user and 10% of the system's utilization -.. per project. To do this, the job scheduler adds (30) minutes priority -.. aging per user and (1) hour of priority aging per project for every (1) -.. percent the user or project is under its fair share value for the prior -.. (8) weeks. Similarly, the job scheduler subtracts priority in the same -.. way for users or projects that are over their fair share. For instance, -.. a user who has personally used 0.0% of the system's utilization over the -.. past (8) weeks who is on a project that has also used 0.0% of the -.. system's utilization will get a (12.5) hour bonus (5 \* 30 min for the -.. user + 10 \* 1 hour for the project). In contrast, a user who has -.. personally used 0.0% of the system's utilization on a project that has -.. used 12.5% of the system's utilization would get no bonus (5 \* 30 min -.. for the user - 2.5 \* 1 hour for the project). -.. -.. ``batch`` Queue Policy -.. ---------------------- -.. -.. The ``batch`` queue is the default queue for production work on Titan. -.. Most work on Titan is handled through this queue. It enforces the -.. following policies: -.. -.. - Limit of (4) *eligible-to-run* jobs per user. -.. - Jobs in excess of the per user limit above will be placed into a -.. *held* state, but will change to eligible-to-run at the appropriate -.. time. -.. - Users may have only (2) jobs in bin 5 *running* at any time. Any -.. additional jobs will be blocked until one of the running jobs -.. completes. -.. -.. .. note:: -.. The *eligible-to-run* state is not the *running* state. -.. Eligible-to-run jobs have not started and are waiting for resources. -.. Running jobs are actually executing. -.. -.. ``killable`` Queue Policy -.. ------------------------- -.. -.. At the start of a scheduled system outage, a *queue reservation* is used -.. to ensure that no jobs are running. In the ``batch`` queue, the -.. scheduler will not start a job if it expects that the job would not -.. complete (based on the job's user-specified max walltime) before the -.. reservation's start time. In constrast, the ``killable`` queue allows -.. the scheduler to start a job even if it will *not* complete before a -.. scheduled reservation. It enforces the following policies: -.. -.. - Jobs will be killed if still running when a system outage begins. -.. - The scheduler will stop scheduling jobs in the ``killable`` queue (1) -.. hour before a scheduled outage. -.. - Maximum-job-per-user limits are the same (i.e., in conjunction with) -.. the ``batch`` queue. -.. - Any killed jobs will be automatically re-queued after a system outage -.. completes. -.. -.. ``debug`` Queue Policy -.. ---------------------- -.. -.. The ``debug`` queue is intended to provide faster turnaround times for -.. the code development, testing, and debugging cycle. For example, -.. interactive parallel work is an ideal use for the debug queue. It -.. enforces the following policies: -.. -.. - Production jobs are not allowed. -.. - Maximum job walltime of (1) hour. -.. - Limit of (1) job per user *regardless of the job's state*. -.. - Jobs receive a (2)-day priority aging boost for scheduling. -.. -.. .. warning:: -.. Users who misuse the ``debug`` queue may have further -.. access to the queue denied. -.. -.. Allocation Overuse Policy -.. ------------------------- -.. -.. Projects that overrun their allocation are still allowed to run on OLCF -.. systems, although at a reduced priority. Like the adjustment for the -.. number of processors requested above, this is an adjustment to the -.. apparent submit time of the job. However, this adjustment has the effect -.. of making jobs appear much younger than jobs submitted under projects -.. that have not exceeded their allocation. In addition to the priority -.. change, these jobs are also limited in the amount of wall time that can -.. be used. For example, consider that ``job1`` is submitted at the same -.. time as ``job2``. The project associated with ``job1`` is over its -.. allocation, while the project for ``job2`` is not. The batch system will -.. consider ``job2`` to have been waiting for a longer time than ``job1``. -.. Also projects that are at 125% of their allocated time will be limited -.. to only one running job at a time. The adjustment to the apparent submit -.. time depends upon the percentage that the project is over its -.. allocation, as shown in the table below: -.. -.. +------------------------+----------------------+--------------------------+------------------+ -.. | % Of Allocation Used | Priority Reduction | Number eligible-to-run | Number running | -.. +========================+======================+==========================+==================+ -.. | < 100% | 0 days | 4 jobs | unlimited jobs | -.. +------------------------+----------------------+--------------------------+------------------+ -.. | 100% to 125% | 30 days | 4 jobs | unlimited jobs | -.. +------------------------+----------------------+--------------------------+------------------+ -.. | > 125% | 365 days | 4 jobs | 1 job | -.. +------------------------+----------------------+--------------------------+------------------+ -.. -.. System Reservation Policy -.. ------------------------- -.. -.. Projects may request to reserve a set of processors for a period of time -.. through the reservation request form, which can be found on the `Special -.. Requests `__ -.. page. If the reservation is granted, the reserved processors will be -.. blocked from general use for a given period of time. Only users that -.. have been authorized to use the reservation can utilize those resources. -.. Since no other users can access the reserved resources, it is crucial -.. that groups given reservations take care to ensure the utilization on -.. those resources remains high. To prevent reserved resources from -.. remaining idle for an extended period of time, reservations are -.. monitored for inactivity. If activity falls below 50% of the reserved -.. resources for more than (30) minutes, the reservation will be canceled -.. and the system will be returned to normal scheduling. A new reservation -.. must be requested if this occurs. Since a reservation makes resources -.. unavailable to the general user population, projects that are granted -.. reservations will be charged (regardless of their actual utilization) a -.. CPU-time equivalent to -.. ``(# of cores reserved) * (length of reservation in hours)``. - -INCITE Allocation Under-utilization Policy -========================================== - -.. note:: - This details an official policy of the OLCF, and must be - agreed to by the following persons as a condition of access to and use - of OLCF computational resources: - - - INCITE Principal Investigators - - **Title:** INCITE Allocation Under-utilization Policy **Version:** 12.10 - -The OLCF has a *pull-back* policy for under-utilization of INCITE -allocations. Under-utilized INCITE project allocations will have -core-hours removed from their outstanding core-hour project balance at -specific times during the INCITE calendar year. The following table -summarizes the current under-utilization policy: - -+-------------+---------------------+-----------------------------------+ -| Date | Utilization to-Date | Forfeited Amount | -+=============+=====================+===================================+ -| May 1 | < 10% | Up to 30% of remaining allocation | -+ +---------------------+-----------------------------------+ -| | < 15% | Up to 15% of remaining allocation | -+-------------+---------------------+-----------------------------------+ -| September 1 | < 10% | Up to 75% of remaining allocation | -+ +---------------------+-----------------------------------+ -| | < 33% | Up to 50% of remaining allocation | -+ +---------------------+-----------------------------------+ -| | < 50% | Up to 33% of remaining allocation | -+-------------+---------------------+-----------------------------------+ - -For example, a 1,000,000 core-hour INCITE project that has utilized only -50,000 core-hours (5% of the allocation) on May 1st would forfeit (0.30 -\* 950,000) = 285,000 core-hours from their remaining allocation.   - -Project Reporting Policy -======================== - -.. note:: - This details an official policy of the OLCF, and must be - agreed to by the following persons as a condition of access to and use - of OLCF computational resources: - - - Principal Investigators (Non-Profit) - - Principal Investigators (Industry) - - **Title:** Project Reporting Policy **Version:** 12.10 - -Principal Investigators of current OLCF projects must submit a quarterly -progress report. The quarterly reports are essential as the OLCF must -diligently track the use of the center's resources. In keeping with -this, the OLCF (and DOE Leadership Computing Facilities in general) -imposes the following penalties for late submission: - -+-----------------+--------------------------------------------------------------------------------------------------------------+ -| Timeframe | Penalty | -+=================+==============================================================================================================+ -| 1 Month Late | Job submissions against offending project will be suspended. | -+-----------------+--------------------------------------------------------------------------------------------------------------+ -| 3 Months Late | Login privileges will be suspended for all OLCF resources for all users associated with offending project. | -+-----------------+--------------------------------------------------------------------------------------------------------------+ - -   - -Non-proprietary Institutional User Agreement Policy -=================================================== - -.. note:: - This details an official policy of the OLCF, and must be - agreed to by the following persons as a condition of access to and use - of OLCF computational resources: - - - Principal Investigators (Non-Profit) - - All Users - - **Title:** Non-proprietary Institutional User Agreement Policy - **Version:** 12.10 - -Users of DOE-designated User Facilities must understand and agree to the -following Institutional User Agreement clause: I understand that my -institution has entered into a User Agreement with UT-Battelle, the -management and operating contractor for the U.S. Department of Energy’s -(DOE) Oak Ridge National Laboratory (ORNL), that governs my research -ORNL’s DOE-designated User Facilities. I have read and understand my -obligations under that Agreement, including the provisions summarized -below. You may check with your institution or contact -accounts@ccs.ornl.gov if you require a copy of your User Agreement. - -Access --------- - -I understand that my access is limited to certain designated areas -and/or systems, and my access may be revoked if I pose a security, -safety, or operational risk. - -Rules and Regulations ------------------------- - -I will follow the applicable ORNL rules, regulations and requirements, -including those requirements of the ORNL User Facility. I will follow -the requirements set forth in training if assigned to me by the ORNL -User Facility. - -Safety and Health -------------------- - -I will take all reasonable precautions to protect the safety and health -of others and comply with all applicable safety and health requirements. - -Intent to Publish -------------------- - -I will use best efforts to publish the results from my use of the ORNL -User Facility in an open scientific journal or significant industry -technical journal or conference proceedings. I will `acknowledge use of -the ORNL User Facility <#olcf-acknowledgement>`__ in the publication and -notify the ORNL User Facility of any publications that result from my -use of the facility. - -Export Control ----------------- - -I will comply with all U.S. Export Control laws and regulations and be -responsible for the appropriate handling and transfer of any export -controlled information, which may require advance U.S. Government -authorization. - -Intellectual Property ------------------------- - -| I will disclose any invention conceived as a part of the work at a - ORNL User Facility and will protect the invention until a patent - application can be filed. I understand that my institution may elect - title to the invention and the U.S. Government retains rights to the - invention. - -Special Requests and Policy Exemptions -====================================== - -To request policy exemptions, please submit the appropriate webform available on -the :ref:`documents-and-forms` page. Special request forms allow a user to: - -- Request Software installations -- Request relaxed queue limits for a job -- Request a system reservation -- Request a disk quota increase -- Request a User Work area purge exemption - -Special requests are reviewed weekly by the OLCF Resource Utilization -Council. Please contact help@olcf.ornl.gov for more information. From aec55e54b57f9935b6058534d377777b682c36e6 Mon Sep 17 00:00:00 2001 From: Stefan Maerz Date: Thu, 9 Jul 2020 10:50:30 -0400 Subject: [PATCH 3/5] Updates to provide some clarification on ownership. --- accounts/AFW_data_policy_guide.rst | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/accounts/AFW_data_policy_guide.rst b/accounts/AFW_data_policy_guide.rst index 662743ca..ab589253 100644 --- a/accounts/AFW_data_policy_guide.rst +++ b/accounts/AFW_data_policy_guide.rst @@ -8,14 +8,12 @@ The NCCS computer systems are operated in support of USAF and only contain data related to scientific research and do not contain personally identifiable information (data that falls under the Privacy Act of 1974 5U.S.C. 552a). Use of NCCS resources to store, manipulate, -or remotely access any national security information is strictly -prohibited. This includes, but is not limited to: classified -information, unclassified controlled nuclear information (UCNI), naval -nuclear propulsion information (NNPI), the design or development of -nuclear, biological, or chemical weapons or any weapons of mass -destruction. Authors/generators/owners of information are responsible -for its correct categorization as sensitive or non-sensitive. Owners of -sensitive information are responsible for its secure handling, +or remotely access classified information, unclassified controlled nuclear +information (UCNI), naval nuclear propulsion information (NNPI), the design +or development of nuclear, biological, or chemical weapons or any weapons +of mass destruction is prohibited. Authors/generators/owners of information +are responsible for its correct categorization as sensitive or non-sensitive. +Owners of sensitive information are responsible for its secure handling, transmission, processing, storage, and disposal on NCCS systems. Principal investigators, users, or project delegates that use NCCS resources, or are responsible for overseeing projects that use NCCS @@ -49,9 +47,7 @@ data with any unauthorized entities nor grant access to data files to any person other than the UNIX “owner” of the data file, except in the following situations: -- When the owner of the data requests a change of ownership for any - reason, e.g., the owner is leaving the project and grants the PI - ownership of the data. +- When the owner of the data requests a change of ownership. - In situations of suspected abuse/misuse computational resources, criminal activity, or cyber-security violations. From 64dac030c3bb87c762e4fd5058745db05cd2f3f4 Mon Sep 17 00:00:00 2001 From: Preston Shires Date: Thu, 9 Jul 2020 11:45:33 -0400 Subject: [PATCH 4/5] lowercase filename, add to index --- accounts/AFW_data_policy_guide.rst | 4 ++++ accounts/index.rst | 1 + 2 files changed, 5 insertions(+) diff --git a/accounts/AFW_data_policy_guide.rst b/accounts/AFW_data_policy_guide.rst index ab589253..2790337a 100644 --- a/accounts/AFW_data_policy_guide.rst +++ b/accounts/AFW_data_policy_guide.rst @@ -1,3 +1,7 @@ +**************************** +AFW Policy Guides +**************************** + Data Use -------- diff --git a/accounts/index.rst b/accounts/index.rst index 16d1ba5c..1f7e68f7 100644 --- a/accounts/index.rst +++ b/accounts/index.rst @@ -11,6 +11,7 @@ Accounts and Projects frequently_asked_questions documents_and_forms olcf_policy_guide + afw_data_policy_guide glossary Additional Resources From 0fc24c697b26e28b3a9429fe901f08b6662e6641 Mon Sep 17 00:00:00 2001 From: Brian Date: Thu, 9 Jul 2020 13:01:04 -0400 Subject: [PATCH 5/5] Update AFW_data_policy_guide.rst Fix some minor issues. --- accounts/AFW_data_policy_guide.rst | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/accounts/AFW_data_policy_guide.rst b/accounts/AFW_data_policy_guide.rst index 2790337a..3a561340 100644 --- a/accounts/AFW_data_policy_guide.rst +++ b/accounts/AFW_data_policy_guide.rst @@ -8,8 +8,9 @@ Data Use Prohibited Data ^^^^^^^^^^^^^^^ -The NCCS computer systems are operated in support of USAF and only -contain data related to scientific research and do not contain +The National Center for Computational Science (NCCS) computer systems +are operated in support of United States Air Force (USAF) 557th Weather Wing +and only contain data related to scientific research and do not contain personally identifiable information (data that falls under the Privacy Act of 1974 5U.S.C. 552a). Use of NCCS resources to store, manipulate, or remotely access classified information, unclassified controlled nuclear @@ -114,8 +115,8 @@ Monitoring and Privacy Users are advised that there is no expectation of privacy of your activities on any system that is owned by, leased or operated by -UT-Battelle on behalf of the U.S. Department of Energy (DOE). The -Company retains the right to monitor all activities on these systems, to +UT-Battelle on behalf of the U.S. Department of Energy (DOE). +UT-Battelle retains the right to monitor all activities on these systems, to access any computer files or electronic mail messages, and to disclose all or part of information gained to authorized individuals or investigative agencies, all without prior notice to, or consent from, @@ -245,7 +246,7 @@ Project Work ^^^^^^^^^^^^ Each project is granted a Project Work directory; these reside in the -center-wide, high-performance parallel file system on large, fast disk +high-performance parallel file system on large, fast disk areas intended for global (parallel) access to temporary/scratch storage. Project Work directories can be accessed by all members of a project and are intended for sharing data within a project. Project Work directories @@ -259,7 +260,7 @@ applicable quotas, backups, purge, and retention timeframes. World Work ^^^^^^^^^^ -Each project has a World Work directory that resides in the center-wide, +Each project has a World Work directory that resides in the high-capacity parallel file system on large, fast disk areas intended for global (parallel) access to temporary/scratch storage. World Work areas can be accessed by all users of the system and are intended for sharing of @@ -319,8 +320,9 @@ Data Purges Data purge mechanisms are enabled on some NCCS file system directories in order to maintain sufficient disk space availability for job execution. By default, these purge mechanisms are disabled on Air Force partnership -file systems. Should the file system exceed critical capacity thresholds; -the NCCS reserves the right to purge files to regain file system stability. +file systems. Should the file system exceed critical capacity thresholds, +the NCCS reserves the right to purge files to regain file system stability. NCCS +will discuss this with Air Force administration before purging data. Storage Space Quotas ^^^^^^^^^^^^^^^^^^^^