the 2nd draft of QC

Author: mariojmdavid

content/06.quality_criteria.md

@@ -0,0 +1,146 @@
+## Quality Criteria
+
+The following sections describe the quality conventions and best
+practices that apply to the development, operation and integration
+phases of a **Service** with a production infrastructure for research,
+such as the EOSC ecosystem. These guidelines rule the **Service** development
+and operation process within the framework of the EOSC-Synergy project.
+
+Some of the criteria in this document is similar or based on the
+document "Software Quality Assurance baseline2 [5], for such cases the
+following tag is added to the criteria: [Ref.5-QC.XyNN] where QC.XyNN is
+the codename of the criteria in that document.
+
+### Integration Testing [SvcQC.Int]
+
+Integration testing refers to the evaluation of the interactions among
+coupled **Service** or parts of a system that cooperate to achieve a given
+functionality.
+
+* **[SvcQC.Int01]** Integration testing outcome MUST guarantee the overall
+operation of the  **Service** whenever new functionality is involved. [Ref.5-QC.Int01].
+
+* **[SvcQC.Int02]** Integration testing SHOULD be automated.
+
+* **[SvcQC.Int03]** Ad-hoc pilot **Service** infrastructures and/or local
+testbeds MAY be used to cope with the integration testing requirements. [Ref.5-QC.Int04].
+
+### Scalability tests [SvcQC.Sca]
+
+Scalability Testing is a non-functional test methodology in which an
+application’s performance is measured in terms of its ability to scale
+up or scale down the number of user requests or other such performance
+measure attributes [9].
+
+* **[SvcQC.Sca01]**
+
+### Elasticity tests [SvcQC.Ela]
+
+* **[SvcQC.Ela01]**
+
+### Acceptance and System tests [SvcQC.Acc]
+
+Test of functionality of the **Service**, tests done before the **Service**
+enters into production. Tests for the public API.
+
+* **[SvcQC.Acc01]**
+
+### Documentation [SvcQC.Doc]
+
+* **[SvcQC.Doc01]** Documentation MUST be available online, easily
+findable and accessible. [Ref.5-QC.Doc03].
+
+* **[SvcQC.Doc02]** Documentation SHOULD have a Persistent Identifier (PID).
+
+* **[SvcQC.Doc03]** Documentation MUST be version controlled. [Ref.5-QC.Doc01.1].
+
+* **[SvcQC.Doc04]** Documentation MUST be updated on new **Service** versions
+involving any change in the installation, configuration or behaviour of
+the **Service**. [Ref.5-QC.Doc04].
+
+* **[SvcQC.Doc05]** Documentation MUST be updated whenever reported
+as inaccurate or unclear. [Ref.5-QC.Doc05].
+
+* **[SvcQC.Doc06]** Documentation MUST have a non-software license.
+
+* **[SvcQC.Doc07]** Documentation MUST be produced according to the
+target audience, varying according to the **Service** specification.
+The identified types of documentation and their RECOMMENDED content are:
+
+  * **[SvcQC.Doc07.2]** Deployment and Administration. [Ref.5-QC.Doc06.3]:
+    * Installation and configuration guides.
+    * Service Reference Card, with the following RECOMMENDED content:
+      * Brief functional description.
+      * List of processes or daemons.
+      * Init scripts and options.
+      * List of configuration files, location and example or template.
+      * Log files location and other useful audit information.
+      * List of ports.
+      * Service state information.
+      * List of cron jobs.
+      * Security information.
+      * FAQs and troubleshooting.
+  * **[SvcQC.Doc07.3]** User. [Ref.5-QC.Doc06.4]:
+    * Detailed User Guide for the **Service**.
+    * Public API documentation (if applicable).
+    * Command-line (CLI) reference (if applicable).
+
+### Policies [SvcQC.Pol]
+
+* **[SvcQC.Pol01]** The **Service** MUST include the following policy documents:
+  * **[SvcQC.Pol01.1]** Acceptable Usage Policy (AUP).
+  * **[SvcQC.Pol01.2]** Access policy.
+  * **[SvcQC.Pol01.3]** Privacy policy.
+
+### Security [SvcQC.Sec]
+
+* **[SvcQC.Sec01]** The **Service** public endpoints and APIs MUST be secured
+with encryption.
+
+* **[SvcQC.Sec02]** Dynamic application security testing (DAST) [6]
+SHALL be performed from the outside, to the **Service** in an operation
+state, to look for security vulnerabilities (e.g. SQL injection,
+cross-site scripting, DDOS). [Ref.5-QC.Sec03].
+
+* **[SvcQC.Sec03]** Manual penetration testing MAY be part of the
+application security  verification effort. [Ref.5-QC.Sec04].
+
+* **[SvcQC.Sec04]** The **Service** SHOULD have an Authentication mechanism.
+
+* **[SvcQC.Sec05]** The **Service** MUST use strong ciphers for encryption.
+
+* **[SvcQC.Sec06]** Credentials used in the **Service** MUST be signed by
+a recognized certification authority.
+
+* **[SvcQC.Sec07]** The **Service** MUST validate the credentials and signatures.
+
+### Automated Deployment [SvcQC.Aud]
+
+* **[SvcQC.Aud01]** Production-ready **Service** SHALL be deployed as
+a workable system with the minimal user or system administrator interaction
+leveraging Infrastructure as Code (IaC) tools.
+
+### Support [SvcQC.Sup]
+
+* **[SvcQC.Sup01]** The **Service** MUST have a tracker or helpdesk
+for operational and users issues.
+
+* **[SvcQC.Sup02]** The **Service** SHOULD have a tracker for the
+underlying software issues. [Ref.5-QC.Man01].
+
+* **[SvcQC.Sup03]** The **Service** SHOULD include an Operational
+Level Agreement (OLA) with the infrastructure where it is integrated.
+
+* **[SvcQC.Sup04]** The **Service** MAY include Service Level
+Agreement (SLA) with user communities.
+
+### Monitoring, Alerts, Metrics [SvcQC.MAM]
+
+* **[SvcQC.MAM01]** The **Service** in an operational production state SHOULD be monitored.
+  * **[SvcQC.MAM01.1]** The **Service** public endpoints SHOULD be monitored.
+  * **[SvcQC.MAM01.2]** The **Service** public APIs SHOULD be monitored.
+  * **[SvcQC.MAM01.3]** The **Service** SHOULD for security related criteria (DAST).
+
+* **[SvcQC.MAM02]** The **Service** SHOULD have alerts.
+
+* **[SvcQC.MAM03]** The **Service** SHOULD have metrics.