This site is currently under development for the RSE-ops community.
Testing and Continuous Integration
Having Continuous Integration means that you continually test your software when a new change
is added.
Legend
Designates a mandatory point (policy).
Designates a recommended point (guideline).
Rationale: Regression tests prevent new features from affecting prior functionality, and they allow developers to have more confidence in their changes when modifying code.
Details: A series of tests (including unit tests) should be runnable by anyone after the package is built to verify a correct build. The installation should also provide tests – which could derive from or be the same as example codes – must be run and installed alongside the library for future verification.
Rationale: An executable test suite gives users confidence that the product gives correct results. It is also recommended that at least a significant subset of the tests be can be run in batch-only environments.
Details: The Git repository must contain test problems an easy way to run them and verify correctness for developers who are working on remote systems. External contributors must also have access to any results from continuous integration pipelines run against their changes before they make a pull request.
Rationale: External contributors should be able to test there code to the same standard as the development team. This ensure easier integration of new contribution and transparency in the process.
Rationale: CI helps to prevent integration problems by identifying issues as they arise instead of allowing them to compound. This process helps reduce rework, cost, and time. Examples of CI tools include Travis CI for projects hosted on GitHub, or GitLab CI for projects hosted on a GitLab instance such as LC-GitLab. Some products may use Azure Pipelines in the future.
Details: During the development process, tests should include compilation across a matrix of compilers and compiler versions, including both open source and vendor-proprietary when available
Rationale: This helps to guard against platform-specific or compiler-specific errors.
Rationale: Code quality checkers help to prevent code constructs that may result in errors in future environments where the code may run. They also reduce noise in build output that may obfuscate real errors. Code style checkers can also help enforce consistent style across a project.
