Skip to content

Commit fab2b27

Browse files
rvermeulenrvermeulen
committed
Add guideline recategorization design document.
1 parent ea3f111 commit fab2b27

File tree

1 file changed

+178
-0
lines changed

1 file changed

+178
-0
lines changed

‎docs/design/guideline_recategorization.md

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Coding Standards: Guideline Recategorization
2+
3+
- [Coding Standards: Guideline Recategorization](#coding-standards-guideline-recategorization)
4+
- [Document management](#document-management)
5+
- [Introduction](#introduction)
6+
- [Design](#design)
7+
- [Guideline Recategorization Plan specification](#guideline-recategorization-plan-specification)
8+
- [Implementation](#implementation)
9+
- [Specification and deviation](#specification-and-deviation)
10+
- [Specification validation](#specification-validation)
11+
- [SARIF rewriting](#sarif-rewriting)
12+
- [Non-MISRA standards](#non-misra-standards)
13+
14+
## Document management
15+
16+
**ID**: codeql-coding-standards/design/grp<br/>
17+
**Status**: Draft
18+
19+
| Version | Date | Author(s) | Reviewer (s) |
20+
| ------- | ---------- | --------------- | ------------ |
21+
| 0.1 | 08/10/2022 | Remco Vermeulen | \<redacted\> |
22+
| 0.2 | 10/25/2022 | Remco Vermeulen | |
23+
24+
## Introduction
25+
26+
Each MISRA guideline belongs to a category that defines a policy to be followed to determine whether a guideline may be violated or not and whether a deviation is required.
27+
The document [MISRA Compliance:2020](https://www.misra.org.uk/app/uploads/2021/06/MISRA-Compliance-2020.pdf) defines the following guideline categories, and their associated policies, in addition to a mechanism to recategorize guidelines.
28+
29+
- Mandatory guidelines - guidelines for which violation is never permitted.
30+
- Required guidelines - guidelines which can only be violated when supported by a deviation.
31+
- Advisory guidelines - recommendations for which violations are identified but are not required to be supported by a deviation.
32+
33+
Guideline recategorization is possible by means of a Guideline Recategorization Plan (GRP). A GRP is a contract between the acquirer and supplier to determine how guidelines are applied.
34+
The GRP defines the additional category Disapplied to be used for Advisory guidelines which are to be ignored. Any other category can be recategorized into stricter categories to ensure that a guideline adheres to the associated policy.
35+
The following table summarizes the possible recategorizations.
36+
37+
| Category | Recategorizations |
38+
| --------- | ------------------------------- |
39+
| Mandatory | |
40+
| Required | Mandatory |
41+
| Advisory | Disapplied, Required, Mandatory |
42+
43+
Other recategorizations, from here on denoted as invalid recategorizations, are not applied and are to be reported to the user.
44+
45+
## Design
46+
47+
Our design includes a Guideline Recategorization Plan specification, logic to apply the category policy to associated guidelines, and a SARIF result rewriter to reflect the new category in the results.
48+
The application of a policy will modify the behavior of a CodeQL queries implementing guidelines as follows:
49+
50+
| Category | Effect |
51+
| ---------- | -------------------------------------------------------------------- |
52+
| Mandatory | Violations are reported, even if a deviation is applicable! |
53+
| Required | Violations are reported unless there exists an applicable deviation. |
54+
| Advisory | Violations are reported unless there exists an applicable deviation. |
55+
| Disapplied | Violations are not reported. |
56+
57+
The SARIF rewriting will update the category of a guideline in a SARIF result file by updating the necessary tag information of a query.
58+
59+
### Guideline Recategorization Plan specification
60+
61+
The Guideline Recategorization Plan specification will build upon the configuration specification introduced for deviations by adding the additional primary section `guideline-recategorizations` to the `codeql-standards.yml` configuration file.
62+
The `guideline-recategorizations` section will be a series of compact mappings in YAML with the keys:
63+
64+
- `rule-id` - the rule identifier that is recategorized.
65+
- `category` - the category assigned to the rule identified by rule-id
66+
67+
Note: We specify the recategorization based on the rule-id instead of the query-id. This can be revised if feedback requires more fine-grained recategorization.
68+
69+
For example:
70+
71+
```yaml
72+
guideline-recategorizations:
73+
- rule-id: “M5-0-17”
74+
category: “mandatory”
75+
```
76+
77+
## Implementation
78+
79+
This section discusses the implementation of the [design](#design).
80+
81+
### Specification and deviation
82+
83+
The implementation will rely on the existing rule meta-data and query exclusion mechanisms to apply policies associated with a rule’s category.
84+
The rule meta-data already includes both the `query-id` and `rule-id` associated with a query and is available during query evaluation.
85+
The rule meta-data needs to be extended with a category that contains the guideline’s category.
86+
87+
For example:
88+
89+
```ql
90+
query =
91+
// `Query` instance for the `pointerSubtractionOnDifferentArrays` query
92+
PointersPackage::pointerSubtractionOnDifferentArraysQuery() and
93+
queryId =
94+
// `@id` for the `pointerSubtractionOnDifferentArrays` query
95+
"cpp/autosar/pointer-subtraction-on-different-arrays" and
96+
ruleId = "M5-0-17" and
97+
category = “required”
98+
```
99+
100+
The category defined by the rule meta-data and the category defined in the `guideline-recategorizations` of the applicable `codeql-standards.yml` configuration file is used to determine the *effective category* of a query.
101+
The *effective category* is the category whose policy is applied during the evaluation of a query.
102+
The policy of a category dictates if a result can be deviated from and implements the effect described in the design section.
103+
The existing exclusion mechanism implemented in the predicate `isExcluded` defined in the `Exclusions.qll` library will be updated to consider the applicable policy of a guideline.
104+
105+
Note: This will change the behavior of deviations which will no longer have an impact on Mandatory guidelines! This, however, will only impact MISRA C rules because there are no MISRA C++ Guidelines with a Mandatory category.
106+
107+
### Specification validation
108+
109+
To assist users with correctly specifying a Guideline Recategorization Plan (GRP) specification we can implement two validations mechanisms that validate the specification at two different points in a GRP life cycle.
110+
The first validation mechanism will perform syntax validation of the specification provided in the guideline-recategorizations section of a `codeql-standards.yml` configuration file and can provide feedback in any editor that supports JSON schemas published at the [JSON schema store](https://www.schemastore.org/json/).
111+
A schema for `codeql-standards.yml` can be extended with the definition of `guideline-category` and the property `guideline-recategorizations`:
112+
113+
```json
114+
{
115+
"$schema": "http://json-schema.org/draft-07/schema",
116+
"additionalProperties": false,
117+
"definitions": {
118+
"guideline-category": {
119+
"enum": [
120+
"mandatory",
121+
"required",
122+
"advisory",
123+
"disapplied"
124+
]
125+
}
126+
},
127+
"properties": {
128+
"report-deviated-alerts": {...},
129+
"deviations": {...},
130+
"deviation-permits": {...},
131+
"guideline-recategorizations": {
132+
"description": "A set of guideline recategorizations",
133+
"type": "array",
134+
"items": {
135+
"type": "object",
136+
"properties": {
137+
"rule-id": {
138+
"type": "string"
139+
},
140+
"category": {
141+
"$ref": "#/definitions/guideline-category"
142+
}
143+
}
144+
}
145+
}
146+
},
147+
"required": [],
148+
"type": "object"
149+
}
150+
```
151+
152+
The second validation mechanism is the generation of a `guideline-recategorization-plan-report.md` containing alerts on semantically incorrect recategorizations.
153+
That is, possible recategorizations that are not described as valid in the introduction.
154+
Semantically invalid recategorizations will be detected by looking at a query’s categorization and its effective categorization (i.e., its applied recategorization).
155+
156+
In addition, an update to the `deviations_report.md` report’s invalidate deviations table will provide feedback to users that apply deviations to guidelines with an effective category equal to `mandatory` which cannot be deviated from.
157+
The changes to generate the new report and update the existing report will be made in the report generation script `scripts/reports/analysis_report.py`.
158+
159+
### SARIF rewriting
160+
161+
The *effective category* of a guideline is a runtime property that is not reflected in the SARIF result file and therefore is not visible in any viewer used to view the results (e.g., [Code Scanning](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning)).
162+
To ensure that users can view the *effective category* we need to rewrite the `@tags` query metadata property in the SARIF result file.
163+
The `@tags` value is a JSON array located at the [JSON path](https://datatracker.ietf.org/wg/jsonpath/about/):
164+
165+
`$.runs[?(@.tool.driver.name="CodeQL")].tool.driver.rules[*].properties.tags`
166+
167+
The category tag has the form `external/<standard>/obligation/<category>`
168+
Each guideline has an `external/<standard>/id/<rule-id>` tag that can be used to determine if a recategorization is applicable by performing a case insensitive compare on the `<rule-id>` extracted from the query’s tags array and the value of the rule-id key in a `guideline-recategorizations` section.
169+
The rewriting replaces the `<category>` part in `external/<standard>/obligation/<category>` with the newly specified category and adds a new tag `external/<standard>/original-obligation/<category>` with the rule’s original category.
170+
171+
The rewrite process translates each entry in the guideline recategorization specification into a [JSON Patch](https://datatracker.ietf.org/doc/html/rfc6902) specific to the processed SARIF file. The JSON Patch is SARIF file specific due to its reliance on [JSON Pointer](https://www.rfc-editor.org/rfc/rfc6901) to locate the obligation tags.
172+
173+
A new SARIF file is created by applying the JSON Patch to the processed SARIF file.
174+
175+
## Non-MISRA standards
176+
177+
Guideline recategorization applies to rules adhering to the MISRA categorizations.
178+
For standards that deviate from these conventions we assume that the rules have a category equivalent to MISRA’s *required* category.

0 commit comments

Comments
 (0)