Skip to content

Contracts7 #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Mar 27, 2023
Merged

Contracts7 #254

merged 17 commits into from
Mar 27, 2023

Conversation

mbaluda
Copy link
Contributor

@mbaluda mbaluda commented Mar 15, 2023

Description

  • Implements the package Contracts7:
    • MSC33-C
    • MSC39-C
    • RULE-12-2
    • RULE-19-1
  • Fixes some typos in false positives annotations
  • Changes M0-2-1 to a share query and fixes the overlap

Change request type

  • Release or process automation (GitHub workflows, internal scripts)
  • Internal documentation
  • External documentation
  • Query files (.ql, .qll, .qls or unit tests)
  • External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

  • No rules added
  • Queries have been added for the following rules:
    • MSC33-C
    • MSC39-C
    • RULE-12-2
    • RULE-19-1
  • Queries have been modified for the following rules:
    • M0-2-1

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

  • The structure or layout of the release artifacts.
  • The evaluation performance (memory, execution time) of an existing query.
  • The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

  • Yes
  • No

🚨🚨🚨
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

  • Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

  • Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Reviewer

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from github-actions bot Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from github-actions bot Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from jsinglet Mar 15, 2023
@github github deleted a comment from github-actions bot Mar 15, 2023
@jsinglet
Copy link
Contributor

🤖 Beep Boop! Matrix Testing for this PR has been completed. If no reports were posted it means this PR does not contain things that need matrix testing!

@github-actions
Copy link

🤖 Beep Boop! Matrix Testing for this PR has been initiated. Please check back later for results.

💡 If you do not hear back from me please check my status! I will report even if this PR does not contain files eligible for matrix testing.

@jsinglet
Copy link
Contributor

🤖 Beep Boop! gcc/cpp/X86_64 Matrix Testing for this PR has been completed. See below for the results!


COMPILE_ERROR_OUTPUT : 
SUITE                : AUTOSAR
COMPILE_PASS         : True
QUERY                : NewDeleteArrayMismatch
RULE                 : A18-5-3
TEST_DIFFERENCE      : 
PACKAGE              : Freed
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
SUITE                : AUTOSAR
COMPILE_PASS         : True
QUERY                : NewArrayDeleteMismatch
RULE                 : A18-5-3
TEST_DIFFERENCE      : 
PACKAGE              : Freed
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
SUITE                : AUTOSAR
COMPILE_PASS         : True
QUERY                : InfeasiblePath
RULE                 : M0-1-2
TEST_DIFFERENCE      : 
PACKAGE              : DeadCode
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
SUITE                : AUTOSAR
COMPILE_PASS         : True
QUERY                : ObjectAssignedToAnOverlappingObject
RULE                 : M0-2-1
TEST_DIFFERENCE      : 
PACKAGE              : Representation
TEST_PASS            : True


@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from github-actions bot Mar 24, 2023
@github github deleted a comment from github-actions bot Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@github github deleted a comment from jsinglet Mar 24, 2023
@jsinglet
Copy link
Contributor

🏁 Beep Boop! Performance testing complete! See below for performance of the last 3 runs vs your PR. Times are based on predicate performance. You can find full graphs and stats in the PR that was created for this test in the release engineering repo.


Release                            : v2.14.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2886922
Mean_Predicate_Execution_Time_Ms   : 108.65344373353406
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1783.906509131339
Total_Serialized_Execution_Time_s  : 2886.922
Mean_Query_Execution_Time_s        : 0.108653443733534
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 237.0
Number_of_Predicates               : 26570

Release                            : v2.14.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4498723
Mean_Predicate_Execution_Time_Ms   : 112.41186906546729
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 1841.844021324711
Total_Serialized_Execution_Time_s  : 4498.723
Mean_Query_Execution_Time_s        : 0.1124118690654672
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 224.0
Number_of_Predicates               : 40020

Release                            : v2.14.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3330223
Mean_Predicate_Execution_Time_Ms   : 125.4557543793558
Median_Predicate_Execution_Time_Ms : 4.0
Standard_Deviation_Ms              : 2125.712933594389
Total_Serialized_Execution_Time_s  : 3330.223
Mean_Query_Execution_Time_s        : 0.1254557543793558
Median_Predicate_Execution_Time_s  : 0.004
Percentile95_Ms                    : 271.0
Number_of_Predicates               : 26545

Release                            : v2.14.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3910243
Mean_Predicate_Execution_Time_Ms   : 97.5706906876934
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1583.9926359710646
Total_Serialized_Execution_Time_s  : 3910.243
Mean_Query_Execution_Time_s        : 0.0975706906876933
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 200.0
Number_of_Predicates               : 40076

Release                            : v2.15.1
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3624097
Mean_Predicate_Execution_Time_Ms   : 128.26845756353083
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 2122.113492946546
Total_Serialized_Execution_Time_s  : 3624.097
Mean_Query_Execution_Time_s        : 0.1282684575635308
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 247.34999999999852
Number_of_Predicates               : 28254

Release                            : v2.15.1
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3838315
Mean_Predicate_Execution_Time_Ms   : 93.4146608581372
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1435.040141045132
Total_Serialized_Execution_Time_s  : 3838.315
Mean_Query_Execution_Time_s        : 0.0934146608581372
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 189.0
Number_of_Predicates               : 41089

Release                            : v2.15.1
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3256343
Mean_Predicate_Execution_Time_Ms   : 114.97980297305888
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1853.6432397269127
Total_Serialized_Execution_Time_s  : 3256.343
Mean_Query_Execution_Time_s        : 0.1149798029730588
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 224.0
Number_of_Predicates               : 28321

Release                            : v2.15.1
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4293575
Mean_Predicate_Execution_Time_Ms   : 105.51398309249976
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1651.868425630411
Total_Serialized_Execution_Time_s  : 4293.575
Mean_Query_Execution_Time_s        : 0.1055139830924997
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 213.0
Number_of_Predicates               : 40692

Release                            : 254
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3827785
Mean_Predicate_Execution_Time_Ms   : 93.55456434070636
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1432.9739617329146
Total_Serialized_Execution_Time_s  : 3827.785
Mean_Query_Execution_Time_s        : 0.0935545643407063
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 190.0
Number_of_Predicates               : 40915

Release                            : 254
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3229813
Mean_Predicate_Execution_Time_Ms   : 112.13849732657454
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1851.0115773644727
Total_Serialized_Execution_Time_s  : 3229.813
Mean_Query_Execution_Time_s        : 0.1121384973265745
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 219.9499999999971
Number_of_Predicates               : 28802


🏁 Below are the slowest predicates for the last 2 releases vs this PR.


Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : cpp
Suite             : cert-default
Predicate         : SSAConstruction#2b11997e::PhiInsertion::definitionHasRedefinition#3#fff
Execution_Time_Ms : 33622

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 35318

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#134#fff#shared
Execution_Time_Ms : 137522

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 35530

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 187498

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 218676

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#ff#antijoin_rhs
Execution_Time_Ms : 214162

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 66623

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 68295

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#134#fff#shared
Execution_Time_Ms : 167904

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : #select#ff#antijoin_rhs
Execution_Time_Ms : 247019

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#134#fff#antijoin_rhs
Execution_Time_Ms : 66032

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 30539

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 34411

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 176681

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 177379

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 59007

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#134#fff#antijoin_rhs
Execution_Time_Ms : 63343

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 68001

Release           : v2.15.1
Run               : 2023-03-17_19-54-48
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnionsUsed#8fe78116::TaggedUnion#f#count_range
Execution_Time_Ms : 29742

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#134#fff#shared
Execution_Time_Ms : 144504

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#134#fff#antijoin_rhs
Execution_Time_Ms : 63760

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 62614

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 53573

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 31711

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 174374

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 31774

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : cpp
Suite             : cert-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 29160

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 178212

Release           : 254
Run               : 2023-03-24_22-27-41
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#ff#antijoin_rhs
Execution_Time_Ms : 218110


@mbaluda
Copy link
Contributor Author

mbaluda commented Mar 25, 2023

@lcartey lcartey dismissed jsinglet’s stale review March 27, 2023 13:25

Comments addressed.

@mbaluda mbaluda added this pull request to the merge queue Mar 27, 2023
Merged via the queue into github:main with commit d9dfdca Mar 27, 2023
@mbaluda mbaluda deleted the Contracts6 branch March 27, 2023 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

3 participants