Skip to content

IES Pipeline conversion for weekly examples update and weekly open Api refresh #2652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Apr 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
3ab4c4d
Converted weekly generation pipeline to 1es
timayabi2020 Mar 26, 2024
f2116da
Updated file to remove invalid paths to referenced yml files
timayabi2020 Mar 26, 2024
8404502
Updated file to reference yml files correclty
timayabi2020 Mar 26, 2024
5f66f10
Corrected file path
timayabi2020 Mar 26, 2024
3b4c0da
Removed self annotation
timayabi2020 Mar 26, 2024
d33786b
Updated weekly update for examples
timayabi2020 Mar 26, 2024
b39fbc8
Corrected path
timayabi2020 Mar 26, 2024
5ef444b
Removed the self annotation
timayabi2020 Mar 26, 2024
61c4345
Rolled back to the original file created during migration
timayabi2020 Mar 26, 2024
52ccec9
Fixed paths
timayabi2020 Mar 26, 2024
1220956
Returned @self
timayabi2020 Mar 26, 2024
98ef8dd
Fixed paths
timayabi2020 Mar 26, 2024
b1fe943
Added @self
timayabi2020 Mar 26, 2024
53badcf
Added credscan suppressions
timayabi2020 Mar 26, 2024
e54f3aa
Corrected credscans declarations
timayabi2020 Mar 26, 2024
f0e821c
Corrected credscan step
timayabi2020 Mar 26, 2024
ee9c7dd
Excluded policheck
timayabi2020 Mar 26, 2024
4db016b
Fixed typo
timayabi2020 Mar 26, 2024
9a1b695
fixed location
timayabi2020 Mar 26, 2024
9ad94fd
Updated credscan file
timayabi2020 Mar 26, 2024
411533e
Excluded files that have secure string conversion
timayabi2020 Mar 26, 2024
d6f5b4a
Suppressed script analyzert secure string rules
timayabi2020 Mar 26, 2024
6128f8d
Increased pipeline timeout
timayabi2020 Apr 2, 2024
d43da5e
Removed autorest installation step from npm because we are using a lo…
timayabi2020 Apr 3, 2024
2fe2b0c
Restricted autorest installation to 3.0.509 version
timayabi2020 Apr 3, 2024
f968e16
Downgraded autorest version
timayabi2020 Apr 3, 2024
595290a
Rolled back agent
timayabi2020 Apr 4, 2024
5356213
Pointed autorest to the latest version
timayabi2020 Apr 4, 2024
9d33a20
Separated pools for source analysis and build
timayabi2020 Apr 4, 2024
5932da7
Rolled back
timayabi2020 Apr 4, 2024
4a416ec
Used custom build agent for 1ES
timayabi2020 Apr 4, 2024
dc71165
Updated pool
timayabi2020 Apr 8, 2024
5234d8a
rolled back to the original build agent
timayabi2020 Apr 9, 2024
28965e8
Merge branch 'dev' of https://github.com/microsoftgraph/msgraph-sdk-p…
timayabi2020 Apr 9, 2024
dab0046
Excluded bin folder from policheck
timayabi2020 Apr 11, 2024
aef4ffe
Testing with change notifications module first
timayabi2020 Apr 11, 2024
e26e760
Added auth helper tests to credscan
timayabi2020 Apr 11, 2024
1937a68
Excluded AuthenticationHelperTests
timayabi2020 Apr 11, 2024
c5b2b23
Added self cert password suppression
timayabi2020 Apr 11, 2024
f152a5d
Disabled rosylyn error
timayabi2020 Apr 11, 2024
e2dab4d
Added back the other modules after testing
timayabi2020 Apr 11, 2024
6f79cca
Merge branch 'dev' into 1esconversion
timayabi2020 Apr 11, 2024
54c2044
Merge branch 'dev' into 1esconversion
timayabi2020 Apr 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .azure-pipelines/config/credscan/credscan-suppressions.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,24 @@
"placeholder": "pa$$w0rd",
"_justification": "[Applications] hard code password in application password HTTP example"
},
{
"placeholder": "P@55w0rd",
"_justification": "[AuthenticationTests] hard code password in AuthenticationTest for self signed certificate"
},
{
"file": [
"src\\Authentication\\Authentication\\test\\Get-MgContext.Tests.ps1",
"src\\Authentication\\Authentication.Test\\Helpers\\AuthenticationHelpersTests.cs",
"src\\Authentication\\Authentication.Test\\Helpers\\AuthenticationHelpersTests.cs"
],
"_justification": "[Authentication] Examples contain random values recognized as secret"
},
{
"file": [
"autorest.powershell\\docs\\samples\\timeswire\\readme.md"
],
"_justification": "[AutoRest] Examples contain random values recognized as secret"
},
{
"file": [
"src\\Applications\\beta\\examples\\New-MgBetaServicePrincipalPasswordSingleSignOnCredentials.md",
Expand All @@ -29,6 +47,12 @@
"src\\DeviceManagement.Actions\\beta\\examples\\Update-MgBetaDeviceManagementVirtualEndpointOnPremisConnectionAdDomainPassword.md"
],
"_justification": "[DeviceManagement.Actions] Examples contain random values recognized as secret"
},
{
"file": [
"tools\\Tests\\loadEnv.md"
],
"_justification": "[ToolsTest] Examples contain random values recognized as secret"
}
]
}
4 changes: 2 additions & 2 deletions .azure-pipelines/config/policheck/policheck-exclusions.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
<!--Each of these exclusions is a folder name -if \[name]\exists in the file path, it will be skipped -->
<!--<Exclusion Type="FolderPathFull">ABC|XYZ</Exclusion>-->
<!--Each of these exclusions is a folder name -if any folder or file starts with "\[name]", it will be skipped -->
<Exclusion Type="FolderPathStart">openApiDocs|docs\exports|src\.*\examples</Exclusion>
<Exclusion Type="FolderPathStart">openApiDocs|docs\exports|src\.*\examples|src\.*\bin|src\.*\generated</Exclusion>
<!--Each of these file types will be completely skipped for the entire scan -->
<!--<Exclusion Type="FileType">.ABC|.XYZ</Exclusion>-->
<!--The specified file names will be skipped during the scan regardless which folder they are in -->
<!--<Exclusion Type="FileName">ABC.TXT|XYZ.CS</Exclusion>-->
<Exclusion Type="FileName">AuthenticationHelpersTests.cs</Exclusion>
</PoliCheckExclusions>
177 changes: 94 additions & 83 deletions .azure-pipelines/weekly-examples-update.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
# Copyright (c) Microsoft Corporation. All Rights Reserved. Licensed under the MIT License. See License in the project root for license information.

trigger: none # disable triggers based on commits.
pr: none # disable as a PR gate.
# This Yaml Document has been converted by ESAI Yaml Pipeline Conversion Tool.
# Please make sure to check all the converted content, it is your team's responsibility to make sure that the pipeline is still valid and functions as expected.
# This pipeline will be extended to the OneESPT template
# If you are not using the E+D shared hosted pool with windows-2022, replace the pool section with your hosted pool, os, and image name. If you are using a Linux image, you must specify an additional windows image for SDL: https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/1es-pipeline-templates/features/sdlanalysis/overview#how-to-specify-a-windows-pool-for-the-sdl-source-analysis-stage
# The Task 'PublishBuildArtifacts@1' has been converted to an output named 'Publish Examples to be reviewed as artifact' in the templateContext section.
trigger: none
pr: none
name: 'PowerShellExamplesUpdateV2 Check'
schedules:
- cron: "0 3 * * FRI" # every Friday at 3AM UTC (off hours for Redmond, Nairobi and Montréal)
displayName: 'PowerShellExamplesUpdateV2'
branches:
include:
- dev
always: true
- cron: "0 3 * * FRI"
displayName: 'PowerShellExamplesUpdateV2'
branches:
include:
- dev
always: true
parameters:
- name: PipelineTimeout
displayName: PipelineTimeout
Expand All @@ -18,77 +21,85 @@ parameters:
- name: BuildAgent
displayName: Build Agent
default: 1es-windows-ps-compute

resources:
repositories:
- repository: msgraph-sdk-powershell
type: github
endpoint: MicrosoftDocs
name: microsoftgraph/msgraph-sdk-powershell
ref: dev

jobs:
- job: PowerShellExamplesUpdateV2
pool:
name: ${{ parameters.BuildAgent }}
timeoutInMinutes: ${{ parameters.PipelineTimeout }}
steps:
- template: ./common-templates/install-sdk.yml
- task: PowerShell@2
name: "ComputeBranch"
displayName: "Compute weekly examples update branch name"
inputs:
targetType: inline
script: |
$branch = "{0}/{1}" -f "WeeklyExamplesUpdate", (Get-Date -Format yyyyMMddHHmm)
Write-Host "Compute branch: $branch"
Write-Host "##vso[task.setvariable variable=WeeklyExamplesBranch;isOutput=true]$branch"

- task: Bash@3
displayName: "Create weekly examples branch"
inputs:
targetType: inline
script: |
git status
git branch $(ComputeBranch.WeeklyExamplesBranch)
git checkout $(ComputeBranch.WeeklyExamplesBranch)
git status

- task: PowerShell@2
displayName: 'Import Examples From API reference - V2'
continueOnError: false
inputs:
targetType: 'filePath'
pwsh: true
filePath: tools\ImportExamples.ps1

- task: PublishBuildArtifacts@1
displayName: 'Publish Examples to be reviewed as artifact'
inputs:
PathtoPublish: 'examplesreport'
ArtifactName: 'ExamplesToBeReviewed'
publishLocation: 'Container'
# StoreAsTar: true

- task: PowerShell@2
displayName: Pushing to github
env:
GITHUB_TOKEN: $(GITHUB_TOKEN)
inputs:
targetType: inline
pwsh: true
script: |
git config --global user.email "GraphTooling@service.microsoft.com"
git config --global user.name "Microsoft Graph DevX Tooling"
git status
git add .
git commit -m "Updating examples"
git push --set-upstream https://$(GITHUB_TOKEN)@github.com/microsoftgraph/msgraph-sdk-powershell.git $(ComputeBranch.WeeklyExamplesBranch)
git status

- template: ./common-templates/create-pr.yml
parameters:
BaseBranch: "dev"
TargetBranch: $(ComputeBranch.WeeklyExamplesBranch)
Title: "[v2] Examples Update"
Body: "This pull request was automatically created by Azure Pipelines. **Important** Check for unexpected deletions or changes in this PR."
repositories:
- repository: msgraph-sdk-powershell
type: github
endpoint: MicrosoftDocs
name: microsoftgraph/msgraph-sdk-powershell
ref: dev
- repository: 1ESPipelineTemplates
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release
extends:
template: v1/1ES.Official.PipelineTemplate.yml@1ESPipelineTemplates
parameters:
pool: $(BuildAgent)
sdl:
credscan:
suppressionsFile: $(Build.SourcesDirectory)/.azure-pipelines/config/credscan/credscan-suppressions.json
policheck:
exclusionFile: $(Build.SourcesDirectory)/.azure-pipelines/config/policheck/policheck-exclusions.xml
customBuildTags:
- ES365AIMigrationTooling
stages:
- stage: stage
jobs:
- job: PowerShellExamplesUpdateV2
timeoutInMinutes: ${{ parameters.PipelineTimeout }}
templateContext:
outputs:
- output: pipelineArtifact
displayName: 'Publish Examples to be reviewed as artifact'
targetPath: 'examplesreport'
artifactName: 'ExamplesToBeReviewed'
publishLocation: 'Container'
steps:
- template: .azure-pipelines/common-templates/install-sdk.yml@self
- task: PowerShell@2
name: "ComputeBranch"
displayName: "Compute weekly examples update branch name"
inputs:
targetType: inline
script: |
$branch = "{0}/{1}" -f "WeeklyExamplesUpdate", (Get-Date -Format yyyyMMddHHmm)
Write-Host "Compute branch: $branch"
Write-Host "##vso[task.setvariable variable=WeeklyExamplesBranch;isOutput=true]$branch"
- task: Bash@3
displayName: "Create weekly examples branch"
inputs:
targetType: inline
script: |
git status
git branch $(ComputeBranch.WeeklyExamplesBranch)
git checkout $(ComputeBranch.WeeklyExamplesBranch)
git status
- task: PowerShell@2
displayName: 'Import Examples From API reference - V2'
continueOnError: false
inputs:
targetType: 'filePath'
pwsh: true
filePath: tools\ImportExamples.ps1
- task: PowerShell@2
displayName: Pushing to github
env:
GITHUB_TOKEN: $(GITHUB_TOKEN)
inputs:
targetType: inline
pwsh: true
script: |
git config --global user.email "GraphTooling@service.microsoft.com"
git config --global user.name "Microsoft Graph DevX Tooling"
git status
git add .
git commit -m "Updating examples"
git push --set-upstream https://$(GITHUB_TOKEN)@github.com/microsoftgraph/msgraph-sdk-powershell.git $(ComputeBranch.WeeklyExamplesBranch)
git status
- template: .azure-pipelines/common-templates/create-pr.yml@self
parameters:
BaseBranch: "dev"
TargetBranch: $(ComputeBranch.WeeklyExamplesBranch)
Title: "[v2] Examples Update"
Body: "This pull request was automatically created by Azure Pipelines. **Important** Check for unexpected deletions or changes in this PR."
Loading