ADMIN: /preview command broken

[mroeschke]

Example: #54545 (comment)

Since the doc builds are green, this might be pointing to an issue with hosting the artifact cc @datapythonista

[lithomas1]

@datapythonista

I thought about this a little, and I think a better alternative to the preview server would be a Github Action that when the user comments preview will download the docs artifact and upload it to OVH object storage via the S3 API, as a bucket (with the bucket name being the PR number of the PR).

If there is already a bucket of that name, the action would just sync the files with the new files.

Then, it would just comment the URL to the bucket for the user to click on.

The advantage of this would be that we'd save on storage and the server costs (we only store PRs that request a preview), and it'd probably also be easier to manage (there wouldn't be auth issues when downloading the artifact inside of Github Actions).

The only disadvantage I can think of at the moment is that to get an updated version of the docs, a user would have to comment /preview again every time they want to see an updated preview.

What do you think?

Here's a little testing script that I used to upload the unzipped website folder

aws --endpoint-url https://s3.bhs.perf.cloud.ovh.net --profile default s3 mb s3://prnum
aws --endpoint-url https://s3.bhs.perf.cloud.ovh.net --profile default s3 cp website s3://prnum --recursive --acl public-read
# Set ACLs to public
aws s3api put-bucket-acl --bucket prnum --acl public-read --endpoint-url https://s3.bhs.perf.cloud.ovh.net
aws --profile default s3 website s3://prnum/ --index-document index.html --endpoint-url=https://s3.bhs.perf.cloud.ovh.net
# Website will be at http://prnum.s3-website.bhs.perf.cloud.ovh.net

EDIT: Typo

[datapythonista]

I'm working on the preview, sorry for the delay, but seems like something changed with the zip format of the artifact that can't be decompressed as before (I think).

@lithomas1 this makes sense, and something like your approach would be my favourite option too, but I couldn't find a way to get an artifact given a PR number with the GitHub API. Seems ridiculous, but the artifacts part of the API is cumbersome, and the only way I could find is to get the artifact given the CI run id, only available during the run. That's why we implemented things the way they are. I don't think the cost of storage is an issue, I don't have the numbers on the top of my head, but we're talking about cents.

Btw, the benchmarks server and the preview server are different, not sure if talking about the benchmarks server was just a typo, but in case you haven't seen it.

[lithomas1]

Yea, sorry my bad about the confusion with the typo.

I think I figured the Github API part out. The basic idea is get latest commit for a PR -> get checks for that PR -> get check matching "Doc Build and Upload" -> extract run id from the html url to the check -> list artifacts from run id

Here's the code

import { Octokit } from "@octokit/core";
const octokit = new Octokit({auth: awef})

//get ref of latest commit for PR
const pr_num = 55104
const commits = await octokit.request('GET /repos/{owner}/{repo}/pulls/{pull_number}/commits', {
    owner: 'pandas-dev',
    repo: 'pandas',
    pull_number: pr_num,
    headers: {
        'X-GitHub-Api-Version': '2022-11-28'
    }
})

console.log(commits['data'][commits['data'].length - 1])

// Get Github Actions Docbuild run for that commit
const ref = commits['data'][commits['data'].length - 1]['sha']

const checks = await octokit.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
    owner: 'pandas-dev',
    repo: 'pandas',
    //ref: 'f4e80a6a73ad908c1cfa3847f59667c6c164df19',
    ref: ref,
    headers: {
      'X-GitHub-Api-Version': '2022-11-28'
    }
})

let run_id;

for (const check of checks['data']['check_runs']) {
    if (check['name'] == 'Doc Build and Upload') {
        // extract the run_id
        // it goes github.com/pandas-dev/pandas/actions/runs/{run_id}/job/blahblah
        const regex = new RegExp("https://github.com/pandas-dev/pandas/actions/runs/([0-9]+)/*");
        console.log(check['html_url'])
        run_id = check['html_url'].match(regex)[1];
    }
}

let artifacts = await octokit.request('GET /repos/{owner}/{repo}/actions/runs/{run_id}/artifacts', {
    owner: 'pandas-dev',
    repo: 'pandas',
    run_id: run_id,
    headers: {
      'X-GitHub-Api-Version': '2022-11-28'
    }
  })

console.log(artifacts['data']['artifacts'])

[datapythonista]

Amazing, great job. I think I missed I could and had to get the laat commit from the PR and use that instead of get the CI runs from the PR directly. Happy to try this approach in the server instead of the current one. The delay after the comment will be a bit bigger, but I think we'll still be in the order of some seconds, which I think it's very reasonable.

[datapythonista]

The previewer is not working again. There was a problem with the token, and it took me some time to realize. Closing here, but I'll still work on implementing the approach described by @lithomas1 above, which is much better than the current one. I updated #50832, which I think it's a better issue than this one to keep track of it.