API Site down 6/13: Angular dependency issue?

[olozzalap]

API site fails to load Angular currently, defaults to unstyled html (see screenshot). I tested on Chrome, Firefox and Safari on multiple different pages.

[llanzo]

I can verify that I've been having the same issue. ( on https://docs.angularjs.org/api )

[kbuechner]

having the same issue

[emilycoco]

+1

[jgalb]

+1

[joaquinnunez]

+1

[mullermw]

+1

[caitp]

Don't enter "+1" guys, you can just "like" the issue now, and it's less spammy that way =)

/CC @jeffbcross maybe you can poke someone to see what's wrong with the google CDN?

[Narretz]

Docs should be back up. You may need to do a hard reload for it to take effect. Thanks for bringing this up.

[olozzalap]

Looks good!

[petebacondarwin]

Hi Everyone! Sorry about the docs outage.

We made a change to our tooling so that we would have a better build experience while working offline. See 6a13460

The aim of this commit was to prevent requests to remote servers for get the release tags and check the CDN versions when the building machine was not connected to the internet. In this cases we would still be able to build but we would not get the correct version information for the "production" docs app.

This commit was merged into master at 14:57 UTC and master builds started using it on our CI server.

Unfortunately our CI server seems to think it is offline at random points during its builds, even though it is permanently connected to the internet. We are not sure why - perhaps its bandwidth or latency is a bit flakey.

So although the builds to master succeeded, resulting in updates to our docs website, the CDN information was missing, which caused the production version of the site to fail to load the angular libraries.

@gkalpak reverted the offending commit at 17:44pm but the master build takes over 25 minutes to run.
@jeffbcross manually reverted the update to the docs website which brought it back online around 18:00 UTC.

Now that the commit has been reverted, future builds should not cause the docs to fail in the same way.

We are looking at alternative mechanisms for improving our offline local builds that will not impact on the CI production builds in this way. We will also look into ways of aborting the CI builds if the internet connection is too flakey to get hold of the CDN versions.

[petebacondarwin]

Thanks @caitp for escalating this

[jeffbcross]

@petebacondarwin was there an e2e test that should have caught this breakage before it was pushed?

[petebacondarwin]

All our e2e tests run against local copies of angular rather than the production copies hosted on the CDN so, no we didn't have a test that would catch this.

Good idea though, I will look at whether we can easily implement it as part of my work of preventing this from happening again...