Skip to content

Simplify local builds with Docker Compose, Also updated README #841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Feb 23, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
source 'https://rubygems.org'
ruby '2.3.1'
gem 'jekyll', '3.5.2'

gem 'jekyll'
gem 'kramdown'
gem 'rack-jekyll'
gem 'rake'
Expand Down
7 changes: 2 additions & 5 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -85,14 +85,11 @@ PLATFORMS

DEPENDENCIES
html-proofer
jekyll (= 3.5.2)
jekyll
jekyll-redirect-from
kramdown
rack-jekyll
rake

RUBY VERSION
ruby 2.3.3p222

BUNDLED WITH
1.16.0
1.16.1
135 changes: 32 additions & 103 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,87 +1,37 @@
# scala-lang.org

This repository contains the _static_ source of [scala-lang.org](http://scala-lang.org).
This repository contains the source for [scala-lang.org](http://scala-lang.org).

It does not contain the source of any content found under the [docs.scala-lang.org](http://docs.scala-lang.org) subdomain (instead, visit the [docs.scala-lang repo](http://github.com/scala/docs.scala-lang) for that source).

This is a static site generated by [Jekyll](https://github.com/mojombo/jekyll), and uses a whole host of open-source tools including a touch of Twitter's Bootstrap.
It does not contain the source for the [docs.scala-lang.org](http://docs.scala-lang.org) subdomain. You can visit the [docs.scala-lang repository](https://github.com/scala/docs.scala-lang) if you are interested in contributing to the Scala documentation site.

## Dependencies

This site uses a Jekyll, a Ruby framework. The required Jekyll version is 3.4.0.

## Building the site

There are two ways to run Jekyll to build the site:

1. Using [Bundler], so Jekyll and accompanying gems are installed only inside this directory.
2. Using globally-installed Jekyll and accompanying gems.

The latter method is the one currently actually used on scala-lang.org. The
former method is likely most convenient for users who already have a different
version of Jekyll installed, or who are comfortable using Bundler and who don't
want anything else installed system-wide.

### Option 1) Building with Bundler

`cd` into the directory where you cloned this repository, then install the
required gems with `bundle install`. This will automatically put the gems into
`./bundle-vendor/bundle`.

Start the server in the context of the bundle:

bundle exec jekyll serve

That's it.

If that doesn't work, to guarantee that your version of Ruby, etc, completely
matches the production environment, you can also use `rbenv`. Start by `cd`ing
into the directory where you cloned this repository, then execute the following:

rbenv install 2.3.1
rbenv local 2.3.1
rbenv rehash
gem install bundle
bundle install # This will automatically put the gems into `./bundle-vendor/bundle`
bundle exec jekyll serve # Start the server in the context of the bundle

From this point, everything else should be the same, regardless of which method
you used to run Jekyll.

### Option 2) Building with global Jekyll

Install Jekyll 3.4.0 on your system using RubyGems:

gem install jekyll -v 3.4.0

After cloning, `cd`ing into the directory where you cloned this repository and
run:
To build the site you can either use [Compose](https://github.com/docker/compose) or [Bundler](https://github.com/bundler/bundler). Compose is a good option if you are just getting
started and want something simple. If you are already familiar with the Ruby ecosystem then Bundler
might be the most comfortable for you.

jekyll serve
Either way the site is built with [Jekyll](https://github.com/jekyll/jekyll) and is typeset mostly in
Markdown.

and watch the output. You should see something like:

Configuration file: /Users/ben/src/scala-lang/_config.yml
Source: /Users/ben/src/scala-lang
Destination: /Users/ben/src/scala-lang/_site
Incremental build: enabled
Generating... done.
Auto-regeneration: enabled for '/Users/ben/src/scala-lang'

### Windows and UTF-8
## Building the site
Make sure you are in the root directory of the cloned repository.
### For Compose:
```
bin/serve
```

If you get `incompatible encoding` errors when generating the site under Windows, then ensure that the
console in which you are running jekyll can work with UTF-8 characters. As described in the blog
[Solving UTF problem with Jekyll on Windows](http://joseoncode.com/2011/11/27/solving-utf-problem-with-jekyll-on-windows/)
you have to execute `chcp 65001`. This command is best added to the `jekyll.bat`-script.
### For Bundler:
```
bundle exec jekyll serve --incremental
```

## Viewing the site

Regardless of your method of running Jekyll, the generated site is available at `http://localhost:4000`.

If you add `--watch` to your Jekyll command line, Jekyll will automatically watch for changes on the filesystem. When you change a file, the console will show that jekyll is regenerating. Wait until it says `done` to refresh your browser.
## Editing the Site

## YAML Front Matter
### YAML Front Matter

The "YAML Front Matter" is nothing more than the header on each page that you intend for Jekyll to parse. It contains information such as the name of the HTML template (layout) chosen for the specific document, and the title of the document. An example YAML front matter might look like:

Expand All @@ -103,31 +53,29 @@ You can use these fields in the YAML front matter later in your document. For ex

`# {{ page.title }}` would be rendered in HTML as, `<h1>My page title</h1>`.

## Markdown
### Recommended Markdown Editor

There are dozens of guides and cheatsheets that cover Markdown syntax out there, though this screenshot from the free OS X Markdown editor, [Mou](http://mouapp.com/), is an excellent and concise reference:

![Mou screen shot](http://25.io/mou/img/1.png)
[Visual Studio Code](https://github.com/Microsoft/vscode) has great support for Scala, Git, and Markdown.

### Linking to internal pages

The least error-prone way to link between documents, to link to local images, or anything else: `[link text]({{ site.baseurl }}/path/to/page/page.html)`
The least error-prone way to make links is to use this format: `[link text]({{ site.baseurl }}/path/to/page/page.html)`

Here, `{{ site.baseurl }}` is a site-wide variable that represents the root directory of the static site. So, to display the Scala logo image, located in `img/scala-logo.png`, one must simply write: `![Img alt text]({{ site.baseurl }}/resources/img/scala-logo.png)`
`{{ site.baseurl }}` is a site-wide variable that represents the root directory of the static site. So, to display the Scala logo image you can simply write: `![Img alt text]({{ site.baseurl }}/resources/img/scala-logo.png)`

## Permalinks
### Permalinks

In this new version of the scala-lang site we've tried to follow a `pretty permalink` style, so that any generated page will have an permalink finishing in a slash character (`/`). This will tell Jekyll to build that particular page as an `index.html` inside a folder with a name as specified in the provided permalink. i.e.: if a page has a permalink as follows:
We trie to follow a [pretty permalink](https://jekyllrb.com/docs/permalinks/) style, so that any generated page will have a link finishing in a slash character (`/`). This will tell Jekyll to build that particular page as an `index.html` inside a folder with a name as specified in the provided permalink. i.e.: if a page has a permalink as follows:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try


`permalink: /what-is-scala/`

This will tell Jekyll to create a `what-is-scala` folder, with an `index.html` file inside. Links to this page will refer to the `{{site.baseurl}}/what-is-scala/`.
This will tell Jekyll to create a `what-is-scala` directory, with an `index.html` file inside. Links to this page will refer to the `{{site.baseurl}}/what-is-scala/`.

## Custom collections and data
### Custom collections and data

In the previous version of the site, data used in different pages was contained in categorized blogs. This has been changed to use custom collections. Every custom collection is a folder starting with an underscore character (`_`), containing a `markdown` file for each member of the collection. As any markdown containing a page in the site, it starts with a YAML front matter containing the data for this item, and can optionally contain markdown text to be rendered as html.
Every [collection](https://jekyllrb.com/docs/collections/) is a directory starting with an underscore character (`_`), containing a Markdown file for each member of the collection. These Markdown files start with a YAML front matter containing the data for this item, and can optionally contain markdown text to be rendered as html.

Right now there are no collections being rendered as specific pages in the site (they're only consumed internally as static data), but in the future this can be changed by specifying the global `output: true` variable in the `_config.yml` custom collections section. You'll also need to specify a layout by using the `defaults` settings in the `_config.yml` file. i.e.:
Right now there are no collections being rendered as specific pages in the site. They are only consumed internally as data. In the future this can be changed by specifying the global `output: true` variable in the `_config.yml` custom collections section. You will also need to specify a layout by using the `defaults` settings in the `_config.yml` file. i.e.:

```
defaults:
Expand All @@ -138,13 +86,11 @@ defaults:
layout: layout_name
```

To access data from a custom collection just refer to `site.<collection_name>`. The collection's name will be the name of its folder sans the underscore character. i.e.: to access the data inside `_downloads`, you can do it as follows:

`site.downloads`
To access data from a custom collection refer to `site.<collection-name>`. The collection's name will be the name of it's directory without the underscore character. i.e.: to access the data inside `_downloads`, use `site.downloads`.

Some of the data has been also modelled as YAML files inside the `_data` folder. Generally for data that is used throughout the site (i.e: the navigation bar links).
Some of our data has been modelled as YAML files inside the `_data` folder. We generally do this for data that is used throughout the whole site. For example we do this for the navigation bar links.

## Resources and Workflow
## The Backend

On every commit to the `scala/scala-lang` repository a [jenkins job](https://scala-webapps.epfl.ch/jenkins/view/All/job/production_scala-lang.org-builder/) will generate the site using jekyll and copy the resulting files to the webserver. **NOTE**: the `rsync` of this job also deletes whatever is in the webserver directory **with explicit exceptions**: we need to keep the files listed below. Kind of a hack.

Expand All @@ -158,20 +104,3 @@ There are additional files on the webserver:
- nightly builds in `/nightly/distributions/`
- nightly api builds in `/nightly/docs-xxx/`
- nightly pdf builds (spec etc) in `/nightly/pdfs`

## Templates

We have the following (general) templates:
_(Note that this is not an exhaustive list.)_

#### page.html

Example YAML front matter with all possible fields:

---
layout: page
title: I Haz Build: An Autobiography of the Build Kitten
by: Scala Jenkins (Build Kitty)
---

[Bundler]: http://bundler.io/
3 changes: 3 additions & 0 deletions bin/bundle
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
command -v docker-compose >/dev/null 2>&1 || { echo >&2 "Please install Docker Compose: https://docs.docker.com/compose/install/"; exit 1; }
docker-compose run --service-ports scala-lang bundle $@
3 changes: 3 additions & 0 deletions bin/check-internal-links
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
command -v docker-compose >/dev/null 2>&1 || { echo >&2 "Please install Docker Compose: https://docs.docker.com/compose/install/"; exit 1; }
docker-compose run --service-ports scala-lang bundle exec htmlproofer ./_site/ --only-4xx --empty-alt-ignore --allow-hash-href --disable-external
3 changes: 3 additions & 0 deletions bin/check-links
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
command -v docker-compose >/dev/null 2>&1 || { echo >&2 "Please install Docker Compose: https://docs.docker.com/compose/install/"; exit 1; }
docker-compose run --service-ports scala-lang bundle exec htmlproofer ./_site/ --only-4xx --empty-alt-ignore --allow-hash-href
3 changes: 3 additions & 0 deletions bin/jekyll
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
command -v docker-compose >/dev/null 2>&1 || { echo >&2 "Please install Docker Compose: https://docs.docker.com/compose/install/"; exit 1; }
docker-compose run --service-ports scala-lang jekyll $@
3 changes: 3 additions & 0 deletions bin/serve
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
command -v docker-compose >/dev/null 2>&1 || { echo >&2 "Please install Docker Compose: https://docs.docker.com/compose/install/"; exit 1; }
docker-compose run --service-ports scala-lang jekyll serve --incremental
12 changes: 12 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
version: "2"
services:
scala-lang:
image: jekyll/jekyll:latest
volumes:
- $PWD:/srv/jekyll
- $PWD/vendor/bundle:/usr/local/bundle
ports:
- 4000:4000
- 35729:35729
- 3000:3000
- 80:4000
Empty file added vendor/bundle/.gitkeep
Empty file.