Skip to content

Rebuild Custom code blocks languages #2338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions js-with-links.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
// const visit = require('unist-util-visit');
// var remark = require('remark');
// var parse = require('remark-parse');
// var html = require('remark-html');
// var stringify = require('remark-stringify');
// var fs = require('fs');

// var file = fs.readFileSync('./src/content/api/node.md', { encoding: 'utf8' });

// remark()
// .use(parse)
// .use(jsWithLinks)
// .use(html)
// .process(file, function(err, file) {
// if (file) {
// console.log('----------------');
// console.log(file.contents);
// console.log('----------------');
// }
// });

// module.exports = jsWithLinks;

// function jsWithLinks(options) {
// return function transformer(tree, file) {
// visit(tree, 'code', visitor);

// function visitor(node) {
// console.log(node)
// remark()
// .use(function() {
// return function t(tr) {
// // tr.children.forEach(ch => console.log(ch))
// };
// })
// .use(html)
// .process(node.value, (err, file) => {
// // console.log('file')
// // console.log(file)
// // console.log(node.value)
// // return file.contents;
// node.value = file.toString()
// });

// // console.log(node)
// }

// // console.log('t',transformed);
// };
// }

// function logType(tree) {
// console.log(JSON.stringify(tree, null, 2));
// }

module.exports = jslinks;

jslinks.displayName = 'jslinks';
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this property should be the same as the code snippet language.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep all the commented code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the original PR text:

File js-with-links.js will be moved to https://github.com/montogeek/remark-refractor as well.

It means commented code will be deleted, as it is right now: https://github.com/montogeek/remark-refractor/blob/master/src/index.js

jslinks.aliases = [];

function jslinks(Prism) {
Prism.languages.jslinks = Prism.languages.extend('javascript', {});

// Prism.hooks.add('wrap', function(env) {
// if (env.type === 'comment') {
// env.content = env.content.value.replace(/\[([^[\]]+?)\]\((.+?)\)/g, match => {
// match = /\[([^[\]]+?)\]\((.+?)\)/.exec(match);
// return `\x3Ca class="code-link" href="${match[2]}"\x3E${match[1]}\x3C/a\x3E`;
// });

// console.log(code)

// env.content.value = env.content.value.replace(/&#x3C;/g, '<')

// env.content.value = code;
// }
// });
}
14 changes: 1 addition & 13 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,19 +48,6 @@
"serve": "npm run build && sirv start ./dist --port 4000",
"deploy": "gh-pages -d dist"
},
"husky": {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will revert

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

"hooks": {
"pre-commit": "lint-staged"
}
},
"lint-staged": {
"*.{js,jsx,md}": [
"npm run lint:js"
],
"*.md": [
"npm run lint:markdown"
]
},
"devDependencies": {
"@octokit/rest": "^15.9.4",
"alex": "^5.1.0",
Expand All @@ -87,6 +74,7 @@
"fontgen-loader": "git://github.com/EugeneHlushko/fontgen-loader.git#a26a73843900ca4b518853952b1fc3c816103512",
"front-matter": "^2.3.0",
"gh-pages": "^1.0.0",
"github": "^10.0.0",
"html-webpack-plugin": "^2.30.1",
"html-webpack-template": "^6.1.0",
"http-server": "^0.10.0",
Expand Down
110 changes: 110 additions & 0 deletions prism-links.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
const refractor = require('refractor/core.js');
const visit = require('unist-util-visit');
const languages = require('prism-languages');

var remark = require('remark');
var parse = require('remark-parse');
var htmlRemark = require('remark-html');

refractor.register(require('refractor/lang/bash.js'));
refractor.register(require('refractor/lang/diff.js'));
refractor.register(require('refractor/lang/yaml.js'));
refractor.register(require('refractor/lang/json.js'));
refractor.register(require('refractor/lang/typescript.js'));
refractor.register(require('refractor/lang/nginx.js'));
refractor.register(require('refractor/lang/ruby.js'));
refractor.register(require('./js-with-links.js'));

// var fs = require('fs');

// var file = fs.readFileSync('./src/content/api/node.md', { encoding: 'utf8' });

// remark()
// .use(parse)
// .use(attacher)
// .use(htmlRemark)
// .process(file, function(err, file) {
// if (err) throw err;
// if (file) {
// console.log('************************************');
// console.log(file.contents);
// console.log('----------------');
// fs.writeFileSync('./index.html', file.contents);
// }
// });

function attacher({ include, exclude } = {}) {
return function transformer(tree, file) {
visit(tree, 'code', visitor);

function visitor(node) {
const { lang } = node;

if (!lang || (include && !~include.indexOf(lang)) || (exclude && ~exclude.indexOf(lang))) {
return;
}

let { data } = node;

if (!data) {
node.data = data = {};
}

try {
data.hChildren = refractor.highlight(node.value, lang);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refractor returns a hast tree.


for (let index = 0; index < data.hChildren.length; index++) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find all comments in the tree and update every one by inject a link element.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename the hChildren to something more meaningful. It can be just hastChildren or children, or hastNodes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can't, otherwise other libraries won't read it.
That is how it works: https://github.com/syntax-tree/mdast-util-to-hast/

const element = data.hChildren[index];

if (element.properties && element.properties.className.includes('comment')) {
const { value } = element.children[0];

let match = /\[([^[\]]+?)\]\((.+?)\)/.exec(value);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Regex find all comments, it is from markdown.js file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we place a comment for this then?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah


if (match !== null) {
const [, text, url] = match;

element.children = [
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node children are:

  1. //
  2. An empty space
  3. Link

{
type: 'text',
value: '//'
},
{
type: 'text',
value: ' '
},
{
type: 'element',
tagName: 'a',
properties: {
href: url,
className: ['code-link']
},
children: [
{
type: 'text',
value: text
}
]
}
];
}
}
}

node.children = [node.data.hChildren];
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will removed this line, it is not necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

} catch (e) {
if (!languages[lang]) {
console.warn('Prism does not support this language: ', lang);
} else {
console.warn('Prism failed to highlight: ', e);
}
}

data.hProperties = data.hProperties || {};
data.hProperties.className = ['hljs', ...(data.hProperties.className || []), `language-${lang}`];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, can we rename the hProperties to something more readable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't, same reason as hChildren, that is how unified ecosystem works.

}
};
}

module.exports = attacher;
2 changes: 1 addition & 1 deletion src/content/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ See [configuration](/configuration) for the options in the configuration file.

## Usage without config file

```sh
```bash
webpack <entry> [<entry>] -o <output>
```

Expand Down
2 changes: 1 addition & 1 deletion src/content/api/node.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ import webpack from 'webpack';

The imported `webpack` function is fed a webpack [Configuration Object](/configuration/) and runs the webpack compiler if a callback function is provided:

``` js-with-links
``` jslinks
const webpack = require("webpack");

webpack({
Expand Down
2 changes: 1 addition & 1 deletion src/content/guides/dependency-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ require('./template/' + name + '.ejs');

webpack parses the `require()` call and extracts some information:

```code
```
Directory: ./template
Regular expression: /^.*\.ejs$/
```
Expand Down
4 changes: 2 additions & 2 deletions src/content/plugins/source-map-dev-tool-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ new webpack.SourceMapDevToolPlugin({

And for cases when source maps are stored in the upper level directory:

```code
```
project
|- dist
|- public
Expand All @@ -108,6 +108,6 @@ new webpack.SourceMapDevToolPlugin({

Will produce the following URL:

```code
```
https://example.com/project/sourcemaps/bundle-[hash].js.map
```
5 changes: 4 additions & 1 deletion src/utilities/process-readme.js
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,8 @@ module.exports = function processREADME(body, options = {}) {
.replace(/<h2[^>]*>/g, '## ')
.replace(/<\/h2>/g, '')
// Drop any comments
.replace(/<!--[\s\S]*?-->/g, '');
.replace(/<!--[\s\S]*?-->/g, '')
// Replace code
.replace(/```console/g, '```bash')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new 2 lines save a lot of Prism warnings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think its worth it replacing content on the fly? i would suggest find replace console -> bash, or is this issue coming from external readme files we fetch?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda have the feeling that we should open a PR in projects using console and change it instead of this because it is not a valid language option in markdown regarding highlighting.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EugeneHlushko Yes, it is from external readmes.
@jeremenichelli Yep :D, will revert

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can keep this, until we replace it in all READMEs.

Copy link
Member Author

@montogeek montogeek Jul 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already talked to @shellscape, console is better because GitHub support is richer than bash.
This is easily fixed with a plugin, just registering it using Prism:
Prism.languages['console'] = Prism.languages.extend('sh', {})

.replace(/```sh/, '```bash');
};
3 changes: 2 additions & 1 deletion webpack.common.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ module.exports = (env = {}) => ({
plugins: [
require('remark-slug'),
require('remark-mermaid'),
require('remark-refractor'),
require('./prism-links'),
// require('remark-refractor'),
[
require('remark-custom-blockquotes'),
{
Expand Down
Loading