Skip to content

Commit a93b939

Browse files
committed
init
1 parent 0172342 commit a93b939

File tree

11 files changed

+355
-322
lines changed

11 files changed

+355
-322
lines changed

package.json

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,17 @@
11
{
2-
"name": "distill-css",
3-
"description": "PACKAGE_DESCRIPTION",
2+
"name": "extract-css-core",
3+
"description": "Extract all CSS from a given url, both server side and client side rendered.",
44
"version": "0.1.0",
55
"homepage": "https://www.projectwallace.com/oss",
6-
"repository": "PACKAGE_REPOSITORY",
7-
"issues": "PACKAGE_ISSUES",
6+
"repository": "https://github.com/bartveneman/extract-css-core",
7+
"issues": "https://github.com/bartveneman/extract-css-core/issues",
88
"license": "MIT",
99
"author": "Bart Veneman",
1010
"keywords": [
11-
"PACKAGE_KEYWORD"
11+
"extract",
12+
"css",
13+
"scrape",
14+
"get-css"
1215
],
1316
"scripts": {
1417
"test": "xo && ava test"
@@ -21,7 +24,25 @@
2124
"node": ">=8.0"
2225
},
2326
"xo": {
24-
"prettier": true
27+
"prettier": true,
28+
"rules": {
29+
"ava/no-skip-test": "warn"
30+
}
31+
},
32+
"prettier": {
33+
"useTabs": true,
34+
"semi": false,
35+
"singleQuote": true,
36+
"bracketSpacing": false,
37+
"proseWrap": "always",
38+
"overrides": [
39+
{
40+
"files": "*.json",
41+
"options": {
42+
"useTabs": false
43+
}
44+
}
45+
]
2546
},
2647
"devDependencies": {
2748
"ava": "^1.3.1",

readme.md

Lines changed: 33 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -16,15 +16,24 @@
1616

1717
### Problem
1818

19-
Existing packages like [get-css](https://github.com/cssstats/cssstats/tree/master/packages/get-css) look at a server-generated piece of HTML and get all the `<link>` and `<style>` tags from it. This works fine for 100% server rendered pages, but apps that employ style injection with JavaScript will not be covered.
19+
Existing packages like
20+
[get-css](https://github.com/cssstats/cssstats/tree/master/packages/get-css)
21+
look at a server-generated piece of HTML and get all the `<link>` and `<style>`
22+
tags from it. This works fine for 100% server rendered pages, but apps that
23+
employ style injection with JavaScript will not be covered.
2024

2125
### Solution
2226

23-
This module uses an instance of Chromium to render a page. This has the benefit that most of the styles can be rendered, even when generated by JavaScript. The [Puppeteer CSSCoverage API](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#coveragestartcsscoverageoptions) is the power behind finding most of the CSS.
27+
This module uses an instance of Chromium to render a page. This has the benefit
28+
that most of the styles can be rendered, even when generated by JavaScript. The
29+
[Puppeteer CSSCoverage API](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#coveragestartcsscoverageoptions)
30+
is the power behind finding most of the CSS.
2431

2532
### Shortcomings
2633

27-
Currently, there is no solution to get the CSS from modules that use [Styled Components](https://www.styled-components.com) or something similar. Any help resolving this issue will be very much appreciated.
34+
Currently, there is no solution to get the CSS from modules that use
35+
[Styled Components](https://www.styled-components.com) or something similar. Any
36+
help resolving this issue will be very much appreciated.
2837

2938
## Installation
3039

@@ -38,57 +47,60 @@ yarn add extract-css-core
3847
## Usage
3948

4049
```js
41-
const extractCss = require("extract-css-core");
50+
const extractCss = require('extract-css-core')
4251

43-
extractCss("http://www.projectwallace.com").then(css => console.log(css));
52+
extractCss('http://www.projectwallace.com').then(css => console.log(css))
4453
```
4554

4655
## API
4756

4857
### extractCss(url, [options])
4958

50-
Extract CSS from a page. Returns a Promise that contains the CSS as a single String.
59+
Extract CSS from a page. Returns a Promise that contains the CSS as a single
60+
String.
5161

5262
### Options
5363

5464
Type: `Object`
5565

5666
#### debug
5767

58-
Type: `Boolean`
59-
Default: `false`
68+
Type: `Boolean` Default: `false`
6069

61-
Set to `true` if you want a Chromium window to open as it works to get all the CSS from the page.
70+
Set to `true` if you want a Chromium window to open as it works to get all the
71+
CSS from the page.
6272

6373
#### waitUntil
6474

65-
Type: `String`
66-
Default: `networkidle2`
75+
Type: `String` Default: `networkidle2`
6776

68-
Can be any value as provided by the [Puppeteer docs](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagegotourl-options).
77+
Can be any value as provided by the
78+
[Puppeteer docs](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagegotourl-options).
6979

7080
#### customBrowser
7181

72-
Type: `Object`
73-
Default: `null`
82+
Type: `Object` Default: `null`
7483

7584
This is useful if you want to run extract-css on AWS Lambda for example.
7685

7786
##### executablePath
7887

79-
Type: `String`
80-
Default: `null`
88+
Type: `String` Default: `null`
8189

8290
Pass in the executable path for a custom Chromium instance.
8391

8492
##### customPuppeteer
8593

86-
Type: `Object`
87-
Default: `null`
94+
Type: `Object` Default: `null`
8895

89-
You probably want to provide [puppeteer-core](https://www.npmjs.com/package/puppeteer-core) for a custom browser instead of [puppeteer](https://www.npmjs.com/package/puppeteer) which brings it's own Chromium instance.
96+
You probably want to provide
97+
[puppeteer-core](https://www.npmjs.com/package/puppeteer-core) for a custom
98+
browser instead of [puppeteer](https://www.npmjs.com/package/puppeteer) which
99+
brings it's own Chromium instance.
90100

91101
## Related
92102

93-
- [extract-css lambda](https://github.com/bartveneman/extract-css) - Extract CSS running as a serverless function
94-
- [get-css](https://github.com/cssstats/cssstats/tree/master/packages/get-css) - The original get-css
103+
- [extract-css lambda](https://github.com/bartveneman/extract-css) - Extract CSS
104+
running as a serverless function
105+
- [get-css](https://github.com/cssstats/cssstats/tree/master/packages/get-css) -
106+
The original get-css

src/index.js

Lines changed: 68 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,85 +1,85 @@
1-
const puppeteer = require("puppeteer");
1+
const puppeteer = require('puppeteer')
22

3-
function InvalidUrlError({ url, statusCode, statusText }) {
4-
this.name = "InvalidUrlError";
5-
this.message = `There was an error retrieving CSS from ${url}.\n\tHTTP status code: ${statusCode} (${statusText})`;
3+
function InvalidUrlError({url, statusCode, statusText}) {
4+
this.name = 'InvalidUrlError'
5+
this.message = `There was an error retrieving CSS from ${url}.\n\tHTTP status code: ${statusCode} (${statusText})`
66
}
77

8-
InvalidUrlError.prototype = Error.prototype;
8+
InvalidUrlError.prototype = Error.prototype
99

1010
module.exports = async (
11-
url,
12-
{ debug = false, waitUntil = "networkidle2", customBrowser = {} } = {}
11+
url,
12+
{debug = false, waitUntil = 'networkidle2', customBrowser = {}} = {}
1313
) => {
14-
const browserOptions = {
15-
headless: debug !== true,
16-
puppeteer
17-
};
14+
const browserOptions = {
15+
headless: debug !== true,
16+
puppeteer
17+
}
1818

19-
// Replace the puppeteer instance if a custom one is provided
20-
// This also means that the executablePath needs to be set to
21-
// a custom path where some chromium instance is running.
22-
if (
23-
customBrowser &&
24-
customBrowser.executablePath &&
25-
customBrowser.customPuppeteer
26-
) {
27-
browserOptions.executablePath = customBrowser.executablePath;
28-
browserOptions.puppeteer = customBrowser.customPuppeteer;
29-
}
19+
// Replace the puppeteer instance if a custom one is provided
20+
// This also means that the executablePath needs to be set to
21+
// a custom path where some chromium instance is running.
22+
if (
23+
customBrowser &&
24+
customBrowser.executablePath &&
25+
customBrowser.customPuppeteer
26+
) {
27+
browserOptions.executablePath = customBrowser.executablePath
28+
browserOptions.puppeteer = customBrowser.customPuppeteer
29+
}
3030

31-
// Setup a browser instance
32-
const browser = await browserOptions.puppeteer.launch(browserOptions);
31+
// Setup a browser instance
32+
const browser = await browserOptions.puppeteer.launch(browserOptions)
3333

34-
// Create a new page and navigate to it
35-
const page = await browser.newPage();
34+
// Create a new page and navigate to it
35+
const page = await browser.newPage()
3636

37-
// Start CSS coverage. This is the meat and bones of this module
38-
await page.coverage.startCSSCoverage();
39-
const response = await page.goto(url, { waitUntil });
37+
// Start CSS coverage. This is the meat and bones of this module
38+
await page.coverage.startCSSCoverage()
39+
const response = await page.goto(url, {waitUntil})
4040

41-
// Make sure that we only try to extract CSS from valid pages.
42-
// Bail out if the response is an invalid request (400, 500)
43-
if (response.status() >= 400) {
44-
await browser.close(); // Don't leave any resources behind
41+
// Make sure that we only try to extract CSS from valid pages.
42+
// Bail out if the response is an invalid request (400, 500)
43+
if (response.status() >= 400) {
44+
await browser.close() // Don't leave any resources behind
4545

46-
return Promise.reject(
47-
new InvalidUrlError({
48-
url,
49-
statusCode: response.status(),
50-
statusText: response.statusText()
51-
})
52-
);
53-
}
46+
return Promise.reject(
47+
new InvalidUrlError({
48+
url,
49+
statusCode: response.status(),
50+
statusText: response.statusText()
51+
})
52+
)
53+
}
5454

55-
// Coverage contains a lot of <style> and <link> CSS,
56-
// but not all...
57-
const coverage = await page.coverage.stopCSSCoverage();
55+
// Coverage contains a lot of <style> and <link> CSS,
56+
// but not all...
57+
const coverage = await page.coverage.stopCSSCoverage()
5858

59-
// Fetch all <style> tags from the page, because the coverage
60-
// API may have missed some JS-generated <style> tags.
61-
// Some of them *were* already caught by the coverage API,
62-
// but they will be removed later on to prevent duplicates.
63-
const styleTagsCss = (await page.$$eval("style", styles => {
64-
// Get the text inside each <style> tag and trim() the
65-
// results to prevent all the inside-html indentation
66-
// clogging up the results and making it look
67-
// bigger than it actually is
68-
return styles.map(style => style.innerHTML.trim());
69-
})).join("");
59+
// Fetch all <style> tags from the page, because the coverage
60+
// API may have missed some JS-generated <style> tags.
61+
// Some of them *were* already caught by the coverage API,
62+
// but they will be removed later on to prevent duplicates.
63+
const styleTagsCss = (await page.$$eval('style', styles => {
64+
// Get the text inside each <style> tag and trim() the
65+
// results to prevent all the inside-html indentation
66+
// clogging up the results and making it look
67+
// bigger than it actually is
68+
return styles.map(style => style.innerHTML.trim())
69+
})).join('')
7070

71-
await browser.close();
71+
await browser.close()
7272

73-
// Turn the coverage Array into a single string of CSS
74-
const coverageCss = coverage
75-
// Filter out the <style> tags that were found in the coverage
76-
// report since we've conducted our own search for them.
77-
// A coverage CSS item with the same url as the url of the page
78-
// we requested is an indication that this was a <style> tag
79-
.filter(styles => styles.url !== url)
80-
// The `text` property contains the actual CSS
81-
.map(({ text }) => text)
82-
.join("");
73+
// Turn the coverage Array into a single string of CSS
74+
const coverageCss = coverage
75+
// Filter out the <style> tags that were found in the coverage
76+
// report since we've conducted our own search for them.
77+
// A coverage CSS item with the same url as the url of the page
78+
// we requested is an indication that this was a <style> tag
79+
.filter(styles => styles.url !== url)
80+
// The `text` property contains the actual CSS
81+
.map(({text}) => text)
82+
.join('')
8383

84-
return Promise.resolve(coverageCss + styleTagsCss);
85-
};
84+
return Promise.resolve(coverageCss + styleTagsCss)
85+
}

test/css-in-js.html

Lines changed: 29 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,38 @@
11
<!DOCTYPE html>
22
<html lang="en">
3-
<head>
4-
<meta charset="UTF-8" />
5-
<title>CodePen - CSS-in-JS example for extract-css-core unit tests</title>
3+
<head>
4+
<meta charset="UTF-8" />
5+
<title>CodePen - CSS-in-JS example for extract-css-core unit tests</title>
66

7-
<style>
8-
html {
9-
color: #f00;
10-
}
11-
</style>
12-
</head>
7+
<style>
8+
html {
9+
color: #f00;
10+
}
11+
</style>
12+
</head>
1313

14-
<body translate="no">
15-
<div id="app"></div>
14+
<body translate="no">
15+
<div id="app"></div>
1616

17-
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/16.8.4/umd/react.production.min.js"></script>
18-
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-dom/16.8.4/umd/react-dom.production.min.js"></script>
19-
<script src="https://unpkg.com/styled-components/dist/styled-components.min.js"></script>
17+
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/16.8.4/umd/react.production.min.js"></script>
18+
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-dom/16.8.4/umd/react-dom.production.min.js"></script>
19+
<script src="https://unpkg.com/styled-components/dist/styled-components.min.js"></script>
2020

21-
<script id="rendered-js">
22-
const Title = styled.h1`
23-
color: blue;
24-
font-family: sans-serif;
25-
font-size: 3em;
26-
`;
21+
<script id="rendered-js">
22+
const Title = styled.h1`
23+
color: blue;
24+
font-family: sans-serif;
25+
font-size: 3em;
26+
`
2727

28-
const App = () => {
29-
return React.createElement(Title, null, "Title");
30-
};
28+
const App = () => {
29+
return React.createElement(Title, null, 'Title')
30+
}
3131

32-
ReactDOM.render(
33-
React.createElement(App, null),
34-
document.querySelector("#app")
35-
);
36-
</script>
37-
</body>
32+
ReactDOM.render(
33+
React.createElement(App, null),
34+
document.querySelector('#app')
35+
)
36+
</script>
37+
</body>
3838
</html>

test/fixture.css

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
body {
2-
color: teal;
2+
color: teal;
33
}

0 commit comments

Comments
 (0)