1
0
mirror of synced 2025-12-25 11:03:37 -05:00
Commit Graph

161 Commits

Author SHA1 Message Date
Mike Surowiec
2de358187c Performance testing NextJS / React (#19540)
* Remove json.parse(json.stringify( usage to improve performance
* Fix missing / duplicate keys in some renders
* Fix missing react-is dependency (only seemed to cause problems from pruning on the heroku deploy)
* Add nextjs tag to datadog-connect config when nextjs is a query param
* Fix router.asPath usage to exclude query param
* Fix styling inconsistencies noticed when testing
* Add a few tests
2021-05-27 15:17:27 -07:00
James M. Greene
542a459c06 Move script-only dependencies to devDependencies (#19542)
* Move lib/search/sync.js to script/search/sync.js

* Move mdast-util-from-markdown to devDeps

* Move lib/redirects/add-redirect-to-frontmatter.js to script/helpers/

* Move mkdirp to devDeps

* Move linkinator to devDeps

* Move rimraf to devDeps

* Fix script/search/sync.js require paths

* Move lib/search/build-records.js to script/search/

* Move lib/search/find-indexable-pages to script/search/

* Fix require paths for build-records

* Fix require paths for find-indexable-pages

* Move lib/search/algolia-get-remote-index-names.js to script/search/

* Movbe lib/search/algolia-search-index.js to script/search/

* Move lib/search/lunr-search-index.js to script/search/

* Move lib/search/lunr-get-index-names.js to script/search/

* Fix Lunr search index paths

* Move lib/search/validate-records.js to script/search/

* Move is-url to devDeps

* Move lib/search/algolia-client.js to script/search/

* Move lib/search/parse-page-sections-into-records.js to script/search/

* Move lib/search/rank.js to script/search/

* Fix path to cached-index-names.json file

* Normalize require for fs.promises
2021-05-25 20:44:19 +00:00
Mike Surowiec
37f73f0bb3 Heroku dev deps (#19431)
* fix: req.csrfToken doesn't always exist (e.g. 500 page)

* feat: update dockerfile and add nextjs to build

* fix: run linter

* move @babel deps -> dev deps

* move webpack looking things from deps -> dev deps

* move pa11y-ci to optional dep

* explicitly include optional deps for pa11y

* allow heroku dev deps to be installed

* fix: update postcss module

* fix: update dockerfile build

* tmp: disable renderReact

* see if another deploy is slower/faster

* move a few more packages to devDeps

* upgrade to package-lock v2

* use dayjs instead of date-fns

* move cross-env to devDeps

* remove unused 'del' package

* commit husky precommit hooks

* add hrtime to clone-for-build.js

* Revert "add hrtime to clone-for-build.js"

This reverts commit 70ee647bacce833f4ed2f621f62c63c1d85e5413.

* update babel/eslint

* fix: remove unused plugin

* try a .slugignore

* fix: heroku-postbuild to use npm run build

* fix: i cannot spell dereferenced

* add .next/cache to heroku cacheDirectories

* test cached build

* remove aws-sdk, see what breaks

* move jest-puppeteer to optional deps

* fix: update browser-test.yml to use newer node version

* move jimp to optional dependencies

* move puppeteer to optional dependencies

* fix: ci optional include

* fix: bad copy pasta

* remove previous react experiment

* update tests/README.md with note about optional deps

* bump node test version back to 14

* convert package-lock back to v1

* fix: use node 15.x to leverage npm optional deps

* fix: optional dep install

* test: see what happens with heroku/nodejs-typescript buildpack

* back to heroku/nodejs buildpack

* move jest to optional

* revert jest move

* remove .slugignore

* cleanup dockerfile, move xlsx-population to optional, add comment about optional deps

* Update Dockerfile

Co-authored-by: James M. Greene <JamesMGreene@github.com>

Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-05-24 15:40:50 -07:00
Mike Surowiec
e46422de6a fix: update husky usage 2021-05-20 13:36:46 -07:00
Mike Surowiec
c4166202a1 feat: upgrade to node v16, package-lock.json v2, remove aws-sdk 2021-05-20 13:32:19 -07:00
Kevin Heis
afa551f2ee Upgrade AJV (#19443)
* Upgrade AJV

* Fixes for AJV upgrade

* lint fix

* Update index.tsx
2021-05-20 17:46:04 +00:00
Kevin Heis
f6acb1f08a Update dependencies (#19408)
* Update dependencies 2

* Lint fixes

* Update graphql packages

* Update changelog-entry.json

* Update @octokit/rest

* Update commander with help from @rachmari

@rachmari

* Upgrade helmet

* Upgrade js-yaml

* Update server.js

* Update cheerio

* Revert "Update cheerio"

This reverts commit 8aa17c39fbf564ee554037d89e7a473027d16984.
2021-05-19 17:32:47 +00:00
Mike Surowiec
e716172ba6 Merge branch 'main' into sponsors-react-3 2021-05-11 13:41:15 -07:00
Mike Surowiec
744051006f Merge branch 'main' into sponsors-react-2 2021-05-11 13:34:38 -07:00
Mike Surowiec
f0fe065cef feat: implement changelog feed, guide section 2021-05-10 15:17:52 -07:00
James M. Greene
ed0f942cf2 Merge branch 'main' into dependabot/npm_and_yarn/lodash-4.17.21 2021-05-07 17:16:11 -05:00
Mike Surowiec
e3ff452f47 feat: add primer/components, wire up SSR for styled-components 2021-05-06 15:07:15 -07:00
Mike Surowiec
8930036e67 Merge branch 'main' into sponsors-react-1 2021-05-06 13:58:54 -07:00
dependabot[bot]
de7cc2db05 Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-06 18:46:59 +00:00
Mike Surowiec
e10c29b1fb feat: use eslint for dependency checks 2021-05-06 08:31:04 -07:00
Mike Surowiec
f3a029b6dd feat: add SupportSection 2021-05-05 15:58:21 -07:00
Mike Surowiec
34dcba51f1 feat: add DefaultLayout, add to sponsors landing page 2021-05-05 14:54:49 -07:00
Mike Surowiec
eaddbc5db7 feat: add nextjs middleware handling (#19139)
* feat: add nextjs middleware handling split

* fix: eslint errors

* fix: filter boolean from csp list

* fix: feature flag nextjs server start

* feat: add prettier rules for ts,tsx files

* fix: remove unnecessary async from next middleware

* fix: next middleware name

* Update tsconfig.json

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* Update next-env.d.ts

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* fix: add typescript linting to lint command

* add comment for unsafe-eval, update webpack to use eval in development

* fix: feature flag typo

Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-05-05 15:23:46 +00:00
James M. Greene
816a7bbc4e Follow-up for removing stale apps (#19056)
* Use existing 'github' script helper to get Octokit instance

* Move @octokit/rest to devDeps

We only use it in scripts, currently

* Removed outdated delete-unused-staging-apps script
2021-04-29 11:09:34 -04:00
dependabot[bot]
7e7f17c3f8 Bump redis from 3.0.2 to 3.1.1 (#5775) 2021-04-28 16:38:05 -07:00
Marc-Andre Giroux
2deb8275da Accept glob patterns for source files in openapi-check script (#18965)
Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-04-26 21:56:12 +00:00
James M. Greene
919054f91c Update npm dependencies to address Dependabot security alerts (#18992)
* npm audit fix

Addresses 1 Dependabot security alert for marked@<2.0.0, which is a dependency of linkinator

https://github.com/advisories/GHSA-4r62-v4vq-hr96

* Update jest to latest, which updates node-notifier@<8.0.1, to address 1 Dependabot security alert

https://github.com/advisories/GHSA-5fw9-fq32-wv5p

* Update rehype-highlight to latest, which updates highlight.js@<10.4.1, to address 1 Dependabot security alert

https://github.com/advisories/GHSA-7wwv-vh3v-89cq
2021-04-26 21:21:09 +00:00
Mike Surowiec
4a931237ea Merge branch 'main' into update-webpack-dep 2021-04-13 16:55:31 -07:00
Mike Surowiec
1c5cd44a83 fix: move ajv from devDeps to deps 2021-04-13 16:49:31 -07:00
Kevin Heis
46c16001c3 Upgrade Primer with translation shim for trashcan icon (#18735)
* Revert "Revert "Upgrade Primer CSS to 16 (#18713)""

This reverts commit c01d3e302e.

* Add a hack to handle trashcan icon in translated content
2021-04-13 18:38:04 +00:00
Mike Surowiec
c01d3e302e Revert "Upgrade Primer CSS to 16 (#18713)"
This reverts commit 49377f9096.
2021-04-13 09:19:28 -07:00
Kevin Heis
49377f9096 Upgrade Primer CSS to 16 (#18713)
* Upgrade primer to 16

* Upgrade colors in scss

* Get SCSS to build

* Initial dark mode support

* trashcan -> trash

* More specific search browser test

* Fix link color on homepage

* Update overrides.scss

* Upgrade some colors to primer variables

* Hide dark mode support for now

* Apply suggestions from code review

* Update overrides.scss
2021-04-13 16:00:46 +00:00
Sarah Schneider
53b7208fe9 Merge branch 'main' into update-webpack-dep 2021-04-09 12:22:18 -04:00
Vanessa Yuen
bbd9f03f6e Populate product-landing "What's new" section using GitHub Changelog (#18328)
Co-authored-by: Jason Etcovitch <jasonetco@github.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>
Co-authored-by: Sarah Schneider <sarahs@users.noreply.github.com>
2021-04-08 20:51:37 +00:00
Sarah Schneider
62955e3843 fix browser tests for webpack 5 2021-04-07 12:05:09 -04:00
Sarah Schneider
d75621aa0e update webpack plugins and remove friendly-errors-webpack-plugin abandonware 2021-04-07 11:26:39 -04:00
Sarah Schneider
583914fc8c sigh retry assert in dependencies 2021-04-07 08:27:44 -04:00
Sarah Schneider
640b9866db move assert to devdeps 2021-04-06 19:19:41 -04:00
Sarah Schneider
274ef8287a update webpack to resolve dependabot alert 2021-04-06 16:33:03 -04:00
Sarah Schneider
107b8e5bc0 Move scheduled tasks code into docs repo (#17926)
Co-authored-by: Rachael Sewell <rachmari@github.com> - https://github.com/github/docs-internal/pull/18453
2021-04-02 12:42:41 -07:00
James M. Greene
d8e0b29fcd Use node-redis for soft-purging cache on deploy (#18474)
* Use [node-]redis as a direct dependency

* Extract Redis client creation to its own module

* Attach extensive logging in the Redis client creation module

* Allow the rate limiter to pass requests when Redis is disconnected

* Update rate-limit-redis

* Default error input to empty object for formatRedisError method

* Provide a name for the rate limiter's Redis client

* Include redis-mock, exclude ioredis/ioredis-mock

* Remove unused RedisAccessor#exists method

* Switch RedisAccessor to use redis/redis-mock

* Provide a name for logging on the Redis page cache

* Remove extraneous trailing space from Redis logging prefix

Our updated use of console.* will already be adding a space after the prefix

* Replace ioredis-mock with redis-mock in tests

* Revert removal of ioredis dependency

* Bind Redis client to async promisified methods

* Extract former RedisAccessor constructor tests to new create-client tests

* Update RedisAccessor tests to work with the callback-based redis client

* Handle formatting Redis errors (or not) with more resiliency

* Remove ioredis as a dependency

* Update Redis soft-purging script to use node-redis

* Do more promisifying for less ugly code

* Add comment explaining TTL of -1

* Make the script executable
2021-03-30 20:40:28 +00:00
James M. Greene
84547e54c7 Use node-redis for page cache (#18421)
* Use [node-]redis as a direct dependency

* Extract Redis client creation to its own module

* Attach extensive logging in the Redis client creation module

* Allow the rate limiter to pass requests when Redis is disconnected

* Update rate-limit-redis

* Default error input to empty object for formatRedisError method

* Provide a name for the rate limiter's Redis client

* Include redis-mock, exclude ioredis/ioredis-mock

* Remove unused RedisAccessor#exists method

* Switch RedisAccessor to use redis/redis-mock

* Provide a name for logging on the Redis page cache

* Remove extraneous trailing space from Redis logging prefix

Our updated use of console.* will already be adding a space after the prefix

* Replace ioredis-mock with redis-mock in tests

* Revert removal of ioredis dependency

* Bind Redis client to async promisified methods

* Extract former RedisAccessor constructor tests to new create-client tests

* Update RedisAccessor tests to work with the callback-based redis client

* Handle formatting Redis errors (or not) with more resiliency
2021-03-29 17:34:22 +00:00
Vanessa Yuen
a8d54c9af7 Improvements in tools to help merge crowdin PRs (#18409)
- add `script/test-render-translation.js` to render all translated content to catch malformed liquid that would cause render errors
- improve test output for `script/fix-translation-errors.js` and `tests/content/lint-files.js`
- make it so `script/reset-translated-file.js` can handle files that have been renamed
2021-03-26 19:21:45 +00:00
James M. Greene
23bcc0bb06 Use node-redis for rate limiter (#18416)
* Use [node-]redis as a direct dependency
* Extract Redis client creation to its own module
* Attach extensive logging in the Redis client creation module
* Allow the rate limiter to pass requests when Redis is disconnected
* Update rate-limit-redis
* Default error input to empty object for formatRedisError method
* Provide a name for the rate limiter's Redis client

Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>
2021-03-25 23:54:25 +00:00
James M. Greene
6e20ed7927 Implement app clustering (#17752)
* Install throng for easy cluster management
* Extract the Express app construction into its own file
* Switch server.js to use app clustering for deployed environments
* Worker count is based on the lesser of process.env.WEB_CONCURRENCY and the count of CPUs
* Reading clustered output is difficult, let's prefix the std{out,err} streams

Co-authored-by: Jason Etcovitch <jasonetco@github.com>
2021-03-19 20:07:46 +00:00
Rachael Sewell
e067616355 add script to read topics spreadsheet (#18175) 2021-03-11 17:20:27 -08:00
James M. Greene
fd7d0eeb1a Add timeout and abort middleware and processing halts (#18177)
* Add middleware to timeout requests after a period

* Add halt-on-dropped-connection middleware to stop the middleware processing stack if the connection was already dropped

* Add a few strategic bail-out spots for dropped connections during the render-page middleware

* Handle 404s and HEAD requests earlier in the page rendering flow

* Add a few more strategic bail-out spots for dropped connections during the render-page middleware

* Add middleware to notice aborted requests

* Add a check for aborted requests into the isConnectionDropped logic

* Reformat comment for consistency

* Handle aborted requests correctly in the error handler

* Explicit returns for consistency
2021-03-09 19:14:02 +00:00
Jason Etcovitch
7e3229e39a Fix translated glossary links (#17734)
* npm start should be ja

* Add special handling for glossary headings

* Don't sort glossary

* Use English slug to maintain anchor, re-add sort
2021-02-12 19:23:40 +00:00
Rachael Sewell
01e52046f0 migrate away from AWS S3 (#17669) 2021-02-12 10:25:45 -08:00
Jason Etcovitch
e68e6a5185 Revert "Revert "Absorb @github-docs/frontmatter (#17783)" (#17793)" (#17799)
* Revert "Revert "Absorb @github-docs/frontmatter (#17783)" (#17793)"

This reverts commit 11baf43eee.

* Move revalidator to depedencies
2021-02-11 11:15:59 -05:00
Jason Etcovitch
6e0c2319b4 Set engines.node to 14+ (#17777)
* Set engines.node

* Add setup-node to browser-test workflow

* Updoot ianwalter/puppeteer-container
2021-02-11 15:40:08 +00:00
Jason Etcovitch
11baf43eee Revert "Absorb @github-docs/frontmatter (#17783)" (#17793)
This reverts commit 1e96c03489.
2021-02-10 20:28:41 +00:00
Jason Etcovitch
1e96c03489 Absorb @github-docs/frontmatter (#17783) 2021-02-10 15:10:46 -05:00
Jason Etcovitch
39e0e0dda1 Use Liquidjs instead of Liquid (#16743)
* Install liquidjs, uninstall liquid

* Comment a bunch of stuff out to get going

* Fix invalid includes

* Fix all includes (path => 'path')

* Get the homepage to render

* Do link-in-list kinda

* Revert "Fix all includes (path => 'path')"

This reverts commit d6fead646353aa5041d9229470a62a1d487456b9.

* Support non-dynamic partials

* Extract getTemplate helper

* Do remaining custom Liquid tags

* Fix some custom tag bugs

* Moar bugs

* Re-add link tag

* Cleaner diff

* Actually fix extended markdown tags

* Fully comment out version matchers

* Smaller diff

* Rely only on Liquid internals for conditionals

* Use new operators option in Liquid engine

* Fix link.js

* Don't need options

* Updoot to the right doot

* Fix some bugs

* Fix another bug

* Pass a test

* Fix the translate bits

* Adjust a test

* Fix another invalid Liquid bug

* Two more borked translations

* Found some more

* Don't need this change

* Revert "Don't need this change"

This reverts commit a916d619747f0492865a69c3e237c97c4d4e7fad.

* This should fix the broken links

* Missed one

* Revert "This should fix the broken links"

This reverts commit e6c2cc0d9055d958706260d57edbe293281c150e.

* Revert "Missed one"

This reverts commit bbe1f23baf16e020f6f7931589decb1afc75dfbd.

* Updoot liquidjs
2021-02-08 12:58:51 -05:00
James M. Greene
f7ec320b64 npm audit fix (#17675) 2021-02-06 23:57:14 +00:00