1
0
mirror of synced 2025-12-21 10:57:10 -05:00
Commit Graph

384 Commits

Author SHA1 Message Date
James M. Greene
84547e54c7 Use node-redis for page cache (#18421)
* Use [node-]redis as a direct dependency

* Extract Redis client creation to its own module

* Attach extensive logging in the Redis client creation module

* Allow the rate limiter to pass requests when Redis is disconnected

* Update rate-limit-redis

* Default error input to empty object for formatRedisError method

* Provide a name for the rate limiter's Redis client

* Include redis-mock, exclude ioredis/ioredis-mock

* Remove unused RedisAccessor#exists method

* Switch RedisAccessor to use redis/redis-mock

* Provide a name for logging on the Redis page cache

* Remove extraneous trailing space from Redis logging prefix

Our updated use of console.* will already be adding a space after the prefix

* Replace ioredis-mock with redis-mock in tests

* Revert removal of ioredis dependency

* Bind Redis client to async promisified methods

* Extract former RedisAccessor constructor tests to new create-client tests

* Update RedisAccessor tests to work with the callback-based redis client

* Handle formatting Redis errors (or not) with more resiliency
2021-03-29 17:34:22 +00:00
James M. Greene
23bcc0bb06 Use node-redis for rate limiter (#18416)
* Use [node-]redis as a direct dependency
* Extract Redis client creation to its own module
* Attach extensive logging in the Redis client creation module
* Allow the rate limiter to pass requests when Redis is disconnected
* Update rate-limit-redis
* Default error input to empty object for formatRedisError method
* Provide a name for the rate limiter's Redis client

Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>
2021-03-25 23:54:25 +00:00
Rachael Sewell
0df04f99c5 deprecate 2.20 version (#18354) 2021-03-23 19:38:04 +00:00
Sarah Schneider
fb0cb29b94 add convenience props to context obj for use in release notes layout 2021-03-19 10:20:14 -04:00
Kevin Heis
2e07070fb9 Add filters params to search endpoint (#18289)
* Add filters params to search endpoint

* Update search.md

* Update middleware/search.js

Co-authored-by: Rachael Sewell <rachmari@github.com>

Co-authored-by: Rachael Sewell <rachmari@github.com>
2021-03-17 21:02:47 +00:00
James M. Greene
a11d0075ef Prevent REALLY garbage request URLs (#18216)
* Block paths that do not normalize well
2021-03-12 02:58:47 +00:00
James M. Greene
c940dcd98b Middleware overhaul! (#18218)
* Middleware overhaul!

- Remove unnecessary 'async' keywords from middleware functions
- Ensure all middleware functions we create have names
- Wrap the method contents of all async middleware functions in a try-catch+next(error) pattern

* Use asyncMiddleware wrapper instead of try-catch+next(error) pattern

* Remove unnecessary try-catch+next(error) pattern from context middleware
2021-03-11 01:12:41 +00:00
James M. Greene
9e31597ee4 Use promisified fs.readFile instead of fs.promises.readFile (#18204)
* Create a readFileAsync wrapper module

* Use the readFileAsync wrapper module instead of fs.promises.readFile

* Fix require path in test
2021-03-10 17:57:10 +00:00
James M. Greene
92ac830a44 Do not timeout requests during testing (#18206)
* Do not load the timeout middleware during testing

* Run the link-check-server in testing mode to avoid loading the timeout middleware

* Revert the change to 'package.json'

* Also consider GITHUB_ACTIONS to be a testing environment to support the link checkers
2021-03-10 11:40:25 -06:00
James M. Greene
fd7d0eeb1a Add timeout and abort middleware and processing halts (#18177)
* Add middleware to timeout requests after a period

* Add halt-on-dropped-connection middleware to stop the middleware processing stack if the connection was already dropped

* Add a few strategic bail-out spots for dropped connections during the render-page middleware

* Handle 404s and HEAD requests earlier in the page rendering flow

* Add a few more strategic bail-out spots for dropped connections during the render-page middleware

* Add middleware to notice aborted requests

* Add a check for aborted requests into the isConnectionDropped logic

* Reformat comment for consistency

* Handle aborted requests correctly in the error handler

* Explicit returns for consistency
2021-03-09 19:14:02 +00:00
Emily Gould
052a540cdb Getting started category landing page (#17841)
* Start building out landing page in index file

* Add articles

* Add a couple more popular articles

* Tweak intro to cover more audiences

* Add TOC back with new organization

* Update content/github/getting-started-with-github/index.md

* Category as product guides section

* Update breadcrumbs.js

* Update server.js

* Update product-landing.html

* Remove TOC listing

* Revert removing TOC

* Update index.md

* Update index.md

* Update content/github/getting-started-with-github/index.md

* Apply suggestions from code review

Co-authored-by: Laura Coursen <lecoursen@github.com>

Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>
Co-authored-by: Laura Coursen <lecoursen@github.com>
2021-03-09 10:04:45 -06:00
James M. Greene
1921855816 Error middleware improvements (#18169)
* Defer logging to Failbot in error middleware until AFTER we finish responding

* If headers have already been written, log to failbot and then delegate to the default error handler

http://expressjs.com/en/guide/error-handling.html#the-default-error-handler

* Comment tweak
2021-03-08 21:56:56 +00:00
Sarah Schneider
063e22d151 re-combine release notes middleware 2021-03-08 15:03:34 -05:00
Sarah Schneider
b3fa73acd8 filter out deprecated patches in release notes middleware 2021-03-04 14:48:38 -05:00
Sarah Schneider
8eb1e69586 separate the middleware for enterprise server vs. github-ae release notes 2021-03-01 17:03:33 -05:00
James M. Greene
e94b52d7b5 Conditionally disable the TLS unauthorized cert check based on REDIS_URL (#17978)
* Conditionally disable the TLS unauthorized cert check based on REDIS_URL

* Update middleware/rate-limit.js
2021-02-24 23:35:39 +00:00
Sarah Schneider
f1336fd26c need to parse the JSON first 2021-02-24 16:33:14 -05:00
Sarah Schneider
08bbe0396f put the got request in a try to prevent timeouts 2021-02-24 10:58:02 -05:00
Sarah Schneider
dce6e7d5bd udpate middleware to read redirects from top-level redirects.json in archival repo 2021-02-22 11:13:52 -05:00
James M. Greene
8148677058 "Enable TLS" for Heroku Redis by ignoring self-signed TLS certificates (#17911)
* 'Enable TLS' for Heroku Redis by ignoring self-signed TLS certificates when creating Redis connections

https://devcenter.heroku.com/articles/heroku-redis#ioredis-module

* Clarify comments
2021-02-18 14:39:18 -06:00
Rachael Sewell
01e52046f0 migrate away from AWS S3 (#17669) 2021-02-12 10:25:45 -08:00
Kevin Heis
a3ad549e39 Block a few more things in AIRGAP (#17657)
* Block a few more things in AIRGAP

* Update middleware/context.js

Co-authored-by: Vanessa Yuen <6842965+vanessayuenn@users.noreply.github.com>

* Update product-landing.html

Co-authored-by: Vanessa Yuen <6842965+vanessayuenn@users.noreply.github.com>
2021-02-09 23:07:23 +00:00
Jason Etcovitch
4a23a3e675 Tweak /events exception reporting (#17776)
* Include fields as metadata instead of error inst

* Include hydroText
2021-02-09 22:24:01 +00:00
Kevin Heis
1918d2ea14 Inline CSRF token in HTML (#17748)
* Inline CSRF token

* Fix tests
2021-02-09 22:08:24 +00:00
Jason Etcovitch
6d20e43444 Add FailBot.report call to events errors (#17773)
* Add FailBot.report call to events errors

* Actually throw the error
2021-02-09 16:27:26 -05:00
Jason Etcovitch
d98bb561c7 Honor Error.status(Code) in handle-errors middleware (#17768)
* Honor error status codes in handle-errors

* Add a test
2021-02-09 14:57:57 -05:00
James M. Greene
4dea2779f3 More explicit terminations of middleware (#17666)
* Be more explicit about next/res.send ending middleware functions

* Add note explaining the one exception
2021-02-03 16:38:41 -06:00
James M. Greene
a40ff31b4b Cache more assets both in browser + Fastly (#17663)
* Cache archived asset JS/CSS for a day

* Don't waste time with ETag calculations

* Add a comment about the expiration length

* Cache images and fonts for a day

* Cache GraphQL data files for a week

* Set Fastly surrogate-key header on all responses

* Update test to account for new response header
2021-02-03 22:16:54 +00:00
Rachael Sewell
fc430896cb add azure to content security policy (#17649) 2021-02-03 10:49:40 -08:00
James M. Greene
0e729d9a75 Cleanup the testing and old code related to Fastly headers (#17643) 2021-02-03 10:43:00 -06:00
James M. Greene
8f133cbf54 Disallow both Fastly AND the browser from caching HTML pages (#17641) 2021-02-02 21:37:41 +00:00
James M. Greene
e53d1b7982 Set cache control for our JS/CSS assets to be kept for 28 days (#17636) 2021-02-02 13:34:37 -06:00
James M. Greene
42b5a90260 Bypass Fastly for all product families EXCEPT GitHub core (#17627) 2021-02-02 17:12:14 +00:00
James M. Greene
6957a4a830 Test bypassing Fastly for the Discussions, Packages, and Actions product families (#17611)
Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>

Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>
2021-02-02 09:43:53 -06:00
Jason Etcovitch
5cabf531da Convert .rc* versions to proper semver, then compare (#17600) 2021-02-01 19:13:35 +00:00
Jason Etcovitch
1bf93673ec Add release notes for GHES 3.0.0.rc2 (#17547)
* Add 0-rc2

* Started adding release notes

* added more line items

* finished initial draft of enterprise 2 fix notes

* some more line items added

* updated known issues and took out rc1 fixes

* added download link bug fix

* added new known issues identified in rc2

* Fix a typo

* Lint YAML

* Tweak sorting for new version schem

* Use today's date

* removed links to PRs and reduced notes to most important ones

* Fix indentation

* bug_fixes => bugs

* spruce it up

* typo

* update without changes

Co-authored-by: Amulya Raja <53104849+amulyaraja@users.noreply.github.com>
Co-authored-by: Maya <maya-ross@github.com>
Co-authored-by: Meg Bird <megbird@github.com>
2021-01-29 15:25:50 -07:00
Kevin Heis
8b86fd8fef Reduce minified JS bundle size from 189kb to 73kb (#17552) 2021-01-28 22:12:04 +00:00
Kevin Heis
d9d61f42ef Add Lunr as a site search option (#17388)
* Add Lunr as a site search option

* Use a class like the Algolia option does

* Lint

* Working on showing real content, prep for highlighting

* Move searching to their own files

* Split out ridiculous fn

* Add highlighting

* Less lunr search code

* A little more tidy

* Then chain on write too

* Mark from server too

* Write a batch of indices

* Update compress.js

* Highlighting with fewer calls

* Update lunr-search.js

* Update lunr-search.js

* Update lunr-search.js

* Update lunr-search.js

* Update lunr-search.js

* Update lunr-search.js
2021-01-28 19:38:22 +00:00
Jason Etcovitch
58319a221c Instrument all of our middleware (#17527)
* Add instrument-middleware.js

* Make it do some fancy require-ing

* Use it

* Prefix names

* dot prefix

* Improve async detection

* Add some comments

* Can't instrument error handler
2021-01-27 17:29:30 -05:00
Jason Etcovitch
591e22d8b1 Rename release notes for 3.0.0-rc to 3.0.0.rc (#17510)
* Change -rc versions to .rc\d*

* Fix the download link

* Rename to 3.0.0.rc1
2021-01-27 15:27:48 +00:00
Sarah Schneider
35e430c96d Merge branch 'main' into make-developer-redirects-static 2021-01-26 11:30:16 -05:00
Sarah Schneider
9f3e2b6017 handle edge case 2021-01-25 16:35:15 -05:00
Sarah Schneider
87bc25d7ca Merge branch 'main' of github.com:github/docs-internal into make-developer-redirects-static 2021-01-25 12:31:23 -05:00
Vanessa Yuen
9bc90cd932 Learning Track navigation banner (#17440)
* add middleware to handle `learn` query param

* add exception to query-less cache key

* add querystring to learning track guides
2021-01-25 16:57:32 +00:00
Kevin Heis
6ae96188f2 Move 404 and 500 images to local (#17432) 2021-01-22 21:49:45 +00:00
Sarah Schneider
3aed23bdc9 Merge branch 'main' into make-developer-redirects-static 2021-01-21 14:14:57 -05:00
Kevin Heis
c51f5390d4 Move search results to use relative urls (#17411)
* Move search results to use relative urls

* ..and now we have real mark tags instead of em tags

Co-authored-by: Chiedo John <2156688+chiedo@users.noreply.github.com>
2021-01-21 17:23:16 +00:00
Sarah Schneider
8198dd4ce7 Merge branch 'main' of github.com:github/docs-internal into make-developer-redirects-static 2021-01-20 15:08:35 -05:00
Kevin Heis
c21a738e95 Make browser search tests green (#17385)
* Make browser search tests green

* Okay... lodash get then _Linter_ ugh
2021-01-20 16:20:32 +00:00
Kevin Heis
2fb2e962bc Move site search to use an endpoint (#17359)
* Move site search to use an endpoint

* Update browser.js

* Update search.js

* Update lib/search/versions.js

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* Fix URLs

Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-01-20 15:37:42 +00:00