1
0
mirror of synced 2025-12-21 19:06:49 -05:00
Commit Graph

56 Commits

Author SHA1 Message Date
Peter Bengtsson
18504871b9 cache full rendering (#25424)
* cache full rendering

* still not working with gzip

* progress progress progress

* smaller

* hacky progress

* small fixes

* wip

* lock file

* wip

* wip

* package-lock updates

* wip

* search DOM in lowercase

* simplify

* with instrument

* improve test coverage

* mutateCheeriobodyByRequest

* fix

* remove renderContentCacheByContex

* disable render caching in sync-search

* diables things in github/github link checker

* gzip lru

* tidying up

* updated

* correct tests

* fix: move userLanguage to LanguagesContext

* Revert "fix: move userLanguage to LanguagesContext"

This reverts commit d7c05d958c71eaad496eb46764eb845d80b866ca.

* contexts ftw

* fixed rendering tests

* oops for got new file

* nits addressed

Co-authored-by: Mike Surowiec <mikesurowiec@users.noreply.github.com>
2022-05-23 12:12:09 +00:00
Grace Park
b98c83e924 REST operations sidebar (#26225)
* testing out a rest operations sidebar

* cleanup

* renamed 5 files

* renamed 5 files

* set redirect_from on 5 files

* renamed 1 files

* renamed 1 files

* renamed 3 files

* renamed 1 files

* renamed 1 files

* renamed 3 files

* renamed 1 files

* renamed 2 files

* renamed 1 files

* renamed 4 files

* renamed 15 files

* renamed 2 files

* renamed 6 files

* renamed 1 files

* renamed 4 files

* renamed 7 files

* renamed 1 files

* renamed 3 files

* renamed 3 files

* renamed 1 files

* renamed 5 files

* renamed 1 files

* renamed 1 files

* renamed 3 files

* renamed 4 files

* renamed 1 files

* renamed 1 files

* renamed 2 files

* renamed 4 files

* renamed 1 files

* renamed 1 files

* renamed 1 files

* renamed 6 files

* renamed 6 files

* renamed 4 files

* move files

* adding more

* updating to add restcontext and start of removing data/reusables/rest-reference

* removed data/reusables

* add a RestMiniTocItem and updating the filtering to add a subcategory so all manually added H3s are in mini tocs in addition to operations

* remove console log

* [WIP]: REST New Proposal Sidebar (#26471)

* saving

* update sidebar

* remove console log

* update guides and overview

* import Category for category level rest pages

* update undefined restOperations

* update restOperationData category and subcategory levels"

* minor updates

* update get mini toc items function

* updating REST context for sidebar

* updating rest data

* remove console logs

* WIP: mini-toc-ing the sidebar

Co-authored-by: Robert Sese <rsese@github.com>

* A little cleanup

* Fix first subcategory link and add some comments

* updating anchor links in sidebar

* adding updates

* remove standalone

* update product and maptopic pages using article context

* add conditional link wrapper

* fix sidebar toggle and versions for enterprise admin

* update versions per subcategory

* Highlight sidebar link for current page

* Update miniToc hash links and hash change tracking

* fix unique key in CollapsibleSection

* Fix list markup

* remove title

* update permissions

* Hide minitocs on landing (#26594)

* hide minitocs on landing page

* simplify page components and remove minitoc from sidebar for guides/overview

* fix carats and category fix

* remove id

Co-authored-by: Grace Park <gracepark@github.com>

* updating content based on versions script check with the OpenAPI

* update script and content files

* update script and content/rest files

* update to add TocLanding

* update script

* update index files

* add codespaces repository-secrets

* remove openapi schema check script

* remove minitocs at the top

* add h2 about the {title} api

* fix tests/unit/openapi-schema.js

* Fix linting tests

* fix search/topics test

* fix tests/unit/pages test

* update rest/reference links in components

* run prettier

* Update components/rest/RestReferencePage.tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Update components/rest/RestReferencePage.tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Update pages/[versionId]/rest/[category]/[subcategory].tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Update pages/[versionId]/rest/[category]/[subcategory].tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Update pages/[versionId]/rest/[category]/[subcategory].tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Update pages/[versionId]/rest/[category]/[subcategory].tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Update tests/unit/openapi-schema.js

Co-authored-by: Rachael Sewell <rachmari@github.com>

* updating comment location

* remove dependabot override

* remove path-utils current product update for rest

* run linter

* remove dependabot.md and remove h2 heading on restreference

* update the correct product to rest for rest pages

* adding comments for updates to path-utils

* remove console log

* REST sidebar: handle legacy v3 redirects (#26686)

* Add script to handle legacy v3 REST redirects

* Run the script

* Handle a redirect to a redirect

* Update REST test URLs

* 'await' and test runs subcategory of checks

* Update REST URLs for routing/developer-site-redirects tests

* Update developer-redirects fixture with new REST URLs

* Resolve merge conflicts

* Update rest-redirects fixture with new REST URLs

* Fix broken links with REST pages re-org

* redirectTo could be undefined

* Fix script for posterity, can't redirect paths with hashes

* Remove invalid hash redirects

* Typically don't need to save one-off scripts

* Undo redirect changes (not necessary for handling v3 redirects)

* Remove script-added redirects

* Update old v3 redirects with new REST URLs

* No more GHES search indexing page

* 'org' not 'organization'

* Update fixture data for new REST URLs

* revert any content directory changes

Co-authored-by: Grace Park <gracepark@github.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>

* Adding test rest (#26750)

* add test to check openapi schema versions and content rest frontmatter versions

* update lib/redirects

* fix test and add error messages

* adding repository secrets

* adding repository-secrets.md

* Revert "update lib/redirects"

This reverts commit 3aafe28265764d5bc09c0c478c8e0ca099c8fbcf.

* remove lib/redirects changes and console logs

* Update lib/rest/index.js

Co-authored-by: Rachael Sewell <rachmari@github.com>

* update unique key

* Rest client side redirects (#26754)

* adding tags subcategory for the rest content repos category

* run prettier

* bug fix for anchor scrolls" (#26892)

* updating width size for rest reference page

* Rest sidebar consolidation (#26862)

* refactor sidebar

* fix articlecontext provider issue on rest product landing page for all versions

* fix a bug, create new component

* revert change to create new component and fix bug

Co-authored-by: Rachael Sewell <rachmari@github.com>

* Set currentAnchor with a hashchange handler (#26923)

* Rest sidebar design tweaks (#26807)

* Rest sidebar design tweaks

* tweak color to subtle

* use muted color and margin for line

* update to design feedback

Co-authored-by: Grace Park <gracepark@github.com>

* Remove cheerio from rest-collapsible (#26948)

* remove cheerio from rest-collapsible

* update type

* adding endswith instead

* use productId instead

* one off edge case for secret-scanning

* Reorganize subcategory and category, Update pre -> div, Add RestContext (#26950)

* reorganize subcategory and category

* add RestContext

* update comment

* update for endpoints page

* add comment

* move object to restcontext

* remove effectiveDate in restcontext

* remove width calculation for rest reference page

* fix adding manual writer's minitocs to sidebar

* update with feedback

* update comment

* update isRestReferencePage

* remove page component and fix bug

* adding back rest/index.tsx

Co-authored-by: Rachael Sewell <rachmari@github.com>

* update content/rest"

* add back design tweak

* update to div

* update margins on rest api reference

* remove page component

* adding tests

* separate product from rest sidebar (#27065)

* separate product from rest sidebar

* Use ProductCollapsibleSections for product pages

* fix tests

Co-authored-by: Robert Sese <rsese@github.com>
Co-authored-by: Grace Park <gracepark@github.com>

* Rest sidebar translations (#27052)

* update translations

* remove general test

Co-authored-by: Robert Sese <rsese@github.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>
2022-04-20 10:19:48 -07:00
Peter Bengtsson
cc79340595 cache individual renderContent instead of page.render (#24955)
* cache individual renderContent() instead of page.render()

* remove commented code

* unbreak unit tests
2022-02-03 16:56:05 +00:00
Kevin Heis
cd958e350e Cache all pages in-memory (#24555) 2022-01-24 18:39:25 +00:00
Rachael Sewell
0da2942dd3 remove storybook (#24122) 2022-01-06 12:11:58 -08:00
Peter Bengtsson
454375234e Mini-toc rendered twice for special case GraphQL pages (#24041) 2022-01-04 17:47:27 +00:00
Kevin Heis
4be9444023 Cache API pages attempt 2 (#23762)
* Update render-page.js

* Update render-page.js

* Update render-page.js

* Update render-page.js

* Consistently use const path = req.pagePath || req.path

* Update render-page.js

* Run memory cache even on test and CI

* feedbacked

* Add an exception for caching the GraphQL Explorer

See https://github.com/github/docs-internal/pull/19611

* pretty-fy

Co-authored-by: Peter Bengtsson <mail@peterbe.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>
Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-12-16 18:14:02 +00:00
Rachael Sewell
45f1a41e79 prerender graphql mutations page (#23757) 2021-12-15 12:28:39 -08:00
Peter Bengtsson
75a85da692 insight into which page has what page.render statsd timer (#23747) 2021-12-15 20:12:26 +00:00
Kevin Heis
553ebdd18a Revert "In memory cache api pages (#23751)" (#23753)
This reverts commit fc08215278.
2021-12-15 19:48:45 +00:00
Kevin Heis
fc08215278 In memory cache api pages (#23751) 2021-12-15 11:21:03 -08:00
Peter Bengtsson
c622faf636 effectiveDate might be falsy (#23002)
Part of #1185
2021-11-19 19:19:13 +00:00
Grace Park
884556eb02 Adding effectiveDate frontmatter property (#22317)
* adding effectiveDate frontmatter property

* Update components/article/ArticlePage.tsx

Co-authored-by: Peter Bengtsson <peterbe@github.com>

* adding validation for correct date

* update to dateTime

* update to year month day

* update error date validation

* moving validation to getArticleContextFromRequest and moving id to div

* remove enteredDate

* Update content/README.md

Co-authored-by: Laura Coursen <lecoursen@github.com>

Co-authored-by: Peter Bengtsson <peterbe@github.com>
Co-authored-by: Laura Coursen <lecoursen@github.com>
2021-10-20 19:40:13 +00:00
Kevin Heis
d431e86dec Storybook lite (#21040)
* Remove storybook

* Update index.js

* Start a storybook outline

* Start a storybook outline

* Start showing errors

* Update storybook.tsx

* Typescript lint

* Name field

* Responsive

* Update storybook.tsx

* Update storybook.tsx

* Update storybook.tsx

* Add BumpLink to storybook

* Update storybook.tsx
2021-08-30 20:34:14 +00:00
James M. Greene
df301cb5da Remove all code related to Redis page caching (#20457)
Towards simplification for https://github.com/github/docs-engineering/issues/678
2021-07-21 20:44:00 +00:00
Mike Surowiec
df38087aed Update 404/500 pages to no longer use liquid templates (#20450)
* Update 404/500 page rendering to no longer use liquid templates

* updating tests

* remove unused path

Co-authored-by: Grace Park <gracepark@github.com>
2021-07-21 12:22:30 -07:00
Grace Park
27aa5d92ea Remove FEATURE_NEXTJS Flag Part 1 (#20176)
* cleanup FEATURE_NEXTJS

* fixing some server tests

* updating article a for server tests

* update h2 to h4 map topic tests

* data off on TOCs

* updating dropdown article versions links

* Update so markdown renders in intros

* updating typo and all server tests are now passing

* remove nextjs feature flag

* head.js tests pass

* updating article-version-picker

* remove nextjs feature flag browser test

* update header.js tests

* fix page-titles.js test

* fix deprecated-enterprise versions

* adding early access

* testing

* getting childTocItem

* fixing table of contents to show child toc items

* updated to 2 because the sidebar article also has the same link

* remove comment

* updating pick

* Update TocLandingContext.tsx

* update package.json and change className to h4 for h2

* updating with mikes feedback

* remove a.active test

* React clean up: Delete unnecessary layouts/includes Part 2 (#20143)

* Delete unnecessary layouts

* setting back tests failing :(

* update layouts

* delete unnecessary includes

* remove github-ae-release-notes and updating layouts

* remove a.active test
2021-07-16 14:54:25 -07:00
Kevin Heis
8a56437c93 Pretty format (#20352)
* Update prettier flow to include JS

* Run prettier

* ...run prettier
2021-07-14 14:35:01 -07:00
Kevin Heis
42e785b0a8 Migrate CommonJS to ESM (#20301)
* First run of script

* Get the app running --- ish

* Get NextJS working

* Remove `node:`

* Get more tests passing in unit directory

* Update FailBot test to use nock

* Update test.yml

* Update Dockerfile

* tests/content fixes

* Update page.js

* Update build-changelog.js

* updating tests/routing

* Update orphan-tests.js

* updating tests/rendering

* Update .eslintrc.js

* Update .eslintrc.js

* Install jest/globals

* "linting" tests

* staging update to server.mjs

* Change '.github/allowed-actions.js' to a ESM export

* Lint

* Fixes for the main package.json

* Move Jest to be last in the npm test command so we can pass args

* Just use 'npm run lint' in the npm test command

* update algolia label script

* update openapi script

* update require on openapi

* Update enterprise-algolia-label.js

* forgot JSON.parse

* Update lunr-search-index.js

* Always explicitly include process.cwd() for JSON file reads pathed from project root

* update graphql/update-files.js script

* Update other npm scripts using jest to pass ESM NODE_OPTIONS

* Update check-for-enterprise-issues-by-label.js for ESM

* Update create-enterprise-issue.js for ESM

* Import jest global for browser tests

* Convert 'script/deploy' to ESM

Co-authored-by: Grace Park <gracepark@github.com>
Co-authored-by: James M. Greene <jamesmgreene@github.com>
2021-07-14 13:49:18 -07:00
Mike Surowiec
a24d01f78a Fix NextJS 500s (#20048)
* feat: allow server to contextualize request when on a /_next/data path

* add client side routing test, run lint

* enable nextjs client side routing
2021-06-22 17:30:40 +00:00
Mike Surowiec
306f6fc75e Enable more react pages (#19935)
* enable more react landing pages

* move nextjs page logic to separate middleware

* enable codespaces landing page + fixes

* enable /education

* enable /admin

* use pathToRegexp to match routes for react rendering

* run lint

* fix: typo in url

* update sidebar test
2021-06-15 13:31:17 -07:00
Grace Park
c2953a5ea7 Merge branch 'main' of https://github.com/github/docs-internal into grace-header-styling 2021-06-11 12:07:23 -07:00
Mike Surowiec
ae33dae11f More react default pages (#19863)
* turn on more default react landing pages
2021-06-11 18:35:41 +00:00
Grace Park
ee34fb0c09 remove line break 2021-06-10 18:20:58 -07:00
Grace Park
e2168885f4 add back spacing 2021-06-10 18:20:08 -07:00
Grace Park
c8ceeb35f2 Merge branch 'main' of https://github.com/github/docs-internal into grace-header-styling 2021-06-10 09:37:13 -07:00
Mike Surowiec
108377b209 add more react default rendered pages, fix toc items (#19783) 2021-06-08 15:35:32 -07:00
Mike Surowiec
37499b80d2 Reactify article page rendering (#19679)
* feat: reactify article page rendering

* feat: add LearningTrackNav

* fix: use plain text title for article
2021-06-07 22:56:33 +00:00
Grace Park
05fb4cb9b2 adding just the styling for header update 2021-06-04 10:29:21 -07:00
Mike Surowiec
6bdbe209c9 Default sponsors+discussions pages to nextjs rendering (#19667)
* default to nextjs rendering on sponsors and discussions pages
2021-06-03 11:17:15 -07:00
James M. Greene
7f65390cf9 Prevent page caching for the GraphQL Explorer page (#19611)
* Prevent page caching for the GraphQL Explorer page

* Add a comment

* Skip caching for airgapped sessions
2021-05-27 22:00:07 +00:00
Mike Surowiec
964aa9977e update logic for whether to render a page with nextjs 2021-05-19 08:27:40 -07:00
Kevin Heis
08ec46d0d1 Render color mode settings from github/github cookie (#19275)
* Add color mode selector in footer

* Interpret color mode from cookie

* Update render-page.js

* Delete color-mode.js

* Update render-page.js

* Make this more resilient to HTML tag changes

* Update scroll-button.scss
2021-05-13 14:45:11 +00:00
Mike Surowiec
e92e11888c feat: add code examples, cleanup and minor fixes 2021-05-11 10:13:15 -07:00
Mike Surowiec
9036301ad1 feat: add CommunityExamples, enable /discussions 2021-05-11 09:10:49 -07:00
Mike Surowiec
b2561a170f fix: run lint 2021-05-05 22:13:31 -07:00
Mike Surowiec
34dcba51f1 feat: add DefaultLayout, add to sponsors landing page 2021-05-05 14:54:49 -07:00
Mike Surowiec
eaddbc5db7 feat: add nextjs middleware handling (#19139)
* feat: add nextjs middleware handling split

* fix: eslint errors

* fix: filter boolean from csp list

* fix: feature flag nextjs server start

* feat: add prettier rules for ts,tsx files

* fix: remove unnecessary async from next middleware

* fix: next middleware name

* Update tsconfig.json

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* Update next-env.d.ts

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* fix: add typescript linting to lint command

* add comment for unsafe-eval, update webpack to use eval in development

* fix: feature flag typo

Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-05-05 15:23:46 +00:00
Sarah Schneider
03ee839af6 Merge branch 'main' into add-layouts-middleware 2021-04-13 16:03:36 -04:00
Sarah Schneider
1623191243 extract layout handling into separate middleware 2021-04-13 14:24:06 -04:00
Mike Surowiec
9185eb3c2a pre-render graphql input-objects
resolves #18269
2021-04-09 08:32:42 -07:00
James M. Greene
22e8d75c91 RedisAccessor tolerance for GET failures (#18586)
* Update RedisAccessor to allow for graceful GET failures, too
* Add unit tests for allowGetFailures behavior
2021-04-05 16:26:46 +00:00
James M. Greene
84547e54c7 Use node-redis for page cache (#18421)
* Use [node-]redis as a direct dependency

* Extract Redis client creation to its own module

* Attach extensive logging in the Redis client creation module

* Allow the rate limiter to pass requests when Redis is disconnected

* Update rate-limit-redis

* Default error input to empty object for formatRedisError method

* Provide a name for the rate limiter's Redis client

* Include redis-mock, exclude ioredis/ioredis-mock

* Remove unused RedisAccessor#exists method

* Switch RedisAccessor to use redis/redis-mock

* Provide a name for logging on the Redis page cache

* Remove extraneous trailing space from Redis logging prefix

Our updated use of console.* will already be adding a space after the prefix

* Replace ioredis-mock with redis-mock in tests

* Revert removal of ioredis dependency

* Bind Redis client to async promisified methods

* Extract former RedisAccessor constructor tests to new create-client tests

* Update RedisAccessor tests to work with the callback-based redis client

* Handle formatting Redis errors (or not) with more resiliency
2021-03-29 17:34:22 +00:00
James M. Greene
fd7d0eeb1a Add timeout and abort middleware and processing halts (#18177)
* Add middleware to timeout requests after a period

* Add halt-on-dropped-connection middleware to stop the middleware processing stack if the connection was already dropped

* Add a few strategic bail-out spots for dropped connections during the render-page middleware

* Handle 404s and HEAD requests earlier in the page rendering flow

* Add a few more strategic bail-out spots for dropped connections during the render-page middleware

* Add middleware to notice aborted requests

* Add a check for aborted requests into the isConnectionDropped logic

* Reformat comment for consistency

* Handle aborted requests correctly in the error handler

* Explicit returns for consistency
2021-03-09 19:14:02 +00:00
Kevin Heis
1918d2ea14 Inline CSRF token in HTML (#17748)
* Inline CSRF token

* Fix tests
2021-02-09 22:08:24 +00:00
James M. Greene
4dea2779f3 More explicit terminations of middleware (#17666)
* Be more explicit about next/res.send ending middleware functions

* Add note explaining the one exception
2021-02-03 16:38:41 -06:00
Vanessa Yuen
9bc90cd932 Learning Track navigation banner (#17440)
* add middleware to handle `learn` query param

* add exception to query-less cache key

* add querystring to learning track guides
2021-01-25 16:57:32 +00:00
James M. Greene
9171d17d23 Ignore query strings and fragment identifiers for Redis cache keys (#17205)
* Ignore query strings and fragment identifiers for Redis cache keys

* Don't serve cached responses if developer is requesting JSON debugging info
2021-01-07 09:42:53 -06:00
James M. Greene
068c472084 Cache rendered pages in Redis (#17106)
* Explicitly set the Redis database number used for rate limiting

* Switch to using ioredis as the Redis client for rate limiting

* Install ioredis-mock as a primary dependency

* Create a Redis BasicAccessor class and tests

* Switch rendered page caching to use Redis for storage

* Add support for additional Redis SET options like TTLs

* Remove currently unused methods

* Rename redis-accessors/basic to redis-accessor and remove extra fluff

* Change default behavior for cache setting to throw if an error occurs

Add option allowSetFailures to facilitate graceful failures

* Allow SET failures to fail gracefully for the rendered page cache

* Remove as-yet unneeded serialization options from RedisAccessor

* Move Redis client construction into RedisAccessor constructor, just pass in databaseNumber as option

* Remove rendered-page-cache in favor of direct RedisAccessor use

* Add tests for RedisAccessor constructor param validations

* Eliminate one roundtrip to Redis for the cached HTML existence check

Are we fast yet?

* Set a rendered page cache TTL of 24 hours
2021-01-06 15:30:51 -06:00
Kevin Heis
dd02bfa007 Merge main 2020-11-16 10:07:53 -08:00