1
0
mirror of synced 2025-12-25 11:03:37 -05:00
Commit Graph

553 Commits

Author SHA1 Message Date
Peter Bengtsson
54991f70c9 connect to ES 7.11 (#30462)
Co-authored-by: Rachael Sewell <rachmari@github.com>
2022-09-06 11:50:40 +00:00
Peter Bengtsson
eda38df4c7 fetch external urls better in link checker (#30387) 2022-09-01 11:46:15 +00:00
Peter Bengtsson
05967a9b68 optionally check external URLs in rendered content (#25429) 2022-08-29 20:43:12 +00:00
Rachael Sewell
a9aa911d84 remove some old logging from openapi script (#30317) 2022-08-26 22:37:23 +00:00
Peter Bengtsson
d977d39536 use prefix for ES indexes for tests (#30288)
Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>
2022-08-26 21:10:30 +00:00
Rachael Sewell
83680f30b0 refactor scripts used for openapi (#30309) 2022-08-26 20:28:37 +00:00
Rachael Sewell
59542a4fcc openapi class bugfix for nullable type (#30270) 2022-08-25 21:59:07 +00:00
Kevin Heis
303d5190db Create a translations test suite (#30000)
* Scaffold files for migration

* Move user-agent into unit suite

* Nothing to move from browser suite

* Migrate tests to translations/content

* Migrate existing translation test to meta

* No graphql tests to migrate

* Migrate lint-translation-reporter

* Migrate lint-translation-reporter

* Remove languages-schema, unused

* Restore languages-schema

* Restore languages-schema

* Migrate rendering

* Migrate routing

* Migrate most of unit

* Remove dead files, comment out tests that aren't expected to work yet

* Migrate from get-redirect

* Migrate page and pages

* Migrate linting code

* Fix lint issues

* Found a few more

* Run prettier

* Move crowdin-config test and helper

* Update crowdin-config.js

* Remove translation linting, crowdin config lint, reduce file count

* Remove test that's been skipped for a year

* Restore linting with note to remove later

* Update lint-translation-reporter.js

* Clean up rendering suite

* Update rendering.js

* Remove excessive describe blocks

* Redirect tests

* Clean up unit

* Remove test that's never called

* Don't compare early access

* Rename test suites

* Update "content" tests

* Update files.js

* Update search.js

* Update files.js

* Update files.js
2022-08-25 12:38:03 -07:00
Rachael Sewell
28423ba069 simplify openapi schema parsing (#30192) 2022-08-24 22:14:39 +00:00
Sarah Edwards
d2a2113c11 REST new user guidance (#29642) 2022-08-23 19:31:01 +00:00
Peter Bengtsson
420ea91b81 cap check-all-english-links and upload log file (#30092) 2022-08-19 18:11:21 +00:00
Rachael Sewell
961f985cfe Revert "Refactor openapi table func" (#30040) 2022-08-17 17:24:11 -07:00
Rachael Sewell
792f4c626a Refactor openapi table func (#29924) 2022-08-17 13:46:15 -07:00
Hector Alfaro
077195f5ab REST: Script for adding a GHEC key to API operations (#29683)
* add draft of ghec api rewrite script

* fix regex to allow for any location of the githubCloudOnly prop

* add better logging

* account for missing x-github key

* skip and log each time dotcom isn't a version

* log oauth categories

* move script to a better spot

* remove oauth logging, add more clarity to logs
2022-08-16 17:40:31 +00:00
Hector Alfaro
54d996f186 Localization: Clean up Microsoft workflow (#29675)
* remove comments that no longer apply

* remove unused matrix variables

* remove CrowdIn-specific step of resetting broken files

* udpate link to Microsoft-specific log

* bring back reset script still in use

* add acceptable ja translation of TOS to rendering test

* add ability to remove files that have been translated but don't exist (wip)

* document what `languageFiles()` returns

* fix path issues for removed translations

* cleaner script execution

* add removed files to CSV report

* add workflow link to PR body for better troubleshooting

* catch missing regex for reporting file removals

* fix capturing groups in regex for csv reporting

* fix link to CSV file for PR

* fix formatting for PR

* pass in branch name from workflow

* put back helper scripts, make msft-specific copies
2022-08-15 17:17:25 +00:00
Rachael Sewell
b552180c04 remove unused properties from decorated rest schemas (#29881) 2022-08-12 00:09:41 +00:00
Sarah Schneider
9236898431 add support for passing an option to open sections by default 2022-08-11 11:21:20 -04:00
Sarah Schneider
db6f3c8890 change toc filenames to index.html 2022-08-10 15:26:13 -04:00
Sarah Schneider
af50d6e859 re-add the dev toc in script/static form 2022-08-10 15:05:23 -04:00
Rachael Sewell
f6d34d019c atomically update index alias and delete old indexes (#29742) 2022-08-09 01:02:55 +00:00
Peter Bengtsson
a1e3866fa5 don't get available languages from session (#29715)
* don't get available languages from session

* update useSession

* one more fix
2022-08-08 11:13:27 -04:00
Peter Bengtsson
f8f20605f4 search api with elasticsearch (#29053)
* indexing records into Elasticsearch

* @elastic/elasticsearch@8.2.0

* mv

* fix the code

* pipe

* lfs

* use sha

* change how index names work

* search api with Elasticsearch

* works

* refactor client

* better snowballing

* remove dbg

* wip

* highlights and fuzzy search

* improvements and upgrade

* also upgrade

* wip

* index more explicitly

* better validation

* change max size

* popularity by ordinal ranking instead

* playing

* fix package-lock

* rearranging

* catchMiddlewareError

* fix indexing

* match_phrase_prefix on wrapped search

* refactorings

* escape html in content

* wip

* search by URL

* update lock file

* v1 api

* inroads on jest testing the search

* valid yaml hopefully

* index fixtures into local elasticsearch

* specific versions to index only

* small fixes

* feedbacked

* fix tests

* use the npm script
2022-08-05 19:40:38 +00:00
Peter Bengtsson
2c8e07843b github/github link checker got config (#29617) 2022-08-04 11:27:39 -04:00
Rachael Sewell
3cbd7103da Remove top-level categories on graphql queries page (#29592) 2022-08-03 20:01:38 +00:00
Rachael Sewell
467459af7c Reactify graphql pages (#28547) 2022-08-03 10:51:21 -07:00
Grace Park
b5f8a6089f Fix REST decorator script (#29542)
* fix decorator script

* run on node 16

Co-authored-by: Grace Park <lugraciecy@gmail.com>
2022-08-03 17:35:02 +00:00
github-openapi-bot
2255a2c8f7 Update OpenAPI Descriptions (#29408)
* Update OpenAPI  Descriptions

* Add decorated OpenAPI schema files

* Add new pages override (#29446)

* update decorated files

Co-authored-by: github-openapi-bot <github-openapi-bot@users.noreply.github.com>
Co-authored-by: Matt Pollard <mattpollard@users.noreply.github.com>
Co-authored-by: Rachael Sewell <rachmari@github.com>
2022-07-28 13:40:47 -07:00
Kevin Heis
2182ff28f3 Next12 now supports ESM (#29295)
* Next12 now supports ESM

* No more michael jackson script extensions

* Fix test running

* Update jest-puppeteer.config.cjs

* Update package.json
2022-07-26 17:53:23 +00:00
Peter Bengtsson
ad4c835316 only sync search records without lunr and no br compression (#29156)
* refactor sync-search-indices

* tidying

* only sync search records without Lunr and no .br compression

* works

* oops
2022-07-20 18:57:34 +00:00
Peter Bengtsson
b0746325db nicer timestamps in Elasticearch index names (#29155) 2022-07-19 15:35:29 +00:00
Peter Bengtsson
44a91fdec5 refactor sync-search-indices (#29152)
* refactor sync-search-indices

* tidying
2022-07-19 15:13:34 +00:00
Peter Bengtsson
b7b22d1685 read search records from any directory (#29153) 2022-07-19 12:24:46 +00:00
Peter Bengtsson
9abd3b7929 remove leftover debugging in index script (#29145) 2022-07-18 20:27:58 +00:00
Peter Bengtsson
e61d71f42f HTML to plain text is broken in various places (#29006)
* HTML to plain text is broken in various places

* remove comment
2022-07-13 21:29:55 +00:00
Peter Bengtsson
88f0b1686f delete script/content-migrations/deduplicate-enterprise-assets.js (#29011)
* delete script/content-migrations/deduplicate-enterprise-assets.js

* update package-lock
2022-07-13 20:48:09 +00:00
Peter Bengtsson
721e68f4ab Index records into Elasticsearch (#27819)
* indexing records into Elasticsearch

* @elastic/elasticsearch@8.2.0

* mv

* fix the code

* pipe

* lfs

* use sha

* change how index names work

* better snowballing

* improvements and upgrade

* index more explicitly

* wip

* no need to save space with elasticsearch

* update package-lock

* Revert "no need to save space with elasticsearch"

This reverts commit 9cdd4d8f4fa7be822bdef3976091c3c640519dec.

* adding a comment

* better comment

* a better comment

* feedbacked

* more comment
2022-07-13 20:32:22 +00:00
Hector Alfaro
95e6f3d3ab Deprecate GHES 3.1 (#28798)
* add 3.1 to deprecated versions

* rewrite img src to use azure blob storage in archive script

Co-authored-by: rachmari <rachmari@users.noreply.github.com>

* remove static files for ghes 3.1

* remove liquid conditionals and content for ghes 3.1

* remove outdated hardware reqs reusable

* Fix liquid conditional uncaught by script

* Close liquid conditionals missed by script

* Apply @mattpollard's suggestions

Co-authored-by: Matt Pollard <mattpollard@users.noreply.github.com>

Co-authored-by: rachmari <rachmari@users.noreply.github.com>
Co-authored-by: Matt Pollard <mattpollard@users.noreply.github.com>
2022-07-11 21:17:09 +00:00
Kevin Heis
05fe4c60cb Upgrade commander (#28613)
* Upgrade commander

* Change import
2022-06-17 16:43:02 +00:00
Bishal Das
139b4f231c "or not" is added 2022-06-15 15:46:54 +05:30
Bishal Das
e6e14dd47b UPDATE the Typos in script/README.md 2022-06-13 17:34:52 +05:30
Robert Sese
b3af722730 Fix daily link checker (#28346)
* reduce linkinator concurrency from 300 to 100

* reduce it and use self-hosted

* try again

* actually do concurrency of 10

* move the override to the Action workflow

* Trying `DISABLE_RENDERING_CACHE`

* back to 100?

* super low concurrency

* 3??

* 1??

* try self-hosted + 100 concurrency

Co-authored-by: Peter Bengtsson <mail@peterbe.com>
2022-06-09 16:48:02 -05:00
Sarah Schneider
14c8fc570e add script 2022-06-06 18:11:17 -04:00
Sarah Schneider
1500b84cd2 support passing an array of file extensions 2022-06-06 18:08:03 -04:00
Mike Surowiec
b4608a86d8 fix: remove usage of readFileAsync (#28053)
* fix: remove usage of readFileAsync

* fix: use same import style

* fix: update tests

Co-authored-by: Peter Bengtsson <mail@peterbe.com>
2022-06-01 14:49:30 +00:00
Peter Bengtsson
31c3fc36b2 don't fail on posting comment from English link checker (#28059)
* don't fail on posting comment from English link checker

* small correction on line break and longer timeout patience

* feedbacked
2022-05-31 18:05:39 +00:00
Peter Bengtsson
7783171b56 get popularity from article path (#28016) 2022-05-26 21:27:09 +00:00
Peter Bengtsson
8d4e1cda62 use shorthand notation (#27919) 2022-05-23 18:22:44 +00:00
Peter Bengtsson
18504871b9 cache full rendering (#25424)
* cache full rendering

* still not working with gzip

* progress progress progress

* smaller

* hacky progress

* small fixes

* wip

* lock file

* wip

* wip

* package-lock updates

* wip

* search DOM in lowercase

* simplify

* with instrument

* improve test coverage

* mutateCheeriobodyByRequest

* fix

* remove renderContentCacheByContex

* disable render caching in sync-search

* diables things in github/github link checker

* gzip lru

* tidying up

* updated

* correct tests

* fix: move userLanguage to LanguagesContext

* Revert "fix: move userLanguage to LanguagesContext"

This reverts commit d7c05d958c71eaad496eb46764eb845d80b866ca.

* contexts ftw

* fixed rendering tests

* oops for got new file

* nits addressed

Co-authored-by: Mike Surowiec <mikesurowiec@users.noreply.github.com>
2022-05-23 12:12:09 +00:00
Peter Bengtsson
d5cef9c5d6 run link checker against localhost:4000 instead of prod (#27757)
* run link checker against localhost:4000 instead of prod

* exclude docs.github.com

* print output if possible

* upload broken_links.md as artifact

* debug broken_links.md
2022-05-19 13:12:07 +00:00
Hector Alfaro
6cbe4d131e clean up unnecessary conditional code 2022-05-13 12:40:07 -04:00