1
0
mirror of synced 2025-12-23 11:54:18 -05:00
Commit Graph

54 Commits

Author SHA1 Message Date
Rachael Sewell
152a2399e2 remove lunr from codebase (#32432) 2022-11-10 21:36:13 +00:00
Rachael Sewell
1256add363 remove enterprise and lunr sync search workflows (#32430) 2022-11-07 22:36:30 +00:00
Peter Bengtsson
96ae1fa44d workflow_dispatch version input for sync search (#32207) 2022-11-01 20:24:58 +00:00
Peter Bengtsson
e94d5e78ef hide some navigational elements from search text (#31607) 2022-10-13 17:02:05 +00:00
Peter Bengtsson
098fb105b2 support filters in api/search/legacy (#31616) 2022-10-12 00:26:18 +00:00
Peter Bengtsson
994c09dc06 use cheerio-to-text (#31479) 2022-10-10 23:25:51 +00:00
Peter Bengtsson
9897635b8b Scraping product landing pages should have a content body (#31431) 2022-10-10 12:00:41 +00:00
Peter Bengtsson
642cb07aee make it easier to spot pages with no scrapable content (#31421) 2022-10-04 18:28:20 +00:00
Peter Bengtsson
ceaa60e384 don't trim searchable text if Elasticsearch (#31383) 2022-10-03 19:55:23 +00:00
Peter Bengtsson
0971a6ebf7 cope with odd 404 errors on /_cat/indices (#31375) 2022-10-03 18:38:32 +00:00
Peter Bengtsson
70f750c8dd create new error if es fails in any way (#31232) 2022-09-28 16:53:07 +00:00
Peter Bengtsson
db52a7e8bd dry-run with scraping and elasticsearch (#31201) 2022-09-27 16:59:33 -04:00
Peter Bengtsson
ff533e4ea3 be more explicit about potential alias updates (#31214) 2022-09-27 20:41:21 +00:00
Peter Bengtsson
8112c47fee merge in domwaiter (#31197) 2022-09-27 14:38:12 -04:00
Peter Bengtsson
a96b9ac779 send statsd timers on ES bulk indexing (#31017) 2022-09-21 18:27:36 +00:00
Peter Bengtsson
921cf5037f dry-run search scrape + index on PRs (#30861) 2022-09-19 14:32:23 +00:00
Peter Bengtsson
a179364cda favor more exact spelling searches (#30741) 2022-09-13 18:59:14 +00:00
Peter Bengtsson
54991f70c9 connect to ES 7.11 (#30462)
Co-authored-by: Rachael Sewell <rachmari@github.com>
2022-09-06 11:50:40 +00:00
Peter Bengtsson
d977d39536 use prefix for ES indexes for tests (#30288)
Co-authored-by: Kevin Heis <heiskr@users.noreply.github.com>
2022-08-26 21:10:30 +00:00
Rachael Sewell
f6d34d019c atomically update index alias and delete old indexes (#29742) 2022-08-09 01:02:55 +00:00
Peter Bengtsson
f8f20605f4 search api with elasticsearch (#29053)
* indexing records into Elasticsearch

* @elastic/elasticsearch@8.2.0

* mv

* fix the code

* pipe

* lfs

* use sha

* change how index names work

* search api with Elasticsearch

* works

* refactor client

* better snowballing

* remove dbg

* wip

* highlights and fuzzy search

* improvements and upgrade

* also upgrade

* wip

* index more explicitly

* better validation

* change max size

* popularity by ordinal ranking instead

* playing

* fix package-lock

* rearranging

* catchMiddlewareError

* fix indexing

* match_phrase_prefix on wrapped search

* refactorings

* escape html in content

* wip

* search by URL

* update lock file

* v1 api

* inroads on jest testing the search

* valid yaml hopefully

* index fixtures into local elasticsearch

* specific versions to index only

* small fixes

* feedbacked

* fix tests

* use the npm script
2022-08-05 19:40:38 +00:00
Kevin Heis
2182ff28f3 Next12 now supports ESM (#29295)
* Next12 now supports ESM

* No more michael jackson script extensions

* Fix test running

* Update jest-puppeteer.config.cjs

* Update package.json
2022-07-26 17:53:23 +00:00
Peter Bengtsson
ad4c835316 only sync search records without lunr and no br compression (#29156)
* refactor sync-search-indices

* tidying

* only sync search records without Lunr and no .br compression

* works

* oops
2022-07-20 18:57:34 +00:00
Peter Bengtsson
b0746325db nicer timestamps in Elasticearch index names (#29155) 2022-07-19 15:35:29 +00:00
Peter Bengtsson
44a91fdec5 refactor sync-search-indices (#29152)
* refactor sync-search-indices

* tidying
2022-07-19 15:13:34 +00:00
Peter Bengtsson
b7b22d1685 read search records from any directory (#29153) 2022-07-19 12:24:46 +00:00
Peter Bengtsson
9abd3b7929 remove leftover debugging in index script (#29145) 2022-07-18 20:27:58 +00:00
Peter Bengtsson
e61d71f42f HTML to plain text is broken in various places (#29006)
* HTML to plain text is broken in various places

* remove comment
2022-07-13 21:29:55 +00:00
Peter Bengtsson
721e68f4ab Index records into Elasticsearch (#27819)
* indexing records into Elasticsearch

* @elastic/elasticsearch@8.2.0

* mv

* fix the code

* pipe

* lfs

* use sha

* change how index names work

* better snowballing

* improvements and upgrade

* index more explicitly

* wip

* no need to save space with elasticsearch

* update package-lock

* Revert "no need to save space with elasticsearch"

This reverts commit 9cdd4d8f4fa7be822bdef3976091c3c640519dec.

* adding a comment

* better comment

* a better comment

* feedbacked

* more comment
2022-07-13 20:32:22 +00:00
Peter Bengtsson
7783171b56 get popularity from article path (#28016) 2022-05-26 21:27:09 +00:00
Peter Bengtsson
18504871b9 cache full rendering (#25424)
* cache full rendering

* still not working with gzip

* progress progress progress

* smaller

* hacky progress

* small fixes

* wip

* lock file

* wip

* wip

* package-lock updates

* wip

* search DOM in lowercase

* simplify

* with instrument

* improve test coverage

* mutateCheeriobodyByRequest

* fix

* remove renderContentCacheByContex

* disable render caching in sync-search

* diables things in github/github link checker

* gzip lru

* tidying up

* updated

* correct tests

* fix: move userLanguage to LanguagesContext

* Revert "fix: move userLanguage to LanguagesContext"

This reverts commit d7c05d958c71eaad496eb46764eb845d80b866ca.

* contexts ftw

* fixed rendering tests

* oops for got new file

* nits addressed

Co-authored-by: Mike Surowiec <mikesurowiec@users.noreply.github.com>
2022-05-23 12:12:09 +00:00
Peter Bengtsson
bd9d85ed0f remove need for 'count-array-values' (#24867) 2022-02-01 21:27:47 +00:00
Peter Bengtsson
3c579b410c delete customRanking (#24659) 2022-01-27 13:02:13 +00:00
Peter Bengtsson
1ab125dd08 draw title from first H1 only (#23266)
Part of #1218
2021-12-01 23:26:08 +00:00
Kevin Heis
5b1ac53379 A few things that Prettier seemed to miss (#23005) 2021-11-19 18:32:20 +00:00
Grace Park
5bf39459ca Updating Search using Primer Components (#22677)
* updating search icon

* remove no fill

* update fill color

* using ActionList and Overlay

* update to ActionList, Overlay, add Label to Breadcrumbs, and change Mark

* add search-results test data and move header search results to right

* make search results scrollable and fix max-width in header search

* update header search and styling updates

* add highlight

* update styling

* styling and lint errors

* fix lint error and fetch

* Update components/Search.tsx

* update label color

* remove bottom margin

Co-authored-by: Rachael Sewell <rachmari@github.com>
2021-11-12 12:25:47 -08:00
Rachael Sewell
0244fda807 search improvements (#22443) 2021-11-10 11:11:28 -08:00
Rachael Sewell
cbcd383323 add search QA tests (#22593) 2021-11-04 14:05:11 -07:00
Rachael Sewell
2f8283a79f remove german search indexes (#22554) 2021-11-02 17:55:58 -07:00
Peter Bengtsson
d4c4c46f95 search sync with popular-pages.json (#22353) 2021-10-26 10:08:14 -07:00
Mike Surowiec
d76c16da19 Scope markdown body (#21082)
* update article content to markdown ui component

* decouple lunr indexing from class name

* remove summary outline none rule, apply utility class instead

* improve typing

* scope more styles down to markdown-body/extended-markdown

* move all markdown-body style overrides to MarkdownContent component

* fix class targeting within css module

* clean up MarkdownContent header style

* rename data-lunr to data-search

* fix: inline code color issue

* fix: update article markdown to work with MarkdownContent
2021-08-30 14:24:49 +00:00
Rachael Sewell
df16267a8f remove cached-index-names file and dry run sync used for Algolia (#21133) 2021-08-25 22:04:26 +00:00
Rachael Sewell
a92853e40d reapply some classes for lunr indexing (#20765) 2021-08-11 15:06:01 +00:00
Grace Park
2dbea3737a Removing algolia (#20633) 2021-08-05 13:10:51 -04:00
Kevin Heis
b29e37318a Remove import x statements (#20594)
* Clear out most import x

* Update rimraf use

* Move up readme blocks in scripts
2021-07-29 20:28:30 +00:00
Kevin Heis
1ed18e1448 Clear out some non-fn references to Algolia (#20592)
* Clear out some non-fn references to Algolia

* Update .github/workflows/dry-run-sync-search-indices.yml

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* Update contributing/search.md

Co-authored-by: James M. Greene <JamesMGreene@github.com>

Co-authored-by: James M. Greene <JamesMGreene@github.com>
2021-07-29 20:20:55 +00:00
Kevin Heis
eddadb875e LFS the Lunr Indexes (#20521)
* LFS the Lunr Indexes

* Add a note about installing Git LFS

* Run "install" command

* Update indexes

* Index names
2021-07-27 22:06:31 +00:00
Mike Surowiec
9386571aa4 fix: use named export for all-versions (#20478)
* fix: use named export for all-versions

* run prettier
2021-07-22 19:29:00 +00:00
Kevin Heis
8a56437c93 Pretty format (#20352)
* Update prettier flow to include JS

* Run prettier

* ...run prettier
2021-07-14 14:35:01 -07:00
Kevin Heis
42e785b0a8 Migrate CommonJS to ESM (#20301)
* First run of script

* Get the app running --- ish

* Get NextJS working

* Remove `node:`

* Get more tests passing in unit directory

* Update FailBot test to use nock

* Update test.yml

* Update Dockerfile

* tests/content fixes

* Update page.js

* Update build-changelog.js

* updating tests/routing

* Update orphan-tests.js

* updating tests/rendering

* Update .eslintrc.js

* Update .eslintrc.js

* Install jest/globals

* "linting" tests

* staging update to server.mjs

* Change '.github/allowed-actions.js' to a ESM export

* Lint

* Fixes for the main package.json

* Move Jest to be last in the npm test command so we can pass args

* Just use 'npm run lint' in the npm test command

* update algolia label script

* update openapi script

* update require on openapi

* Update enterprise-algolia-label.js

* forgot JSON.parse

* Update lunr-search-index.js

* Always explicitly include process.cwd() for JSON file reads pathed from project root

* update graphql/update-files.js script

* Update other npm scripts using jest to pass ESM NODE_OPTIONS

* Update check-for-enterprise-issues-by-label.js for ESM

* Update create-enterprise-issue.js for ESM

* Import jest global for browser tests

* Convert 'script/deploy' to ESM

Co-authored-by: Grace Park <gracepark@github.com>
Co-authored-by: James M. Greene <jamesmgreene@github.com>
2021-07-14 13:49:18 -07:00