improve link checker (#6797)

* improve link checker

* use ref instead of base_ref

* remove base, always compare to master for modified

* add failing to test

* don't do quiet for testing

* switch error to 404 not 403

* yes to both

* turn off verbose mode

* fix

* actually check things

* fix outstanding link problems

* revet change to run for everything

* use new format

* ignore gitbook failures

* switch back to only running on master

.github/workflows/doc-link-check.json

@@ -45,11 +45,15 @@
       "reason": "Test only scaffold connector",
       "pattern": "destinations/scaffold-"
     },
+    {
+      "reason": "Returns a 403 for many valid pages",
+      "pattern": "https://mvnrepository.com/artifact/"
+    },
     {
       "reason": "Archived articles aren't actively maintained.",
       "pattern": "archive/"
     }
   ],
   "retryOn429": false,
-  "aliveStatusCodes": [200, 206, 401, 403, 429, 503]
+  "aliveStatusCodes": [200, 206, 429, 503, 0]
 }

.github/workflows/doc-link-check.yml

@@ -1,5 +1,6 @@
 # Perform link check on all markdown files
-name: Doc Link Checker (Full)
+
+name: Doc Link Checker
 
 on:
   push:
@@ -12,10 +13,17 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@master
+      # check all files on master
       - uses: gaurav-nelson/github-action-markdown-link-check@v1
+        if: github.ref == 'refs/heads/master'
         with:
-          use-quiet-mode: 'no'
-          use-verbose-mode: 'yes'
+          use-quiet-mode: 'yes'
           check-modified-files-only: 'no'
           config-file: .github/workflows/doc-link-check.json
-          base-branch: ${{ github.base_ref }}
+#      # check changed files for branches
+#      - uses: gaurav-nelson/github-action-markdown-link-check@v1
+#        if: github.ref != 'refs/heads/master'
+#        with:
+#          use-quiet-mode: 'yes'
+#          check-modified-files-only: 'yes'
+#          config-file: .github/workflows/doc-link-check.json

docs/deploying-airbyte/on-kubernetes.md

@@ -10,8 +10,8 @@ Airbyte allows scaling sync workloads horizontally using Kubernetes. The core co
 
 For local testing we recommend following one of the following setup guides:
 
-* [Docker Desktop \(Mac\)](https://docs.docker.com/desktop/kubernetes/)
-* [Minikube](https://minikube.sigs.k8s.io/docs/start/)
+* [Docker Desktop \(Mac\)](https://docs.docker.com/desktop/kubernetes)
+* [Minikube](https://minikube.sigs.k8s.io/docs/start)
   * NOTE: Start Minikube with at least 4gb RAM with `minikube start --memory=4000`
 * [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/)
 

docs/integrations/sources/amplitude.md

@@ -38,7 +38,7 @@ The Amplitude connector should gracefully handle Amplitude API limitations under
 * Amplitude Secret Key
 
 ### Setup guide
-
+<!-- markdown-link-check-disable-next-line -->
 Please read [How to get your API key and Secret key](https://help.amplitude.com/hc/en-us/articles/360058073772-Create-and-manage-organizations-and-projects#view-and-edit-your-project-information).
 
 ## Changelog

docs/integrations/sources/iterable.md

@@ -51,7 +51,7 @@ The Iterable connector should not run into Iterable API limitations under normal
 * Iterable API Key
 
 ### Setup guide
-
+<!-- markdown-link-check-disable-next-line -->
 Please read [How to find your API key](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-#creating-api-keys).
 
 ## CHANGELOG

docs/integrations/sources/klaviyo.md

@@ -44,6 +44,7 @@ The Klaviyo connector should not run into Klaviyo API limitations under normal u
 
 ### Setup guide
 
+<!-- markdown-link-check-disable-next-line -->
 Please follow these [steps](https://help.klaviyo.com/hc/en-us/articles/115005062267-How-to-Manage-Your-Account-s-API-Keys#your-private-api-keys3) to obtain Private API Key for your account.
 
 ## CHANGELOG

docs/integrations/sources/mixpanel.md

@@ -47,9 +47,10 @@ The Mixpanel connector should not run into Mixpanel API limitations under normal
 * Project region `US` or `EU`
 
 ### Setup guide
-
+<!-- markdown-link-check-disable-next-line -->
 Please read [Find API Secret](https://help.mixpanel.com/hc/en-us/articles/115004502806-Find-Project-Token-).
 
+<!-- markdown-link-check-disable-next-line -->
 Select the correct region \(EU or US\) for your Mixpanel project. See detail [here](https://help.mixpanel.com/hc/en-us/articles/360039135652-Data-Residency-in-EU)
 
 ## CHANGELOG

docs/integrations/sources/trello.md

@@ -42,7 +42,7 @@ The Trello connector should not run into Trello API limitations under normal usa
 * Trello API Key
 
 ### Setup guide
-
+<!-- markdown-link-check-disable-next-line -->
 Please read [How to get your APIs Token and Key](https://developer.atlassian.com/cloud/trello/guides/rest-api/authorization/#using-basic-oauth) or you can log in to Trello and visit [Developer API Keys](https://trello.com/app-key/).
 
 ## Changelog