1
0
mirror of synced 2026-01-04 09:04:47 -05:00
Commit Graph

2364 Commits

Author SHA1 Message Date
Davin Chia
fd5c5be352 Inject LOG_LEVEL for docker. (#4498) 2021-07-02 16:57:12 +08:00
Abhi Vaidyanatha
d985fe73f4 Add note about rate limits with the HubSpot API (#4491)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 23:04:35 -07:00
Davin Chia
436471c699 Github Store should not error if the file cannot be found. (#4455)
Fixes #4411.

There are two cases here:

- No internet access
- The file cannot be found.

In both cases, we return an empty list instead. This turns this operation into a noop and Airbyte then uses the list bundled with the current version.
2021-07-02 10:55:47 +08:00
Abhi Vaidyanatha
5d56d00f1a Add 7-1-2021 Slack release summary (#4482)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 19:01:55 -07:00
Abhi Vaidyanatha
64a8247b83 Add Airflow demo script for one click deployment (#4451)
* Add Airflow Demo Example

* Finalize script instructions

* Add down.sh and run gradle format

* Note about down.sh

* Address review comments

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 18:38:21 -07:00
Sherif A. Nada
9708e4c8de 🐛 bugfix Source Okta: Fix endless loop while syncing data from logs stream (#4464)
Co-authored-by: sabifranjo <sabifranjo@gmail.com>
Co-authored-by: Sabolc Franjo <sabolc.franjo@ev-box.com>
2021-07-01 17:39:51 -07:00
Abhi Vaidyanatha
885da9b27a Make experience required more vague for Tech Writer job. (#4480)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 15:44:01 -07:00
Abhi Vaidyanatha
0963c27578 Add changelog summaries for the last 3 updates on main changelog doc. (#4479)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-02 09:22:20 +11:00
Abhi Vaidyanatha
7f8e2c22a2 Add platform changelog for 0.27.0 (#4477)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 14:55:56 -07:00
Abhi Vaidyanatha
aa5066081c Add 7/1/2021 connector changelog (#4476)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 14:49:37 -07:00
Abhi Vaidyanatha
b94d2eb4c4 Fix pubsub description YAML frontmatter (#4475)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-01 14:32:02 -07:00
Eugene
4399656ec9 🎉 Source Cockroachdb: Added new source for Cockroachdb (#4382)
* [1705] Added cockroachdb source connector
2021-07-01 22:04:21 +03:00
Christophe Duong
53b02e0499 Tweak .env in kube/overlays to connect to external db (#4403)
* Mirror .env to connect to external db
2021-07-01 19:27:33 +02:00
Subodh Kant Chaturvedi
586eeb70c2 minor fix to upgrade airbyte doc (#4467) 2021-07-01 22:44:21 +05:30
Yevhenii
83ac92b212 Source Facebook Marketing: update cursor_field for Ads Insights Streams (#4437)
* Big Fix: updated_time is an attribute of ad in FB ad insights call

Co-authored-by: harshithmullapudi <harshithmullapudi@gmail.com>
Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
2021-07-01 17:50:29 +03:00
Subodh Kant Chaturvedi
097c9548c0 Doc for Automatic Upgrade of Airbyte (#4453)
* Revert "manual airbyte upgrade doc (#4435)"

This reverts commit dedab30a93.

* update doc

* upgrade version
2021-07-01 18:31:34 +05:30
Subodh Kant Chaturvedi
7de4bbf13c Bump version: 0.26.4-alpha → 0.27.0-alpha (#4458) v0.27.0-alpha 2021-07-01 18:06:51 +05:30
Subodh Kant Chaturvedi
e19a90857d enable automatic migration for kube on or above 0.26.5-alpha (#4454)
* enable autoamtic migration for kube on or above 0.26.5-alpha

* update log message

* address review comment

* address review comment

* handle dev version
2021-07-01 16:40:21 +05:30
Christophe Duong
8b6093ce61 Configure kube pod resources for workers/syncs (#4381)
* Configure kube pod resources for workers/syncs
2021-07-01 12:25:14 +02:00
Davin Chia
b9f5632108 Better logging for Job Submission errors. (#4452)
Log the job id, job status and connection id if there is any job submission error to allow devs to easily investigate what is going on.

Without this it's impossible to remove the offending job. This is bad since the job submitter is currently set up to pull the oldest job, which means it will spin on the offending job, stopping all job submission.
2021-07-01 17:59:56 +08:00
Christophe Duong
45b47f498c set limits on production kube containers
* Set limits individually by container
2021-07-01 10:43:32 +02:00
Charles
d78a985cb3 Separate config validation from FileSystemConfigPersistence (#4169) 2021-06-30 21:59:24 -07:00
John Lafleur
f3acd45b44 Update SUMMARY.md 2021-07-01 13:38:23 +11:00
John Lafleur
681452a8d8 Update recruiter.md 2021-07-01 13:37:54 +11:00
John Lafleur
1aed32433e Update and rename code-of-conduct.md to slack-code-of-conduct.md (#4450)
* Update and rename code-of-conduct.md to slack-code-of-conduct.md

* Update SUMMARY.md
2021-07-01 13:32:30 +11:00
Sherif A. Nada
0f171a11f7 fix formatting, remove slack singer, add okta connectors (#4448) 2021-06-30 14:18:54 -07:00
Abhi Vaidyanatha
da36e22272 Add missing Sources and Destinations to Catalog Table. (#4423)
* Add missing Sources and Destinations to Catalog Table.

* Add Snowflake Source.

* Address review comments

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-06-30 13:27:26 -07:00
Sherif A. Nada
a826850d08 "Slack CDK" --> Slack 2021-06-30 13:20:27 -07:00
Loc Nguyen
acf0c207de 🎉 New source: Chargebee (#3410)
* Add source-chargebee/

* Add schemas

* Remove debug messages

* Remove airbyte_secret specification to fix UI error: API key not found

* Add Invoice stream

* Add airbyte_secret back

* Rename streams for consistency with source.py

* Add pagination and incremental syncing

* Add backoff

* Add invoice schema

* Minor JSON bug fix

* Add orders; Clean up source.py

* Change format to remove datetime string with integer (according to Chargebee documentation)

* Change format to remove datetime string with integer (according to Chargebee documentation)

* Change self.params to be instance var (to reset every Stream instance); Add Stream logging; Add Plan

* Change self.params to be instance var (to reset every Stream instance); Add Stream logging; Add Plan

* Add Plan

* Add Addon

* Reformat

* Rename files so they are recognizeable in the UI

* Update airbyte-integrations/connectors/source-chargebee/setup.py

Thanks! TIL!! 🙏

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-06-30 22:53:52 +03:00
etsybaev
60c53e330c 🐛 Source MySQl: Fixed MySQL CDC Decimal handling (#4315)
* [4105] Fixed MySQL CDC Decimal handling (store decimal value in Kafka as a string instead of binary value)
2021-06-30 21:47:17 +03:00
Marcos Marx
265e7f79d8 Normalization: remove dedup cdc excluded (#4297)
* change stream processor

* integraton tests

* add integration tests

* format gradle file

* add excluded files

* change catalog and msgs

* add cdc messages

* solve cdc excluded problem with tests

* remove .egg files

* remove time import

* tab stream_processor

* uncommented local test

* add tests for dbt!

* add excluded files

* add missing snowflake file

* add pg, bq and snowflake

* chris comments

* test comment

* pytest parametrize tests

* bump normalization version

* formating

* run test for all destinations
2021-06-30 14:59:13 -03:00
Charles
335f5edef8 copy seed configurations from config/init to server (#4417)
Co-authored-by: subodh <subodh1810@gmail.com>
2021-06-30 10:38:19 -07:00
Charles
8b01d981b6 filter request body in logs for deployment/import (#4394) 2021-06-30 10:36:15 -07:00
Subodh Kant Chaturvedi
dedab30a93 manual airbyte upgrade doc (#4435) 2021-06-30 23:02:57 +05:30
Daniel Dubinin
0a11d88b0b Source Oracle DB: Create comprehensive data type tests (#4127)
* oracle comprehensive test
2021-06-30 13:04:42 +03:00
Davin Chia
2113828560 Load local docker images into KIND. (#4426)
We were previously running the Kube acceptance test on images pulled from Dockerhub and not images built as part of the acceptance tests build.

This PR fixes that by explicitly loading the images into KIND. This happened because KIND does not have access local docker agent.

We stopped publishing dev images in PR:4425 so there are other updates for tests that were previously using these images.

Since we are no longer publishing dev images - I also went through and removed all the images on DockerHub - I'm pretty confident this is actually, finally, using locally built docker images.
2021-06-30 15:51:14 +08:00
Subodh Kant Chaturvedi
4b6465d853 fix versioning for server in docker compose (#4430) 2021-06-30 13:05:33 +05:30
Jared Rhizor
19020d97b3 preserve kube db after deleting db pod (#4424) 2021-06-29 22:47:07 -07:00
Abhi Vaidyanatha
9138e58bf1 Add CentOS 8 setup guide. (#4421)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-06-29 20:48:44 -07:00
Davin Chia
c5a02810d1 Various build updates. (#4425)
* Stop pushing dev images. Pin actions to the v2 release track to get latest updates.

* Update gradle git workflow comments.
2021-06-30 11:16:00 +08:00
Davin Chia
52bd5c96f0 🎉 Add Minio support to Kube. (#4365)
Implement logging to and reading from Minio. Use the same S3 client for this.

Configure Airbyte Kube Prod and Staging to use Minio by default, so Airbyte Kube is a standalone deployment.

Also update documentation to reflect this.
2021-06-30 09:42:50 +08:00
Marcos Marx
398bf6674a Documentation: Slack issue template (#4337)
* slack issue template docs

* abhi comments and add section items
2021-06-29 19:47:55 -03:00
Abhi Vaidyanatha
60be058c11 Add note about repeated rows when attempting incremental append syncs with unchanged data (#4309)
* Add incremental append note about syncs with unchanged data

* Update SQL statement to illustrate how incremental append could work.

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-06-29 15:32:29 -07:00
Abhi Vaidyanatha
33a80deb0a Fix broken links in docs. (#4419)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-06-29 14:51:46 -07:00
Sherif Nada
0a0a3b97fe fix gitbook formatting in tutorial 2021-06-29 14:49:00 -07:00
Sherif Nada
5a59e27f94 add destination tutorial to SUMMARY.md 2021-06-29 14:45:05 -07:00
Marcos Marx
4bace33322 Add comments on issue templates and reframe bug-report template (#3203)
* add comments and reframe bug-report issue

* add suggestion using details tag for log

* change abhi comments

* add more example os vrson/instance

* correct tag

* remove hidden comments and add submit pr

* correct some points

* correct details log block
2021-06-29 18:37:18 -03:00
Sherif A. Nada
c2b3c23a92 respond to post-merge code review for destination docs (#4418) 2021-06-29 14:35:22 -07:00
Sherif A. Nada
921263845e 📚 Add a tutorial for how to build a java destination (#4384)
Co-authored-by: LiRen Tu <tuliren@gmail.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>

Co-authored-by: LiRen Tu <tuliren@gmail.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
2021-06-29 13:54:37 -07:00
Subodh Kant Chaturvedi
0e194fb331 use old seed mount (#4413) 2021-06-30 01:21:48 +05:30