jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-23 11:57:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,118 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

170 Commits

Author SHA1 Message Date
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Artem Inzhyyants
b5bc65276c Source Mailchimp: handle unknown error in EmailActivity (#25717) 
* Source Mailchimp: handle unknown error in EmailActivity

* Source Mailchimp: update docs

* Source Mailchimp: refactor

* auto-bump connector version

---------

Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-03 14:18:53 +02:00
btkcodedev
f6bd91018b 🎉Source MailChimp: New Stream Automations (#23290) 
* bump connector version

* Source: MailChimp, New Stream Automations

* fix: Schema error fix

* Fixes formatting.

* fix: Fixed acceptance test schema errors

* fix: Update schema, update dockerlabel

* refactor: Update docs

* refactor: Update DockerVersionTag

* fix: Update EOF and configured catalog

* fix: Schema, Update docs

* fix: Schema on catalog

* fix: update DocketVersionTag

* fix: schema, update docs, bump docker version

* fix: spec, update doc in streams.py

* fix: schema compatability for null

* update connector version to 0.4.0

* Update Dockerfile

* recreate source spec

* restore source spec form main

* rollback source spec from main

* source spec

* run gradle config seed

* fix schema and remove unused code

* fix docs

* bump connector seed file versino

* auto-bump connector version

---------

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: nataly <nataly@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2023-04-24 15:07:18 -03:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254) 
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
 2023-04-18 22:15:11 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Ella Rohm-Ensing
900327726f Add bypasses for extra fields test (#24552) 
* Add bypass for extra fields test

* Add bypass for extra fields test
 2023-03-26 20:16:49 -05:00
Marcos Marx
133077797d Source Mailchimp - Add Reports stream #23464 (#24112) 
* init commit

* update doc

* add reports stream

* Bump mailchimps connector docker image

* fix tests

* update abnormal stream state for report mailchimp

* add mailchimp allowedHosts

* run config seed files

---------

Co-authored-by: Haithem Souala <haithem.souala@melting-point.fr>
Co-authored-by: Haithem Souala <haithem.souala@woopit.fr>
 2023-03-15 16:27:19 -03:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084) 
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2023-02-24 16:13:42 -05:00
Augustin
fbd6dbf091 python-connectors: remove explicit dependencies to connector-acceptance-test build (#23332) 2023-02-22 21:21:38 +01:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594) 
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
 2023-02-08 13:01:16 -08:00
Denys Davydov
ef164d1c5b Source Mailchimp: revert extra logging (#22405) 
* #1058 source Mailchimp: revert extra logging

* #1058 source Mailchimp: upd changelog

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-02-06 23:06:43 +02:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846) 
Rename source acceptance test to connector acceptance test
 2023-02-02 11:38:19 +01:00
Denys Davydov
29cbc0a811 Source Mailchimp: add extra logs (#22228) 
* #1058 source mailchimp: add extra logs

* #1058 source mailchimp: upd changelog

* #1058 source mailchimp: flake fix

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-02-02 11:24:56 +02:00
Ella Rohm-Ensing
148dcee811 Turn off default httpavailabilitystrategy for source-mailchimp (GA) (#22014) 
* Turn off default for source-mailchimp (GA)

* Update streams.py

* Update mailchimp.md

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-30 16:18:27 +00:00
Denys Davydov
86157be611 Source Mailchimp - fix the way request params are built (#20765) 
* 1058 source mailchimp - fix the way request params are built

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-09 22:30:50 +01:00
Mahdi Dibaiee
34e85e3cf1 source-mailchimp: set primary_key for EmailActivity (#19023) 
* source-mailchimp: set primary_key for EmailActivity

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-05 11:47:41 -05:00
Brian Lai
3521802545 Update Python Source connectors to use the latest airbyte-cdk minor version (#18261) 
* update all connectors currently only adhering to patch versions to pull latest minor version

* also add greenhouse

* fix scaffolds
 2022-10-20 16:06:02 -04:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585) 
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
 2022-10-11 11:04:23 -04:00
Evan Tahler
49cb3360de Remove redundant title labels from connector specs (#17544) 
* Remove redundant title labels from connector specs

* Manually update specs

* add env variable

* Remove debugging log
 2022-10-05 12:58:38 -07:00
Augustin
0f618f3503 migrate GA source connectors to per-stream state (#17326) 2022-09-29 14:45:35 +02:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233) 
* Bump year in license short to 2022

* remove protocol from cdk
 2022-05-26 15:00:42 -03:00
Sherif A. Nada
cc46255190 cull base-python and base-python-test (#12596) 2022-05-05 13:48:15 -07:00
Anna Lvova
9bfc21f91d Source Mailchimp: update documentation (#11352) 
* update doc

* bump version

* upd doc

* update doc

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* update doc

* Update docs/integrations/sources/mailchimp.md

Co-authored-by: Andy <andy@airbyte.io>

* auto-bump connector version

Co-authored-by: Andy <andy@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-04-21 14:56:26 +03:00
Richard Lee
d3582296cf Source Mailchimp: correct schema description url property (#12095) 2022-04-19 11:42:14 -03:00
Anna Lvova
686e9edf12 Source Mailchimp: unit tests (#11632) 
* add tests

* up

* up

* add tests

* add test for EmailActivity

* upd test requirements

* upd test requirements

* upd tests

* format

* mock request for ConnectionError

* fix test

* add timeout

* remove default from spec

* remove enum from oneOf

* bump version

* bump version

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-04-12 10:54:02 +03:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763) 
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
 2022-04-11 20:51:37 -07:00
Ari Bajo
656ccd5b74 🐛 Source Mailchimp: fix campaign's stream normalization (#10975) 
* Remove type from x-discriminator node

* Bump version in Dockerfile

* Update changelog

* Update source definition and specs
 2022-03-23 11:15:35 +01:00
Anna Lvova
d09829b21c 🎉 Source Mailchimp: support oauth flow (#7159) 
* add mailchimp oauth support

* add PR

* fix creds

* upd spec

* format

* upd creds

* upd auth for different creds

* rename creds

* rename creds

* change ref in campaigns.json

* upd timeout_seconds

* merge

* add oauth java part

* add java test

* bump version

* update spec

* add anotation

* upd spec

* upd spec

* upd

* upd tests

* format

* upd

* upd

* add state

* add invalid_config

* bump version

* format
 2022-01-17 12:20:58 +02:00
Iryna Grankova
93d2c78ddf Update fields in source-connectors specifications: mailchimp (#9000) 2021-12-23 14:51:41 +02:00
VitaliiMaltsev
faa4d9a3c0 🎉 Source Mailchimp: updated Mailchimp schemas (#7975) 
* create mailchimp schemas

* remove star import

* refactoring

* resolve merge conflicts

* update mailchimp cdk version

* revert changes from JsonToAvroSchemaConverter

* revert changes from JsonToAvroSchemaConverter

* bump Source Mailchimp

Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com>
 2021-12-13 11:36:01 +02:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
Marcos Marx
8c1a5d57c7 Source Mailchimp: correct output schema to handle date-time values (#5481) 
* correct output schema to handle date-time values

* add docs
 2021-08-18 19:13:50 -03:00
LiRen Tu
2906ec287a CI: Add action to check broken doc links (#5254) 
* Add action to check broken doc links

* Ignore localhost

* Update config

* Fix broken links

* Use quiet mode

* Ignore PR link

* Fix more broken links

* Fix more broken links

* Fix more broken links

* Verify pattern

* Fix more broken links

* Separate full and pr check

* Update pattern

* Test invalid link

* Remove invalid link
 2021-08-07 14:28:02 -07:00
Dmytro
abde7c7727 Remove json schema parameter (#4907) 
source-acceptance-test framework not longer required json_schema parameters
   from catalog file. This parameter is verbose, makes reading config file
   complicated and could be misleading when debugging acceptance test issues.

Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
 2021-08-04 18:16:59 +03:00
vovavovavovavova
1e450005d2 Source Mailchimp: fix primary key for email activities (#5137) 
* upgrade primary key for mailchimp email activities

* bump version
 2021-08-04 09:54:21 +03:00
vovavovavovavova
283731ba6c Source Mailchimp: handle records with no no "activity" field in response (#5024) 
* cover if no "activity" field in response

* add new file for sat pass

* one line

* apply cosmetics suggestion

* bump version

* upd doc log
 2021-07-28 17:27:55 +03:00
Eugene Kulak
9de53688e3 SAT: remove deprecated validate_output_from_all_streams from configs (#5010) 
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-07-27 03:02:32 -04:00
Sherif A. Nada
b86ebf63fb Automatically pull latest SAT & build connector image in SAT runner (#4969) 2021-07-26 11:56:19 -07:00
vovavovavovavova
6fdf1ef6ae Mailchimp fix url-base (#4621) 
* minimal change to show acceptance test failure

* exactly fix

* bump version and readme

* upd
 2021-07-08 18:15:54 +03:00
Marcos Marx
77f7096d14 🐛 Source Mailchimp: use datacenter parameter from apikey info (#4285) 
* replace datacenter parameter using apikey info

* change to use dynamic property

* add dynamic property

* bump version

* change config versino
 2021-06-24 21:55:13 -03:00
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973) 
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-09 13:01:45 -07:00
vovavovavovavova
123355bd04 secondary PR: update acceptance config variable (#3930) 2021-06-08 09:09:35 +03:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
vovavovavovavova
b8e050e1ec Source Mailchimp: Get Member activities (#3415) 
* add logic for new stream: email activities (campaign dependent)

* de-duplicate pagination logic

* MailChimp: CDK version (before acceptance test)

* add test files

* applying requested changes + incremental for email reports (requested)

* no-formatted test modification

* save changes for test

* upd tests + build

* update accordinately to requested changes + readme

* restore original READMe & update mailchimps

* upd requested changes

* rm metadata _links from schemas

* rm links from configured_catalog

* fixup unittest

* requested changes from Sherif

* upd: describe why acceptance tests are commented for incremental

* bump version
 2021-05-27 13:37:16 +03:00
Davin Chia
9460cb4422 Update all of Pydantic to 1.6.2 per Dependabot. (#3408) 
* Update all to 1.6.2.

* Publish new airbyte-cdk version.

* Use repr instead of str for exceptions.

* Use rc.

* Edit test.

* Bump for SAT.

* Format.
 2021-05-14 16:44:49 +08:00
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Davin Chia
f660b0a946 Add template generation for Santa aka CDK. (#3034) 
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.

General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.

Checked in and added tests for the generated modules.
 2021-04-25 18:02:33 +08:00
Charles
f445fdb5b2 match styling for spotlessApply and format (#3017) 
* as a java developer I want to be able to run spotlessApply without changing styles in python code
 2021-04-23 09:21:41 -07:00
Sherif A. Nada
2d2d0173d6 Update codegen template READMEs (#2873) 2021-04-13 11:41:52 -07:00
Jared Rhizor
0609011e05 fix connector base versioning (#2726) 
* fix connector base versioning

* only run integration tests in non-bases directories

* upgrade datamodel-code-generator which coincidentally has already fixed this issue with generating List of Lists

* bump forgotten hubspot version

* fix build
 2021-04-02 22:46:06 -07:00