1
0
mirror of synced 2025-12-21 19:11:14 -05:00
Commit Graph

127 Commits

Author SHA1 Message Date
Alexandre Girard
677fa9756d Small fixes to support python3.8 (#23653)
* Use ** instead of |

* Using typing type

* Using typing type

* Using typing type

* Revert "Merge branch 'master' into alex/support_3.8"

This reverts commit 5c7581518e, reversing
changes made to 5cb47f8c70.

* Revert "Merge branch 'master' into alex/support_3.8"

This reverts commit 5cb47f8c70, reversing
changes made to 4058fce754.

* Revert "Merge branch 'master' into alex/support_3.8"

This reverts commit e1d109905a, reversing
changes made to da881ef0d7.

* reset changes

* undo borked publish

* downgrade bumpversion.cfg and Dockerfile too

* explicitely support >=3.8

* update readme
2023-03-01 19:14:27 -08:00
Serhii Lazebnyi
3d8f81a3e4 Connector Health: Fixed SAT for Amazon Ads, Faker, Hubspot, Instagram, Pinterest, Sentry, Zendesk Talk (#23600)
* Fixed SAT for some connecotrs

* Fix unittest pinterest

* add allowedHosts

* Updated unittest for instagram

* Fix expected records instagram
2023-03-01 10:09:19 +01:00
Sherif A. Nada
a595bb4ddd Source Faker: export json files (#23468) 2023-02-28 09:55:31 -08:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084)
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-02-24 16:13:42 -05:00
Evan Tahler
25542493ed source-faker v2.0.3 to test publication (#23259)
* `source-faker` v2.0.2 to test publication

* add PR number to changelog

* it's 2.0.3 now!

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-21 12:02:29 -08:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
darynaishchenko
93186e7bc3 updated expected records for stream purchases (#22404) 2023-02-07 15:44:02 +02:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
Evan Tahler
e4f4412873 source-faker to Beta + Fix AllowedHosts Checks (#22117)
* beta faker + no allowed hosts

* V2.0.1

* fixup python checks

* add back allowedHosts

* simplify

* simplify releaseStage

* bump expected records

* More mock logger methods

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-01-31 06:13:19 +00:00
Evan Tahler
2022f7d2a8 [faker] decouple stream state (#20492)
* [faker] decouple stream state

* add PR #

* commit Stream instantiate changes

* fixup expected record

* skip backward test for this version too

* Apply suggestions from code review

Co-authored-by: Augustin <augustin@airbyte.io>

* lint

* Create realistic datasets of 10GB, 100GB, and 1TB in size (#20558)

* Faker CSV Streaming utilities

* readme

* don't do a final pipe to jq or you will run out or ram

* doc

* Faker gets 250% faster (#20741)

* Faker is 250% faster

* threads in spec + lint

* pass tests

* revert changes to record helper

* cleanup

* update expected_records

* bump default records-per-slice to 1k

* enforce unique email addresses

* cleanup

* more comments

* `parallelism` and pass tests

* update expected records

* cleanup notes

* update readme

* update expected records

* auto-bump connector version

Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-01-02 18:04:44 -06:00
Yevhen Sukhomud
2a5d0344e7 Fix formatting (#20570)
* Fix formatting

* try fixing quote escapes?

* unignore sql files

Co-authored-by: Edward Gao <edward.gao@airbyte.io>
2022-12-16 13:43:42 -08:00
Evan Tahler
92ad0fdc07 Destination doc and warning updates (#20110)
* Doc updates

* Bigquery Denormalized

* bump faker for change

* ignore missing strict-encrypt connectors from ci check

* Apply suggestions from code review

Co-authored-by: Augustin <augustin@airbyte.io>

* Fix MD titles

Co-authored-by: Augustin <augustin@airbyte.io>
2022-12-06 12:10:13 -08:00
Evan Tahler
1e160b2858 Faker V0.3 - CDK, consistent naming, determinism, display name, and SAT (#19490)
* better comment to show sync types

* WIP - basics in place

* test cleanup

* emit `AirbyteEstimateTraceMessage`

* assert estimates emitted

* unit tests passing

* format

* flake

* check tests

* add newline

* lint

* check in with new file name

* bump acceptance test yml

* acceptance test progress

* deterministic purchases

* SAT making progress

* better disable

* adjust yaml

* change abnormal_state.json format

* per-stream state

* products for abnormal state

* products have state + incremental

* purchases state = total_user_records

* update docs and name

* disable proper version

* typo

* try disable 0.2.1

* kwargs

* v1.0.0

* lint

* backward_compatibility_tests_config

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-12-01 14:53:38 -08:00
Evan Tahler
05c99c1693 Faker emits AirbyteEstimateTraceMessage (#19197)
* Faker emits `AirbyteEstimateTraceMessage`

* update pr

* fix SAT

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-15 15:58:22 -08:00
Evan Tahler
c81ea485ce Faster Faker (#18021)
* Faster Faker

* unit tests

* tests passing

* update pr link

* guard against small ranges

* Fixup product timezones

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-15 13:31:12 -07:00
Serhii Chvaliuk
6a8b053fb5 Use locally-installed qemu rather than docker-installed version (2) (#17889)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-12 17:46:24 +03:00
Evan Tahler
2d049f18cf Use locally-installed qemu rather than docker-installed version (#17848)
* Use locally-installed qemu rather than docker-installed version

* Bump faker for test

* Update docs/integrations/sources/faker.md

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>

* auto-bump connector version [ci skip]

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-11 10:32:57 -07:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Evan Tahler
dcfcb75d0f AirbyteLogMessage.stack_trace for logging messages with related (non-fatal) errors (#16479)
* Test log message from faker

* AirbyteLogMessage gains stack_trace

* Fixup spacing

* bump python protocol

* fixup additionalProperties in faker spec

* bump faker version

* update docs

* use lineSeparator vs \r\n

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 16:01:57 -07:00
Evan Tahler
edb74ec282 Faker Source emits times in proper ISO format (#13695)
* Faker Source emits times in ISO format

* fix product times

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-10 17:21:26 -07:00
Marcos Marx
adf4b6df25 run gradlew add headers, correct formatting (#13460) 2022-06-03 16:17:03 -03:00
Evan Tahler
0562e9558c Bump faker to test publication flow (#13298)
* Bump faker to test publication flow

* Add PR link

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-27 19:19:44 -07:00
Evan Tahler
40d1dc1421 Faker source gains purchases and products (#13248)
* Faker source gains purchases and products

* ranges start at 1

* fix nits

* lint

* user_ids start at 1

* v0.1.3

* fix IDs in test output recods

* auto-bump connector version

* lint

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-27 16:14:59 -07:00
Evan Tahler
e3cb566f79 Bump faker version to test M1 Connector builds (#13235)
* Bump faker version to test M1 Connector builds

* add PR link to changelog

* Remove birhtday, it  doesn't obey seed

* bump readme

* more birthday to remove

* remove image specificity in publish-command

* Set AMI for testing.

* test with including buildx emulators

* auto-bump connector version

* add TODO comment

Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-27 10:36:05 -07:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233)
* Bump year in license short to 2022

* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Marcos Marx
3113ef44a3 Bump Google Ads version and run format (#12087)
* combine all

working locally

missing fields in click view and keyqord_view

add date to campaigns

add fields to click_view

test

* configured_catalog campaigns

* rem keywordview

* conflicts

* string type

* remove network type

* change ad group id to integer

* fix tests

* bump connector version

* bump connector config doc vesion

* run format

Co-authored-by: Konrad <konrad@PB-Mac8.local>
Co-authored-by: schlattk <konrad@pensionbee.com>
2022-04-18 21:15:29 -03:00
Evan Tahler
8293ce3fbe 🎉 New Source: Faker (#11738)
* Faker WIP

* Update catalog to handle dates better

* Adding unit tests for faker source

* WIP  - tests mostly passing

* add docs

* bump python version and fix unit tests

* test array types

* remove comment

* better python map

* update `moduleDirectory`

* simplify intiilization\

* use `ConfiguredAirbyteCatalog` in test rather than custom dict class

* Tests passing by using deterministic time

* Bump birthdays

* Update airbyte-integrations/connectors/source-faker/integration_tests/acceptance.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* remove bootstrap and stronger types

* better incremental support

* fixup un-used imports

* bump to test codecov

* Add connector to medatata files

Co-authored-by: George Claireaux <george@claireaux.co.uk>
2022-04-14 15:21:30 -07:00