1
0
mirror of synced 2025-12-21 02:51:29 -05:00
Commit Graph

15 Commits

Author SHA1 Message Date
Aaron ("AJ") Steers
83ecbe0fc3 CI: apply pre-commit format fix from #49806 (#49852) 2024-12-18 14:05:43 -08:00
Alexandre Girard
906d7a4ed3 refactor(source-faker): Replace AirbyteLogger with logging.Logger (#38256) 2024-05-20 11:11:14 -07:00
Aaron ("AJ") Steers
80456ef0b6 Source Faker: Enable zero-config execution by making 'count' an optional config param (#36167) 2024-03-21 11:02:07 -07:00
Evan Tahler
94097d0325 source-faker: cursor = updated_at & remove records_per_sync (#27684)
* source-faker: cursor = updated_at & remove records_per_sync

* metadata

* flake
2023-06-24 15:31:53 -04:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Evan Tahler
2022f7d2a8 [faker] decouple stream state (#20492)
* [faker] decouple stream state

* add PR #

* commit Stream instantiate changes

* fixup expected record

* skip backward test for this version too

* Apply suggestions from code review

Co-authored-by: Augustin <augustin@airbyte.io>

* lint

* Create realistic datasets of 10GB, 100GB, and 1TB in size (#20558)

* Faker CSV Streaming utilities

* readme

* don't do a final pipe to jq or you will run out or ram

* doc

* Faker gets 250% faster (#20741)

* Faker is 250% faster

* threads in spec + lint

* pass tests

* revert changes to record helper

* cleanup

* update expected_records

* bump default records-per-slice to 1k

* enforce unique email addresses

* cleanup

* more comments

* `parallelism` and pass tests

* update expected records

* cleanup notes

* update readme

* update expected records

* auto-bump connector version

Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-01-02 18:04:44 -06:00
Evan Tahler
1e160b2858 Faker V0.3 - CDK, consistent naming, determinism, display name, and SAT (#19490)
* better comment to show sync types

* WIP - basics in place

* test cleanup

* emit `AirbyteEstimateTraceMessage`

* assert estimates emitted

* unit tests passing

* format

* flake

* check tests

* add newline

* lint

* check in with new file name

* bump acceptance test yml

* acceptance test progress

* deterministic purchases

* SAT making progress

* better disable

* adjust yaml

* change abnormal_state.json format

* per-stream state

* products for abnormal state

* products have state + incremental

* purchases state = total_user_records

* update docs and name

* disable proper version

* typo

* try disable 0.2.1

* kwargs

* v1.0.0

* lint

* backward_compatibility_tests_config

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-12-01 14:53:38 -08:00
Evan Tahler
05c99c1693 Faker emits AirbyteEstimateTraceMessage (#19197)
* Faker emits `AirbyteEstimateTraceMessage`

* update pr

* fix SAT

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-15 15:58:22 -08:00
Evan Tahler
c81ea485ce Faster Faker (#18021)
* Faster Faker

* unit tests

* tests passing

* update pr link

* guard against small ranges

* Fixup product timezones

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-15 13:31:12 -07:00
Evan Tahler
dcfcb75d0f AirbyteLogMessage.stack_trace for logging messages with related (non-fatal) errors (#16479)
* Test log message from faker

* AirbyteLogMessage gains stack_trace

* Fixup spacing

* bump python protocol

* fixup additionalProperties in faker spec

* bump faker version

* update docs

* use lineSeparator vs \r\n

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 16:01:57 -07:00
Evan Tahler
edb74ec282 Faker Source emits times in proper ISO format (#13695)
* Faker Source emits times in ISO format

* fix product times

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-10 17:21:26 -07:00
Evan Tahler
40d1dc1421 Faker source gains purchases and products (#13248)
* Faker source gains purchases and products

* ranges start at 1

* fix nits

* lint

* user_ids start at 1

* v0.1.3

* fix IDs in test output recods

* auto-bump connector version

* lint

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-27 16:14:59 -07:00
Evan Tahler
e3cb566f79 Bump faker version to test M1 Connector builds (#13235)
* Bump faker version to test M1 Connector builds

* add PR link to changelog

* Remove birhtday, it  doesn't obey seed

* bump readme

* more birthday to remove

* remove image specificity in publish-command

* Set AMI for testing.

* test with including buildx emulators

* auto-bump connector version

* add TODO comment

Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-27 10:36:05 -07:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233)
* Bump year in license short to 2022

* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Evan Tahler
8293ce3fbe 🎉 New Source: Faker (#11738)
* Faker WIP

* Update catalog to handle dates better

* Adding unit tests for faker source

* WIP  - tests mostly passing

* add docs

* bump python version and fix unit tests

* test array types

* remove comment

* better python map

* update `moduleDirectory`

* simplify intiilization\

* use `ConfiguredAirbyteCatalog` in test rather than custom dict class

* Tests passing by using deterministic time

* Bump birthdays

* Update airbyte-integrations/connectors/source-faker/integration_tests/acceptance.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* remove bootstrap and stronger types

* better incremental support

* fixup un-used imports

* bump to test codecov

* Add connector to medatata files

Co-authored-by: George Claireaux <george@claireaux.co.uk>
2022-04-14 15:21:30 -07:00