* Source TPI Central: add a check for HTTP URLs in config
* Update changelog
* Fix tests
* Add missing attributes to fix the test that wasn't working
* Add HTTPS URL check to SAT tests
* Remove previously added changes
* added new source for ip2whois
* updated documentation link
* fixes for review comments
* format connector
* run format
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add new source: The Guardian API
* Add documentation
* Fix custom paginator, it now stops without throwing an error
* Update the-guardian-api.md with PR number and link
* Remove catalog file, add titles to all properties in spec.yaml
* Add incremental sync, change parameter names
* format
* remove order from spec
* add guardian to source def
* auto-bump connector version
Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
* switch to light theme for yaml editor
* adjust colors again
* rename colors
* fix config menu theme
* use gradient again for background
* undo change to comment out react query dev tools
it seems github APIs want us to use longer shas in some cases where a
7 character short sha is still entirely valid
So just set the minimum short sha length to 10
* convex source
* fixed unit tests
* use the smaller test instance
* fix integration tests and pagination
* fix tableName casing in api call
* fix logo
* more shapes -- thinking about moving this to an internal api
* use new /json_schemas api
* use new APIs
* remove unused start_date
* doc skeleton
* fix deletes with incremental sync
* fix test of json_schema
* fix expected records
* merge
* version and docs
* fix test
* fix docs url
* mention that you need to ask convex to enable it for your account
* docs
* two stage pagination for initial sync. some unit tests not working
* fixed all the unit tests, but we need to push backends before acceptance tests work
* fix snapshot pagination
* fix integration tests
* some checkboxes from the PR description
* Update airbyte-integrations/connectors/source-convex/source_convex/source.py
Co-authored-by: Sujay Jayakar <sujayakar314+github@gmail.com>
* Update airbyte-integrations/connectors/source-convex/source_convex/source.py
Co-authored-by: Sujay Jayakar <sujayakar314+github@gmail.com>
* stop storing the delta_has_more in state
* fix mypy
* use UTC timestamps
* rename instance_name to deployment_name
* include data format request param
* remove dev docker image
* fix unit test
* documentation
* .
* code review
* documentation and change deployment_name to deployment_url
* fix pip dependency
* regenerate the spec file
Co-authored-by: Sujay Jayakar <sujayakar314+github@gmail.com>
* change splitter to be single bar, and rename props
* add logs viewer to testing panel
* make log display height 100%
* remove comment
* clean up some styling, and add record count to tab title
* cleanup + only render paginator and slice selector when necessary
* move selected slice/page state into context and fix bug with state between streams
* fix tab keys
* pull LogsDisplay out into its own component to simplify ResultDisplay
* simplify ResultDisplay prop
* add yaml validation and error marking logic to yaml editor
* generify YamlEditor
* Revert "generify YamlEditor"
This reverts commit decf6d6957.
* disable download and test buttons while yaml is invalid
* remove console log and add comments
* change splitter to be single bar, and rename props
* add logs viewer to testing panel
* make log display height 100%
* remove comment
* clean up some styling, and add record count to tab title
* cleanup + only render paginator and slice selector when necessary
* move selected slice/page state into context and fix bug with state between streams
* fix tab keys
* pull LogsDisplay out into its own component to simplify ResultDisplay
* simplify ResultDisplay prop
Co-authored-by: Tim Roes <tim@airbyte.io>
* Extract Operation API
* Extract scheduler API
* Format
* extract source api
* Extract source definition api
* Add path
* Extract State API
* extract webbackend api
* extract webbackend api
* extract workspace api
* Extract source definition specification api
* Add improvements to "Building a connector the hard way"
* add log_error() formatting to pass SAT tests
* Update to new version of acceptance-test-config.yml
* Edit tutorial directory to match tutorial
* Change permissions on acceptance-test-docker.sh
This reverts commit 40b2d98053.
* Adds TLS check to mongodb destination and migrates util constants
* Migrates MongodbSourceUitls to general purprose Utils file
* Updates expected_spec.json to include SSH tunnel
* Bumps connector version and removes connector from being hidden in UI
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Extract Operation API
* Extract scheduler API
* Format
* extract source api
* Extract source definition api
* Add path
* Extract State API
* extract webbackend api
* extract webbackend api
* extract workspace api
* Extract Operation API
* Extract scheduler API
* Format
* extract source api
* Extract source definition api
* Add path
* Extract State API
* extract webbackend api
* extract webbackend api
* Fix build
* 🪟🔧 Refactor FrequentlyUsedDestinations
We need to reuse this component in an experiment to suggest sources.
This PR covers the refactor so it's not destination-dependant and can be
used later on.
Demo: https://www.loom.com/share/c207ab2a53c146bd8e4fe57a57660a6b
* PR comments
* solve conflicts
* solve conflict in json schema
* bump to version 0.1.8 for the changes of this pr
* change ad account id in the schemas
* query to include data plane attributes (#18531)
* query to include data plane attributes
* rename functions
* fix java build
* more renaming fix
* Fix unit tests in source relational db (#18789)
* Fix unit tests
* Add extra test case for record count > 1
* Store record count in variable
* ci: use custom test-reporter action to upload job results (#18004)
* ci: use custom action to upload job results
* Correct coinmarket spec (#18790)
* correct coinmarket spec
* remove duplicate support normalization from source spec
* rollback coinmarketcap version in source def seed
* update connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197)
* Remove lines that always add Postgres to list of destinations
* Parameterize all tests in test_ephemeral by destination
* 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734)
* 🐛 Source Facebook Marketing: reduce request limit after specific error
* 🐛 Source Facebook Marketing: bump version; update docs
* 🐛 Source Facebook Marketing: add test
* 🐛 Source Facebook Marketing: increase timeout
* [charts/airbyte-cron] Cleanup env vars (#18787)
* [charts/airbyte-cron] Cleanup env vars
* Remove unused env var
* Use equalsIgnoreCase (#18810)
* Bump helm chart version reference to 0.40.40 (#18815)
Co-authored-by: perangel <perangel@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>
* 🐛Destination Google Sheets: Fix empty headers list (#18729)
* Fix empty headers list
* Updated PR number
* Bumped version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* 🐛Source Exchange Rates: Fix handling error during check connection (#18726)
* Fix handling error during check connection
* Updated PR number
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add normalization changelog and bump normalization version in platform (#18813)
* Remove ConfigPersistence usage from SecretsMigrator (#18747)
* remove config persistence from seeding logic (#18749)
* Remove the bulk actions from ConfigPersistence (#18800)
* hide ConfigPersistence inside ConfigRepository to discourage use (#18803)
* ci: add job and run id to test reports (#18832)
* Bump Airbyte version from 0.40.17 to 0.40.18 (#18827)
Co-authored-by: grishick <grishick@users.noreply.github.com>
* 🪟🔧 Remove styled components (round 1) (#18766)
* refactor EditorHeader (untested)
* refactor BaseClearView
* delete unused Subtitle
* refactor ConfirmationModal
* refactor Arrow
* refactor BulkHeader
* refactor CatalogTreeSearch
* refactor StreamFieldTable
* refactor StreamHeader
* refactor ConnectorIcon
* refactor TreeRowWrapper
* refactor DeleteBlock
* refactor EmptyResourceBlock
* revert unintended element change
* fixed acceptance tests (#18699)
* 🪟🔧 Reactor Breadcrumbs component to use anchors (#18764)
* refactor breadcrumbs to use actual links
* PR comments on styles
* increase test timeout for some webapp tests to prevent flakes (#18807)
* Remove "Filters and Segments" from Google Analytics v4 (#18508)
Filters and Segments info was incorrectly added to the Google Analytics v4 connector instead of the Google Analytics (Universal Analytics) Connector.
* Add notes about EU OAUth (#18835)
EU OAuth is not fully tested so adding a note to account for that.
* 🪟🐛 Fix: visual regression in ConnectorIcon (#18849)
* fix visual regression
* remove unused prop
* Add links to demo page (#18828)
* mysql-source:fix tinyint unsigned handling (#18619)
* mysql-source:fix tinyint unsigned handling
* update doc
* format
* upgrade version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* 🪟🎉 Allow environment specific sections in docs (#18829)
* Allow environment specific sections in docs
* Change syntax to lower case
* ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables (#18809)
* ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables
* ci: replace github set-ouput with new syntax in ./tools/bin/
* Add connection ID to span (#18852)
* edited connector docs (#18855)
* 🪟🔧 Upgrade husky to 8.0.1 (#18719)
* Upgrade Husky
* Upgrade Husky
* Upgrade Husky
* Upgrade Husky
* Upgrade Husky
* ci: replace GITHUB_OUTPUT with GITHUB_ENV for multiline variables (#18857)
* Avoid NPE when adding connection ID to trace (#18856)
* Filter exit errors by operation name (#18850)
* add label
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
Co-authored-by: Liren Tu <tuliren@gmail.com>
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Greg Solovyev <grishick@users.noreply.github.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: perangel <perangel@gmail.com>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: perangel <perangel@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Charles <charles@airbyte.io>
Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>
Co-authored-by: darynaishchenko <80129833+darynaishchenko@users.noreply.github.com>
Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com>
Co-authored-by: Tyler B <104733644+TBernstein4@users.noreply.github.com>
Co-authored-by: Yowan Ramchoreeter <26179814+YowanR@users.noreply.github.com>
Co-authored-by: Tim Roes <tim@airbyte.io>
Co-authored-by: Subodh Kant Chaturvedi <subodh1810@gmail.com>
Co-authored-by: Volodymyr Pochtar <v.pochtar@gmail.com>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
* method yielding airbytemessage
* move to Stream
* update abstract source
* reset
* missing file
* add test docker image
* script to run acceptance tests with local cdk
* Update conftest to use a different image
* extract to method
* dont use a different image tag
* Always install local cdk
* break the cdk
* get path from current working directory
* or
* ignore unit test
* debug log
* Revert "AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689)"
This reverts commit bf06decf73.
* build from the top
* Update source-acceptance-test
* fix
* copy setup
* some work on the gradle plugin
* reset to master
* delete unused file
* delete unused file
* reset to master
* optional argument
* delete dead code
* use latest cdk with sendgrid
* fix sendgrid dockerfile
* break the cdk
* use local file
* Revert "break the cdk"
This reverts commit 600c195541.
* dont raise an exception
* reset to master
* unit tests
* missing test
* more unit tests
* newline
* reset to master
* remove files
* reset
* Update abstract source
* remove method from stream
* convert to airbytemessage
* unittests
* Update
* unit test
* remove debug logs
* Revert "remove debug logs"
This reverts commit a1a139ef37.
* Revert "Revert "remove debug logs""
This reverts commit b1d62cdb60.
* Revert "reset to master"
This reverts commit 3fa6a004c1.
* Revert "Revert "reset to master""
This reverts commit 5dac7c2804.
* reset to master
* reset to master
* test
* Revert "test"
This reverts commit 2f91b803b0.
* test
* Revert "test"
This reverts commit 62d95ebbb5.
* test
* Revert "test"
This reverts commit 27150ba341.
* format
* Generate source
* Add candidates stream
* Fix docs
* Add offers stream
* Fix schema
* Use a single primary_key field
* Add departments stream
* Add sample state
* Fix company ID number
* Update PR number
* Update configured catalog
* Fix schema
* Add newlines
* Add title to properties in spec.yaml
* run format and add recruitee to source def
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add support to clear unsupported protocol flag to StandardSyncPersistence
* Fix internal records scope
* Plug clearUnsupportedProtocolVersionFlag on connector update
* Update UPDATED_AT when clearing protocol flags
Logic in this class is going to have to change as part of two big upcoming projects:
- column selection
- progress bars
To prepare for this, I've gone ahead and refactored the run method for readability. This is a monster function. The current function is too long and contains several operational abstractions, increasing unnecessary complexity. This is the core of what we do, so it's important to ensure this code is extremely understandable.
Ultimately we want to probably want to break the run method up into two or more separate classes - one that deals with replication and one that deals with outputs - for better testing, readability and isolation. This sets the stage for that.
I have intentionally NOT removed or touched any logic, nor have I put thought into consolidating the function signatures to preserve as much of the pre-existing logic and keep the changeset small and reviewable.
This changeset only renames and moves code around.
* draft for fastbill
* adding documentation
* fixing common errors
* make requested changes according to review
* requested changes according to review
* compressing code even more
* format
* run format
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
* Extract Operation API
* Extract scheduler API
* Format
* extract source api
* Extract source definition api
* Add path
* Extract State API
* Add missing binder
* fix type
* Adding transactions for klarna source integration
* Fixing tests
* Adding payout stream
* Finalizing connector
* Adding new lines af EOF
* Applying code review suggestions. Adding doc for klarna. Adding title in spec.yaml
* run format
* format files
* add to source def
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* allow for spec to be defined in the source.yaml manifest instead of an external file
* make spec a component within the language to get schema validation and rework the code for better testing
* fix formatting and extra method
* pr feedback and add some more test
* pr feedback
* bump airbyte-cdk version
* bump version
* gradle format
* remove from manifest spec
