jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-21 19:11:14 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

82 Commits

Author SHA1 Message Date
Daryna Ishchenko
30d7b0231e feat(source-file): update to airbyte cdk version 7 (#68588) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-10-27 12:40:46 +02:00
Morgan
c8d22f9e9c fix(source-file): make client _is_zip method case-insensitive (#55205) 2025-03-06 16:30:37 +02:00
Aaron ("AJ") Steers
83ecbe0fc3 CI: apply pre-commit format fix from #49806 (#49852) 2024-12-18 14:05:43 -08:00
Alfredo Garcia
af77e73cef Adding reader options for Excel: header & skiprows (#48804) 2024-12-10 12:32:23 -06:00
Carlos Alvarenga
bc36dd2052 Source file: Fix error on zipped files (#39909) 
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2024-08-01 16:10:24 -03:00
Natik Gadzhi
30643ae71f chore: replace AirbyteLogger with logging.Logger in connectors (#40215) 
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
 2024-06-27 18:02:01 +02:00
Daryna Ishchenko
395effcf1d 🐛 Source File: add engine fastparquet to parquet reader options (#37799) 2024-05-07 17:23:40 +03:00
Anatolii Yatsuk
df17c85858 🏥 Source Connectors: Pin airbyte-cdk version to ^0 (#36267) 2024-03-20 21:46:57 -04:00
Julien COUTAND
a886ace52f ✨ Source File: support ZIP file (#32354) 
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
 2024-02-15 18:33:19 +01:00
mjgatz
f6b4436206 ✨ Source File: add fixed width file format support (#34678) 
Co-authored-by: mgreene <michael.greene@gravie.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
 2024-02-13 19:58:35 +01:00
Joe Reuter
f29234a2db airbyte-lib: Refactor connectors (#34552) 2024-01-30 03:22:13 -06:00
Augustin
0b33caecda Revert "[skip ci] formatting: add missing license headers (#33250)" (#33289) 2023-12-11 11:38:37 +01:00
Augustin
60c1cc01ad [skip ci] formatting: add missing license headers (#33250) 2023-12-11 10:15:18 +01:00
Pedro S. Lopez
d4ef599838 ✨ source-file: prevent local file usage on cloud deployment mode (#30984) 
Co-authored-by: pedroslopez <pedroslopez@users.noreply.github.com>
 2023-10-16 13:00:30 -07:00
Denys Davydov
809d86bbb6 🐛 Source File: handle parse errors when running discovery (#30579) 2023-09-25 03:07:58 -05:00
Serhii Lazebnyi
0bdd8445cf Remove debug mode for pytest config (#27157) 
* Deleted pytest log level

* Force log level for smart open to error

* Bump version

* Bump version

* Delete extra space
 2023-06-08 19:17:15 +02:00
Augustin
d8820d6e80 Fix python connectors formatting errors with black / isort (#26951) 
* format source-tiktok-marketing

* format source-twilio

* format source-instgram

* format source-amazon-ads

* format source-google-search-console

* format source-stripe

* format source-google-analytics-data-api

* format source-facebook-marketing

* format source-zendesk-support

* format source-file
 2023-06-02 09:26:04 -05:00
Daryna Ishchenko
2a1d7f37e2 🐛Source File: added parser error handling (#26275) 
* added parser error handling

* updated versions, added changelog

* Automated Change

---------

Co-authored-by: darynaishchenko <darynaishchenko@users.noreply.github.com>
 2023-05-25 19:02:57 +03:00
Serhii Chvaliuk
50c7cb438d Source File: add retry on SSHException('Error reading SSH protocol banner') - 2 (#26210) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-05-18 00:42:33 +03:00
Serhii Chvaliuk
63887f7fe0 Source File: add retry on SSHException('Error reading SSH protocol banner') (#26115) 
Signed-off-by: Serhii Chvaliuk <grubberr@gmail.com>
 2023-05-16 20:03:37 +03:00
Artem Inzhyyants
fc6165ee1c Source File: Check if reader options is JSON object (#26117) 
* Source File: check if reader options is JSON object

* Source File: bump version; update docs

* Source File: update error message

* Source File: format

* Source File: test fix

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-16 12:03:36 +00:00
Evan Tahler
3679dcd9d0 🐛 Fix date-time for airbyte types (#25965) 
* Fix airbyte types

* update firebolt test fixture

* Bump changelog PR ids

* update metadata files

* fix firebolt test

* fix file-source for date-time overload

* fix source-file changelog

* it's a date, not date-time

* another date

* auto-bump connector version

* manually bump definitions

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-12 01:15:30 +00:00
Evan Tahler
63d0402111 🐛 source-file GCP token is a secret (#25819) 
* source-file GCP token is a secret

* docs and changelog

* bump metadata

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-04 20:49:28 -07:00
Artem Inzhyyants
814b4f7f59 Source File: Handle network errors (#25641) 
* Source File: Handle network errors

* Source File: Add unit test

* Source File: bump version + docs update

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-01 15:52:05 +02:00
Artem Inzhyyants
a3fa2b15dd Source File: Fix OOM; read Excel files in chunks (#25575) 
* Source File: Use openpyxl to read excel files in chunks

* Source File: bump version

* Source File: update docs

* Source File Secure: bump version

* Source File Secure: add docstring

* Source File: use column names from reader options

* Source File: refactor; use pandas for non xlsx formats

* Source File: reformat

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-01 14:45:36 +02:00
Serhii Lazebnyi
3bfcddac2d 🎉Source File: Add datatime format parsing support for csv files (#25445) 
* Add datatime format parsing support

* Updated subject in docs

* Bump file secure

* PR number bumped

* Fix typo

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-04-25 11:57:10 +02:00
Ali Hassan
d7a3b2a04f 🐛 Source File: fix column data types for numerical values (#23759) 
* Fix column data types for numerical values

* Update unit tests

* Update Dockerfile version

* update dockerfile version for source-file-secure

* Update source-file in seed file

* Update doc

* Update source_specs.yaml

---------

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2023-04-13 18:37:39 -03:00
Ujjwal Tyagi
812453fd8e Source File: fix example in Reader Options in specification (#24525) 
* Update file.md

* Update spec.json

* Update spec.json

* update versions and changelog

* bump version number

* update changelog and rebase

* auto-bump connector version

* fix integration test spec.json file

* remove unused files

* restore acceptance test

* convert all examples to string

* convert example to string

* regenerate seed

* change to airbyte.com

* Update source_specs.yaml

* Delete source_definitions.yaml

* regenerate source spec file

* update spec for file-secure

---------

Co-authored-by: Mal Hancock <mallory@archangelic.space>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2023-04-10 17:20:56 -03:00
Daryna Ishchenko
8bb84f2912 Source File: decrease give up rate, added better error message (#24588) 
* added fields description

* removed traceback from user messages

* bump version, added changelog

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-03-30 18:20:10 +03:00
Denys Davydov
9fe07dcf5d Source file: do not read whole file on check and discover (#24278) 
* #1681 source file: do not read whole file on check and discover

* #1681 source file: upd changelog

* #1681 source file: add allowed hosts

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-03-22 10:06:02 +02:00
Daryna Ishchenko
5f0d9f0f6f Source Files: decrease give up rate (#23723) 
* updated spec, docs, error messages

* added change log

* updated error message in unit tests

* added allowedHosts

* refactored error message and url descriprion

* updated example for url field in spec

* bump source-file verion for source-file-secure

* auto-bump connector version

* updated spec.json for source-file-secure

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-03-10 11:13:05 +02:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594) 
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
 2023-02-08 13:01:16 -08:00
Artem Inzhyyants
10d7bea363 Source File: Fix special characters bug (#21012) 
* Source File: Fix special characters bug

* Source File: bump version; update docs

* Source File: update SSH credentials in unit tests

* Source File: bump version (secure)

* Source File: update resources manually
 2023-01-04 20:58:42 +01:00
Artem Inzhyyants
54377e4eec Source File: increase timeout OC 1197 (#20740) 
* Source File: increase timeout OC 1197

* Source File: docs update; bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-21 14:28:40 -06:00
resoliwan
88c9844265 Source file: fix pandas read_csv header none issue. (#18587) 
* Source file: fix pandas read_csv header none issue.

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2022-12-02 19:43:03 -03:00
Serhii Chvaliuk
fce9184d86 Source File: using AirbyteConnectionStatus for "check" command instead of Exception (#19222) 2022-11-10 14:22:52 +01:00
Baz
91a10dae57 🐛 Source File: add retry logic for Connection reset by peer - 104 error (#18428) 2022-10-27 16:05:07 +03:00
Artem Inzhyyants
0daeb8dd7a 🐛 Source File: fix check method (#18481) 
* 🐛 Source File: fix check method

* 🐛 Source File: bump version

* 🐛 Source File: test for wrong format

* 🐛 Source File: docs update

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-10-26 18:34:10 +02:00
Serhii Chvaliuk
5b81897b96 Source File: add validate_and_transform of config (#18116) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-10-19 22:10:51 +03:00
Marcos Marx
70dd9a85cf 🎉 Source Zendesk Chat: engagements data fix infinity looping + gradlew format (#18121) 
* fix infinity looping in chats

* added more meaninful to variable name

* bump docker version

* auto-bump connector version

* run format

Co-authored-by: Roberto Bonnet <robertojuarezwp@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-10-18 15:14:02 -03:00
Arsen Losenko
a28836e0c8 Source File: Handle UnicodeDecodeError in case of incorrect filetype (#17994) 
* Handle UnicodeDecodeError in case of incorrect filetype

* Update changelog and Dockerfile

* Update changelog with PR number

* Move try/except close to the end of method

* Add test and test files
 2022-10-17 12:20:03 +03:00
Eugene
33dfd5d6be Fixed "Ensure no file change" build issue. Only checkstyle changes, no logic chnages (#17885) 2022-10-12 15:06:28 +03:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585) 
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
 2022-10-11 11:04:23 -04:00
Baz
8c4b61c13b 🐛 Source File: add data presence validation for HTTPS for check_connection (#17504) 2022-10-03 18:44:56 +03:00
Denys Davydov
d780141f89 Source File: fix schema generation for json files containing an array (#16772) 
* #547 oncall Source File: fix schema generation for json files containing arrays

* source file: upda changelog

* #547 oncall: source file - upgrade source-file-secure

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-09-19 12:18:05 +03:00
Arsen Losenko
35257a0ec1 Source File: specify pyxlsb as engine for Excel file parsing (#15568) 
* Specify engine for excel files

* Add test for the chagne related to pyxlsb

* Update changelog and bump connector version

* Update PR link and number

* Update spec to include 'excel_binary' option, formatted files

* Update spec for source-file-secure to match changes made in source-file spec
 2022-08-26 17:34:16 +03:00
Denys Davydov
81bfb5c350 Source file: fix csv schema discovery (#15870) 
* #174 source file: fix csv schema discovery

* #174 source file: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-08-23 12:38:08 +03:00
Serhii Chvaliuk
e9fa2c4476 🎉 Source File: Convert 'nan' to 'null' (#15768) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-08-20 09:53:36 +03:00
Serhii Chvaliuk
f4adef7927 🎉 Source File: cache binary stream to file for discover (#15698) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-08-17 15:19:30 +03:00
Serhii Chvaliuk
5eef51ae15 🎉 Source File: cache binary stream to file (#15501) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-08-11 11:56:53 +03:00