jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-21 11:01:41 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,485 Commits 5,109 Branches 518 Tags
bmoric/schema-mapping-postgres
Commit Graph

104 Commits

Author SHA1 Message Date
Gireesh Sreepathi
6b4f215c6d Destination Postgres: DV2 Beta (#34372) 
## What
* Adding postgres DV2 beta launch option
 2024-01-22 09:07:13 -08:00
Edward Gao
d610ad1f48 Destination postgres (dv2): add indexes to raw table (#34236) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
 2024-01-18 16:21:24 -08:00
Edward Gao
006338257c Destination postgres: DV2 beta implementation (#34177) 2024-01-17 10:32:21 -08:00
Gireesh Sreepathi
d29cb2d41e Publish CDK and fix postgres to use cdk (#34135) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
 2024-01-10 20:48:14 -04:00
Gireesh Sreepathi
a0ece1231e destination-postgres: Add tunnel heartbeats and keepalive (#33875) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
 2024-01-10 15:42:32 -08:00
Edward Gao
216d2b2e70 Destination postgres: enable in-connector normalization (#33873) 2024-01-04 14:15:33 -04:00
Gireesh Sreepathi
af4f5ae570 Destination Postgres - fix tests (#33507) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
 2023-12-18 17:51:16 -08:00
Evan Tahler
9210547af5 [Docs] No Deduped + Hostory, Append + Deduped is the future! (#29114) 
* [Docs] No `Deduped + Hostory`, `Append + Deduped` is the future!

* fix links
 2023-08-08 15:07:49 -07:00
Evan Tahler
f455c1288d Java DB Destination connector licenses to Elv2 (#27781) 
* Java DB Destination connector licenses to Elv2

* PR id for docs

* fix redshift tagging
 2023-06-29 12:26:24 -05:00
Evan Tahler
4c26efb6b8 destination-postgres docs warning for large data (#25817) 
* destination-postgres warning for large data

* :::caution
 2023-05-05 09:15:29 -07:00
Ryan Fu
73900fad3e JDBC connections to support checkpointing (#24604) 
* Shifted logic within JDBC connections to support checkpointing

* Bumped version numbers and updated changelog
 2023-04-05 18:32:28 +00:00
Sophia Wiley
5512befeb1 Docs: updated links from .io to .com (#23652) 
* updated links

* edited contributors link

* deleted line about CDK in docs
 2023-03-06 17:27:55 +01:00
Werner Fouché
05390991ec Update postgres.md (#18432) 2022-10-25 15:09:53 -03:00
Tyler B
40896d00c0 Re-direct Postgres guide to active page, current link is 404'd (#18103) 
The current  "Refer to this guide..." link goes to a 404 page on postgres. The new link i put in is the equivalent of the old site. It can be compared using the wayback machine: https://web.archive.org/web/20220121223117/https://jdbc.postgresql.org/documentation/head/connect.html
 2022-10-18 10:47:39 -04:00
Rodi Reich Zilberman
248da78ba7 Make error more descriptive in case of no encryption state (#17299) 
* Make error more descriptive in case of no encryption state

* Fix failing test

* Change error message on strict-encrypt postgres destination as well. Fix failing test

* update versions and changelogs

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-09-28 15:26:09 -07:00
Amruta Ranade
a6af9611a6 added SSL Tunnel method rec (#16782) 2022-09-15 17:36:32 -04:00
Adam Fletcher
80397adbe1 🐛 Destination Postgres: fix missing database name URL Encoding (#16046) 
* Fix missing database name URL Encoding in Postgres destination

* add unit test

* bump connector version and update doc

* auto-bump connector version [ci skip]

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-09-08 21:17:29 -03:00
Rodi Reich Zilberman
fc13413c9b Enforce secure connection in destination-postgres-strict-encrypt (#16260) 
* Enforce secure connection in destination-postgres-strict-encrypt

* bump docker image values and update changelog.

* Typo

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-09-02 00:47:19 -07:00
andriikorotkov
d63a24ed41 🎉 Destination postgres: Add SSL certificates and update normalization (#14743) 
* added ssl certificates for postgres source

* added command for remove client private key after transformation to encrypted key with .pk8 extension

* added connection with CA and client certificates for postgres destination

* updated code style

* moved common methods to the common class

* moved common methods to the common class

* fixed remarks

* updated postgres source tests

* added minor changes to spec and added fixes to password mechanism

* updated postgres source tests

* updated strict-encrypt postgres source and destination and added tests for SSL certificates for all postgres connectors

* fixed check style

* updated documentation and versions of connectors

* updated ordrs in test spec

* fixed minor remarks in specs and expected_specs

* fixed minor remarks in specs and expected_specs

* fixed Dockerfile

* fixed remarks

* fixed remarks

* fixed remarks

* fixed remarks

* fixed remarks

* fixed code style

* fixed connectors version in definition file

* updated postgres destination normalization

* updated postgres destination tests

* fixed code style for postgres source and destination

* pulled master changes

* removed allow mode for destination-postgres-strect-encrypt

* updated connectors version

* fixed custom DBT transformation and enabled test for it

* updated normalization version

* updated keystore password generation method

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-08-15 16:32:37 +03:00
Jimmy Ma
6653cd30a3 Bump destination-postgres to 0.3.21 (#14479) 
* Bump destination-postgres to 0.3.21

Goal is to trigger a publish of the arm64 build

* Update destination/postgres changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-07-14 16:58:46 -07:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596) 
This reverts commit a2c194a11f.
 2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369) 
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
 2022-07-08 14:18:27 -05:00
Oleksandr Sheheda
44a43f0338 Improve check connection performance for all JDBC-related DBs (#12820) 
* removed redundant read permission check for destination

* bump docker image versions, update definitions and changelog

* bump docker image versions, update definitions and changelog

* update changelog

* fix for MSSQL and Oracle tests

* fix for MSSQL strict encrypt

* fixed conflicts

* revert versions that were not published
 2022-05-24 16:06:01 +03:00
Eugene
6565315a04 Destination-Postgres: updated postgres.md documentation as per BETA template (#12279) 
* Updated postgres.md documentation as per BETA template
 2022-04-29 09:55:11 +03:00
noahkawasaki-airbyte
d46058e456 🎉 Destination Postgres: Add jdbc_url_params input (#12195) 
* Add jdbc_url_params input for Destination Postgres
 2022-04-25 08:12:31 -07:00
Greg Solovyev
63f6fc5712 Update destination-postgres version in Dockerfile and docs (#12290) 
* Update destination-postgres version in Dockerfile and docs

* Manually bump destination-postgres version
 2022-04-22 18:44:52 -07:00
Sreeram Venkitesh
893b4e4c7a Docs: Fixed typo in postgres destination doc (#12106) 2022-04-18 18:24:42 -03:00
VitaliiMaltsev
a0b7dd0289 🐛 Destination Redshift and Postgres handle custom namespace with '-' (#11729) 
* fix for jdk 17

* add CHANGELOG

* JdbcBufferedConsumerFactory resolve namespace with dash

* add CHANGELOG

* bump versions

* bump postgres version

* update destination-jdbc version

Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com>
 2022-04-07 13:08:57 +03:00
girarda
5d8cb41150 Refactor to enable support for optional JDBC parameters for all JDBC destinations (#10421) 
* refactoring to allow testing

* MySQLDestination uses connection property map instead of url arguments

* Update jdbc destinations

* A little more generic

* reset to master

* reset to master

* move to jdbcutils

* Align when multiline

* Align when multiline

* Update postgres to use property map

* Move tests to AbstractJdbcDestinationTest

* clean

* Align when multiline

* return property map

* Add postgres tests

* update clickhouse

* reformat

* reset

* reformat

* fix test

* reformat

* fix bug

* Add mssql tests

* refactor test

* fix oracle destination test

* oracle tests

* fix redshift acceptance test

* Pass string

* Revert "Pass string"

This reverts commit 697821738c.

* Double deserialization

* Revert "Double deserialization"

This reverts commit ee8d75245b.

* try updating json_operations

* Revert "try updating json_operations"

This reverts commit c8022c2994.

* json parse

* Revert "json parse"

This reverts commit 11a6725eaa.

* Revert "Revert "Double deserialization""

This reverts commit 213f47acc4.

* Revert "Revert "Revert "Double deserialization"""

This reverts commit 66822454af.

* move to constant

* Add comment

* map can be constant

* Add comment

* move map

* hide in method

* no need to create new map

* no need to create new map

* no need to create new map

* enably mysql test

* Update changelogs

* Update changelog

* update changelog

* Bump versions

* bump version

* disable dbt support

* update spec

* update other oracle tests

* update doc

* bump seed

* fix source test

* update seed spec file

* fix expected spec
 2022-02-28 09:25:04 -08:00
Lake Mossman
3d8a0dc048 Add ExitOnOutOfMemoryError to java connectors and bump versions (#10256) 2022-02-14 15:49:15 -08:00
Brian Leonard
f5beea1c81 Documentation: postgres version update for creating index (#9766) 2022-01-24 21:07:38 -03:00
oneshcheret
fcf7491fee 🐛 Validate incorrect handling '\n' symbols in ssh key (#8371) 
* 🐛 Validate \n in ssh key

* bump versions for ssh key connectors

* update versions for ssh key connectors

* temporal fix for checking failed tests

* revert temp changes and destination oracle version

* bump versions in config for ssh key related connectors
 2021-12-03 20:11:51 +02:00
Sherif A. Nada
efb5151011 🐛 Make all JDBC destinations (SF, RS, PG, MySQL, MSSQL, Oracle) handle wide rows by using byte-based record buffering (#7719) 2021-11-08 19:26:32 -08:00
Sherif A. Nada
10889d8338 Add user creation instructions to Postgres destination 2021-10-28 15:44:03 -07:00
Abhi Vaidyanatha
ae32ecbb27 GitBook: [master] 186 pages and 77 assets modified 2021-10-08 21:17:47 +00:00
Abhi Vaidyanatha
c8c736b575 📚 Docs Refresh: Postgres Destination (#6675) 
* Docs Refresh: Postgres Destination

* Add SSL note.
 2021-10-03 23:05:56 -07:00
Abhi Vaidyanatha
d732eaf80e Add information on which destinations support Incremental - Deduped History in their docs (#6031) 
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-09-16 16:40:12 -07:00
Charles
8ad43afb07 SSH for Postgres Destination (#5743) 
Co-authored-by: George Claireaux <phlair@users.noreply.github.com>
 2021-09-07 17:06:25 -07:00
Andrii Leonets
b18bd439d0 🐛 Destination Postgres: fix \u0000(NULL) value processing (#5336) 
* fix \u0000(NULL) value processing for Postgres + move postgres impl of SqlOperations to PostgresSqlOperations.

* changelog + format

* incr release version

* Add generic solution to adopt messages for a destination + remove unnecessary serialization

* revert version for build

* minor review fixes

* format

* add comments

* format

* incr version
 2021-08-30 21:41:02 +03:00
Davin Chia
42686add8a Release connectors with namespace change. (#2990) 
Release all connectors affected by namespace change. Includes all JDBC sources and destinations.

Also add documentation for normalisation. Prerequisite to actually releasing 0.21.0-alpha.
 2021-04-21 11:35:08 +08:00
Shrif Nada
ff458bbd9d GitBook: [master] 111 pages modified 2021-01-26 04:14:55 +00:00
Jared Rhizor
4ab0464274 add postgres troubleshooting docs (#1653) 
* add postgres troubleshooting docs

* update headings

* add to other db sources

* move connection troubleshooting to faq
 2021-01-25 10:56:09 -08:00
Christophe Duong
194d6cb5d0 Add metadata prefix to destination internal columns (#1708) 
* Add metadata prefix to destination internal columns

* Bumpversion all destinations for new normalization behavior

* Fix acceptance tests
 2021-01-19 17:21:15 +01:00
Charles
01cdeafdb1 release v0.8.0-alpha (#1339) 2020-12-16 14:01:57 -08:00
John Lafleur
a5f509bf49 GitBook: [master] 15 pages and 4 assets modified 2020-11-25 19:01:56 +00:00
Christophe Duong
206d3cbea8 Naming conventions managed in destinations (#1060) 
Handled field and table identifier conversions as necessary in order to successfully in destinations
 2020-11-25 18:53:23 +01:00
Charles
4d05a01802 Remove copy and pasted instruction code (#789) 2020-11-02 14:35:30 -08:00
Jared Rhizor
5f0b0a4101 add snowflake destination (#784) 
* add snowflake destination

* dry

* temporarily run in PR build

* fmt

* fix nits

* finish fixing nits

* propate timestamp

* stop running snowflake tests in prs

* Update docs/integrations/destinations/snowflake.md

Co-authored-by: Charles <giardina.charles@gmail.com>

* use non-transient tables

* separate knowledge of integration name from individual destination tests

Co-authored-by: Charles <giardina.charles@gmail.com>
 2020-11-02 11:51:15 -08:00
John Lafleur
7ef914a213 GitBook: [john_specs_suggestion] 5 pages modified (#666) 
* GitBook: [john_specs_suggestion] 5 pages modified

* GitBook: [john_specs_suggestion] 34 pages modified

Co-authored-by: John Lafleur <john@dataline.io>
 2020-10-21 17:28:08 +11:00
Charles
57c0325f03 remove data type mappings (#646) 2020-10-20 01:56:25 -07:00