Commit Graph

143 Commits

Author SHA1 Message Date
sbliu
8dd9692ce6 feat(TUP-30114) Update Copyrights to 2021 for master code (#4102) 2021-03-04 18:05:41 +08:00
vdrokov
ac30cd0908 TESB-31044: Data service's endpoint can't be updated if passed as a context variable (#4018) 2021-02-03 09:58:46 +02:00
pyzhou
e2a87b440d feat(TDI-45463):Add Delta Lake SQL Template (#4009)
* feat(TDI-45463):Add Delta Lake SQL Template

* correct sqltemplate

* wrong place of using delta
2021-02-02 17:32:30 +08:00
apoltavtsev
03314aeee8 feat(TESB-29949) Pass the data source to a job using a context variable 2020-11-19 09:31:33 +01:00
wang wei
ccb9137df5 fix(TDI-45038): add "timestamp" info for component start event for studio audit log(#3742) (#3783) 2020-11-02 10:18:04 +08:00
chmyga
5a8ef4838a Dchmyga/tdi 44962 backport to master (#3704)
* Revert "fix(TDI-44826): Missing line in Resume log (#3640)" (#3691)

This reverts commit cf52e1e004.

Co-authored-by: Dmytro Chmyga <dmytro.chmyga@synapse.com>

* fix(TDI-44962): fix stream closed exception (#3692)

* Recreate FileWriter with append after writing header

* Add csv writer to shared writer map

Co-authored-by: Dmytro Chmyga <dmytro.chmyga@synapse.com>

Co-authored-by: Dmytro Chmyga <dmytro.chmyga@synapse.com>
2020-10-12 14:30:39 +03:00
wang wei
470175ccf6 fix(TDI-44937): TalendDate.formatDateLocale()gives random value for FR country code in java 11.(#3679) 2020-10-12 10:15:12 +08:00
vyu-talend
16fc19571c fix(TDI-44726):add some necessary annotation. (#3666) (#3682) 2020-09-27 16:14:50 +08:00
wang wei
c68419a4e2 fix(TDI-44623): [OBSERVABILITY] : add PID and (Thread-ID OR UUID) information (#3614) (#3662)
* fix(TDI-44623): [OBSERVABILITY] : add PID and (Thread-ID OR UUID)
information

* decrease the expensive call number for getting pid

* fix(TDI-44807): Observability: initialize moment only when necessary

* fix(TDI-44806): Observability: Add customization for event frequency
2020-09-18 15:06:44 +08:00
hcyi
40da0e6196 feat(TUP-23827): Runtime Lineage (#3646)
* feat(TUP-23827):Runtime Lineage for studio.

* fix(TUP-28635):[bug] output path always be changed when path contains
2020-09-17 14:55:09 +08:00
pyzhou
62bcd2d641 fix(TDI-44826): Missing line in Resume log (#3639) 2020-09-14 17:10:39 +08:00
Colm O hEigeartaigh
df577338cd TUP-28307 - Replace toLowerCase().equals() with equalsIgnoreCase (#3552) 2020-09-03 10:08:18 +08:00
wang wei
56f900f470 fix(TDI-44171): Observability: Add "Component" label to component-related events
* fix(TDI-44171): Observability: Add "Component" label to
component-related events

* fix(TDI-44172): Observability: Create new "component execution" event
and emit it frequently

* fix(TDI-44172): fix some issue

* fix(TDI-44172): fix some issue

* remove the unuseful code in loop to avoid performance loss
2020-06-10 16:28:36 +08:00
AlixMetivier
be546937be fix(TBD-10269): force return in case of bad job design (#3287) 2020-05-25 10:26:37 +02:00
AlixMetivier
890f446373 fix(TBD-10269): method getDate can work with java.sql.Timestamp/Date (#3196)
* fix(TBD-10269): method getDate can work with java.sql.Timestamp/Date

* removed file
2020-05-04 09:48:45 +02:00
Zhiwei Xue
dc1271edf7 fix(TBD-10370):"Hadoop configuration jar not found" when run spark job (#3221) 2020-04-17 10:07:47 +02:00
bhe-talendbj
4f32054c45 bugfix(TUP-25495) Update to dom4j 2.1.1 (#2959)
* bugfix(TUP-25495) Upgrade dom4j to 2.1.1

* bugfix(TUP-25495) Upgrade dom4j to 2.1.1

* bugfix(TUP-25495) Add dom4j license

* bugfix(TUP-25495) Add back version number of jar

* bugfix(TUP-25495) Revert plug.xml

* bugfix(TUP-25495) Keep dom4j-1.6.1.jar

* bugfix(TUP-25495) Upgrade jaxen to 1.1.6 and add keep jaxen-1.1.1.jar

* bugfix(TUP-25495) Upgrade jaxen to 1.1.6 and add keep jaxen-1.1.1.jar

* bugfix(TUP-25495) Upgrade jaxen to 1.1.6 and add keep jaxen-1.1.1.jar
2020-01-07 15:15:13 +08:00
wang wei
56a48fa889 Revert "fix(TDI-43101) : support to generate a sample logging file for runtime lineage (#2855)" (#2899)
This reverts commit d889b21c6d.
2019-12-10 15:12:35 +08:00
Antoine Parent
ab6aa521e7 fix(TBD-9669): add method signature for class RunStat (#2888)
* fix(TBD-9669): add method signature for class RunStat

add a new signature for the method "log"

* fix(TBD-9669): add method signature for failed tujs

fix variable name as it lead to compile error
2019-12-09 10:16:15 +01:00
Richard Lecomte
b5240d1762 Rlecomte/fix/tdi 42188 harden deserialization 2 (#2884)
* fix : TDI-42188

Harden Deserializaion (RunTrace)
InputStream in a try-with-resource block (GetJarsToRegister)

* fix : TDI-42188

Harden Deserializaion (RunTrace)
InputStream in a try-with-resource block (GetJarsToRegister)

* fix : TDI-42188

Copy expectedTypes to avoid security leak
Better type checking

* fix : TDI-42188

Deserialized type checked in GetJarsToRegister

* fix : TDI-42188

Removed deserialized type checking in GetJarsToRegister
2019-12-05 10:25:30 +01:00
kjwang
d95d058e75 Kjwang/fix tup 22966 remove hard coded key in password encrypt util 1 (#2829)
* feat(TUP-24862) Key rotation

* feat(TUP-24862) Add key rotation

* feat(TUP-24862) Refactor

* feat(TUP-24862) Refactor

* feat(TUP-24862) Fix logger

* feat(TUP-24862) Add Key rotation

* feat(TUP-24862) add test case

* feat(TUP-24862) Fix test case

* feat(TUP-24862) Fix password encrypt util

* feat(TUP-24862) Resolve comments

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966

* kjwang/Fix TUP-22966 Remove hard-coded key in PasswordEncryptUtil_1
https://jira.talendforge.org/browse/TUP-22966
2019-12-05 15:00:46 +08:00
wang wei
d889b21c6d fix(TDI-43101) : support to generate a sample logging file for runtime lineage (#2855)
* fix(TDI-33262): align slf4j version to 1.8.0-beta4

* Revert "fix(TDI-33262): align slf4j version to 1.8.0-beta4"

This reverts commit 68bb6ba4dd.

* bump the version as we update the jar which remove log4j1 dependency and
then use slf4j

* fix(TDI-43101): [Research] Generate a sample logging file for runtime
lineage

* fix(TDI-43101): fix it
2019-11-26 18:11:20 +08:00
wang wei
273fe48fa7 fix(TDI-43211): SharedDBConnectionLog4j use log4j1 api, should use slf4j (#2841) 2019-11-22 11:01:21 +08:00
bhe-talendbj
9a6fe39e7d feat(TUP-24862) encryption key rotation (#2806)
* feat(TUP-24862) Key rotation

* feat(TUP-24862) Add key rotation

* feat(TUP-24862) Refactor

* feat(TUP-24862) Refactor

* feat(TUP-24862) Fix logger

* feat(TUP-24862) Add Key rotation

* feat(TUP-24862) add test case

* feat(TUP-24862) Fix test case

* feat(TUP-24862) Fix password encrypt util

* feat(TUP-24862) Resolve comments

* feat(TUP-24862) Resolve comments
2019-11-21 10:13:37 +08:00
AlixMetivier
655418754a add method to fix arguments (#2788) 2019-10-24 13:14:47 +02:00
Roman
96a90bc9d4 fix(TDI-43022): change LN method in Routines (#2751) 2019-10-23 07:41:17 +03:00
bhe-talendbj
755fed6f00 feat(TUP-21518) Remove CryptoHelper (#2655)
* feat(TUP-21518) initial change from cryptohelper to AESEncryption

feat(TUP-21518) Replace CryptoHelper

feat(TUP-21518) Revert migration related changes and changes to old hardcoded key of AESEncryption, tracked by https://jira.talendforge.org/browse/TUP-22966

feat(TUP-21518) Add CryptoHelperWrapper which will be intended for
migration

feat(TUP-21518) Add methods to CryptoHelperWrapper

feat(TUP-21518) Fix Separate migration code

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Format source code

* feat(TUP-21518) Uptake new daikon lib

* feat(TUP-21518) initial change from cryptohelper to AESEncryption

feat(TUP-21518) Replace CryptoHelper

feat(TUP-21518) Revert migration related changes and changes to old hardcoded key of AESEncryption, tracked by https://jira.talendforge.org/browse/TUP-22966

feat(TUP-21518) Add CryptoHelperWrapper which will be intended for
migration

feat(TUP-21518) Add methods to CryptoHelperWrapper

feat(TUP-21518) Fix Separate migration code

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Format source code

feat(TUP-21518) Uptake new daikon lib

feat(TUP-21518) Rename encryption key

* feat(TUP-21518) Update system property

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Update log level

* feat(TUP-21518) Set encryption key system property only if it is not set

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Set up default keys

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Setup default encryption keys

* feat(TUP-21518) Update crypto-utils

* feat(TUP-21518) Merge changes

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Fix test case and load default keys for tac and
jobserver

* feat(TUP-21518) Fix load key file

* feat(TUP-21518) Revert pom_minimal and server

* feat(TUP-21518) Fix test case

* feat(TUP-21518) Fix test case

* feat(TUP-21518) Fix test case

* feat(TUP-21518) Fix ElementParameterParserTest

* fix(TDQ-16616): refactor the code to make it reuseable

* feat(TUP-21518) Fix TUP-24580

* feat(TUP-21518) Fix TUP-24578

* feat(TUP-21518) Add JDBC connection type

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Refactor

* feat(TUP-21518) Fix log

* feat(TUP-21518) Fix format of source code

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Load migration.token.encryption.key from local jar

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Fix TUP-24868
2019-10-18 09:36:24 +08:00
wang wei
15216b8be3 fix(TDI-42183): Avoid locale dependent checks (#2715)
* fix(TDI-42183): Avoid locale dependent checks

* fix(TDI-42183): declare the code come from apache commons lang
2019-09-23 18:04:06 +08:00
wang wei
1f385ee333 fix(TDI-42187): Fix Double Checked Locking in GlobalResource (#2708)
* fix(TDI-42187): Fix Double Checked Locking in GlobalResource

* fix(TDI-42187): use more readable method
2019-09-23 17:42:49 +08:00
wang wei
7b21c010e2 fix(TDI-42192): Use SecureRandom in system routines (#2707) 2019-09-18 18:34:00 +08:00
Richard Lecomte
2aae24829a fix : TDI-42776 (#2692)
* fix : TDI-42776

Numeric sequence is now thread safe
2019-08-30 10:50:32 +02:00
Emmanuel GALLOIS
a26002f9a2 feat(TDI-40339): add Snowflake SQL Templates (#2633)
* feat(TDI-40339): add Snowflake Merge template
* feat(TDI-40339): add Snowflake table management templates
* fix(TDI-40339): correct create target table template
* fix(TDI-40339): remove the OR REPLACE statement
2019-08-16 11:23:22 +02:00
wang wei
bea0b2a13b fix(TDI-42522): avoid 65535 issue (#2540) 2019-08-16 11:59:59 +08:00
Irene Wang
dcd7bf8010 DEVOPS-6106 Update copyright year to 2019 2019-06-10 17:34:30 +02:00
wang wei
5d627f8d64 fix(TDI-42071): PoC for job runtime information publishing (#2454) 2019-06-05 17:58:06 +08:00
kjwang-talend
f01cd163ef TUP-23201 Need change of encryption for jobs (#2445)
https://jira.talendforge.org/browse/TUP-23201
2019-05-29 15:59:08 +08:00
kjwang-talend
f807ca1839 TUP-23004 Fail to decrypte password (#2411)
* TUP-23004 Fail to decrypte password
https://jira.talendforge.org/browse/TUP-23004

* TUP-23004 Fail to decrypte password
https://jira.talendforge.org/browse/TUP-23004
2019-05-14 17:01:59 +08:00
kjwang-talend
97dd532fdd Fix TUP-21510 Update PasswordEncryptUtil (#2380)
* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil
https://jira.talendforge.org/browse/TUP-21510

* Fix TUP-21510 Update PasswordEncryptUtil (Update slf4j version)
https://jira.talendforge.org/browse/TUP-21510
2019-05-09 11:52:58 +08:00
TKaczynski
eb33120cac fix(TBD-7687): tFileInputDelimited issue with separator (#2314)
* fix(TBD-7687): tFileInputDelimited issue with separator

add a null instead of delete the column

* Update StringUtils.java

* Update StringUtils.java

* Update StringUtils.java

* Update StringUtils.java
2019-04-04 13:53:30 +02:00
chmyga
82f48b9ec9 fix(TDI-41681): fix XMLHelper (#2263) 2019-03-19 10:09:41 +02:00
Dmytro Sylaiev
a101784871 Dsylaiev/tdi 41808 fix showing pass nzload (#2276)
* fix(TDI-41808): Add password for logging constant

* fix(TDI-41808): Rename constant

* chore(TDI-41808): Update year in copyright
2019-03-07 11:12:46 +02:00
Tao Tao
4e2069f2f5 Revert XMLHelper.java changed in TUP-21232. (#2217) 2019-01-22 19:20:24 +08:00
Tao Tao
e5deedc9a8 workItem(TUP-21307):Duplicate code in IdGenerator + TalendString (#2193)
* workItem(TUP-21307):Duplicate code in IdGenerator + TalendString

https://jira.talendforge.org/browse/TUP-21307

* workItem(TUP-21307):Duplicate code in IdGenerator + TalendString

https://jira.talendforge.org/browse/TUP-21307

* workItem(TUP-21307):Use SecureRandom + StringBuilder in IdGenerator +
TalendString

https://jira.talendforge.org/browse/TUP-21307
2019-01-15 18:30:29 +08:00
Tao Tao
d287b431b5 workItem(TUP-21232):Secure SAXParserFactory configuration (#2185)
* workItem(TUP-21232):Secure SAXParserFactory configuration

https://jira.talendforge.org/browse/TUP-21232

* workItem(TUP-21232):Secure SAXParserFactory configuration

https://jira.talendforge.org/browse/TUP-21232
2019-01-15 11:30:26 +08:00
Pierre Teyssier
db0a79887b TDI-41577 fix wheather strings 2018-12-25 17:12:27 +08:00
chmyga
036b6b090f fix(TDI-31052): tRowGenerator Numeric.random (#2043)
* fix(TDI-31052): tRowGenerator Numeric.random

* throw RuntimeException if max value is less than or equal to min

* fix(TDI-31052): tRowGenerator Numeric.random

* throw exception only for max < min values

* fix(TDI-31052): tRowGenerator Numeric.random

* Fix javadoc
2018-11-30 16:19:55 +02:00
mbasiuk-talend
b608c2a769 fix(TDI-40846): create isDateStrict in routines (#1823) 2018-10-10 17:11:17 +08:00
wang wei
61b14bd1f5 fix(TDI-40981): Resuming log header miss last column name (#1866) 2018-09-30 10:10:31 +08:00
Yu
aa5e7e2806 Add epoch time parser (#1840) 2018-09-03 18:19:07 +02:00
kcoepeau
ab8c9b7875 fix(TBD-7393): job jar in job server does not have ./ (#1789) 2018-07-20 15:28:32 +02:00