Compare commits

...

189 Commits

Author SHA1 Message Date
vyu-talend
950fb468d6 fix(TPS-3690):patch for TDI-43446 2020-01-13 15:39:00 +08:00
hwang-talend
ecdf7a18b0 bugfix(TUP-24413):The tab name of resource is changed (#4298) 2020-01-13 10:26:15 +08:00
jzhao
6eddb2cd20 feat(TDI-43395):update dom4j to 2.1.1 (#4296)
* feat(TDI-43395):update dom4j

* feat(TDI-43395):update dom4j depends jaxen

* feat(TDI-43395):dom4j api changed

* fix compiler error
* remove useless code of txmlmap

* feat(TDI-43395):add migration task for bonita components after dom4j
update
2020-01-10 17:18:26 +08:00
Dmytro Sylaiev
0bd016b1c5 Dsylaiev/tdi 42875 upgrade t google dataproc crm (#4212)
* feat(TDI-42875): Upgrade google dataproc

* Add region enum

* Update zones

* Make file property use file widget

* feat(TDI-42875): Implement generated code for feature

* feat(TDI-42875): Remore IF in required operator

* feat(TDI-42875): Use region for stopping cluster either

* feat(TDI-42875): Add commons-codec in dependencies
2020-01-09 13:21:31 +02:00
hwang-talend
93b97ab0ed bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4307)
bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2020-01-09 17:11:18 +08:00
hwang-talend
0e08ab1a54 bugfix(TUP-24510):Studio can't use JDBC connection to trace job (#4273)
bugfix(TUP-24510):Studio can't use JDBC connection to trace job stats&logs
2020-01-09 10:50:17 +08:00
Mike Yan
c3f79cd79e feat(TESB-25283)[CI] Support Maven based Versioning for ESB Use cases (#4057)
* feat(TESB-25283)[CI] Support Maven based Versioning for ESB Use cases

* fix(TESB-25283):update osgi pom template to skip deploy
2020-01-09 10:48:24 +08:00
jiezhang-tlnd
29c1c47d97 fix(TUP-21660)add slf4j-simple.jar to Bundle-ClassPath in the plugin (#4303)
org.talend.sdk.component.studio-integration
2020-01-09 10:01:33 +08:00
sponomarova
ccdaa1af1c fix(TBD-9317): clean up components from org.talend.libraries.pig dependency (#4012)
* fix(TBD-9317): clean up components from org.talend.libraries.pig
2020-01-08 12:58:55 +02:00
hcyi
25857d0c10 fix(TUP-25229):Some TCOMPV0's components should not has "Use this (#4299)
Component" in Connection Component
2020-01-07 18:12:15 +08:00
jiezhang-tlnd
0f3a3dda19 fix(TUP-25702)TOS:Log4j:cannot print info on console when run job (#4301)
* fix(TUP-25702)TOS:Log4j:cannot print info on console when run job
https://jira.talendforge.org/browse/TUP-25720

* fix(TUP-25702)TOS:Log4j:cannot print info on console when run job
2020-01-07 16:21:14 +08:00
zshen-talend
943dbe61b2 fix(TDQ-17709): create number limit text controller (#4278)
* fix(TDQ-17709): create number limit text controller

* fix(TDQ-17709): improve code and add some lost modify

* fix(TDQ-17709): improve the code about the limit

* fix(TDQ-17709): remove unused code
2020-01-07 11:27:30 +08:00
vyu-talend
2c7dca1458 Vyu/tdi 43446 fix compilation error for tmap (#4291)
* Fix(TDI-43446):fix compilation error.

* Fix(TDI-43446):fix issues.

* Fix(TDI-43446):fix a bug written by myself.

* Fix(TDI-43446):fix an issue.

* Fix(TDI-43446):fix the exception problem.

* Fix(TDI-43446):fix issue.
2020-01-06 17:36:59 +08:00
jiezhang-tlnd
06a4ca938c fix(TUP-21660)SLF4J: Failed to load class (#4300)
"org.slf4j.impl.StaticLoggerBinder"
2020-01-06 09:59:15 +08:00
Zhiwei Xue
217360420f feat(TUP-24722):Force each studio on the same project to install the same patch (#4270)
* feat(TUP-24722):Force each studio on the same project to install the
same patch

* feat(TUP-24722):refactor util class.
2020-01-03 18:14:51 +08:00
hwang-talend
9bf91aca62 bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4281)
* bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2020-01-03 17:30:21 +08:00
jiezhang-tlnd
155973785f fix(TUP-25572) improve the performance of import job (#4295)
* fix(TUP-25572) improve the performance of import job

* fix(TUP-25572) improve the performance of import job
2020-01-03 16:43:45 +08:00
jiezhang-tlnd
5e0f8e9194 fix(TUP-25338)Run job use sap miss jar slf4j-api-1.7.16 and there is no (#4292)
* fix(TUP-25338)Run job use sap miss jar slf4j-api-1.7.16 and there is no
pop up let me download or browse in.
https://jira.talendforge.org/browse/TUP-25338

* fix(TUP-25338) avoid multiple versions

* avoid multiple module versions

* remove the same version jar only about slf4j-api

* remove the same version jar only about slf4j-api

* avoid multiple versions of slf4j-api
2020-01-03 16:36:29 +08:00
Jane Ding
64b049164c feat(TUP-24057)There is no error message when studio can't connect to (#4297)
customer artifact repository
https://jira.talendforge.org/browse/TUP-24057
2020-01-03 11:25:19 +08:00
Emmanuel GALLOIS
1be6c8fdd4 chore(TDI-40449): removed unused jars (#2815)
* chore(TDI-40449): removed unused jars

* chore(TDI-40449): add version to jars w/o version

* revert(TDI-40449): remove external_sort jar

* Revert: chore(TDI-40449): removed external_sort.jar

* chore(TDI-40449): remove external_sort

* chore(TDI-40449): remove jnp-client and talend-bigdata-launcher

* chore(TDI-40449): fix maven paths

* chore(TDI-40449): restore boxjavalibv2-v3.0.10.jar

* chore(TDI-40449): restore talend-webservice-input-1.0.0-20170301.jar

* chore(TDI-40449): restore geopip

* chore(TDI-40449): restore talend-bigdata-launcher-1.1.0-20160405.jar

* chore(TDI-40449): bump ws-commons-util

* chore(TDI-40449): restore tMom jars needed

* fix(TDI-40449) : prepare servicenow for publishing
- force maven-deploy-plugin to 2.8+
- bump version to 1.0.1
- reformat code to Talend standard
- add license header
- remove main method containing credentials(not critical)

* chore(TDI-40449): fixup some components

* chore(TDI-40449): mavenized google-storage-utils

* chore(TDI-40449): cleaning talendssl ant file

* chore(TDI-40449): reverting changes
- SugarCRM is deprecated and will be removed in this release

* chore(TDI-40449): fix version typo/group for asm

* chore(TDI-40449): rename google storage utils

* chore(TDI-40449): fix dependencies to the future google-storage-utils
- will be built during https://jira.talendforge.org/browse/TDI-43489

* chore(TDI-40449): fix javassist GAV dependency

* chore(TDI-40449): fix GAV for tMOM/tWebService
- revert change on tWebService_java.xml
- fix GAV for accuracy on tMOM

* chore(TDI-40449): remove last jars
- See https://jira.talendforge.org/browse/TDI-43490

* chore(TDI-40449): fix url

* chore(TDI-40449): distribution management needed for google storage utils on ci-common

* chore(TDI-40449): cleaning xml files

* chore(TDI-40449): update copyright

* chore(TDI-40449): last clean
- i swear

Co-authored-by: Pierre Teyssier <pteyssier@talend.com>
2020-01-02 15:56:13 +08:00
jiezhang-tlnd
73e7d360d0 fix(TUP-25524)issue for TOS Log4j (#4294) 2019-12-31 15:54:13 +08:00
hcyi
ec490c589e fix(TUP-25382):More than one dialog pop up when quickly click the button more than once. (#4269)
* fix(TUP-25382):More than one dialog pop up when quickly click the button
more than once.

* fix(TUP-25382):More than one dialog pop up when quickly click the button
more than once.
2019-12-31 11:41:46 +08:00
hwang-talend
2a67448f58 bugfix(TUP-24029):[FTP] Connection wizard add timeout property (#3864) (#3982)
* bugfix(TUP-24029):[FTP] Connection wizard add timeout property (#3864)

bugfix(TUP-24029):[FTP] Connection wizard add timeout property

* feature(TUP-24029): [FTP] Connection wizard add timeout property

* feature(TUP-24768):[bug] no unit for timeout parameter for ftp wizard.

* feature(TUP-24768):[bug] timeout value support to type non-digit.

* feature(TUP-24768):[bug] timeout value support to type non-digit.
2019-12-30 18:08:49 +08:00
Jane Ding
617095c83b feat(TUP-24057)There is no error message when studio can't connect to (#4290)
customer artifact repository
https://jira.talendforge.org/browse/TUP-24057
2019-12-30 17:49:20 +08:00
mbasiuk-talend
1c857a778a Mbasiuk/tdi 43095 improve set section (#4241)
* feat(TDI-43095): improve set section for tELTTeradataOutput

* feat(TDI-43095): improve SET section for tELTOutput generic

* feat(TDI-43095): extend logic for USE_UPDATE_STATEMENT, fix typo

* feat(TDI-43095): make changes according to PR comments
2019-12-30 11:41:34 +02:00
hcyi
568c60df0f fix(TUP-23672):Check connection button for tacokit integration to Studio (#4271) 2019-12-30 10:43:01 +08:00
hwang-talend
425dce5d7a bugfix(TUP-21691):Run job fail after drag it out from a folder. (#4265)
bugfix(TUP-21691):Run job fail after drag it out from a folder.
2019-12-30 10:09:50 +08:00
kjwang
30b9a6af9d Fix TUP-24523 Online Help in studio - refactor help features (#4227)
* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523
2019-12-27 18:20:39 +08:00
hcyi
c782cdc070 fix(TUP-24405):stats/logs database ui is not right when choose teradata (#4277)
database
2019-12-27 17:36:37 +08:00
hcyi
7bd4d48476 fix(TUP-24727):Use shortcut F1 to search help document on component then (#4285)
throw errors
2019-12-27 17:25:22 +08:00
Zhiwei Xue
99ad6fb48b fix(TUP-25223):Child job maven version is incorrect if it is opened by (#4280)
tRunjob
2019-12-27 17:25:13 +08:00
jiezhang-tlnd
92c5a956b8 Jzhang/bugfix/tup 25572 avoid loop method on log4j (#4259)
* avoid loop method

* fix(TUP-25572)avoid loop method to improve performance

* avoid loop method to improve performance

* update junits

* fix jar start with "talend-bigdata"

* fix(TUP-25541)bridge jar not included in class path for log4j1 jar
needed in child process but not in father's.

* update class path order of log4j

* fix dateviewer error on oracle db

* fix data viewer, and conflict spark-assembly jar and log4j jar

* update class path order for spark-assembly

* avoid repeating for child module and refactor code

* refactor code for sort class path

* add junits

* fix tuj tMap_04_persistence_04_n_lookups
2019-12-27 17:15:41 +08:00
Zhiwei Xue
a3daac36e9 fix(TUP-25328): When move folder out of folder will have error :Resource (#4274)
*** not exist
2019-12-27 10:59:42 +08:00
hcyi
b7c5bb914c fix(TUP-24625):implement lazy migration. (#4174)
* fix(TUP-24625):Implement jobs lazy migration.

* fix(TUP-24625):When/If lazy migration will be done following fix won't
be needed anymore on TUP-21055 .

* fix(TUP-24625):remove job migration code when start-up if implement lazy
migration.

* fix(TUP-24625):remove migration code when start-up if implement lazy
migration.

* fix(TUP-24625):improve to implement lazy migration.
2019-12-26 16:59:49 +08:00
Chao MENG
3f9f37bf90 fix(TUP-24905): Studio automatically set password when set up multiple (#4272)
fix(TUP-24905): Studio automatically set password when set up multiple login connections
https://jira.talendforge.org/browse/TUP-24905
2019-12-26 14:55:48 +08:00
hzhao-talendbj
24471c3ed3 feat(TUP-25120)Merge nexus proxy setup (#4211)
* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup
2019-12-26 11:16:49 +08:00
bhe-talendbj
e592a52dfa bugfix(TUP-25409) Improve building speed of CI builder (#4253)
* bugfix(TUP-25409) Generate source code for main job only

* bugfix(TUP-25409) Resolve comments
2019-12-25 14:51:13 +08:00
hzhao-talendbj
b0d4990091 bugfix(TUP-25035)Implicit tContextLoad feature (with selection from (#4268)
file) uses a hardcoded encoding of ISO-8859-15
2019-12-25 11:40:29 +08:00
Zhiwei Xue
4766e40047 fix(TUP-23331):Studio title change to 'Setup Traces' after I click Setup (#4262)
Traces
2019-12-24 18:15:46 +08:00
vyu-talend
78bbfc61fb Fix(TDI-36397):Reset default value for 'Batch mode' and migration. (#4267)
* Fix(TDI-36397):Reset default value for 'Batch mode' and migration.

* Fix(TDI-36397):register migration task to plugin.xml.
2019-12-24 17:26:54 +08:00
Zhiwei Xue
f3bd46488e fix(TUP-25206):NPE thrown in error log when close job (#4263) 2019-12-24 16:45:58 +08:00
chmyga
36f98068e5 fix(TDI-43319): CDC Oracle wrong settings (#4219)
* Add migration task for old jobs with OracleCDC component
2019-12-23 15:35:50 +02:00
hwang-talend
41cbeeb140 bugfix(TUP-22500):jars already existed on nexus still try to upload (#4152)
bugfix(TUP-22500):jars already existed on nexus still try to upload again
2019-12-23 10:17:32 +08:00
Chao MENG
c96bae71da fix(TUP-25502): import dialog i18n error (#4260)
fix(TUP-25502): import dialog i18n error
https://jira.talendforge.org/browse/TUP-25502
2019-12-23 10:11:58 +08:00
vyu-talend
eedf273635 Fix(TDI-43323):Fix issue of duplicated variables. (#4233) 2019-12-23 10:00:57 +08:00
hcyi
d047644d8b fix(TUP-25575):Upgrade tacokit to 1.1.15. (#4258) 2019-12-20 17:28:10 +08:00
vyu-talend
bd04ae1b54 Fix(TDI-42408):rename for tGPGDecrypt. (#4254) 2019-12-20 15:40:59 +08:00
chmyga
8641ad63d1 feat(TDI-40351): Add alternative schema name (#4007)
*  feat(TDI-40351): Add alternative schema name

 * Add "use alternative schema" property for db output components

* feat(TDI-40351): Add alternative schema name

* Move alternate schema to advanced settings

* Rename alternate schema properties

* feat(TDI-40351): Add alternative schema name

* fix problems for ParAccel, PostgreSql, PostgresPlus, Redshift

* feat(TDI-40351): Add alternative schema name

* Remove schema field when use existing connection is checked
2019-12-19 14:18:07 +02:00
chmyga
9f80f3f7ff feat(TUP-25053): use alternate schema in DB Output (#4090)
* Add usage of alternate schema during table selection
2019-12-19 14:17:13 +02:00
wang wei
bbb2bddea5 fix(TDI-43396): Compile error for S3 and tFileFetch TUJs when log4j is not enabled at default(#4238) 2019-12-19 11:35:10 +08:00
jiezhang-tlnd
ad193ddfd8 fix(TUP-25483)color default setting not work on ubuntu (#4251)
https://jira.talendforge.org/browse/TUP-25483
2019-12-18 17:57:27 +08:00
jiezhang-tlnd
8ee5787201 fix(TUP-25541)tMap TUJs are broken for the ClassNotFound (#4249)
* fix(TUP-25541)tMap TUJs are broken for the ClassNotFound

* fix(TUP-25541)tMap TUJs are broken for the ClassNotFound
2019-12-18 17:26:44 +08:00
hcyi
7f91106ec7 fix(TUP-25411):Table name and field should in double quotes in (#4242)
tELTPostgresqlMap.
2019-12-18 14:53:09 +08:00
hwang-talend
f7607f6e8a bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4221)
bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2019-12-18 14:43:22 +08:00
hwang-talend
5027f0fd3b bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4220)
bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2019-12-18 14:41:51 +08:00
jiezhang-tlnd
504a5477bc fix(TUP-25530)Log4j error when execute a job on datatabricks (#4243)
* fix(TUP-25530)Log4j error when execute a job on datatabricks

* Log4j error when execute a job on datatabricks
2019-12-17 19:17:09 +08:00
jiezhang-tlnd
196bec65e5 fix(TUP-25504)Active log4j by default (#4237)
* fix(TUP-25504)Active log4j by default
https://jira.talendforge.org/browse/TUP-25504

* fix tos log4j default
2019-12-16 16:27:10 +08:00
jiezhang-tlnd
c6fdf4da5b fix log4j failed junits (#4239) 2019-12-16 15:09:39 +08:00
jiezhang-tlnd
f52c9c2584 fix(TBD-9677)under the log4j2, Run some job with spark 2.4 local get (#4235) 2019-12-13 14:27:47 +01:00
jiezhang-tlnd
0fb3315eee feat(TUP-24651)Add choice between log4j and log4j2 in project settings (#4173)
* feat(TUP-24651)Add choice between log4j and log4j2 in project settings

* add warnings

* fix(TUP-25319): When we deactivate the log4j option, DI need use default
jars log4j2 needed

* remove system property and enable the feature by defult

* fix(TUP-25222)NoClassDefFoundError is thrown when switch between 2 log4j
versions for tRunJob if Use independent Process is checked

* add dialog for log4j version changed

* issue of tRunjob with unactivate log4j

* fix junit for log4j

* fix issue:get error when config match rules on TDM component
"tMatchGroup"

* fix log4j default issue for new project
2019-12-13 17:22:43 +08:00
Jane Ding
ba12b6577a Fix junits failure (#4229)
caused by org.eclipse.emf.ecore.resource.Resource$IOWrappedException:
The object 'testauto@talend.com' is not contained in a resource
2019-12-13 16:50:11 +08:00
hwang-talend
ce7bfa060f bugfix(TUP-25106):when creating an onsubjobOk link with tELTMap, this (#4224)
bugfix(TUP-25106):when creating an onsubjobOk link with tELTMap, this generate an incorrect dsiplay
2019-12-13 15:29:07 +08:00
Mike Yan
a8155b659c fix(TESB-27266):Exclude tdm-lib-di from REST DS, but keep package info (#4230) 2019-12-13 14:20:07 +08:00
Zhiwei Xue
f0c4517bbc fix(TUP-24719): improve the exception handling of signature failure. (#4226) 2019-12-13 10:07:20 +08:00
bhe-talendbj
247fc84baa bugfix(TUP-25237) Need to clear modules per job cache before generate pom (#4216) 2019-12-12 15:26:18 +08:00
Zhiwei Xue
1b8fe81314 fix(TUP-24719): fix possible NPE. (#4225) 2019-12-11 19:13:04 +08:00
Zhiwei Xue
5d32586b46 feat(TUP-24719):Studio updates need to install new components by (#4171)
project/license type
2019-12-11 16:05:36 +08:00
jiezhang-tlnd
4596f2a3a8 fix(TUP-25457)Harden RunTrace deserialization (#4223)
https://jira.talendforge.org/browse/TUP-25457
2019-12-11 14:55:32 +08:00
kjwang
f5001bcddf Kjwang/ tup 25343 database can not dataview (#4215)
* TUP-25343 [BUG] database can't dataview after set
routine.encryption.key.v2
https://jira.talendforge.org/browse/TUP-25343

* TUP-25343 [BUG] database can't dataview after set
routine.encryption.key.v2
https://jira.talendforge.org/browse/TUP-25343
2019-12-11 11:25:26 +08:00
jiezhang-tlnd
15f87a595f Jzhang/tup 25461 log4j2 default setting for level (#4222)
* log4j2 setting default on log level

* log4j2 setting default on log level
2019-12-11 11:00:36 +08:00
jiezhang-tlnd
74d0a5fc4d fix(TUP-25453)color setting for log4j2 (#4214)
* fix(TUP-25453)color setting for log4j2

* color setting for log4j2
2019-12-10 16:28:07 +08:00
wang wei
9a7f158ea8 fix(TDI-43101): revert it as only a poc research topic, should not merge (#4213) 2019-12-10 15:12:09 +08:00
Liu Xinquan
76aff58dc9 TDQ-17742 change the dialog title to Enter a new license key (#4195)
* TDQ-17742 change the dialog title to Enter a new license key

* TDQ-17742 change the dialog title to Enter a new license key
2019-12-09 15:56:39 +08:00
jiezhang-tlnd
ab1c2ac982 fix(TUP-25460)schema hardcoded in guess query in tDBInput component (#4204)
https://jira.talendforge.org/browse/TUP-25406
2019-12-09 14:21:17 +08:00
hcyi
856a9343b8 fix(TUP-25218):tELTXXXMap: Output table : Expression & Column set to (#4200)
blank on Mac OS X。
2019-12-09 11:06:01 +08:00
hzhao-talendbj
1db0ec3664 feat(TUP-24382)Enable HTTPS with authentication (#4168)
* feat(TUP-24382)Enable HTTPS with authentication

* feat(TUP-24382)Enable HTTPS with authentication

* feat(TUP-24382)Enable HTTPS with authentication

* feat(TUP-24382)Enable HTTPS with authentication
2019-12-06 17:46:27 +08:00
Dmytro Sylaiev
b39ca1b28e fix(TDI-43085): Bump jackson version to 2.10.1 (#4192)
* Bump http-utils lib version
2019-12-06 11:24:03 +02:00
jiezhang-tlnd
54e445bdd2 bugfix(TUP-25350)Show if for opened list: active values are not updated (#4199)
until you deselect component
https://jira.talendforge.org/browse/TUP-25350
2019-12-06 11:38:08 +08:00
Jane Ding
9cbe98d4ef feat(TUP-24849)Remove jars from the studio (#4107)
https://jira.talendforge.org/browse/TUP-24849
2019-12-05 12:13:51 +08:00
vyu-talend
26e88b26be Fix(TDI-42424):Improve for tELTPostgresqlOutput generate wrong update (#3615)
* Fix(TDI-42424):Improve for tELTPostgresqlOutput generate wrong update
query

* Fix(TDI-42424)

* Fix(TDI-42424)

* Fix(TDI-42424)

* Fix(TDI-42424):make modifications for new suquery.

* Fix(TDI-42424):fix some issues.

* Fix(TDI-42424):fix bug.

* Fix(TDI-42424):fix issue.
2019-12-05 10:43:25 +08:00
vyu-talend
6217c3042d Vyu/tdi 40599 enable use hint in merge (#4177)
* Fix(TDI-40599):A demo for current new feature.

* Fix(TDI-40599):rename 'MERGE'
2019-12-05 10:09:23 +08:00
wang wei
2c8df34e97 fix(TDI-43266): Studio returns exit code 0 when OutOfMemoryError in routine occurs (#4178)
* fix(TDI-43266): Studio returns exit code 0 when OutOfMemoryError in
routine occurs

* fix(TDI-43266): main thread should stop at once when subthread throw
outofmemory error
2019-12-05 09:31:09 +08:00
vyu-talend
86b8e75e73 Fix(TDI-43264):Fix issues. (#4182) 2019-12-03 16:33:40 +08:00
wang wei
f2bcb6d71f fix(TDI-43091): Mysql8:SSLException when run job which contain ELT component(#4181) 2019-12-03 11:41:27 +08:00
hcyi
0403d4d7d4 fix(TUP-25227):Changing the joblet name is not reflected properly. (#4188) 2019-12-02 16:07:51 +08:00
Chao MENG
4198afd1d2 fix(TUP-25344): can't logon project after change workspace if set log4j2 (#4183)
fix(TUP-25344): can't logon project after change workspace if set log4j2
https://jira.talendforge.org/browse/TUP-25344
2019-12-02 10:29:53 +08:00
wang wei
a971215151 fix(TDI-43296): Generate a sample logging file for runtime lineage cause compile error(#4185) 2019-11-29 12:03:14 +08:00
Chao MENG
14c59926c6 fix(TUP-24749): [7.2.1] issue to download component jar via https proxy (#4167)
fix(TUP-24749): [7.2.1] issue to download component jar via https proxy
https://jira.talendforge.org/browse/TUP-24749
2019-11-29 10:22:52 +08:00
bhe-talendbj
374903d266 bugfix(TUP-25257) Fix missing jobid for trunjob (#4166)
* bugfix(TUP-25257) Fix missing jobid for trunjob

* bugfix(TUP-25257) Fix missing jobid for trunjob

* bugfix(TUP-25257) Refactor

* bugfix(TUP-25257) Refactor
2019-11-28 15:06:33 +08:00
bhe-talendbj
abab035072 bugfix(TUP-25186) Revert previous pr (#4175) 2019-11-28 15:04:01 +08:00
hwang-talend
03c3e264e6 bugfix(TUP-24937):Propagate changes in tELTInput schema to tELTMap (#4165) 2019-11-28 14:34:01 +08:00
jiezhang-tlnd
948eb33247 fix(TUP-25119)add jar needed and (#4122)
* fix(TUP-25119)add jar needed and
log4j.xml
https://jira.talendforge.org/browse/TUP-25119

* fix(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119

* fix(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119

* fix(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119

* feat(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119
2019-11-28 10:11:26 +08:00
wang wei
70b86c3519 fix(TDI-43101) : support to generate a sample logging file for runtime lineage (#4164)
* fix(TDI-33262): Support for Apache Log4j2

* use release version

* fix(TDI-43101): [Research] Generate a sample logging file for runtime
lineage

* fix(TDI-43101): fix it

* fi something

* add a system property to disable this feature
2019-11-26 18:11:43 +08:00
Dmytro Sylaiev
57ef3ab3f6 fix(TDI-41651): Add where clause support for update query (#4172) 2019-11-26 10:50:30 +02:00
Jane Ding
bb0dc580a2 fix(TUP-24732)Studio Code tab doesn't generate code/show errors in Job (#4148)
https://jira.talendforge.org/browse/TUP-24732
2019-11-26 10:44:05 +08:00
Dmytro Sylaiev
2f1029eba9 fix(TDI-41651): Provide possibility to specify alias for output table… (#3645)
* fix(TDI-41651): Provide possibility to specify alias for output table ELT

* Do the same fix as mssqlELT

* Small refactoring for query codeGen

* fix(TDI-41651): Fix issues

* Correct notion in javaxml file

* Rename cleanUp method

* Use stringUtils in alias for input table

* fix(TDI-41651): Add getters for inner class

* fix(TDI-41651): Add missing property in pom.xml

* fix(TDI-41651): Add missing i18n label

* fix(TDI-41651): Remove use alias for output field

* fix(TDI-41651): Fix conflicts
2019-11-25 14:59:46 +02:00
Dmytro Sylaiev
fff59f3b2f feat(TDI-43088): Update cxf deps for msCRM and webservice components (#4131)
* feat(TDI-43088): Update cxf deps for msCRM and webservice components

* feat(TDI-43088): Update talend-ws usage of CXF
2019-11-25 13:03:45 +02:00
Chao MENG
712db2045d feat(TUP-24720): Remove component manager in the studio (#4108)
* feat(TUP-24720): Remove component manager in the studio

https://jira.talendforge.org/browse/TUP-24720

* feat(TUP-24720): Remove component manager in the studio

https://jira.talendforge.org/browse/TUP-24720
2019-11-25 17:08:52 +08:00
mbasiuk-talend
96bc7ed4c5 fix(TDI-43212): improve code generation for query string (#4149) 2019-11-25 08:49:40 +02:00
Roman
f749ac3e2a feat(TDI-42766): Redshift SSO support with Ping Federate
* feat(TDI-42766): remove code related to SSO without login/pass

* feat(TDI-42766): add migration task
2019-11-22 12:54:03 +02:00
jiezhang-tlnd
91e83eedf0 Jzhang/feat/tup 24894 automatically review changes slf4j log4j dependencies5 (#4140)
* feat(TUP-24894)add some bridge jars mostly for DB

* feat(TUP-24894)add library

* feat(TUP-24894)add some jar mostly for TBD needs

* feat(TUP-24894)add some jar mostly for DB needs

* feat(TUP-24894)add some jar mostly for DB needs

* feat(TUP-24894)add junits
2019-11-22 11:42:02 +08:00
wang wei
4d5844376e fix(TDI-43211): SharedDBConnectionLog4j use log4j1 api, should use slf4j (#4141) 2019-11-22 11:01:33 +08:00
Dmytro Sylaiev
e56ccf3f62 fix(TDI-43036): Wait for threads finish for OnComponentOK and RunIf for multithead iterate execution (#4075) 2019-11-21 12:08:15 +02:00
hzhao-talendbj
e04ef82e30 Hengzhao/bugfix/tup 25035 (#4157)
* feat(TUP-25035)Implicit tContextLoad feature (with selection from file)
uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15
2019-11-21 14:59:30 +08:00
Zhiwei Xue
273a5025c7 fix(TUP-25186):tRunJob broken after migrating project from 6.3.1 (#4145) 2019-11-21 14:41:05 +08:00
wang wei
612e5a5c27 fix(TDI-43189): Some tJDBC components use tCompV0, but they still keep the folder in localprovider(#4139) 2019-11-20 15:38:14 +08:00
vyu-talend
38f544be0a Fix(TDI-34953):Show SQL query in debug log for Db components (#4084)
* Fix(TDI-34953):issues fixed.

* Fix(TDI-34953):Fix the reopen issues(1,2,6).

* Fix(TDI-34953):Fix printing order.
2019-11-20 14:09:35 +08:00
wang wei
3b872b62e5 fix(TDI-43174): tMSSqlinput process datetimeoffset data type failed when use DST timezone as localtime (#4138) 2019-11-20 13:50:19 +08:00
wang wei
1910ed40e5 fix(TDI-39084): fail to get sap table parameter for tsapbapi and throw NPE (#4062)
* fix(TDI-39084): fail to get sap table parameter for tsapbapi and throw
NPE

Conflicts:
	main/plugins/org.talend.repository/plugin.xml

* correct the version after merge origin/master

* fix migration
2019-11-20 11:07:55 +08:00
AlixMetivier
d0912599ec csv option were not called (#4150) 2019-11-18 15:27:30 +01:00
hcyi
ee20cf02ca fix(TUP-25218):tELTXXXMap: Output table : Expression & Column set to (#4142)
blank if you add a Column without Expression on Mac OS X.
2019-11-17 22:12:43 +08:00
hcyi
5427bc53f1 fix(TUP-25099):NPE when using joblets from reference Projects. (#4132) 2019-11-17 22:07:05 +08:00
AlixMetivier
9b4bfe85be feat(TBD-9231): dataset can be used in left outer and inner (#4137)
* dataset can be used in left outer and inner

* refacto

* refacto

* wip

* Update MapperComponent.java
2019-11-15 10:59:33 +01:00
hwang-talend
b5916d9122 bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job (#4126)
bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job
2019-11-15 17:20:46 +08:00
bhe-talendbj
5e9f08c9ff bugfix(TUP-25109) redshift guess schema by executing query. (#4129) 2019-11-15 16:23:21 +08:00
Chao MENG
8e977f5110 fix(TUP-25052): File json miss source schema when after do export to (#4123)
fix(TUP-25052): File json miss source schema when after do export to context.
https://jira.talendforge.org/browse/TUP-25052
2019-11-15 15:40:47 +08:00
mbasiuk-talend
90c7f820b9 feat(TDI-41954): add commit and rollback to unified (#4103) 2019-11-15 08:39:25 +02:00
kjwang
1d5f0bacc6 Fix TUP-24842 Switch jobscript and run job get error : NoClassDefFoundError: org/talend/daikon/crypto/Encryption (#4085)
* Fix TUP-24842 Switch jobscript and run job get error :
NoClassDefFoundError: org/talend/daikon/crypto/Encryption
https://jira.talendforge.org/browse/TUP-24842

* Fix TUP-24842 Switch jobscript and run job get error :
NoClassDefFoundError: org/talend/daikon/crypto/Encryption
https://jira.talendforge.org/browse/TUP-24842
2019-11-15 14:28:54 +08:00
pyzhou
8a3ef5e9ad fix(TDI-43058):tFileFetch apply trustStore (#4061)
* fix(TDI-43058):tFileFetch apply trustStore

* Add support for Trust Store

* The password of truststore and keystore can be empty.

* fix(TDI-43058):remove catch

* Revert "fix(TDI-43058):remove catch"

This reverts commit 542547ca40.

* fix(TDI-43058):throw IOException

* fix throw error
2019-11-14 10:13:12 +08:00
AlixMetivier
47bd697fbe added override to getShouldGenerateDataset (#4135) 2019-11-13 14:52:35 +01:00
jiezhang-tlnd
926cb4f36b feat(TUP-24894) add back some jars to the end classpath list. (#4128) 2019-11-13 16:02:52 +08:00
kjwang
2ffffcc703 TUP-25101 The key store password missing of component "tSetKeyStore" (#4111)
when import into studio
https://jira.talendforge.org/browse/TUP-25101
2019-11-13 10:54:23 +08:00
Dmytro Sylaiev
c010bc5084 Dsylaiev/tdi 41794 fix null string t write json (#3258)
* fix(TDI-41794): Fix processing null strings json-lib

* fix(TDI-41794): Remove extra files

* fix(TDI-41794): Rename default standard to legacy

* fix(TDI-41794): Fix WriteJSON component to reuse library

* Add dropdown list propety

* fix(TDI-41794): Add a migration task

* chore(TDI-41794): Add missing copyright in migration class

* fix(TDI-41794): Use StringBuilders in toString methods

* Avoid of using StringBuffers where not needed

* chore(TDI-41794): Minor refactoring for JSONUtils

* fix(TDI-41794): Fix null processing for nested json

* fix(TDI-41794): Update writeJsonField dependency

* fix(TDI-41794): Fix null strings for save root node

* Fix null strings for arrays

* fix(TDI-41794): Fix null string processing for arrays with jsonObject

* fix(TDI-41794): Fix tExtractJSONField dependency

* Fix writeJsonField java.xml file format

* fix(TDI-41794): Change adv param line number

* fix(TDI-41794): Bump the library version

* fix(TDI-41794): Fix UI to use checkbox

* Change code generation to use checkbox

* Correct the lib version

* fix(TDI-41794): Fix migration task

* fix(TDI-41794): Remove debug information

* fix(TDI-41794): Fix merge conflict

* fix(TDI-41794): Correct plugin.xml file

* fix(TDI-41794): Don't wrap nulls when checkbox is hidden

* fix(TDI-41794): Fix compile error on component when JSON is array
2019-11-12 15:51:30 +02:00
wang wei
3dc2aac9bb fix(TDI-43167): add back slf4j bridge for special components for the unactive log4j case.(#4127) 2019-11-12 14:55:04 +08:00
apoltavtsev
9f7ad6f9dd TESB-27110 Missing tSendMail dependencies are added 2019-11-08 10:22:57 +01:00
wang wei
7426a93b49 fix(TDI-33262): Support for Apache Log4j2 (#3970)
* fix(TDI-33262): Support for Apache Log4j2

* use release version
2019-11-07 18:04:47 +08:00
jiezhang-tlnd
c3e5d003be commons-logging.jar need not to remove (#4109) 2019-11-07 10:58:19 +08:00
SunChaoqun
cf5ea4e526 TESB-27309:During the publishing with CI feature we got an error of (#4097)
generating context
2019-11-07 10:25:27 +08:00
hwang-talend
4914a382fc bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job (#4099)
bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job
2019-11-07 10:15:12 +08:00
jiezhang-tlnd
30caab1c84 feat(TUP-24894)Automatically review / changes slf4j/log4j dependencies (#4106) 2019-11-06 17:35:37 +08:00
pyzhou
da989e3f8c fix(TDI-43105):tmap jdbc join timestamp (#4092) 2019-11-06 17:13:37 +08:00
vyu-talend
77754871b5 Fix(TDI-43073&TDI-42135):fix some issues. (#4067)
* Fix(TDI-43073&TDI-42135):fix some issues.

* Fix(TDI-43073):fix some issues

* Fix(TDI-43073):Fix for reopen.

* Fix(TDI-43073):recommit the omitted file.
2019-11-06 17:11:45 +08:00
Jane Ding
b024dd0ec0 Jding/tup 24732 studio code tab doesn't show error (#4094)
* fix(TUP-24732)Studio Code tab doesn't generate code/show errors in Job
https://jira.talendforge.org/browse/TUP-24732

* fix(TUP-24732)Studio Code tab doesn't generate code/show errors in Job
https://jira.talendforge.org/browse/TUP-24732

* fix(TUP-24732)[7.2.1] Studio Code tab doesn't generate code/show errors
in Job
https://jira.talendforge.org/browse/TUP-24732
2019-11-06 10:52:11 +08:00
chmyga
b4d3e5ebea fix(TDI-41358): Date pattern readonly (#3161)
* Make date pattern readonly if type is not date
2019-11-05 12:47:59 +02:00
jiezhang-tlnd
29f74206d5 feat(TUP-24894)Automatically review / changes slf4j/log4j dependencies (#4068)
* feat(TUP-24894)Automatically review / changes slf4j/log4j dependencies
https://jira.talendforge.org/browse/TUP-24894

* fix(TUP-24894)

* format code

* add the condition for esb

* add junits

* remove esb filter
2019-11-05 18:00:45 +08:00
clesaec
445c9fc74b TDI-42900 : usage of Thread.contextClassLoader (get/set) instead of S… (#3877)
* TDI-42900 : usage of Thread.contextClassLoader (get/set) instead of SystemClassLoader (with no cast, nor protected method call)
2019-11-05 10:57:20 +01:00
AlixMetivier
9f0b02c045 fix(TBD-9492):fix condition that broke graphical build of nodes (#4100)
* fix condition that broke graphical build of nodes

* added comments
2019-11-04 11:20:09 +01:00
Max
e3f4be3219 TBD-9462:fixed (#4079) 2019-11-04 11:18:37 +02:00
kjwang
dc47689710 kjwang/Fix TUP-24221 Fix a junit (#4088)
https://jira.talendforge.org/browse/TUP-24221
2019-11-04 17:04:17 +08:00
Chao MENG
936cf7958f fix(TUP-20939): Actions are called twice due 2 sets of ElementParameters (#4096)
https://jira.talendforge.org/browse/TUP-20939
2019-11-04 17:02:11 +08:00
hwang-talend
8f842b2155 bugfix(TUP-24937):Propagate changes in tELTInput schema to tELTMap (#4086)
bugfix(TUP-24937):Propagate changes in tELTInput schema to tELTMap
2019-11-04 14:31:05 +08:00
hcyi
db23bf8ec1 fix(TUP-24625):Configuration migration doesn't work for tacokit (#4083)
components.
2019-11-04 12:01:01 +08:00
jiezhang-tlnd
99ba1911f3 fix(TUP-24786)Jobs are failing with the error "Cannot find or load the (#4077)
main class" when imported the project export from 6.3 to 7.2
https://jira.talendforge.org/browse/TUP-24786
2019-11-04 11:52:36 +08:00
Roman
24ef38e2da fix(TDI-42015): change label for hex and octal numbers (#4078) 2019-10-29 16:50:57 +02:00
AlixMetivier
190acc37f8 refacto (#4081) 2019-10-29 14:34:01 +01:00
AlixMetivier
c9dd09c733 fix(TBD-9372): fix code generation error when tMap has only one input (#4080)
* methods added to know if tMap can work in dataset or not

* Update MapperComponent.java

remove commented code

* refacto

* refacto

* delete commented code

* refacto

* fix case when there is only one input to tMap

* Update MapperComponent.java
2019-10-29 14:25:54 +01:00
AlixMetivier
263b021fca fix(TBD-9372): methods added to know if tMap can work in dataset or not (#4074)
* methods added to know if tMap can work in dataset or not

* Update MapperComponent.java

remove commented code

* refacto

* refacto

* delete commented code

* refacto
2019-10-28 14:26:52 +01:00
Zhiwei Xue
76fbb18558 fix(TUP-24623):fix regression. (#4064) 2019-10-28 15:39:39 +08:00
bhe-talendbj
d89f4d424f bugfix(TUP-24923) Commandline can't execute imported job (#4060)
* bugfix(TUP-24923) Commandline can't execute imported job

* bugfix(TUP-24923) Fix junit
UpgradePasswordEncryptionAlg4ItemMigrationTaskTest
2019-10-28 10:11:04 +08:00
hcyi
c8b397b4a6 fix(TUP-24708): improve for Tacokit exists Help Content . (#4059) 2019-10-25 18:54:44 +08:00
Chao MENG
bd0d7e0afd fix(TBD-9366): tELTHiveMap doesn't pass context variable properly (#4066)
fix(TBD-9366): tELTHiveMap doesn't pass context variable properly
https://jira.talendforge.org/browse/TBD-9366
2019-10-25 18:51:55 +08:00
SunChaoqun
f36de6e55a TESB-27139:"Could not find the context Default" is shown in runtime when (#4056)
* TESB-27139:"Could not find the context Default" is shown in runtime when
deploy a route into runtime

* TESB-27139:"Could not find the context Default" is shown in runtime when
deploy a route into runtime

* TESB-27139:"Could not find the context Default" is shown in runtime when
deploy a route into runtime
2019-10-25 18:48:40 +08:00
Mike Yan
a5a41af6cf fix (TESB-25283): backport docker esb fixes to master (#3844)
* fix(TESB-24597): Fixed OSGi job build missing OSGI-INF folder issue

* TESB-26736:DS OSGi bundles are failing on runtime after the docker patch

* fix (TESB-25283): backport docker esb fixes to master

* fix(TESB-27046): Job run failed with pure DI license in studio
2019-10-25 16:02:10 +08:00
hwang-talend
c5b23dd28f bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job (#4038)
bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job
2019-10-24 10:17:11 +08:00
clesaec
fec8a84ca1 TDI-42493 : correct versions (#4063) 2019-10-23 13:50:25 +02:00
slushatel
f77ff96de2 Sbovsunovskyi/tdi 43065 t dtd validator and utf8 (#4034)
* fix(TDI-43065): tDTDValidator fails with tDTDValidator_1 Invalid byte 2 of 3-byte UTF-8 sequence when XML file encoded in UTF-8 contains multi-bytes characters

* fix(TDI-43065): tDTDValidator fails with tDTDValidator_1 Invalid byte 2 of 3-byte UTF-8 sequence when XML file encoded in UTF-8 contains multi-bytes characters

* fix(TDI-43065): tDTDValidator fails with tDTDValidator_1 Invalid byte 2 of 3-byte UTF-8 sequence when XML file encoded in UTF-8 contains multi-bytes characters. Change default encoding to system's 'file.encoding'
2019-10-23 11:04:21 +03:00
clesaec
df4e255819 TDI-42493 : comment to english, automapping ok (#3933)
* TDI-42493 : comment to english, automapping ok, add transformation
2019-10-23 08:44:05 +02:00
kjwang
5cf8a97e87 kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading (#4058)
2
https://jira.talendforge.org/browse/TUP-24212
2019-10-23 14:29:36 +08:00
hcyi
792b1ccc01 fix(TUP-24885):tELTTeradataMap: Output table : Expression & Column set (#4048)
to blank if you add a Column without Expression.
2019-10-23 10:09:53 +08:00
Redwene
6507179daa fix(TBD-9366) : tELTHiveMap doesn't pass context variable properly (#4024)
* fix(TBD-9366) : typo and blank line

* fix(TBD-9366) : remove commented code

* fix(TBD-9366) : tELTHiveMap doesn't pass context variable properly

* fix(TBD-9366): tELTHiveMap doesn't pass context variable properly
2019-10-22 15:00:39 +02:00
Dmytro Sylaiev
3627e9fc1f fix(TDI-40977): Fix https proxy usage for custom mode (#4052) 2019-10-22 13:28:07 +03:00
SunChaoqun
572f642911 TESB-27139:"Could not find the context Default" is shown in runtime when (#4032)
* TESB-27139:"Could not find the context Default" is shown in runtime when
deploy a route into runtime

* TESB-27139:"Could not find the context Default" is shown in runtime when
deploy a route into runtime
2019-10-22 11:20:30 +08:00
hcyi
e06da9671f fix(TUP-24917):Upgrade tacokit to 1.1.14. (#4050) 2019-10-22 10:11:18 +08:00
Mike Yan
dc1ec06b42 fix(TESB-27030): Exclude jms library when it is used from tRunJob (#4026)
* fix(TESB-27030): Avoid to add activemq-all lib in OSGi bundle

* fix(TESB-27030): Exclude jms library when it is used from tRunJob
2019-10-21 15:47:18 +08:00
Chao MENG
7dfac8dc4f fix(TUP-24148): tNetsuiteInput throws cast error when switching from use (#4036)
fix(TUP-24148): tNetsuiteInput throws cast error when switching from use tNetsuiteConnection to use this component
https://jira.talendforge.org/browse/TUP-24148
2019-10-21 10:43:58 +08:00
hwang-talend
f398c2d5f8 bugfix(TUP-24355):Default Context environment appended after migrating (#4037)
bugfix(TUP-24355):Default Context environment appended after migrating
2019-10-18 17:57:26 +08:00
Zhiwei Xue
67a155faa3 fix(TUP-24623):Compilation error on the job with project references (#4010) 2019-10-18 15:54:20 +08:00
zshen-talend
8fe63a7afc feat(TDQ-17432): combined all masking components to use one migration (#4028)
* feat(TDQ-17432): combined all masking components to use one migration

* feat(TDQ-17432): support MR and streaming job
2019-10-18 14:28:08 +08:00
kjwang
52665b6347 kjwang/Fix a compile error for TUP-21518 (#4035) 2019-10-18 12:00:13 +08:00
hzhao-talendbj
1dfd7ea0ec fix(TUP-24613)routine section does not allow user to edit required (#4011)
checkbox
2019-10-18 10:06:53 +08:00
bhe-talendbj
7b35b37054 feat(TUP-21518) Remove CryptoHelper (#3812)
* feat(TUP-21518) Remove CryptoHelper

feat(TUP-21518) Remove CryptoHelper

feat(TUP-21518) Revert changes of migration

feat(TUP-21518) Refactor migration code

feat(TUP-21518) Rename AESEncryption

feat(TUP-21518) Refactor migration code

feat(TUP-21518) Format source code

feat(TUP-21518) Format source code

* feat(TUP-21518) Uptake new daikon lib and format source code

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Should encrypt data with aes

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Fix upgradePassowrdEncryptionAlg test

* fix(TDQ-16616): refactor the code to make it reuseable

* fix(TDQ-16616): refactor the code to make it reuseable

* feat(TUP-21518) Fix TUP-24597 and TUP-24611

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Fix JDBC migration

* kjwang/Fix TUP-24212 Create migration for CryptoHelper to AES upgrading
https://jira.talendforge.org/browse/TUP-24212

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Fix format of source code

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments

* feat(TUP-21518) Resolve comments
2019-10-18 09:36:36 +08:00
Dmytro Sylaiev
d99fd9c2e5 Dsylaiev/tdi 40977 add setting local properties for t set proxy (#3899)
* chore(TDI-42791): Refactor creation connection ftp

* chore(TDI-42791): Fix compile error after refactoring

* feat(TDI-40977): Add use global or local proxy properties

* feat(TDI-40977): Change UI in tSetProxy

* feat(TDI-40977): Add lib dependency to the tSetProxy component

* feat(TDI-40977): Save default proxy selector to use job-scope

* feat(TDI-40977): Modify tSetProxy component to reuse talend-proxy lib

* feat(TDI-40977): Disable FTP proxy when not global settings used

* Add http-proxy support for SFTP (as an example)

* feat(TDI-40977): Apply proxy feature for FTPS and SFTP connection

* feat(TDI-40977): Move thread-specific to advanced properties

* fix(TDI-40977): Fix footer conflicts

* chore(TDI-40977): Fix indentation

* feat(TDI-40977): Include proxy jar to studio build

* feat(TDI-40977): Fix incorrect master merge

* feat(TDI-40977): Fix proxy lib dependency for ftp components

* feat(TDI-40977): Fix tSetProxy ui row numbers

* feat(TDI-40977): Fix proxy selection for FTPS and SFTP

* feat(TDI-40977): Support basic https auth

* feat(TDI-40977): Fix request old apache-commons lib
2019-10-17 13:29:05 +03:00
Dmytro Sylaiev
072795ea1b Add talend proxy library (#3876)
* feat(TDI-40977): Add talend-proxy library

* feat(TDI-40977): Correct dep conflict with loggers

* feat(TDI-40977): Add posibility to use all-threads setting

* feat(TDI-40977): Add method to get proxy

* feat(TDI-40977): Support port -1 as port mask

* feat(TDI-40977): Remove redundant optional in interface

* feat(TDI-40977): Make library thread-safe

* feat(TDI-40977): Unsnaphot version

* fix(TDI-40977): Fix arrayindexofbounds when port is not specified

* feat(TDI-40977): Fix thread-scoped issue

* fix(TDI-40977): Fix NPE on java11 with commons-lang
2019-10-17 11:37:15 +03:00
jiezhang-tlnd
ceb555dc51 feat(TUP-24819)change brandings for Talend 7.3.1 (#4027) 2019-10-17 15:07:15 +08:00
hwang-talend
67fb0f4d12 bugfix(TUP-24728):Date and Time is not displaying when the Preferred (#4000)
* bugfix(TUP-24728):Date and Time is not displaying when the Preferred
Language is French

* bugfix(TUP-24728):Date and Time is not displaying when the Preferred
Language is French

* bugfix(TUP-24728):Date and Time is not displaying when the Preferred
Language is French

* bugfix(TUP-24728):Date and Time is not displaying when the Preferred
Language is French
2019-10-16 10:46:12 +08:00
slushatel
47deaa8326 fix(TDI-43010): [ADLSgen2] Short term solution for columns selector i… (#3979)
* fix(TDI-43010): [ADLSgen2] Short term solution for columns selector in Studio. Hide 'CSV schema'field

* fix(TDI-43010): [ADLSgen2] Short term solution for columns selector in Studio. Hide 'CSV schema'field

* fix(TDI-43010): [ADLSgen2] Short term solution for columns selector in Studio. Hide 'CSV schema'field. Refactoring

* fix(TDI-43010): [ADLSgen2] Short term solution for columns selector in Studio. Hide 'CSV schema'field. Add Output component check
2019-10-15 18:22:49 +03:00
hcyi
94128e4cbf fix(TUP-24784):Build error with tRunJob on a spark job. (#4025) 2019-10-15 19:38:58 +08:00
vyu-talend
935b6c3996 Fix(TDI-41289):Add reuse tS3Connection. (#3969)
* Fix(TDI-41289):Add reuse tS3Connection.

* Fix(TDI-41289):Fix a bug in UI.

* Fix(TDI-41289):Fixed can't not use existing connection and Assume Role
at the same time.

* Fix(TDI-41289):Make Encrypt hidden when use existing connection.

* Fix(TDI-41289):Make the code more elegant.

* Fix(TDI-41289):optimize code and fix UI issues.

* Fix(TDI-41289)

* Fix(TDI-41289):Fix the presentation logic of Encrypt.

* Fix(TDI-41289):Fix a logical issue in tROBE
2019-10-15 15:54:13 +08:00
hwang-talend
08b538ff57 bugfix(TUP-24563):Specifying a custom MVN URI in a Bean still ends up (#4015)
bugfix(TUP-24563):Specifying a custom MVN URI in a Bean still ends up turning into a reference to the default 6.0.0-SNAPSHOT version
2019-10-14 16:50:18 +08:00
Redwene
033fd28354 Revert " fix(TBD-9366): tELTHiveMap doesn't pass context variable properly (#4013)" (#4020)
This reverts commit 00505282ac.
2019-10-14 10:46:56 +02:00
Redwene
00505282ac fix(TBD-9366): tELTHiveMap doesn't pass context variable properly (#4013)
* fix(TBD-9366) : typo and blank line

* fix(TBD-9366) : remove commented code

* fix(TBD-9366) : tELTHiveMap doesn't pass context variable properly
2019-10-14 09:20:39 +02:00
Jane Ding
c33d8c379b fix(TUP-24540)java.lang.NoSuchMethodError: (#4018)
org.slf4j.spi.LocationAwareLogger.log
https://jira.talendforge.org/browse/TUP-24540
2019-10-14 11:04:35 +08:00
hwang-talend
64a0bf8012 bugfix(TUP-23100):Add warning in the tRunjob for recursive jobs (#3974)
bugfix(TUP-23100):Add warning in the tRunjob for recursive jobs
2019-10-12 14:59:09 +08:00
jiezhang-tlnd
195eaa06e7 fix(TUP-24809)log4j2 should not be selected an enable when log4j is (#4008)
inactive in project setting
https://jira.talendforge.org/browse/TUP-24809
2019-10-12 11:36:05 +08:00
kjwang
d59aa3ca97 kjwang/Fix TUP-24748 CLONE - Modification in Context Variables is not (#4009)
propagating to all jobs
https://jira.talendforge.org/browse/TUP-24748
2019-10-12 11:27:07 +08:00
hcyi
21c18c0374 fix(TUP-24800):Upgrade tacokit to release version of 1.1.13 (#4005) 2019-10-12 11:15:32 +08:00
pyzhou
4d5a9e18b8 fix(TDI-43053):unify mvn uri (#4014) 2019-10-12 11:12:56 +08:00
hzhao-talendbj
d7b6cfecf2 fix(TUP-24680)Context variable type disappears after changing default (#3984)
* fix(TUP-24680)Context variable type disappears after changing default
context environment

* fix(TUP-24680)Context variable type disappears after changing default
context environment

* fix(TUP-24680)Context variable type disappears after changing default
context environment

* fix(TUP-24680)Context variable type disappears after changing default
context environment
2019-10-11 11:09:43 +08:00
jiezhang-tlnd
08c045ee3a fix(TUP-24704)TOS 7.3.1: after import a 7.2.1 project context disappears in Java Debug (#3996)
* fix(TUP-24704)TOS 7.3.1: after import a 7.2.1 project context disappears
in Java Debug
https://jira.talendforge.org/browse/TUP-24740

* fix(TUP-24740)TOS 7.3.1: after import a 7.2.1 project context disappears
in Java Debug
https://jira.talendforge.org/browse/TUP-24740
2019-10-10 19:19:19 +08:00
614 changed files with 10432 additions and 6550 deletions

57
PATCH_RELEASE_NOTE.md Normal file
View File

@@ -0,0 +1,57 @@
---
version: 7.1.1
module: https://talend.poolparty.biz/coretaxonomy/42
product:
- https://talend.poolparty.biz/coretaxonomy/23
---
# TPS-3690
| Info | Value |
| ---------------- | ---------------- |
| Patch Name | Patch\_20200113\_TPS\-3690\_v1\-7.1.1 |
| Release Date | 2020-01-13 |
| Target Version | 20181026_1147-V7.1.1 |
| Product affected | Talend Studio |
## Introduction
This is a self-contained patch.
**NOTE**: For information on how to obtain this patch, reach out to your Support contact at Talend.
## Fixed issues
This patch contains the following fixes:
- TPS-3690 [7.1.1]tMap temp data with dynamic field error(TDI-43446)
## Prerequisites
Consider the following requirements for your system:
- Talend Studio 7.1.1 must be installed.
## Installation
### Installing the patch using Software update
1) Logon TAC and switch to Configuration->Software Update, then enter the correct values and save referring to the documentation: https://help.talend.com/reader/f7Em9WV_cPm2RRywucSN0Q/j9x5iXV~vyxMlUafnDejaQ
2) Switch to Software update page, where the new patch will be listed. The patch can be downloaded from here into the nexus repository.
3) On Studio Side: Logon Studio with remote mode, on the logon page the Update button is displayed: click this button to install the patch.
### Installing the patch using Talend Studio
1) Create a folder named "patches" under your studio installer directory and copy the patch .zip file to this folder.
2) Restart your studio: a window pops up, then click OK to install the patch, or restart the commandline and the patch will be installed automatically.
### Installing the patch using Commandline
Execute the following commands:
1. Talend-Studio-win-x86_64.exe -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002 --talendDebug
2. initRemote {tac_url} -ul {TAC login username} -up {TAC login password}
3. checkAndUpdate -tu {TAC login username} -tup {TAC login password}

View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>org.talend.help.tos.components.feature</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.eclipse.pde.FeatureBuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.eclipse.pde.FeatureNature</nature>
</natures>
</projectDescription>

View File

@@ -0,0 +1 @@
bin.includes = feature.xml

View File

@@ -0,0 +1,94 @@
<?xml version="1.0" encoding="UTF-8"?>
<feature
id="org.talend.help.tos.components.feature"
label="Talend COMPONENTS help feature"
version="7.3.1.qualifier"
provider-name=".Talend SA.">
<description url="http://www.example.com/description">
[Enter Feature Description here.]
</description>
<copyright url="http://www.example.com/copyright">
[Enter Copyright Description here.]
</copyright>
<license url="http://www.example.com/license">
[Enter License Description here.]
</license>
<url>
<update label="CORE Talend update site for milestones" url="http://talendforge.org/core/updatesite/testing/"/>
</url>
<plugin
id="org.talend.help.azurestorage"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.filedelimited"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.filterrow"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.googledrive"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.jdbc"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.jira"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.marketo"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.salesforce"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.snowflake"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.splunk"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
</feature>

View File

@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.talend.studio</groupId>
<artifactId>tdi-studio-se</artifactId>
<version>7.3.1-SNAPSHOT</version>
<relativePath>../../../</relativePath>
</parent>
<artifactId>org.talend.help.tos.components.feature</artifactId>
<packaging>eclipse-feature</packaging>
</project>

View File

@@ -9,6 +9,7 @@
</url>
<requires>
<import feature="org.talend.tos.feature" version="0.0.0" match="greaterOrEqual"/>
<import feature="org.talend.help.di.feature" version="0.0.0" match="greaterOrEqual"/>
</requires>
<plugin id="org.talend.designer.maven.tos" download-size="0" install-size="0" version="0.0.0" fragment="true"/>
<plugin id="org.talend.presentation.onboarding" download-size="0" install-size="0" version="0.0.0" unpack="false"/>

View File

@@ -53,16 +53,6 @@
<plugin id="org.talend.designer.xmlmap.nl" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="org.talend.expressionbuilder" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.expressionbuilder.nl" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="org.talend.help.azurestorage" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.filedelimited" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.filterrow" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.googledrive" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.jdbc" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.jira" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.marketo" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.salesforce" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.snowflake" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.splunk" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.repository.generic" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.sdk.component.studio-integration" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
</feature>

View File

@@ -329,14 +329,26 @@ end_Hash.put("<%=node.getUniqueName() %>", System.currentTimeMillis());
}
// Else, that means the component is a ON_COMPONENT_OK virtual component typed.
}
if(generateMethodCall) {
/*check if parralel iterate call not finished from this component */
boolean parallelIterate = false;
for (IConnection iterateConn : iterateConnSet) {
parallelIterate = "true".equals(ElementParameterParser.getValue(iterateConn, "__ENABLE_PARALLEL__"));
if (parallelIterate) {
if (codePart.equals(ECodePart.END)) {
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
%>
mtp_<%=iterateTargetNodeName %>.waitForEndOfQueue();
<%
}
}
}
%>
<%=outgoingConn.getTarget().getUniqueName() %>Process(globalMap);
<%
}
}
}
if (outgoingConn.getLineStyle().equals(EConnectionType.RUN_IF)) {
%>
@@ -344,12 +356,27 @@ end_Hash.put("<%=node.getUniqueName() %>", System.currentTimeMillis());
<%//send the true status to socket
if(stat){
%>
if(execStat){
if(execStat){
runStat.updateStatOnConnection("<%=outgoingConn.getUniqueName() %>", 0, "true");
}
<%}%>
<%=outgoingConn.getTarget().getUniqueName() %>Process(globalMap);
}
<%
}
/*check if parralel iterate call not finished from this component */
boolean parallelIterate = false;
for (IConnection iterateConn : iterateConnSet) {
parallelIterate = "true".equals(ElementParameterParser.getValue(iterateConn, "__ENABLE_PARALLEL__"));
if (parallelIterate) {
if (codePart.equals(ECodePart.END)) {
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
%>
mtp_<%=iterateTargetNodeName %>.waitForEndOfQueue();
<%
}
}
}
%>
<%=outgoingConn.getTarget().getUniqueName() %>Process(globalMap);
}
<%
//send the false status to socket
@@ -376,7 +403,7 @@ end_Hash.put("<%=node.getUniqueName() %>", System.currentTimeMillis());
parallelIterate = "true".equals(ElementParameterParser.getValue(iterateConn, "__ENABLE_PARALLEL__"));
if (parallelIterate) {
if (codePart.equals(ECodePart.END)) {
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
%>
mtp_<%=iterateTargetNodeName %>.waitForEndOfQueue();

View File

@@ -54,7 +54,9 @@
jobFolderName = jobFolderName + JavaResourcesHelper.getJobFolderName(process.getName(), process.getVersion());
String jobClassPackageFolder = codeGenArgument.getCurrentProjectName().toLowerCase() + '/' + jobFolderName;
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J_ACTIVATE__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J_ACTIVATE__"));//log4j enable
boolean isLog4j2Enabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J2_ACTIVATE__"));//log4j2 enable
boolean isLog4j1Enabled = !isLog4j2Enabled;//log4j1 enable
boolean exist_tParallelize = false;
List<? extends INode> tParallelizeList = process.getNodesOfType("tParallelize");
@@ -449,6 +451,7 @@
<%if(isLog4jEnabled){%>
if(!"".equals(log4jLevel)){
<%if(isLog4j1Enabled){%>
if("trace".equalsIgnoreCase(log4jLevel)){
log.setLevel(org.apache.log4j.Level.TRACE);
}else if("debug".equalsIgnoreCase(log4jLevel)){
@@ -465,6 +468,26 @@
log.setLevel(org.apache.log4j.Level.OFF);
}
org.apache.log4j.Logger.getRootLogger().setLevel(log.getLevel());
<%}%>
<%if(isLog4j2Enabled){%>
if("trace".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.TRACE);
}else if("debug".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.DEBUG);
}else if("info".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.INFO);
}else if("warn".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.WARN);
}else if("error".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.ERROR);
}else if("fatal".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.FATAL);
}else if ("off".equalsIgnoreCase(log4jLevel)){
org.apache.logging.log4j.core.config.Configurator.setLevel(log.getName(), org.apache.logging.log4j.Level.OFF);
}
org.apache.logging.log4j.core.config.Configurator.setLevel(org.apache.logging.log4j.LogManager.getRootLogger().getName(), log.getLevel());
<%}%>
}
log.info("TalendJob: '<%=codeGenArgument.getJobName()%>' - Start.");
<%}%>
@@ -499,6 +522,13 @@
.filter(it -> it.startsWith("monitoring.audit.logger.properties."))
.forEach(key -> properties_<%=jobCatcherNode.getUniqueName()%>.setProperty(key.substring("monitoring.audit.logger.properties.".length()), System.getProperty(key)));
<%if(isLog4j1Enabled) {%>
org.apache.log4j.Logger.getLogger("audit").setLevel(org.apache.log4j.Level.DEBUG);
<%}%>
<%if(isLog4j2Enabled) {%>
org.apache.logging.log4j.core.config.Configurator.setLevel("audit", org.apache.logging.log4j.Level.DEBUG);
<%}%>
auditLogger_<%=jobCatcherNode.getUniqueName()%> = org.talend.job.audit.JobEventAuditLoggerFactory.createJobAuditLogger(properties_<%=jobCatcherNode.getUniqueName()%>);
}
<%
@@ -773,6 +803,25 @@ if(execStat) {
startTime = System.currentTimeMillis();
<%
boolean isCustomProxySettingsUsed = false;
List<? extends INode> proxyNodes = process.getNodesOfType("tSetProxy");
for (int i = 0; i < proxyNodes.size(); i++) {
INode proxyNode = proxyNodes.get(i);
String useCustomProxyCheckboxValue = ElementParameterParser.getValue(proxyNode, "__PROXY_GLOBAL__");
if (useCustomProxyCheckboxValue != null && !"true".equals(useCustomProxyCheckboxValue)) {
isCustomProxySettingsUsed = true;
break;
}
}
if (isCustomProxySettingsUsed) {
%>
java.net.ProxySelector proxySelectorOnStartJob = java.net.ProxySelector.getDefault();
<%
}
//1. send the begin msg
for (INode statCatcherNode : process.getNodesOfType("tStatCatcher")) {
@@ -995,6 +1044,13 @@ this.globalResumeTicket = true;//to run tPostJob
<%= ElementParameterParser.getValue(process, "__FOOTER_CODE__")%>
<%
if (isCustomProxySettingsUsed) {
%>
java.net.ProxySelector.setDefault(proxySelectorOnStartJob);
<%
}
%>
end = System.currentTimeMillis();
if (watch) {

View File

@@ -141,7 +141,9 @@ for (INode node : process.getNodesOfType("tRESTClient")) {
boolean talendEsbJobFactory = actAsProvider || !process.getNodesOfType("tRouteInput").isEmpty();
boolean talendEsbJob = talendEsbJobFactory || actAsConsumer || ProcessorUtilities.isEsbJob(process);
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J_ACTIVATE__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J_ACTIVATE__"));//log4j enable
boolean isLog4j2Enabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J2_ACTIVATE__"));//log4j2 enable
boolean isLog4j1Enabled = !isLog4j2Enabled;//log4j1 enable
if (talendMdmJob) {
talendJobInterfaces += ", TalendMDMJob"; // Talend MDM job
@@ -156,7 +158,14 @@ if (talendEsbJobFactory) {
public class <%=className%> implements <%=talendJobInterfaces%> {
<%if(isLog4jEnabled){%>
static {System.setProperty("TalendJob.log", "<%=className%>.log");}
<%if(isLog4j1Enabled){%>
private static org.apache.log4j.Logger log = org.apache.log4j.Logger.getLogger(<%=className%>.class);
<%}%>
<%if(isLog4j2Enabled){%>
private static org.apache.logging.log4j.Logger log = org.apache.logging.log4j.LogManager.getLogger(<%=className%>.class);
<%}%>
<%}%>
protected static void logIgnoredError(String message, Throwable cause) {

View File

@@ -94,7 +94,7 @@ public class JavaRoutineSynchronizer extends AbstractRoutineSynchronizer {
private void syncRoutineItems(Collection<RoutineItem> routineObjects, boolean forceUpdate) throws SystemException {
for (RoutineItem routineItem : routineObjects) {
syncRoutine(routineItem, true, true, forceUpdate);
syncRoutine(routineItem, true, forceUpdate);
}
syncSystemRoutine(ProjectManager.getInstance().getCurrentProject());
}

View File

@@ -1,88 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.bigdata/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="GoogleStorageUtils.jar" />
<property name="component.name" value="tGSPut" />
<property name="author.name" value="jzhao" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
<pathelement location="${component.plugin.home}/tBigQueryBulkExec/jets3t-0.9.0.jar" />
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -1,127 +0,0 @@
package org.talend.gs.util;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Set;
import org.jets3t.service.model.GSObject;
import org.jets3t.service.utils.Mimetypes;
public class GSObjectUtil {
public List<GSObject> genGSObjectList(List<GSObject> objects, File file,
String keyParent, boolean isGenFileObject, boolean isGenFolderObject)
throws NoSuchAlgorithmException, IOException {
if (file.isDirectory()) {
if (keyParent != null && !"".equals(keyParent)) {
if (keyParent.trim().lastIndexOf("/") != keyParent.trim()
.length() - 1) {
keyParent = keyParent + "/";
}
if(isGenFolderObject){
objects.add(new GSObject(keyParent));
}
}
File[] files = file.listFiles();
for (File f : files) {
if (f.isDirectory()) {
objects = genGSObjectList(objects, f,
keyParent + f.getName() + "/", isGenFileObject,
isGenFolderObject);
} else {
objects = genGSObjectList(objects, f,
keyParent + f.getName(), isGenFileObject,
isGenFolderObject);
}
}
} else {
if (isGenFileObject) {
GSObject obj = new GSObject(file);
obj.setKey(keyParent);
objects.add(obj);
}
}
return objects;
}
public java.util.Map<String,String> genFileFilterList(java.util.List<java.util.Map<String,String>> list,String localdir,String remotedir){
if (remotedir != null && !"".equals(remotedir)) {
if (remotedir.trim().lastIndexOf("/") != remotedir.trim()
.length() - 1) {
remotedir = remotedir + "/";
}
}
java.util.Map<String,String> fileMap=new HashMap<String,String>();
for (java.util.Map<String, String> map : list) {
java.util.Set<String> keySet = map.keySet();
for (String key : keySet){
String tempdir = localdir;
String filemask = key;
String dir = null;
String mask = filemask.replaceAll("\\\\", "/") ;
int i = mask.lastIndexOf('/');
if (i!=-1){
dir = mask.substring(0, i);
mask = mask.substring(i+1);
}
if (dir!=null && !"".equals(dir)) tempdir = tempdir + "/" + dir;
mask = mask.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*");
final String finalMask = mask;
java.io.File[] listings = null;
java.io.File file = new java.io.File(tempdir);
if (file.isDirectory()) {
listings = file.listFiles(new java.io.FileFilter() {
public boolean accept(java.io.File pathname) {
boolean result = false;
if (pathname != null && pathname.isFile()) {
result = java.util.regex.Pattern.compile(finalMask).matcher(pathname.getName()).find();
}
return result;
}
});
}
if(listings == null || listings.length <= 0){
System.err.println("No match file("+key+") exist!");
}else{
String localFilePath = "";
String newObjectKey = "";
for (int m = 0; m < listings.length; m++){
if (listings[m].getName().matches(mask)){
localFilePath = listings[m].getAbsolutePath();
if(map.get(key)!=null && map.get(key).length()>0){
newObjectKey = remotedir+map.get(key);
}else{
newObjectKey = remotedir+listings[m].getName();
}
fileMap.put(localFilePath, newObjectKey);
}
}
}
}
}
return fileMap;
}
public List<GSObject> genObjectByFileMap(java.util.Map<String,String> fileMap) throws Exception, IOException{
Set<String> localFiles=fileMap.keySet();
List<GSObject> objects=new ArrayList<GSObject>();
for(String localFilePath:localFiles){
GSObject object=new GSObject(new File(localFilePath));
object.setKey(fileMap.get(localFilePath));
objects.add(object);
}
return objects;
}
public void initMimeTypes() throws IOException {
InputStream mimetypesFile = this.getClass().getResourceAsStream(
"/resource/mime.types");
Mimetypes.getInstance().loadAndReplaceMimetypes(mimetypesFile);
}
}

View File

@@ -4,7 +4,7 @@
<groupId>com.microsoft.azure</groupId>
<artifactId>adal4j</artifactId>
<version>1.1.1-patch</version>
<version>1.1.1-20191012</version>
<packaging>jar</packaging>
<name>adal4j</name>
<description>
@@ -105,7 +105,7 @@
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>

View File

@@ -33,10 +33,15 @@
<version>3.0.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
<version>1.7.25</version>
</dependency>
<!-- Spring 3 dependencies -->
<dependency>

View File

@@ -27,7 +27,8 @@ import java.util.List;
import javax.servlet.http.HttpServletRequest;
import org.apache.commons.lang3.text.WordUtils;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
@@ -41,7 +42,7 @@ import org.json.JSONObject;
*/
public class JSONHelper {
private static Logger logger = Logger.getLogger(JSONHelper.class);
private static Logger logger = LoggerFactory.getLogger(JSONHelper.class);
JSONHelper() {
// PropertyConfigurator.configure("log4j.properties");

View File

@@ -1,88 +1,88 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="external_sort.jar" />
<property name="component.name" value="tSortOut" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="external_sort.jar" />
<property name="component.name" value="tSortOut" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -0,0 +1,88 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (C) 2006-2020 Talend Inc. - www.talend.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components</groupId>
<artifactId>google-storage-utils</artifactId>
<version>1.0.0</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
</properties>
<name>Google Storage Utils</name>
<description>Talend Helper library for Google Storage</description>
<url>
https://github.com/Talend/tdi-studio-se/
</url>
<licenses>
<license>
<name>Apache-2.0</name>
<url>
http://www.talendforge.org/modules/licenses/APACHE_v2.txt
</url>
</license>
</licenses>
<dependencies>
<dependency>
<groupId>net.java.dev.jets3t</groupId>
<artifactId>jets3t</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
</repository>
</distributionManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
<resource>
<directory>src/main/resources</directory>
<targetPath>resource</targetPath>
</resource>
</resources>
</build>
</project>

View File

@@ -0,0 +1,123 @@
package org.talend.gs.util;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Set;
import org.jets3t.service.model.GSObject;
import org.jets3t.service.utils.Mimetypes;
public class GSObjectUtil {
public List<GSObject> genGSObjectList(List<GSObject> objects, File file, String keyParent, boolean isGenFileObject,
boolean isGenFolderObject) throws NoSuchAlgorithmException, IOException {
if (file.isDirectory()) {
if (keyParent != null && !"".equals(keyParent)) {
if (keyParent.trim().lastIndexOf("/") != keyParent.trim().length() - 1) {
keyParent = keyParent + "/";
}
if (isGenFolderObject) {
objects.add(new GSObject(keyParent));
}
}
File[] files = file.listFiles();
for (File f : files) {
if (f.isDirectory()) {
objects = genGSObjectList(objects, f, keyParent + f.getName() + "/", isGenFileObject,
isGenFolderObject);
} else {
objects = genGSObjectList(objects, f, keyParent + f.getName(), isGenFileObject, isGenFolderObject);
}
}
} else {
if (isGenFileObject) {
GSObject obj = new GSObject(file);
obj.setKey(keyParent);
objects.add(obj);
}
}
return objects;
}
public java.util.Map<String, String> genFileFilterList(java.util.List<java.util.Map<String, String>> list,
String localdir, String remotedir) {
if (remotedir != null && !"".equals(remotedir)) {
if (remotedir.trim().lastIndexOf("/") != remotedir.trim().length() - 1) {
remotedir = remotedir + "/";
}
}
java.util.Map<String, String> fileMap = new HashMap<String, String>();
for (java.util.Map<String, String> map : list) {
java.util.Set<String> keySet = map.keySet();
for (String key : keySet) {
String tempdir = localdir;
String filemask = key;
String dir = null;
String mask = filemask.replaceAll("\\\\", "/");
int i = mask.lastIndexOf('/');
if (i != -1) {
dir = mask.substring(0, i);
mask = mask.substring(i + 1);
}
if (dir != null && !"".equals(dir))
tempdir = tempdir + "/" + dir;
mask = mask.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*");
final String finalMask = mask;
java.io.File[] listings = null;
java.io.File file = new java.io.File(tempdir);
if (file.isDirectory()) {
listings = file.listFiles(new java.io.FileFilter() {
public boolean accept(java.io.File pathname) {
boolean result = false;
if (pathname != null && pathname.isFile()) {
result = java.util.regex.Pattern.compile(finalMask).matcher(pathname.getName()).find();
}
return result;
}
});
}
if (listings == null || listings.length <= 0) {
System.err.println("No match file(" + key + ") exist!");
} else {
String localFilePath = "";
String newObjectKey = "";
for (int m = 0; m < listings.length; m++) {
if (listings[m].getName().matches(mask)) {
localFilePath = listings[m].getAbsolutePath();
if (map.get(key) != null && map.get(key).length() > 0) {
newObjectKey = remotedir + map.get(key);
} else {
newObjectKey = remotedir + listings[m].getName();
}
fileMap.put(localFilePath, newObjectKey);
}
}
}
}
}
return fileMap;
}
public List<GSObject> genObjectByFileMap(java.util.Map<String, String> fileMap) throws Exception, IOException {
Set<String> localFiles = fileMap.keySet();
List<GSObject> objects = new ArrayList<GSObject>();
for (String localFilePath : localFiles) {
GSObject object = new GSObject(new File(localFilePath));
object.setKey(fileMap.get(localFilePath));
objects.add(object);
}
return objects;
}
public void initMimeTypes() throws IOException {
InputStream mimetypesFile = this.getClass().getResourceAsStream("/resource/mime.types");
Mimetypes.getInstance().loadAndReplaceMimetypes(mimetypesFile);
}
}

View File

@@ -7,7 +7,7 @@
<groupId>net.sf.json-lib</groupId>
<artifactId>json-lib</artifactId>
<packaging>jar</packaging>
<version>2.4.3-talend</version>
<version>2.4.4-talend</version>
<name>json-lib</name>
<properties>

View File

@@ -25,12 +25,12 @@ import java.util.Iterator;
import java.util.Set;
import java.util.TreeSet;
import net.sf.json.util.JSONUtils;
import net.sf.json.util.JsonEventListener;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import net.sf.json.util.JSONUtils;
import net.sf.json.util.JsonEventListener;
/**
* Base class for JSONObject and JSONArray.
*
@@ -227,7 +227,8 @@ abstract class AbstractJSON implements JSON {
}
protected Object _processValue( Object value, JsonConfig jsonConfig ) {
if( JSONNull.getInstance().equals( value ) ) {
if( JSONNull.getInstance().equals( value ) &&
((jsonConfig.getJsonStandard() != JsonStandard.WRAP_NULL_STRINGS) || !"null".equals(value))) {
return JSONNull.getInstance();
} else if( Class.class.isAssignableFrom( value.getClass() ) || value instanceof Class ) {
return ((Class) value).getName();

View File

@@ -2391,7 +2391,7 @@ public final class JSONObject extends AbstractJSON implements JSON, Map, Compara
}
try{
Iterator keys = keys();
StringBuffer sb = new StringBuffer( "{" );
StringBuilder sb = new StringBuilder( "{" );
while( keys.hasNext() ){
if( sb.length() > 1 ){
@@ -2460,7 +2460,7 @@ public final class JSONObject extends AbstractJSON implements JSON, Map, Compara
return this.toString();
}
Iterator keys = keys();
StringBuffer sb = new StringBuffer( "{" );
StringBuilder sb = new StringBuilder( "{" );
int newindent = indent + indentFactor;
Object o;
if( n == 1 ){
@@ -2549,7 +2549,7 @@ public final class JSONObject extends AbstractJSON implements JSON, Map, Compara
if( o instanceof JSONArray ){
((JSONArray) o).element( value, jsonConfig );
}else{
setInternal( key, new JSONArray().element( o )
setInternal( key, new JSONArray().element( o, jsonConfig )
.element( value, jsonConfig ), jsonConfig );
}
}

View File

@@ -86,7 +86,6 @@ public class JsonConfig {
private boolean handleJettisonEmptyElement;
private boolean handleJettisonSingleElementArray;
private boolean ignoreDefaultExcludes;
//private boolean ignoreJPATransient;
private boolean ignoreTransientFields;
private boolean ignorePublicFields = true;
private boolean javascriptCompliant;
@@ -110,6 +109,7 @@ public class JsonConfig {
private Map typeMap = new HashMap();
private List ignoreFieldAnnotations = new ArrayList();
private boolean allowNonStringKeys = false;
private JsonStandard jsonStandard = JsonStandard.LEGACY;
public JsonConfig() {
}
@@ -1246,7 +1246,20 @@ public class JsonConfig {
this.newBeanInstanceStrategy = newBeanInstanceStrategy == null ? DEFAULT_NEW_BEAN_INSTANCE_STRATEGY
: newBeanInstanceStrategy;
}
/**
* Sets the config to wrap "null" strings as strings instead of JsonNull.
*
*/
public void setJsonStandard(JsonStandard wrapNullStringValues) {
this.jsonStandard = wrapNullStringValues;
}
public JsonStandard getJsonStandard() {
return jsonStandard;
}
/**
* Sets a PropertyExclusionClassMatcher to use.<br>
* Will set default value (PropertyExclusionClassMatcher.DEFAULT) if null.<br>

View File

@@ -0,0 +1,14 @@
package net.sf.json;
public enum JsonStandard {
/**
* Out of date standard used to be default before
*/
LEGACY,
/**
* Updated standard due to RFC 7159 to not unwrap "null" strings (keep quotations)
*/
WRAP_NULL_STRINGS
}

View File

@@ -24,6 +24,8 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import org.apache.commons.beanutils.DynaBean;
import net.sf.ezmorph.MorphUtils;
import net.sf.ezmorph.MorpherRegistry;
import net.sf.ezmorph.bean.MorphDynaBean;
@@ -36,10 +38,9 @@ import net.sf.json.JSONNull;
import net.sf.json.JSONObject;
import net.sf.json.JSONString;
import net.sf.json.JsonConfig;
import net.sf.json.JsonStandard;
import net.sf.json.regexp.RegexpUtils;
import org.apache.commons.beanutils.DynaBean;
/**
* Provides useful methods on java objects and JSON values.
*
@@ -124,7 +125,7 @@ public final class JSONUtils {
public static String getFunctionBody( String function ) {
return RegexpUtils.getMatcher( FUNCTION_BODY_PATTERN, true ).getGroupIfMatches( function, 1 );
}
/**
* Returns the params of a function literal.
*/
@@ -605,7 +606,7 @@ public final class JSONUtils {
return input.startsWith( SINGLE_QUOTE ) && input.endsWith( SINGLE_QUOTE ) ||
input.startsWith( DOUBLE_QUOTE ) && input.endsWith( DOUBLE_QUOTE );
}
public static boolean isJsonKeyword( String input, JsonConfig jsonConfig ) {
if( input == null ){
return false;
@@ -615,7 +616,7 @@ public final class JSONUtils {
"false".equals( input ) ||
(jsonConfig.isJavascriptCompliant() && "undefined".equals( input ));
}
/**
* Throw an exception if the object is an NaN or infinite number.
*
@@ -754,7 +755,77 @@ public final class JSONUtils {
return quote( value.toString() );
}
/**
public static String jsonToStandardizedString(JSON json, JsonStandard standard) {
switch (standard) {
case WRAP_NULL_STRINGS:
if (json.isArray()) {
JSONArray jsonArray = (JSONArray) json;
return jsonArrayToWrappedNullStrings(jsonArray);
} else if (!JSONNull.getInstance().equals(json)) {
return jsonToWrappedNullStrings((JSONObject) json);
}
default:
return json.toString();
}
}
/**
*
* @return plain String from JSONObject (@see JSONObject#toString()), but wrap null strings to quotation
*/
private static String jsonToWrappedNullStrings(JSONObject json) {
if (json.isNullObject()) {
return JSONNull.getInstance()
.toString();
}
try {
Iterator keys = json.keys();
StringBuilder sb = new StringBuilder("{");
while (keys.hasNext()) {
if (sb.length() > 1) {
sb.append(',');
}
Object o = keys.next();
sb.append(quote(o.toString()));
sb.append(':');
sb.append(valueToStringWrappedNullStrings(json.get(o)));
}
sb.append('}');
return sb.toString();
} catch (Exception e) {
return null;
}
}
private static String jsonArrayToWrappedNullStrings(JSONArray jsonArray) {
final String separator = ",";
StringBuilder sb = new StringBuilder("[");
for (int i = 0; i < jsonArray.size(); i++) {
if (i > 0) {
sb.append(separator);
}
sb.append(JSONUtils.valueToStringWrappedNullStrings(jsonArray.get(i)));
}
return sb.append("]").toString();
}
private static String valueToStringWrappedNullStrings(Object o) {
if ("null".equals(o)) {
return quote(o.toString());
} else if (o instanceof JSONArray) {
return jsonArrayToWrappedNullStrings((JSONArray) o);
} else if (o instanceof JSONObject) {
return jsonToWrappedNullStrings((JSONObject) o);
} else {
return valueToString(o);
}
}
/**
* Finds out if n represents a BigInteger
*
* @return true if n is instanceOf BigInteger or the literal value can be

View File

@@ -22,6 +22,8 @@ import net.sf.json.JSONException;
import net.sf.json.JSONFunction;
import net.sf.json.JSONNull;
import net.sf.json.JSONObject;
import net.sf.json.JsonConfig;
import net.sf.json.JsonStandard;
import net.sf.json.util.JSONUtils;
import nu.xom.Attribute;
import nu.xom.Builder;
@@ -29,7 +31,6 @@ import nu.xom.Document;
import nu.xom.Element;
import nu.xom.Elements;
import nu.xom.Node;
import nu.xom.ProcessingInstruction;
import nu.xom.Serializer;
import nu.xom.Text;
import org.apache.commons.lang.ArrayUtils;
@@ -163,6 +164,10 @@ public class XMLSerializer {
*/
private boolean useLongDecimals;
/**
* The config parameter to wrap "null" strings as strings instead of JsonNull.
*/
private JsonStandard jsonStandard;
/**
* flag for if parse empty elements as empty strings
*/
@@ -372,17 +377,19 @@ public class XMLSerializer {
return JSONNull.getInstance();
}
String defaultType = getType( root, JSONTypes.STRING );
JsonConfig config = new JsonConfig();
config.setJsonStandard(jsonStandard);
if( isArray( root, true ) ){
json = processArrayElement( root, defaultType );
if( forceTopLevelObject ){
String key = removeNamespacePrefix( root.getQualifiedName() );
json = new JSONObject().element( key, json );
json = new JSONObject().element(key, json, config);
}
}else{
json = processObjectElement( root, defaultType );
if( forceTopLevelObject ){
String key = removeNamespacePrefix( root.getQualifiedName() );
json = new JSONObject().element( key, json );
json = new JSONObject().element(key, json, config);
}
}
}catch( JSONException jsone ){
@@ -660,6 +667,10 @@ public class XMLSerializer {
this.useLongDecimals = useLongDecimals;
}
public void setJsonStandard(JsonStandard jsonStandard) {
this.jsonStandard = jsonStandard;
}
/**
* Writes a JSON value into a XML string with UTF-8 encoding.<br>
*
@@ -1264,14 +1275,16 @@ public class XMLSerializer {
}
private void setOrAccumulate( JSONObject jsonObject, String key, Object value ) {
JsonConfig config = new JsonConfig();
config.setJsonStandard(jsonStandard);
if( jsonObject.has( key ) ){
jsonObject.accumulate( key, value );
jsonObject.accumulate(key, value, config);
Object val = jsonObject.get( key );
if( val instanceof JSONArray ){
((JSONArray) val).setExpandElements( true );
}
}else{
jsonObject.element( key, value );
jsonObject.element( key, value, config);
}
}
@@ -1279,9 +1292,10 @@ public class XMLSerializer {
String clazz = getClass( element );
String type = getType( element );
type = (type == null) ? defaultType : type;
JsonConfig config = new JsonConfig();
config.setJsonStandard(jsonStandard);
if( hasNamespaces( element ) && !skipNamespaces ){
jsonArray.element( simplifyValue( null, processElement( element, type ) ) );
jsonArray.element(simplifyValue(null, processElement(element, type)), config);
return;
}else if( element.getAttributeCount() > 0 ){
if( isFunction( element ) ){
@@ -1289,10 +1303,10 @@ public class XMLSerializer {
String[] params = null;
String text = element.getValue();
params = StringUtils.split( paramsAttribute.getValue(), "," );
jsonArray.element( new JSONFunction( params, text ) );
jsonArray.element(new JSONFunction(params, text));
return;
}else{
jsonArray.element( simplifyValue( null, processElement( element, type ) ) );
jsonArray.element(simplifyValue(null, processElement(element, type)), config);
return;
}
}
@@ -1300,10 +1314,10 @@ public class XMLSerializer {
boolean classProcessed = false;
if( clazz != null ){
if( clazz.compareToIgnoreCase( JSONTypes.ARRAY ) == 0 ){
jsonArray.element( processArrayElement( element, type ) );
jsonArray.element(processArrayElement(element, type), config);
classProcessed = true;
}else if( clazz.compareToIgnoreCase( JSONTypes.OBJECT ) == 0 ){
jsonArray.element( simplifyValue( null, processObjectElement( element, type ) ) );
jsonArray.element(simplifyValue(null, processObjectElement( element, type)), config);
classProcessed = true;
}
}
@@ -1339,12 +1353,12 @@ public class XMLSerializer {
jsonArray.element( new JSONFunction( params, text ) );
}else{
if( isArray( element, false ) ){
jsonArray.element( processArrayElement( element, defaultType ) );
jsonArray.element(processArrayElement(element, defaultType), config);
}else if( isObject( element, false ) ){
jsonArray.element( simplifyValue( null, processObjectElement( element,
defaultType ) ) );
jsonArray.element(simplifyValue(null, processObjectElement(element,
defaultType)), config);
}else{
jsonArray.element( trimSpaceFromValue( element.getValue() ) );
jsonArray.element(trimSpaceFromValue(element.getValue()), config);
}
}
}

View File

@@ -17,16 +17,25 @@
package net.sf.json.util;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Objects;
import junit.framework.TestCase;
import net.sf.json.JSONArray;
import net.sf.json.JSONException;
import net.sf.json.JSONFunction;
import net.sf.json.JSONObject;
import net.sf.json.JsonConfig;
import net.sf.json.JsonStandard;
/**
* @author Andres Almiray <aalmiray@users.sourceforge.net>
*/
public class TestJSONUtils extends TestCase {
private static Map<String, String> valuesMap;
public static void main( String[] args ) {
junit.textui.TestRunner.run( TestJSONUtils.class );
}
@@ -35,6 +44,14 @@ public class TestJSONUtils extends TestCase {
super( name );
}
public void setUp() throws Exception {
super.setUp();
valuesMap = new LinkedHashMap<>();
valuesMap.put("key1", "null");
valuesMap.put("key2", "not_null");
}
public void testDoubleToString_infinite() {
assertEquals( "null", JSONUtils.doubleToString( Double.POSITIVE_INFINITY ) );
}
@@ -177,4 +194,48 @@ public class TestJSONUtils extends TestCase {
// ok
}
}
public void testNullStringsWrapped() {
JsonConfig config = new JsonConfig();
config.setJsonStandard(JsonStandard.WRAP_NULL_STRINGS);
JSONObject jsonObject = new JSONObject();
jsonObject.putAll(valuesMap, config);
String resultingString = JSONUtils.jsonToStandardizedString(jsonObject, JsonStandard.WRAP_NULL_STRINGS);
assertFalse("Wrapping null strings standard's broken", Objects.equals(jsonObject.toString(), resultingString));
assertTrue(resultingString.contains("\"null\""));
}
public void testNullStringsUnwrapped() {
JSONObject jsonObject = new JSONObject();
jsonObject.putAll(valuesMap);
String resultingString = JSONUtils.jsonToStandardizedString(jsonObject, JsonStandard.WRAP_NULL_STRINGS);
assertEquals(jsonObject.toString(), resultingString);
}
public void testNullStringsOnArrayWrapped() {
JsonConfig config = new JsonConfig();
config.setJsonStandard(JsonStandard.WRAP_NULL_STRINGS);
JSONArray jsonArray = new JSONArray();
jsonArray.add("abc");
jsonArray.add("null", config);
jsonArray.add(null);
String resultingString = JSONUtils.jsonToStandardizedString(jsonArray, JsonStandard.WRAP_NULL_STRINGS);;
assertFalse("Wrapping null strings standard's broken", Objects.equals(jsonArray.toString(), resultingString));
assertTrue(resultingString.contains("\"null\""));
}
public void testNullStringsOnArrayUnwrapped() {
JSONArray jsonArray = new JSONArray();
jsonArray.add("abc");
jsonArray.add("null");
jsonArray.add(null);
String resultingString = JSONUtils.jsonToStandardizedString(jsonArray, JsonStandard.LEGACY);
assertEquals(jsonArray.toString(), resultingString);
}
}

View File

@@ -4,7 +4,7 @@
<groupId>org.neo4j.talend</groupId>
<artifactId>neo4j-talend-component</artifactId>
<version>1.3-20171206</version>
<version>1.3-20191012</version>
<packaging>jar</packaging>
<name>Neo4j Talend </name>
@@ -100,12 +100,23 @@
<artifactId>neo4j-import-tool</artifactId>
<version>${neo4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<!-- Talend log4j library -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
<scope>provided</scope>
<version>1.2.17</version>
<scope>test</scope>
</dependency>
<!-- JUnit -->
<dependency>

View File

@@ -1,6 +1,7 @@
package org.neo4j.talend;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.neo4j.graphdb.DynamicLabel;
import org.neo4j.graphdb.index.IndexHits;
import org.neo4j.helpers.collection.MapUtil;
@@ -23,7 +24,7 @@ public class Neo4jBatchDatabase {
/**
* The logger
*/
private static Logger log = Logger.getLogger(Neo4jBatchDatabase.class);
private static Logger log = LoggerFactory.getLogger(Neo4jBatchDatabase.class);
/**
* Name of the field index for the importId

View File

@@ -1,7 +1,8 @@
package org.neo4j.talend;
import org.apache.commons.lang3.StringUtils;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.neo4j.graphdb.DynamicRelationshipType;
@@ -11,7 +12,7 @@ import java.util.Map;
public class Neo4jBatchInserterRelationship extends Neo4jBatchInserterAbstract {
private static Logger log = Logger.getLogger(Neo4jBatchInserterRelationship.class);
private static Logger log = LoggerFactory.getLogger(Neo4jBatchInserterRelationship.class);
private String relationshipTypeField;
private String direction;

View File

@@ -1,7 +1,8 @@
package org.neo4j.talend;
import org.apache.commons.lang3.StringUtils;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.neo4j.tooling.ImportTool;
import java.io.BufferedWriter;
@@ -18,7 +19,7 @@ public class Neo4jImportTool {
/**
* The logger
*/
private final static Logger log = Logger.getLogger(Neo4jImportTool.class);
private final static Logger log = LoggerFactory.getLogger(Neo4jImportTool.class);
protected final static String HEADERS_KEY = "HEADERS";
protected final static String FILE_KEY = "FILE";

View File

@@ -1,95 +1,125 @@
<!--
Copyright (C) 2006-2020 Talend Inc. - www.talend.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components</groupId>
<artifactId>components-servicenow</artifactId>
<name>servicenow</name>
<version>1.0.0</version>
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components</groupId>
<artifactId>components-servicenow</artifactId>
<name>servicenow</name>
<version>1.0.1</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
</properties>
<description>Talend Helper library for ServiceNow components</description>
<url>
https://github.com/Talend/tdi-studio-se/
</url>
<licenses>
<license>
<name>Apache-2.0</name>
<url>
http://www.talendforge.org/modules/licenses/APACHE_v2.txt
</url>
</license>
</licenses>
<dependencies>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.6</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.3</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient-cache</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20150729</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.0</version>
<type>jar</type>
<optional>true</optional>
</dependency>
</dependencies>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
</repository>
</distributionManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
</plugins>
<dependencies>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.6</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.3</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient-cache</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20150729</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.0</version>
<type>jar</type>
<optional>true</optional>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
</resources>
<!-- TODO
<testResources>
<testResource>
<directory>src/test/java</directory>
</testResource>
</testResources>
-->
</build>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
</resources>
</build>
</project>

View File

@@ -1,274 +1,269 @@
package org.talend.servicenow;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.apache.http.HttpResponse;
import org.apache.http.ParseException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
public class Util {
private HttpClient client;
private String baseurl;
public Util(HttpClient client, String baseurl) {
this.client = client;
this.baseurl = baseurl;
}
@SuppressWarnings("rawtypes")
public static void main(String[] args) throws ClientProtocolException, IOException {
boolean exception = true;
org.apache.http.impl.client.CloseableHttpClient client = null;
while(exception) {
try {
//login
org.apache.http.client.CredentialsProvider credsProvider = new org.apache.http.impl.client.BasicCredentialsProvider();
credsProvider.setCredentials(
new org.apache.http.auth.AuthScope(new org.apache.http.HttpHost("sandbox.service-now.com")), new org.apache.http.auth.UsernamePasswordCredentials("talend", "talend")
);
client = org.apache.http.impl.client.HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();
//get metadata
Util util = new Util(client,"https://sandbox.service-now.com");
Object result = util.getMetadata("incident");
System.out.println(result);
System.out.println(((Map)result).size());
} catch(Exception e) {
System.out.println(e.getMessage());
continue;
} finally {
client.close();
}
exception = false;
}
}
private Map<String,String> typeMapping = new HashMap<String,String>();
{
typeMapping.put("string", "id_String");
typeMapping.put("boolean", "id_Boolean");
typeMapping.put("integer", "id_Integer");
typeMapping.put("decimal", "id_BigDecimal");
typeMapping.put("float", "id_Float");
typeMapping.put("glide_date_time", "id_Date");
typeMapping.put("glide_date", "id_Date");
typeMapping.put("glide_time", "id_Date");
}
public static class ColumnMetadata {
private String name;
private String type;
private int maxlength;
private String pattern;
public ColumnMetadata(String name, String type, int maxlength, String pattern) {
this.name = name;
this.type = type;
this.maxlength = maxlength;
this.pattern = pattern;
}
public String getName() {
return name;
}
public String getType() {
return type;
}
public int getMaxLength() {
return maxlength;
}
public String getPattern() {
return pattern;
}
public String toString() {
return name + ":" + type + ":" + maxlength;
}
}
public Map<String,ColumnMetadata> getMetadata(String tablename) throws ClientProtocolException, IOException {
Map<String,ColumnMetadata> result = new HashMap<String,ColumnMetadata>();
List<String> relation = getRelationship(tablename);
StringBuilder sb = new StringBuilder();
for(String tname : relation) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_dictionary");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tname);
sb.append("&sysparm_fields=element,internal_type,max_length,active");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
List<Map<String, String>> info = extractResponse4MultiRowFromArray(response);
for(Map<String, String> row : info) {
String element = row.get("element");
boolean active = Boolean.parseBoolean(row.get("active"));
if(element!=null && !element.isEmpty() && active) {
String talend_type = null;
String pattern = null;
String servicenow_type = (String)row.get("internal_type");
if(servicenow_type!=null && !servicenow_type.isEmpty()) {
talend_type = typeMapping.get(servicenow_type);
//need date pattern
if("glide_date_time".equals(servicenow_type)) {
pattern = "yyyy-MM-dd HH:mm:ss";
} else if("glide_date".equals(servicenow_type)) {
pattern = "yyyy-MM-dd";
} else if("glide_time".equals(servicenow_type)) {
pattern = "HH:mm:ss";
}
}
if(talend_type == null) {
talend_type = "id_String";
}
int mlength = 64;
String maxlength = (String)row.get("max_length");
if(maxlength!=null && !maxlength.isEmpty()) {
mlength = Integer.parseInt(maxlength);
}
ColumnMetadata column = new ColumnMetadata(element,talend_type,mlength,pattern);
result.put(element, column);
}
}
}
return result;
}
private List<String> getRelationship(String tablename) throws ClientProtocolException, IOException {
LinkedList<String> result = new LinkedList<String>();
StringBuilder sb = new StringBuilder();
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tablename);
sb.append("&sysparm_fields=name,super_class");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
Map<String,String> info = extractResponse4OneRowFromArray(response);
result.add(info.get("name"));
String superclass = info.get("super_class");
while(superclass!=null && !superclass.isEmpty()) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object/");
sb.append(superclass);
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_fields=name,super_class");
httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
response = this.client.execute(httpget);
info = extractResponse4OneRowFromObject(response);
result.add(info.get("name"));
superclass = info.get("super_class");
}
Collections.reverse(result);
return result;
}
private Map<String,String> extractResponse4OneRowFromArray(HttpResponse response) throws ParseException, IOException {
validateResponse(response);
Map<String,String> result = new HashMap<String,String>();
org.json.JSONArray array = (org.json.JSONArray)getResult(response);
for(int i=0;i<array.length();i++) {
org.json.JSONObject row = (org.json.JSONObject)array.get(i);
for(String key : row.keySet()) {
result.put(key, (String)row.get(key));
}
}
return result;
}
private Map<String,String> extractResponse4OneRowFromObject(HttpResponse response) throws ParseException, IOException {
validateResponse(response);
Map<String,String> result = new HashMap<String,String>();
org.json.JSONObject object = (org.json.JSONObject)getResult(response);
for(String key : object.keySet()) {
result.put(key, (String)object.get(key));
}
return result;
}
private List<Map<String, String>> extractResponse4MultiRowFromArray(HttpResponse response) throws ParseException, IOException {
validateResponse(response);
List<Map<String, String>> result = new ArrayList<Map<String,String>>();
org.json.JSONArray array = (org.json.JSONArray)getResult(response);
for(int i=0;i<array.length();i++) {
Map<String,String> element = new HashMap<String,String>();
org.json.JSONObject row = (org.json.JSONObject)array.get(i);
for(String key : row.keySet()) {
element.put(key, (String)row.get(key));
}
result.add(element);
}
return result;
}
private void validateResponse(HttpResponse response) {
if(response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Fail to get the table metadata. The operation has returned the code : " + response.getStatusLine() + ".");
}
}
private Object getResult(HttpResponse response) throws ParseException, IOException {
String responseBody = org.apache.http.util.EntityUtils.toString(response.getEntity());
org.json.JSONObject json = new org.json.JSONObject(responseBody);
return json.get("result");
}
/**
* Copyright (C) 2006-2019 Talend Inc. - www.talend.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.talend.servicenow;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.apache.http.HttpResponse;
import org.apache.http.ParseException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
public class Util {
private HttpClient client;
private String baseurl;
public Util(HttpClient client, String baseurl) {
this.client = client;
this.baseurl = baseurl;
}
private Map<String, String> typeMapping = new HashMap<String, String>();
{
typeMapping.put("string", "id_String");
typeMapping.put("boolean", "id_Boolean");
typeMapping.put("integer", "id_Integer");
typeMapping.put("decimal", "id_BigDecimal");
typeMapping.put("float", "id_Float");
typeMapping.put("glide_date_time", "id_Date");
typeMapping.put("glide_date", "id_Date");
typeMapping.put("glide_time", "id_Date");
}
public static class ColumnMetadata {
private String name;
private String type;
private int maxlength;
private String pattern;
public ColumnMetadata(String name, String type, int maxlength, String pattern) {
this.name = name;
this.type = type;
this.maxlength = maxlength;
this.pattern = pattern;
}
public String getName() {
return name;
}
public String getType() {
return type;
}
public int getMaxLength() {
return maxlength;
}
public String getPattern() {
return pattern;
}
public String toString() {
return name + ":" + type + ":" + maxlength;
}
}
public Map<String, ColumnMetadata> getMetadata(String tablename) throws ClientProtocolException, IOException {
Map<String, ColumnMetadata> result = new HashMap<String, ColumnMetadata>();
List<String> relation = getRelationship(tablename);
StringBuilder sb = new StringBuilder();
for (String tname : relation) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_dictionary");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tname);
sb.append("&sysparm_fields=element,internal_type,max_length,active");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
List<Map<String, String>> info = extractResponse4MultiRowFromArray(response);
for (Map<String, String> row : info) {
String element = row.get("element");
boolean active = Boolean.parseBoolean(row.get("active"));
if (element != null && !element.isEmpty() && active) {
String talend_type = null;
String pattern = null;
String servicenow_type = (String) row.get("internal_type");
if (servicenow_type != null && !servicenow_type.isEmpty()) {
talend_type = typeMapping.get(servicenow_type);
//need date pattern
if ("glide_date_time".equals(servicenow_type)) {
pattern = "yyyy-MM-dd HH:mm:ss";
} else if ("glide_date".equals(servicenow_type)) {
pattern = "yyyy-MM-dd";
} else if ("glide_time".equals(servicenow_type)) {
pattern = "HH:mm:ss";
}
}
if (talend_type == null) {
talend_type = "id_String";
}
int mlength = 64;
String maxlength = (String) row.get("max_length");
if (maxlength != null && !maxlength.isEmpty()) {
mlength = Integer.parseInt(maxlength);
}
ColumnMetadata column = new ColumnMetadata(element, talend_type, mlength, pattern);
result.put(element, column);
}
}
}
return result;
}
private List<String> getRelationship(String tablename) throws ClientProtocolException, IOException {
LinkedList<String> result = new LinkedList<String>();
StringBuilder sb = new StringBuilder();
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tablename);
sb.append("&sysparm_fields=name,super_class");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
Map<String, String> info = extractResponse4OneRowFromArray(response);
result.add(info.get("name"));
String superclass = info.get("super_class");
while (superclass != null && !superclass.isEmpty()) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object/");
sb.append(superclass);
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_fields=name,super_class");
httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
response = this.client.execute(httpget);
info = extractResponse4OneRowFromObject(response);
result.add(info.get("name"));
superclass = info.get("super_class");
}
Collections.reverse(result);
return result;
}
private Map<String, String> extractResponse4OneRowFromArray(HttpResponse response)
throws ParseException, IOException {
validateResponse(response);
Map<String, String> result = new HashMap<String, String>();
org.json.JSONArray array = (org.json.JSONArray) getResult(response);
for (int i = 0; i < array.length(); i++) {
org.json.JSONObject row = (org.json.JSONObject) array.get(i);
for (String key : row.keySet()) {
result.put(key, (String) row.get(key));
}
}
return result;
}
private Map<String, String> extractResponse4OneRowFromObject(HttpResponse response)
throws ParseException, IOException {
validateResponse(response);
Map<String, String> result = new HashMap<String, String>();
org.json.JSONObject object = (org.json.JSONObject) getResult(response);
for (String key : object.keySet()) {
result.put(key, (String) object.get(key));
}
return result;
}
private List<Map<String, String>> extractResponse4MultiRowFromArray(HttpResponse response)
throws ParseException, IOException {
validateResponse(response);
List<Map<String, String>> result = new ArrayList<Map<String, String>>();
org.json.JSONArray array = (org.json.JSONArray) getResult(response);
for (int i = 0; i < array.length(); i++) {
Map<String, String> element = new HashMap<String, String>();
org.json.JSONObject row = (org.json.JSONObject) array.get(i);
for (String key : row.keySet()) {
element.put(key, (String) row.get(key));
}
result.add(element);
}
return result;
}
private void validateResponse(HttpResponse response) {
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException(
"Fail to get the table metadata. The operation has returned the code : " + response.getStatusLine()
+ ".");
}
}
private Object getResult(HttpResponse response) throws ParseException, IOException {
String responseBody = org.apache.http.util.EntityUtils.toString(response.getEntity());
org.json.JSONObject json = new org.json.JSONObject(responseBody);
return json.get("result");
}
}

View File

@@ -1,95 +1,95 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="sugarCRMManagement.jar" />
<property name="component.name" value="tSugarCRMInput" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
<pathelement location="${component.plugin.home}/tWebServiceInput/axis.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/commons-discovery-0.2.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/wsdl4j-1.5.1.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/saaj.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/jaxrpc.jar" />
<pathelement location="${component.plugin.home}/tXMLRPCInput/commons-logging-1.1.jar" />
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}" />
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="sugarCRMManagement.jar" />
<property name="component.name" value="tSugarCRMInput" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
<pathelement location="${component.plugin.home}/tWebServiceInput/axis.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/commons-discovery-0.2.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/wsdl4j-1.5.1.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/saaj.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/jaxrpc.jar" />
<pathelement location="${component.plugin.home}/tXMLRPCInput/commons-logging-1.1.jar" />
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}" />
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -7,7 +7,7 @@
<groupId>org.talend.components.lib</groupId>
<artifactId>talend-db-exasol</artifactId>
<version>2.1.4</version>
<version>2.1.5</version>
<packaging>jar</packaging>
<name>talend-db-exasol</name>
@@ -33,10 +33,22 @@
</distributionManagement>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>

View File

@@ -27,15 +27,15 @@ import java.util.Collections;
import java.util.Date;
import java.util.List;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* This class provides an IMPORT command for the EXASol database.
* @author Jan Lolling, jan.lolling@cimt-ag.de
*/
public class EXABulkUtil {
private static Logger logger = Logger.getLogger(EXABulkUtil.class);
private static Logger logger = LoggerFactory.getLogger(EXABulkUtil.class);
public static final String CSV = "CSV";
public static final String FBV = "FBV";
public static final String ORA = "ORA";
@@ -83,14 +83,6 @@ public class EXABulkUtil {
private int sourceIdentifierCase = 0; // 0 = unchanged, 1 = lower case, 2 = upper case
private boolean onlyBuildSQLCode = false;
public void setDebug(boolean debug) {
if (debug) {
logger.setLevel(Level.DEBUG);
} else {
logger.setLevel(Level.INFO);
}
}
private String createNumberFormat(Integer length, Integer precision, boolean hasGroups) {
if (length != null && length.intValue() > 0) {
StringBuilder sb = new StringBuilder();

View File

@@ -39,7 +39,6 @@ public class TestEXABulkUtil {
truncateStat.execute("truncate table " + errorTable);
truncateStat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setTransferSecure(true);
u.setConnection(connection);
u.setTable(table);
@@ -83,7 +82,6 @@ public class TestEXABulkUtil {
}
truncateStat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setConnection(connection);
u.setTable(table);
u.setRemoteFileUrl("sftp://172.16.214.132/home/tisadmin/Downloads/");
@@ -130,7 +128,6 @@ public class TestEXABulkUtil {
}
truncateStat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setConnection(connection);
u.setTable(table);
u.setDbmsSourceType("JDBC");
@@ -170,7 +167,6 @@ public class TestEXABulkUtil {
}
truncateStat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setConnection(connection);
u.setTable(table);
u.setDbmsSourceType("ORA");
@@ -211,7 +207,6 @@ public class TestEXABulkUtil {
}
stat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setConnection(connection);
u.setTable(table);
u.setDbmsSourceType("JDBC");
@@ -250,7 +245,6 @@ public class TestEXABulkUtil {
}
stat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setConnection(connection);
u.setTable(table);
u.setDbmsSourceType("JDBC");
@@ -290,7 +284,6 @@ public class TestEXABulkUtil {
}
stat.close();
EXABulkUtil u = new EXABulkUtil();
u.setDebug(true);
u.setConnection(connection);
u.setTable(table);
u.setDbmsSourceType("EXA");

View File

@@ -4,7 +4,7 @@
<groupId>org.talend</groupId>
<artifactId>talend-httputil</artifactId>
<name>talend-httputil</name>
<version>1.0.4</version>
<version>1.0.5</version>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
@@ -20,7 +20,7 @@
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.9</version>
<version>2.10.1</version>
</dependency>
<dependency>

View File

@@ -6,7 +6,7 @@
<groupId>org.talend.libraries</groupId>
<artifactId>job-audit</artifactId>
<version>1.0</version>
<version>1.1</version>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
@@ -50,28 +50,43 @@
<dependencies>
<dependency>
<groupId>org.talend.daikon</groupId>
<artifactId>daikon-audit</artifactId>
<version>0.31.8</version>
<artifactId>audit-common</artifactId>
<version>1.8.0</version>
</dependency>
<!--
<dependency>
<groupId>org.talend.daikon</groupId>
<artifactId>audit-logback</artifactId>
<version>0.31.8</version>
<artifactId>audit-log4j1</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.3.0-alpha4</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>test</scope>
</dependency>
-->
<dependency>
<groupId>org.talend.daikon</groupId>
<artifactId>audit-log4j2</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.12.1</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.12.1</version>
<optional>true</optional>
</dependency>
</dependencies>
</project>

View File

@@ -18,13 +18,30 @@ public class JobEventAuditLoggerFactory {
final AuditConfigurationMap config = AuditConfiguration.loadFromProperties(properties);
AbstractBackend logger = null;
String loggerClass = "org.talend.logging.audit.logback.LogbackBackend";
//load log4j2 implement firstly
String loggerClass = "org.talend.logging.audit.log4j2.Log4j2Backend";
try {
final Class<?> clz = Class.forName(loggerClass);
logger = (AbstractBackend) clz.getConstructor(AuditConfigurationMap.class).newInstance(config);
} catch (ReflectiveOperationException e) {
throw new RuntimeException("Unable to load backend " + loggerClass, e);
}
final Class<?> clz = Class.forName(loggerClass);
logger = (AbstractBackend) clz.getConstructor(AuditConfigurationMap.class).newInstance(config);
} catch (ReflectiveOperationException e) {
// do nothing
}
//load log4j1 implement if not found log4j2
if (logger == null) {
loggerClass = "org.talend.logging.audit.log4j1.Log4j1Backend";
try {
final Class<?> clz = Class.forName(loggerClass);
logger = (AbstractBackend) clz.getConstructor(AuditConfigurationMap.class).newInstance(config);
} catch (ReflectiveOperationException e) {
// do nothing
}
}
if(logger == null) {
throw new RuntimeException("Unable to load backend : " + loggerClass);
}
final DefaultAuditLoggerBase loggerBase = new DefaultAuditLoggerBase(logger, config);

View File

@@ -8,7 +8,8 @@ public class JobAuditLoggerTest {
public static void main(String[] args) {
Properties props = new Properties();
props.setProperty("root.logger", "audit");
String root_logger_name = "audit";
props.setProperty("root.logger", root_logger_name);
props.setProperty("encoding", "UTF-8");
props.setProperty("application.name", "Talend Studio");
props.setProperty("service.name", "Talend Studio Job");
@@ -19,6 +20,10 @@ public class JobAuditLoggerTest {
props.setProperty("appender.file.maxsize", "52428800");
props.setProperty("appender.file.maxbackup", "20");
props.setProperty("host", "false");
org.apache.logging.log4j.core.config.Configurator.setLevel(root_logger_name, org.apache.logging.log4j.Level.DEBUG);
//org.apache.log4j.Logger.getLogger("audit").setLevel(org.apache.log4j.Level.DEBUG);
final JobAuditLogger logger = JobEventAuditLoggerFactory.createJobAuditLogger(props);
Context context = JobContextBuilder.create().jobName("fetch_from_s3_every_day").jobId("jobid_123")
.jobVersion("0.1").connectorType("tXMLMAP").connectorId("tXMLMap_1")

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend-mscrm</artifactId>
<version>3.4-20190513</version>
<version>3.4-20191012</version>
<packaging>jar</packaging>
<name>talend-mscrm</name>
@@ -109,7 +109,7 @@
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>adal4j</artifactId>
<version>1.1.1-patch</version>
<version>1.1.1-20191012</version>
</dependency>
</dependencies>
<build>

View File

@@ -22,7 +22,8 @@ import org.apache.axis2.transport.http.HTTPConstants;
import org.apache.axis2.transport.http.HTTPTransportConstants;
import org.apache.axis2.transport.http.HttpTransportProperties;
import org.apache.axis2.transport.http.HttpTransportProperties.ProxyProperties;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.datacontract.schemas._2004._07.system_collections_generic.KeyValuePairOfEndpointTypestringztYlk6OT;
import org.talend.ms.crm.sdk.OnlineAuthenticationPolicy;
import org.talend.ms.crm.sdk.OrganizationServiceStubWrapper;
@@ -61,7 +62,7 @@ import com.microsoft.schemas.xrm._2011.contracts.discovery.RetrieveOrganizationR
*/
public class MSCRMClient {
static Logger logger = Logger.getLogger(MSCRMClient.class.getName());
static Logger logger = LoggerFactory.getLogger(MSCRMClient.class.getName());
/**
* Microsoft account (e.g. youremail@live.com) or Microsoft Office 365 (Org ID e.g.

View File

@@ -38,7 +38,8 @@ import org.apache.http.impl.client.SystemDefaultHttpClient;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.CoreConnectionPNames;
import org.apache.http.params.HttpParams;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.w3c.dom.Document;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
@@ -53,7 +54,7 @@ public final class DeviceIdManager {
/**
* Logger
*/
static final Logger Log = Logger.getLogger(DeviceIdManager.class.getName());
static final Logger Log = LoggerFactory.getLogger(DeviceIdManager.class.getName());
private static final Random RandomInstance = new Random();

View File

@@ -16,7 +16,8 @@ import javax.wsdl.WSDLException;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPathExpressionException;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.xml.sax.SAXException;
/**
@@ -25,7 +26,7 @@ import org.xml.sax.SAXException;
*/
public final class OnlineAuthenticationPolicy {
static Logger logger = Logger.getLogger(OnlineAuthenticationPolicy.class.getName());
static Logger logger = LoggerFactory.getLogger(OnlineAuthenticationPolicy.class.getName());
/**
* Construct an Instance of the OnlineAuthenticationPolicy class.

View File

@@ -44,7 +44,8 @@ import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.CoreConnectionPNames;
import org.apache.http.params.HttpParams;
import org.apache.http.util.EntityUtils;
import org.apache.log4j.Logger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.xml.sax.SAXException;
@@ -55,7 +56,7 @@ import org.xml.sax.SAXException;
*/
public final class WsdlTokenManager {
static Logger logger = Logger.getLogger(WsdlTokenManager.class.getName());
static Logger logger = LoggerFactory.getLogger(WsdlTokenManager.class.getName());
private final String DeviceTokenTemplate = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" + " <s:Envelope "
+ " xmlns:s=\"http://www.w3.org/2003/05/soap-envelope\""

View File

@@ -0,0 +1,55 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components.lib</groupId>
<artifactId>talend-proxy</artifactId>
<version>1.0.0</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
</properties>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
<snapshots><enabled>true</enabled></snapshots>
<releases><enabled>false</enabled></releases>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</distributionManagement>
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
</project>

View File

@@ -0,0 +1,20 @@
package org.talend.proxy;
import org.apache.commons.lang3.tuple.ImmutablePair;
import org.apache.commons.lang3.tuple.Pair;
public class ProxyCreds {
private Pair<String, String> credPair;
public ProxyCreds(String userName, String pass) {
this.credPair = new ImmutablePair<>(userName, pass);
}
public String getUser() {
return credPair.getKey();
}
public String getPass() {
return credPair.getValue();
}
}

View File

@@ -0,0 +1,36 @@
package org.talend.proxy;
import java.net.Proxy;
import java.util.HashMap;
import java.util.Map;
/**
* Use only inside of ThreadLocal
*/
public class ProxyHolder {
private Map<String, Proxy> proxyMap;
public ProxyHolder() {
proxyMap = new HashMap<>();
}
/**
*
* @param proxy HTTP or SOCKS proxy instance to use
* @param host without protocol
* @param port -1 to apply proxy for every port
*/
public void putNewHost(Proxy proxy, String host, int port) {
if (port != -1) {
proxyMap.put(host + ":" + port, proxy);
} else {
proxyMap.put(host, proxy);
}
}
public Map<String, Proxy> getProxyMap() {
return proxyMap;
}
}

View File

@@ -0,0 +1,41 @@
package org.talend.proxy;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.util.HashMap;
import java.util.Map;
public class TalendProxyAuthenticator extends Authenticator {
private static TalendProxyAuthenticator instance;
public static synchronized TalendProxyAuthenticator getInstance() {
if (instance == null) {
instance = new TalendProxyAuthenticator();
}
return instance;
}
private TalendProxyAuthenticator() {
}
private Map<String, ProxyCreds> proxyCredsMap = new HashMap<>();
public synchronized void addAuthForProxy(String host, String port, String userName, String pass) {
proxyCredsMap.put(host + ":" + port, new ProxyCreds(userName, pass));
}
public synchronized ProxyCreds getCredsForProxyURI(String proxyURI) {
return proxyCredsMap.get(proxyURI);
}
@Override
protected synchronized PasswordAuthentication getPasswordAuthentication() {
String requestURI = super.getRequestingHost() + ":" + super.getRequestingPort();
if (proxyCredsMap.containsKey(requestURI)) {
return new PasswordAuthentication(proxyCredsMap.get(requestURI).getUser(), proxyCredsMap.get(requestURI).getPass().toCharArray());
} else {
return super.getPasswordAuthentication(); //don't use authentication
}
}
}

View File

@@ -0,0 +1,98 @@
package org.talend.proxy;
import java.io.IOException;
import java.net.Proxy;
import java.net.ProxySelector;
import java.net.SocketAddress;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import org.apache.log4j.Logger;
public class TalendProxySelector extends ProxySelector {
private static TalendProxySelector instance;
private static final Logger log = Logger.getLogger(TalendProxySelector.class);
private ThreadLocal<ProxyHolder> threadLocalProxyHolder;
private ProxyHolder globalProxyHolder;
public static synchronized TalendProxySelector getInstance() {
if (instance == null) {
instance = new TalendProxySelector();
}
return instance;
}
private TalendProxySelector() {
globalProxyHolder = new ProxyHolder();
}
public synchronized void addProxySettings(Proxy proxy, boolean threadSpecific, String host, int port) {
if (threadSpecific) {
if (threadLocalProxyHolder == null) {
threadLocalProxyHolder = new ThreadLocal<>();
}
if (threadLocalProxyHolder.get() == null) {
ProxyHolder newProxyHolder = new ProxyHolder();
threadLocalProxyHolder.set(newProxyHolder);
}
threadLocalProxyHolder.get().putNewHost(proxy, host, port);
} else {
globalProxyHolder.putNewHost(proxy, host, port);
}
}
/**
* Finds and return proxy was set for specific host
* @param uriString host:port
* @return Optional of Proxy if such proxy setting was set
*/
public synchronized Proxy getProxyForUriString(String uriString) {
if (proxyHolderContainsHost(globalProxyHolder, uriString)) {
log.debug("All threads proxy " + globalProxyHolder.getProxyMap().get(uriString) + " is using to connect to URI " + uriString);
return globalProxyHolder.getProxyMap().containsKey(uriString) ? globalProxyHolder.getProxyMap().get(uriString) :
globalProxyHolder.getProxyMap().get(uriString.substring(0, uriString.lastIndexOf(":")));
} else if (threadLocalProxyHolder != null && proxyHolderContainsHost(threadLocalProxyHolder.get(), uriString)) {
log.debug("Proxy " + threadLocalProxyHolder.get().getProxyMap().get(uriString) + " is using to connect to URI " + uriString);
return threadLocalProxyHolder.get().getProxyMap().containsKey(uriString) ?
threadLocalProxyHolder.get().getProxyMap().get(uriString) :
threadLocalProxyHolder.get().getProxyMap().get(uriString.substring(0, uriString.lastIndexOf(":")));
} else {
log.debug("No proxy is using to connect to URI " + uriString);
return Proxy.NO_PROXY;
}
}
@Override
public List<Proxy> select(URI uri) {
String uriString = uri.getHost();
if (uri.getPort() != -1) {
uriString += ":" + uri.getPort() ;
}
log.debug("Network request hadling from Talend proxy selector. Thread " + Thread.currentThread().getName() + ". URI to connect: " + uriString);
return Collections.singletonList(getProxyForUriString(uriString));
}
@Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
if (ioe != null) {
log.warn("Connect failed when use Talend ProxySelector to the URI:" + uri.toString(), ioe);
} else {
log.warn("Connect failed when use Talend ProxySelector to the " + uri);
}
}
private static boolean proxyHolderContainsHost(ProxyHolder holder, String uriString) {
return holder != null &&
(holder.getProxyMap().containsKey(uriString)
|| (uriString.contains(":") && holder.getProxyMap().containsKey(uriString.substring(0, uriString.indexOf(":")))));
}
}

View File

@@ -4,12 +4,12 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend-ws</artifactId>
<version>1.0.1-20190204</version>
<version>1.0.1-20191112</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<cxf.version>3.1.1</cxf.version>
<cxf.version>3.3.4</cxf.version>
<odata.version>4.3.0</odata.version>
<slf4j.version>1.7.12</slf4j.version>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
@@ -90,42 +90,42 @@
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-core</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-simple</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-tools-common</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-wsdl</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-bindings-soap</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-databinding-jaxb</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http-jetty</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.talend.libraries</groupId>

View File

@@ -2,7 +2,7 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talendMsgMailUtil-1.1-20170419</artifactId>
<artifactId>talendMsgMailUtil-1.1-20191012</artifactId>
<name>talendMsgMailUtil</name>
<version>6.0.0</version>
<packaging>jar</packaging>
@@ -37,6 +37,11 @@
</distributionManagement>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>

View File

@@ -12,8 +12,8 @@ import org.apache.poi.hsmf.datatypes.MAPIProperty;
import org.apache.poi.hsmf.datatypes.StringChunk;
import org.apache.poi.hsmf.datatypes.Types;
import org.apache.poi.hsmf.exceptions.ChunkNotFoundException;
import org.apache.log4j.Logger;
import org.apache.log4j.Level;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class MsgMailUtil {
@@ -24,6 +24,15 @@ public class MsgMailUtil {
public MsgMailUtil() {
}
private enum Level {
DEBUG,
INFO,
TRACE,
FATAL,
WARN,
ERROR
}
public MsgMailUtil(String fileName, String outAttachmentPath)
throws IOException {
@@ -31,8 +40,8 @@ public class MsgMailUtil {
this.outAttachmentPath = outAttachmentPath;
}
public void activeLog(Logger log, String position) {
this.log = log;
public void activeLog(String logger_name, String position) {
this.log = LoggerFactory.getLogger(logger_name);
this.position = position;
}
@@ -113,23 +122,20 @@ public class MsgMailUtil {
}
}
public void processLog(Level level, String message) {
private void processLog(Level level, String message) {
message = position + " - " + message;
if (this.log != null) {
switch (level.toInt()) {
case Level.TRACE_INT:
switch (level) {
case TRACE:
log.trace(message);
break;
case Level.ERROR_INT:
case ERROR:
log.error(message);
break;
case Level.FATAL_INT:
log.fatal(message);
break;
case Level.INFO_INT:
case INFO:
log.info(message);
break;
case Level.WARN_INT:
case WARN:
log.warn(message);
break;
default:

View File

@@ -2,9 +2,9 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend_DB_mssqlUtil-1.3-20190523</artifactId>
<version>6.0.0</version>
<groupId>org.talend.components.lib</groupId>
<artifactId>talend_DB_mssqlUtil</artifactId>
<version>1.4</version>
<packaging>jar</packaging>
<name>talend_DB_mssqlUtil</name>

View File

@@ -16,6 +16,8 @@ import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import java.util.TimeZone;
import java.text.SimpleDateFormat;
import java.text.ParseException;
/**
* DOC Administrator class global comment. Detailled comment
@@ -117,10 +119,15 @@ public class MSSqlGenerateTimestampUtil {
return null;
}
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
{
sdf.setTimeZone(TimeZone.getTimeZone("GMT"));
}
/**
* parse datetimeoffset string to date.
* datetimeoffset string show as YYYY-MM-DD hh:mm:ss[.nnnnnnn] [+|-]hh:mm
* and no enough infomation for the DST decision,so no DST consider[not sure if 100% support for DST].
*
* @param datetimeOffsetString
* @return
@@ -131,13 +138,17 @@ public class MSSqlGenerateTimestampUtil {
String offsetString = datetimeOffsetString.substring(idx + 1);
int offset = TimeZone.getTimeZone("GMT" + offsetString).getRawOffset();
// get local timezone, also consider the DST
TimeZone local = TimeZone.getDefault();
int localOffset = local.getOffset(new java.util.Date().getTime());
long milliseconds = java.sql.Timestamp.valueOf(datetimeString).getTime();
long millisecondsToGMT1970_01_01 = milliseconds - offset + localOffset;
return new java.util.Date(millisecondsToGMT1970_01_01);
long millisecondsWithOffset = 0l;
try {
millisecondsWithOffset = sdf.parse(datetimeString).getTime();
} catch(ParseException e) {
throw new RuntimeException(e.getMessage());
}
//get GMT's millisecond number since GMT 1970_01_01
long milliseconds4GMT_to_GMT_1970_01_01 = millisecondsWithOffset - offset;
return new java.util.Date(milliseconds4GMT_to_GMT_1970_01_01);
}
}

View File

@@ -1,10 +1,10 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<groupId>org.talend.components.lib</groupId>
<artifactId>talendsap</artifactId>
<name>talendsap</name>
<version>1.0.1</version>
<version>1.0.2</version>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>

View File

@@ -90,6 +90,7 @@ public class DocumentExtractor {
List<Element> tablesAndChangingElements = new ArrayList<Element>(3);
tablesAndChangingElements.add(functionElement.element("TABLES"));
tablesAndChangingElements.add(functionElement.element("CHANGING"));
tablesAndChangingElements.add(functionElement.element("OUTPUT"));
for(Element tablesOrChangingElement : tablesAndChangingElements) {
if (tablesOrChangingElement == null) {
@@ -121,6 +122,10 @@ public class DocumentExtractor {
}
result.add(row);
}
if(!result.isEmpty()) {
return result;
}
}
return result;

View File

@@ -49,11 +49,11 @@ public class DocumentHelper {
}
}
public void addSingleParameter(String name, String value, boolean isChanging) {
public void addSingleParameter(String name, String value, SAPParameterType parameter_type) {
if(value == null) {
value = "";
}
if (isChanging) {
if (parameter_type == SAPParameterType.CHANGING) {
correctChanging();
changing.addElement(name).setText(value);
} else {
@@ -62,8 +62,8 @@ public class DocumentHelper {
}
}
public void addStructParameter(String name, boolean isChanging) {
if (isChanging) {
public void addStructParameter(String name, SAPParameterType parameter_type) {
if (parameter_type == SAPParameterType.CHANGING) {
correctChanging();
currentStruct = changing.addElement(name);
} else {
@@ -79,13 +79,16 @@ public class DocumentHelper {
currentStruct.addElement(name).setText(value);
}
public void addTableParameter(String name, boolean isChanging) {
if (isChanging) {
public void addTableParameter(String name, SAPParameterType parameter_type) {
if (parameter_type == SAPParameterType.CHANGING) {
correctChanging();
currentTable = changing.addElement(name);
} else {
} else if(parameter_type == SAPParameterType.TABLES) {
correctTables();
currentTable = tables.addElement(name);
} else {
correctInput();
currentTable = input.addElement(name);
}
}
@@ -108,18 +111,18 @@ public class DocumentHelper {
DocumentHelper helper = new DocumentHelper();
helper.setFunctionName("READ_TABLE_FUNCTION");
helper.addSingleParameter("ID", "1", true);
helper.addSingleParameter("NAME", "gaoyan", false);
helper.addSingleParameter("ID", "1", SAPParameterType.CHANGING);
helper.addSingleParameter("NAME", "gaoyan", SAPParameterType.IMPORT);
helper.addStructParameter("INFO", true);
helper.addStructParameter("INFO", SAPParameterType.CHANGING);
helper.addStructChildParameter("ID", "2");
helper.addStructChildParameter("NAME", "wangwei");
helper.addStructParameter("INFO1", false);
helper.addStructParameter("INFO1", SAPParameterType.IMPORT);
helper.addStructChildParameter("ID1", "4");
helper.addStructChildParameter("NAME1", "momo");
helper.addTableParameter("TABLE1", false);
helper.addTableParameter("TABLE1", SAPParameterType.TABLES);
for (int i = 0; i < 200000; i++) {
helper.addTableRow();
helper.addTableRowChildParameter("c1", i + "");
@@ -132,7 +135,7 @@ public class DocumentHelper {
helper.addTableRowChildParameter("c8", "wangwei" + i);
}
helper.addTableParameter("TABLE2", false);
helper.addTableParameter("TABLE2", SAPParameterType.TABLES);
for (int i = 0; i < 2; i++) {
helper.addTableRow();
helper.addTableRowChildParameter("ID4", i + "");

View File

@@ -0,0 +1,8 @@
package org.talend.sap;
public enum SAPParameterType {
IMPORT,
CHANGING,
TABLES,
EXPORT
}

View File

@@ -1,88 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="talendssl.jar" />
<property name="component.name" value="tLDAPInput" />
<property name="author.name" value="wliu" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -149,6 +149,8 @@
<IMPORT NAME="Driver-V5R3_V6R1" MODULE="jt400_V5R3.jar" MVN="mvn:org.talend.libraries/jt400_V5R3/6.0.0" REQUIRED_IF="DB_VERSION == 'jt400_V5R3.jar'" />
<IMPORT NAME="Driver-V6R1_V7R2" MODULE="jt400_V6R1.jar" MVN="mvn:org.talend.libraries/jt400_V6R1/6.0.0" REQUIRED_IF="DB_VERSION == 'jt400_V6R1.jar'" />
<IMPORT NAME="Driver-V7R1_V7R3" MODULE="jt400-9.8.jar" MVN="mvn:net.sf.jt400/jt400/9.8" REQUIRED_IF="DB_VERSION == 'jt400-9.8.jar'" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -301,6 +301,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
@@ -308,6 +309,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.sql.SQLException e) {
@@ -431,6 +433,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
@@ -438,6 +441,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().updating(dbLog.var("nb_line"));%>
} catch(java.sql.SQLException e) {
@@ -584,6 +588,7 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -659,6 +664,7 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -833,11 +839,13 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
}else{
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().updating(dbLog.var("nb_line"));%>
}
} catch(java.sql.SQLException e) {
@@ -914,12 +922,14 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
}else {
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>

View File

@@ -58,7 +58,7 @@ USER.NAME=Username
USE_COMMIT_CONTROL.NAME=Use commit control
USE_EXISTING_CONNECTION.NAME=Using existing connection
USE_FIELD_OPTIONS.NAME=Use field options
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.V5R2_V5R4=V5R2 to V5R4 (Deprecated)
DB_VERSION.ITEM.V5R3_V6R1=V5R3 to V6R1 (Deprecated)

View File

@@ -123,6 +123,9 @@
<IMPORT NAME="common2.6" MODULE="commons-lang-2.6.jar" MVN="mvn:commons-lang/commons-lang/2.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-lang-2.6.jar" REQUIRED="true"/>
<IMPORT NAME="common-logging1.1.3" MODULE="commons-logging-1.1.3.jar" MVN="mvn:commons-logging/commons-logging/1.1.3" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-logging-1.1.3.jar" REQUIRED="true"/>
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -139,6 +139,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -228,6 +229,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -345,6 +347,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -406,6 +409,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try{
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -501,6 +505,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updateFlag_<%=cid%>=pstmtUpdate_<%=cid %>.executeUpdate();
if(updateFlag_<%=cid%> != 0) {
nb_line_<%=cid%>++;
@@ -567,6 +572,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -645,6 +651,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>

View File

@@ -52,7 +52,7 @@ FIELD_OPTIONS.ITEM.UPDATE_KEY=Key in update
FIELD_OPTIONS.ITEM.DELETE_KEY=Key in delete
FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
NB_LINE.NAME=Number of line
NB_LINE_UPDATED.NAME= NB Line Updated

View File

@@ -1,240 +0,0 @@
MaxMind GeoIP License version 1.6, March 23rd, 2007
MAXMIND END-USER LICENSE AGREEMENT
By clicking on the words "I Agree" below, you agree that your use of the
MaxMind GeoIP database products and services is subject to the terms and
conditions set forth in this MaxMind End User License Agreement (this
"Agreement").
MaxMind, Inc. ("MaxMind") provides a line of database services and products
that provide the geographic information and other data associated with
specific Internet protocol addresses. These database services and products
are referred to in this Agreement individually as the "GeoIP Database" and
collectively as the "GeoIP Databases". The data available through the GeoIP
Databases is referred to in this Agreement as the "GeoIP Data". Some
products in the MaxMind GeoIP Line may include MaxMind's proprietary computer
programs used to query the GeoIP Databases (the "GeoIP Programs").
ADDITIONAL POLICIES.
The following policies are incorporated into this End User Agreement by
reference and provide additional terms and conditions related to specific
services and products provided by MaxMind:
minFraud Service Terms of Use
Each of these policies may be amended at any time and such amendments
shall be binding and effective as indicated below in the section on
Changes To The Agreement.
LIMITED GRANT OF RIGHTS.
In accordance with the terms of this Agreement, MaxMind hereby grants you a
non-exclusive, non-transferable limited license to access and use the GeoIP
Databases and GeoIP Data for your own internal Restricted Business
purposes. Restricted Business purposes are limited to customizing website
content, fraud prevention, geographic reporting, and similar business
purposes. You agree to use the GeoIP Data and GeoIP Databases only in a
manner that is consistent with applicable laws.
RESTRICTIONS ON USE.
Except as expressly permitted in this Agreement, you may not, nor may you
permit others to:
(a) copy any portion of the GeoIP Databases except as reasonably required
for using the GeoIP Database as permitted hereunder,
(b) allow anyone other than yourself or your employees to access the GeoIP
Databases, or any portion thereof, without MaxMind's express written
permission,
(c) use the GeoIP Databases to develop a database, infobase, online or
similar database service, or other information resource in any media (print,
electronic or otherwise, now existing or developed in the future) for sale
to, distribution to, display to or use by others,
(d) create compilations or derivative works of the GeoIP Databases,
(e) use the GeoIP Databases in any fashion that may infringe any copyright,
intellectual property right, contractual right, or proprietary or property
right or interest held by MaxMind,
(f) store in a retrieval system accessible to the public, transfer,
publish, distribute, display to others, broadcast, sell, or sublicense the
GeoIP Databases, or any portion thereof,
(g) remove or obscure any copyright notice or other notice or terms of use
contained in the GeoIP Databases,
(h) use the GeoIP Data to create or otherwise support the transmission of
unsolicited, commercial email,
(i) remove, disable, avoid, circumvent, or defeat any functionality in the
GeoIP Databases or GeoIP Programs designed to limit or control access to or
use of the GeoIP Databases or GeoIP Data,
(j) use, copy or otherwise access any portion of the GeoIP Data for which
you have not made payment to MaxMind. If for any reason, you access such
GeoIP Data, these terms and conditions apply to your use of such data and
you agree to pay all applicable charges, or
(k) copy, reverse engineer, decompile, disassemble, derive source code,
modify or prepare derivative works of the GeoIP Programs.
OWNERSHIP AND INTELLECTUAL PROPERTY RIGHTS.
(a) Trade Secrets and Confidential Information. You acknowledge and agree
that any and all confidential information and materials from which MaxMind
derives actual or potential economic value constitutes MaxMind's
confidential and proprietary trade secrets (collectively, "Trade Secrets").
You further acknowledge and agree that MaxMind's Trade Secrets include, but
are not limited to, the GeoIP Data contained in the GeoIP Databases and the
technology used in the GeoIP Programs. You shall maintain any information
learned about MaxMind's Trade Secrets as confidential and shall not disclose
such information or permit such information to be disclosed to any person or
entity. With respect to all such information, you shall exercise the same
degree of care to protect MaxMind's Trade Secrets that you exercise with
respect to protecting your own confidential information, and in no event
less than reasonable care.
(b) Ownership. All intellectual property rights including copyrights, moral
rights, trademarks, trade secrets, proprietary rights to the GeoIP
Databases, GeoIP Data and GeoIP Programs are exclusively owned by MaxMind.
You acknowledge and agree that you obtain no right, title or interest
therein. You hereby assign to MaxMind all copyrights, intellectual property
rights, and any other proprietary or property rights or interests in and to
any work created in violation of this Agreement.
FEES.
MaxMind's current fee schedule for using the GeoIP Databases and related
services is posted on the MaxMind website (url: http://www.maxmind.com).
You are responsible for paying all fees associated with the use of the GEOIP
Databases.
CHANGES TO THE AGREEMENT.
MaxMind may amend this Agreement at any time. Any such amendment(s) shall
be binding and effective upon the earlier of (i) the date that is thirty
(30) days after posting of the amended Agreement on MaxMind's web site, or
(ii) the date that MaxMind provides notice to you of the amended Agreement
pursuant to the notice provisions in this Agreement; except that changes to
charges and payment terms may be made only upon 30 days' prior written
notice to you. You may immediately terminate this Agreement upon notice to
MaxMind if a change is unacceptable to you. Your continued use of the GeoIP
Data, GeoIP Databases or GeoIP Programs following notice to you of a change
shall constitute your acceptance of the change.
LIMITATION ON LIABILITY.
MAXMIND'S MAXIMUM TOTAL LIABILITY FOR ALL OCCURRENCES (IF ANY), TAKING PLACE
DURING ANY TWELVE-MONTH PERIOD (OR A PORTION THEREOF, IF THIS AGREEMENT IS NOT
IN EFFECT FOR TWELVE MONTHS), ARISING OUT OF OR IN ANY WAY RELATED TO THE
AUTHORIZED OR UNAUTHORIZED ACTS OF MAXMIND'S EMPLOYEES OR MAXMIND'S PERFORMANCE
OR NONPERFORMANCE OF THE SERVICES PROVIDED HEREIN, INCLUDING (BUT NOT LIMITED
TO) ERRORS OF DESIGN OR ERRORS WHICH ARE DUE SOLELY TO MALFUNCTION OF
MAXMIND-CONTROLLED MACHINES OR FAILURES OF MAXMIND OPERATORS, MAXMIND
PROGRAMMERS OR MAXMIND-DEVELOPED PROGRAMS, SHALL BE LIMITED TO GENERAL MONEY
DAMAGES IN AN AMOUNT NOT TO EXCEED THE TOTAL AMOUNT PAID BY YOU FOR SERVICES
PROVIDED BY MAXMIND UNDER THIS AGREEMENT DURING SAID TWELVE-MONTH PERIOD (OR
DURING SUCH SHORTER PERIOD THAT THIS AGREEMENT IS IN EFFECT). YOU AGREE THAT THE
FOREGOING SHALL CONSTITUTE YOUR EXCLUSIVE REMEDY. YOU HEREBY RELEASE MAXMIND,
ITS OFFICERS, EMPLOYEES AND AFFILIATES FROM ANY AND ALL OBLIGATIONS, LIABILITIES
AND CLAIMS IN EXCESS OF THIS LIMITATION.
NO CONSEQUENTIAL DAMAGES.
UNDER NO CIRCUMSTANCES INCLUDING NEGLIGENCE SHALL MAXMIND OR ANY RELATED PARTY OR SUPPLIER BE
LIABLE FOR INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR PUNITIVE DAMAGES;
OR FOR LOSS OF PROFITS, REVENUE, OR DATA; THAT ARE DIRECTLY OR INDIRECTLY
RELATED TO THE USE OF, OR THE INABILITY TO USE THE SITE OR SERVICES, WHETHER
IN AN ACTION IN CONTRACT, TORT, PRODUCT LIABILITY, STRICT LIABILITY, STATUTE
OR OTHERWISE EVEN IF MAXMIND HAS BEEN ADVISED OF THE POSSIBILITY OF THOSE
DAMAGES.
NO WARRANTIES.
THE GEOIP DATABASES, GEOIP DATA AND GEOIP PROGRAMS ARE FURNISHED ON AN "AS
IS", AS-AVAILABLE BASIS. MAXMIND MAKES NO WARRANTY, EXPRESS OR IMPLIED,
WITH RESPECT TO THE CAPABILITY OF THE GEOIP DATABASES, GEOIP DATA AND GEOIP
PROGRAMS OR THE ACCURACY OR THE COMPLETENESS OF THE GEOIP DATA. ALL
WARRANTIES OF ANY TYPE, EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT OF
THIRD PARTY RIGHTS ARE EXPRESSLY DISCLAIMED. MAXMIND DOES NOT WARRANT THAT
THE SERVICES WILL MEET ANY PARTICULAR CRITERIA OF PERFORMANCE OR QUALITY, OR
THAT THE SITE IS FREE OF OTHER HARMFUL COMPONENTS. NEVERTHELESS, MAXMIND
SHALL MAKE COMMERCIALLY REASONABLE EFFORTS TO MAINTAIN THE SITE FREE OF
VIRUSES AND MALICIOUS CODE.
GOVERNING LAW.
This Agreement shall be governed and interpreted pursuant to the laws of the
Commonwealth of Massachusetts, applicable to contracts made and to be
performed wholly in Massachusetts, without regard to principles of conflicts
of laws.
You specifically consent to personal jurisdiction in Massachusetts in connection
with any dispute between you and MaxMind arising out of this Agreement or
pertaining to the subject matter hereof. You agree that the exclusive venue for
any dispute hereunder shall be in the state and federal courts in Boston,
Massachusetts.
NOTICES
Notices given under this Agreement shall be in writing and sent by
facsimile, e-mail, or by first class mail or equivalent. MaxMind shall
direct notice to you at the facsimile number, e-mail address, or physical
mailing address (collectively, "Address") you provided in the registration
process. You shall direct notice to MaxMind at the following address:
MaxMind, Inc.
PO Box 230074
Boston MA 02123-0074
Email: legal@maxmind.com
Fax: (815) 301-8737
Either party may change its Address for notice at anytime by giving notice
of the new Address as provided in this section.
COMPLETE AGREEMENT.
This Agreement represents
the entire agreement between you and MaxMind with respect to the subject
matter hereof and supersedes all previous representations, understandings or
agreements, oral or written, between the parties regarding the subject
matter hereof.
APPLICABLE LAWS.
You agree to use the GeoIP Data and GeoIP Databases only in a
manner that is consistent with applicable laws.
ASSIGNMENT.
You may not assign your rights in this Agreement without MaxMind's prior
written consent.
SEVERABILITY.
Should any provision of this Agreement be held void, invalid or inoperative,
such decision shall not affect any other provision hereof, and the remainder
of this Agreement shall be effective as though such void, invalid or
inoperative provision had not been contained herein.
FAILURE TO ENFORCE.
The failure of MaxMind to enforce any provision of these terms and
conditions shall not constitute or be construed as a waiver of such
provision or of the right to enforce it at a later time.
FACSIMILES.
If you submit a document which includes your facsimile signature, MaxMind
may treat the facsimile signature as an original of your signature.
CAPTIONS.
The section headings used herein, are for convenience only and shall have no
force or effect upon the construction or interpretation of any provision
hereof.

View File

@@ -211,10 +211,8 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java_DOM4J1.6" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar"
REQUIRED="true" BundleID="" />
<IMPORT NAME="Java_JAXEN1.1" MODULE="jaxen-1.1.1.jar" MVN="mvn:org.talend.libraries/jaxen-1.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/jaxen-1.1.1.jar"
REQUIRED="true" BundleID="" />
<IMPORT NAME="Java_DOM4J_2.1.1" MODULE="dom4j-2.1.1.jar" MVN="mvn:org.dom4j/dom4j/2.1.1" REQUIRED="true" BundleID="" />
<IMPORT NAME="Java_JAXEN_1.1.6" MODULE="jaxen-1.1.6.jar" MVN="mvn:jaxen/jaxen/1.1.6" REQUIRED="true" BundleID="" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -215,6 +215,8 @@
</TEMPLATES>
<IMPORTS>
<IMPORT NAME="Driver-MysqlConnector" MODULE="mysql-connector-java-5.1.30-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.30-bin/6.0.0" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="(USE_SHARED_CONNECTION == 'true' AND SPECIFY_DATASOURCE_ALIAS=='false')" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="(USE_SHARED_CONNECTION == 'true' AND SPECIFY_DATASOURCE_ALIAS=='false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS/>

View File

@@ -73,7 +73,7 @@ FIELD_OPTIONS.ITEM.DELETE_KEY=Key in delete
FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
PROPERTIES.NAME=Additional JDBC Parameters
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
NB_LINE_UPDATED.NAME=Number Of Updated Lines
NB_LINE_INSERTED.NAME=Number Of Inserted Lines

View File

@@ -121,14 +121,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -427,14 +427,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -178,14 +178,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -164,6 +164,8 @@
<IMPORTS>
<IMPORT NAME="Driver-MysqlConnector" MODULE="mysql-connector-java-5.1.30-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.30-bin/6.0.0" REQUIRED_IF="DB_VERSION == 'MYSQL_5'" />
<IMPORT NAME="Driver-MysqlConnector_Old" MODULE="mysql-connector-java-5.1.0-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.0-bin/6.0.0" REQUIRED="false" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS/>

View File

@@ -763,6 +763,7 @@ if(columnList != null && columnList.size() > 0) {
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
<%
if (!("").equals(numPerInsert ) && !("0").equals(numPerInsert )) {
@@ -776,6 +777,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -853,6 +855,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -979,6 +982,7 @@ if(columnList != null && columnList.size() > 0) {
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
@@ -986,6 +990,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().updating(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -1137,6 +1142,7 @@ if(columnList != null && columnList.size() > 0) {
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -1212,6 +1218,7 @@ if(columnList != null && columnList.size() > 0) {
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -1389,11 +1396,13 @@ if(columnList != null && columnList.size() > 0) {
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
}else{
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().updating(dbLog.var("nb_line"));%>
}
} catch(java.lang.Exception e) {
@@ -1472,12 +1481,14 @@ if(columnList != null && columnList.size() > 0) {
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
}else {
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>
@@ -1625,6 +1636,7 @@ if(columnList != null && columnList.size() > 0) {
%>
int count_on_duplicate_key_<%=cid%> = 0;
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
count_on_duplicate_key_<%=cid%> = pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().insertingOnDuplicateKeyUpdating(null);%>
} catch(java.lang.Exception e) {
@@ -1690,6 +1702,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().insertingOnDuplicateKeyUpdating(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {

View File

@@ -82,7 +82,7 @@ FIELD_OPTIONS.ITEM.DELETE_KEY=Key in delete
FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
PROPERTIES.NAME=Additional JDBC Parameters
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
NB_LINE_UPDATED.NAME=Number Of Updated Lines
NB_LINE_INSERTED.NAME=Number Of Inserted Lines

View File

@@ -56,7 +56,7 @@ imports="
if(isLog4jEnabled){
shareDBClass = "SharedDBConnectionLog4j";
%>
SharedDBConnectionLog4j.initLogger(log,"<%=cid%>");
SharedDBConnectionLog4j.initLogger(log.getName(),"<%=cid%>");
<%
}
%>

View File

@@ -243,6 +243,8 @@
<IMPORT NAME="Driver-Ojdbc8" MODULE="ojdbc8-12.2.0.1.jar" MVN="mvn:com.oracle.jdbc/ojdbc8/12.2.0.1" REQUIRED_IF="(DB_VERSION == 'ORACLE_18')" />
<IMPORT NAME="Driver-Ojdbc7" MODULE="ojdbc7.jar" MVN="mvn:org.talend.libraries/ojdbc7/6.0.0" REQUIRED_IF="(DB_VERSION == 'ORACLE_12')" />
<IMPORT NAME="Driver-Ojdbc6" MODULE="ojdbc6.jar" MVN="mvn:org.talend.libraries/ojdbc6/6.0.0" REQUIRED_IF="(DB_VERSION == 'ORACLE_11')" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -46,6 +46,10 @@ String batchSize=ElementParameterParser.getValue(node,"__BATCH_SIZE__");
boolean useExistingConnection = "true".equals(ElementParameterParser.getValue(node,"__USE_EXISTING_CONNECTION__"));
boolean useAlternateSchema = "true".equals(ElementParameterParser.getValue(node, "__USE_ALTERNATE_SCHEMA__"));
String alternateSchema = ElementParameterParser.getValue(node, "__ALTERNATE_SCHEMA__");
String rejectConnName = null;
List<? extends IConnection> rejectConns = node.getOutgoingConnections("REJECT");
if(rejectConns != null && rejectConns.size() > 0) {
@@ -198,8 +202,16 @@ if(useExistingConnection) {
String connection = ElementParameterParser.getValue(node,"__CONNECTION__");
String conn = "conn_" + connection;
String schema = "dbschema_" + connection;
if(useAlternateSchema) {
%>
dbschema_<%=cid%> = <%=alternateSchema%>;
<%
} else {
%>
dbschema_<%=cid%> = (String)globalMap.get("<%=schema%>");
<%
}
%>
dbschema_<%=cid%> = (String)globalMap.get("<%=schema%>");
conn_<%=cid%> = (java.sql.Connection)globalMap.get("<%=conn%>");
<%dbLog.conn().useExistConn("conn_"+cid+".getMetaData().getURL()", "conn_"+cid+".getMetaData().getUserName()");%>
<%

View File

@@ -145,7 +145,7 @@
NUM_ROW="50"
REPOSITORY_VALUE="SCHEMA"
REQUIRED="false"
SHOW_IF="((USE_EXISTING_CONNECTION == 'false') "
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') "
>
<DEFAULT>""</DEFAULT>
</PARAMETER>
@@ -266,7 +266,6 @@
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') and (CONNECTION_TYPE != 'ORACLE_WALLET')">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="NOTE"
FIELD="LABEL"
@@ -274,6 +273,26 @@
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') and (CONNECTION_TYPE != 'ORACLE_WALLET')">
<DEFAULT>*Note: Example for Additional JDBC Parameters: "parameterName1=value1&amp;&amp;parameterName2=value2"</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="USE_ALTERNATE_SCHEMA"
FIELD="CHECK"
NUM_ROW="15"
SHOW_IF="USE_EXISTING_CONNECTION == 'true'"
>
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="ALTERNATE_SCHEMA"
FIELD="TEXT"
NUM_ROW="20"
REQUIRED="false"
SHOW_IF="(USE_EXISTING_CONNECTION == 'true') and (USE_ALTERNATE_SCHEMA == 'true')"
>
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="ENCODING"
FIELD="ENCODING_TYPE"

View File

@@ -453,6 +453,7 @@ IMetadataColumn talendDynCol = null;
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -594,6 +595,7 @@ IMetadataColumn talendDynCol = null;
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().updating(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -758,6 +760,7 @@ IMetadataColumn talendDynCol = null;
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -838,6 +841,7 @@ IMetadataColumn talendDynCol = null;
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -1023,11 +1027,13 @@ IMetadataColumn talendDynCol = null;
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
}else{
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().updating(dbLog.var("nb_line"));%>
}
} catch(java.lang.Exception e) {
@@ -1116,6 +1122,7 @@ IMetadataColumn talendDynCol = null;
}else {
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>

View File

@@ -77,7 +77,7 @@ FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
LOCAL_SERVICE_NAME.NAME=Local Service Name
CONVERT_COLUMN_TABLE_TO_UPPERCASE.NAME=Convert columns and table to uppercase
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.ORACLE_18=Oracle 18
DB_VERSION.ITEM.ORACLE_12=Oracle 12-7
@@ -118,3 +118,5 @@ SUPPORT_NULL_WHERE.NAME=Support null in "SQL WHERE" statement
USE_TIMESTAMP_FOR_DATE_TYPE.NAME = Use Timestamp for Date type
NB_LINE_REJECTED.NAME=Number Of Rejected Lines
USE_ALTERNATE_SCHEMA.NAME=Use alternate schema
ALTERNATE_SCHEMA.NAME=Schema

View File

@@ -234,14 +234,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -589,11 +589,11 @@
<%if(setFieldDelimiter) {
%>
loadJobBuilder_<%=cid%>.setFormatOptions(csvOptions_<%=cid%>.setFieldDelimiter(<%=fieldDelimiter%>).build());
csvOptions_<%=cid%>.setFieldDelimiter(<%=fieldDelimiter%>);
<%
}
%>
loadJobBuilder_<%=cid%>.setFormatOptions(csvOptions_<%=cid%>.build());
loadJobBuilder_<%=cid%>.setNullMarker("\\N");
com.google.cloud.bigquery.Job job_<%=cid%> = bigquery_<%=cid%>.create(com.google.cloud.bigquery.JobInfo.of(loadJobBuilder_<%=cid%>.build()));
job_<%=cid%> = job_<%=cid%>.waitFor(com.google.cloud.RetryOption.initialRetryDelay(org.threeten.bp.Duration.ofSeconds(1)));

View File

@@ -190,7 +190,7 @@
<IMPORT NAME="google-oauth-client-1.19.0.jar" MODULE="google-oauth-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-oauth-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-oauth-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.19.0.jar" MODULE="google-http-client-jackson2-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-jackson2-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-jackson2-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar" MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1" REQUIRED="true" />
<!-- REQUIRED FOR GOOGLE STORAGE -->
<IMPORT NAME="jets3t-0.9.1" MODULE="jets3t-0.9.1.jar" MVN="mvn:org.talend.libraries/jets3t-0.9.1/6.0.0" REQUIRED="true" />
<IMPORT NAME="commons-logging-1.1.1" MODULE="commons-logging-1.1.1.jar" MVN="mvn:org.talend.libraries/commons-logging-1.1.1/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.1.1.v201101211721.jar" REQUIRED="true" />

View File

@@ -178,7 +178,7 @@
<IMPORT NAME="google-oauth-client-1.19.0.jar" MODULE="google-oauth-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-oauth-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-oauth-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.19.0.jar" MODULE="google-http-client-jackson2-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-jackson2-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-jackson2-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar" MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1" REQUIRED="true" />
<IMPORT NAME="google-cloud-bigquery-1.32.0.jar" MODULE="google-cloud-bigquery-1.32.0.jar" MVN="mvn:com.google.cloud/google-cloud-bigquery/1.32.0" REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'" />
<IMPORT NAME="google-http-client-jackson-1.23.0.jar" MODULE="google-http-client-jackson-1.23.0.jar" MVN="mvn:com.google.http-client/google-http-client-jackson/1.23.0" REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'" />
<IMPORT NAME="threetenbp-1.3.3.jar" MODULE="threetenbp-1.3.3.jar" MVN="mvn:org.threeten/threetenbp/1.3.3" REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'" />

View File

@@ -195,8 +195,8 @@
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar"
MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true"/>
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9" REQUIRED="true"/>
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1" REQUIRED="true"/>
<IMPORT NAME="google-cloud-bigquery-1.32.0.jar" MODULE="google-cloud-bigquery-1.32.0.jar"
MVN="mvn:com.google.cloud/google-cloud-bigquery/1.32.0"
REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'"/>

View File

@@ -37,8 +37,8 @@
<PARAMETER NAME="DB_VERSION" FIELD="CLOSED_LIST" NUM_ROW="1">
<ITEMS DEFAULT="BONITA_523">
<ITEM NAME="BONITA_523" VALUE="antlr-2.7.6.jar;asm-1.5.3.jar;asm-3.1.jar;asm-attrs-1.5.3.jar;cglib-nodep-2.1_3.jar;commons-collections-2.1.1.jar;commons-logging-1.0.4.jar;dom4j-1.6.1.jar;ehcache-1.2.3.jar;groovy-all-1.7.0.jar;h2-1.2.132.jar;hibernate-3.2.6.ga.jar;jline-0.9.94.jar;jta-1.1.jar;novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar" />
<ITEM NAME="BONITA_531" VALUE="novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar;hibernate-core-3.5.6-Final.jar;commons-collections-3.2.2.jar;ehcache-core-2.2.0.jar;hibernate-commons-annotations-3.2.0.Final.jar;hibernate-search-3.2.1.Final.jar;javassist-3.8.0.GA.jar;lucene-core-2.9.3.jar;slf4j-api-1.6.1.jar;dom4j-1.6.1.jar;h2-1.2.132.jar;jta-1.1.jar;antlr-2.7.6.jar;commons-logging-1.1.1.jar" />
<ITEM NAME="BONITA_523" VALUE="antlr-2.7.6.jar;asm-1.5.3.jar;asm-3.1.jar;asm-attrs-1.5.3.jar;cglib-nodep-2.1_3.jar;commons-collections-2.1.1.jar;commons-logging-1.0.4.jar;dom4j-2.1.1.jar;ehcache-1.2.3.jar;groovy-all-1.7.0.jar;h2-1.2.132.jar;hibernate-3.2.6.ga.jar;jline-0.9.94.jar;jta-1.1.jar;novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar" />
<ITEM NAME="BONITA_531" VALUE="novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar;hibernate-core-3.5.6-Final.jar;commons-collections-3.2.2.jar;ehcache-core-2.2.0.jar;hibernate-commons-annotations-3.2.0.Final.jar;hibernate-search-3.2.1.Final.jar;javassist-3.8.0.GA.jar;lucene-core-2.9.3.jar;slf4j-api-1.6.1.jar;dom4j-2.1.1.jar;h2-1.2.132.jar;jta-1.1.jar;antlr-2.7.6.jar;commons-logging-1.1.1.jar" />
<ITEM NAME="BONITA_561" VALUE="BONITA_561" />
<ITEM NAME="BONITA_5101" VALUE="BONITA_5101" />
<ITEM NAME="BONITA_652" VALUE="BONITA_652" />
@@ -177,7 +177,7 @@
<IMPORT NAME="cglib_nodep_213" MODULE="cglib-nodep-2.1_3.jar" MVN="mvn:org.talend.libraries/cglib-nodep-2.1_3/6.0.0" REQUIRED="false" />
<IMPORT NAME="commons_collections_211" MODULE="commons-collections-2.1.1.jar" MVN="mvn:org.talend.libraries/commons-collections-2.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-collections-2.1.1.jar" REQUIRED="false" />
<IMPORT NAME="commons_logging_104" MODULE="commons-logging-1.0.4.jar" MVN="mvn:org.talend.libraries/commons-logging-1.0.4/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.0.4.v201101211617.jar" REQUIRED="false" />
<IMPORT NAME="dom4j_161" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar" REQUIRED_IF="DB_VERSION=='BONITA_561'" BundleID="" />
<IMPORT NAME="dom4j_2.1.1" MODULE="dom4j-2.1.1.jar" MVN="mvn:org.dom4j/dom4j/2.1.1" REQUIRED_IF="DB_VERSION=='BONITA_561'" BundleID="" />
<IMPORT NAME="ehcache_123" MODULE="ehcache-1.2.3.jar" MVN="mvn:org.talend.libraries/ehcache-1.2.3/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ehcache/lib/ehcache-1.2.3.jar" REQUIRED="false" />
<IMPORT NAME="groovy_all_170" MODULE="groovy-all-1.7.0.jar" MVN="mvn:org.talend.libraries/groovy-all-1.7.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.groovy/lib/groovy-all-1.7.0.jar" REQUIRED="false" />

Some files were not shown because too many files have changed in this diff Show More