Compare commits

...

154 Commits

Author SHA1 Message Date
sponomarova
d4f1f09b0d Merge branch 'master' into fix/TBD-9317 2020-01-20 17:47:59 +02:00
Mike Yan
2ed876833f fix(TESB-27912):[CI]Cannot skip docker for DemoRESTConsumer (#4349) 2020-01-20 18:24:10 +08:00
hzhao-talendbj
05f87d09e3 workItem(TUP-21195)replace all forms of subjob with subJob (#4348)
* workItem(TUP-21195)replace all forms of subjob with subJob

* workItem(TUP-21195)replace all forms of subjob with subJob

* workItem(TUP-21195)replace all forms of subjob with subJob
2020-01-20 16:54:43 +08:00
hwang-talend
444d28c48d bugfix(TUP-25764):Win:Cannot load component "tImpalaOutput" error in (#4324)
error log list if Studio store path contain space
2020-01-20 14:59:05 +08:00
Chao MENG
a7d010413d fix(TUP-25152): component does not display in palette, after deployed to (#4343)
the studio 7.2.1

https://jira.talendforge.org/browse/TUP-25152
2020-01-20 11:07:41 +08:00
jiezhang-tlnd
6c4981c2bf fix(TUP-25843)Hide log4j/log4j2 combox when it is not enabled for new (#4338)
created project after 7.3.1.
https://jira.talendforge.org/browse/TUP-25843
2020-01-20 11:02:43 +08:00
Chao MENG
ae3e6bf7be fix(TUP-25545): tWorkdayInput --> Mode and WQL Query can't edit when (#4330)
use metadata

https://jira.talendforge.org/browse/TUP-25545
2020-01-20 10:58:48 +08:00
hwang-talend
730d9cb8e9 bugfix(TUP-25768):Ubuntu:Switch to a new workspace not work if Studio (#4336)
store path contain space
2020-01-20 10:30:37 +08:00
Chao MENG
e58754832e feat(TUP-25226): Make the hadoop conf jar path a contextualizable parameter in the Hadoop conf wizard (#4203)
* feat(TUP-25226): Make the hadoop conf jar path a contextualizable
parameter in the Hadoop conf wizard

https://jira.talendforge.org/browse/TUP-25226

* fix(TUP-25585): [BUG]Data viewer failed for HDFS

https://jira.talendforge.org/browse/TUP-25585

* feat(TUP-25226): Make the hadoop conf jar path a contextualizable
parameter in the Hadoop conf wizard

https://jira.talendforge.org/browse/TUP-25226

* feat(TUP-25226): Make the hadoop conf jar path a contextualizable
parameter in the Hadoop conf wizard

https://jira.talendforge.org/browse/TUP-25226

* feat(TUP-25226): Make the hadoop conf jar path a contextualizable
parameter in the Hadoop conf wizard

https://jira.talendforge.org/browse/TUP-25226

* Revert "feat(TUP-25226): Make the hadoop conf jar path a contextualizable parameter in the Hadoop conf wizard"

This reverts commit b04d1bdca4.

* Revert "feat(TUP-25226): Make the hadoop conf jar path a contextualizable parameter in the Hadoop conf wizard"

This reverts commit 0fd1ce627b.

* Revert "feat(TUP-25226): Make the hadoop conf jar path a contextualizable parameter in the Hadoop conf wizard"

This reverts commit 434c102f8c.
2020-01-19 16:10:08 +08:00
msjian
d6927c33fa feat(TDQ-17294): support snowflake guess schema well (#4337) 2020-01-19 15:23:51 +08:00
hcyi
7ea06495a6 fix(TUP-25533):Talend Component Kit Components cannot be renamed in (#4341)
Studio.
2020-01-19 14:47:56 +08:00
Jane Ding
8b4dd6398d fix(TUP-21555)Error pops up if run job during opening another job in git (#4342)
* fix(TUP-21555)Error pops up if run job during opening another job in git
project
https://jira.talendforge.org/browse/TUP-21555

* fix(TUP-21555)Error pops up if run job during opening another job in git
project
https://jira.talendforge.org/browse/TUP-21555
2020-01-19 14:24:13 +08:00
apoltavtsev
5e65e4ed26 fix(TESB-27583): Maven nature is added to child Job for OSGI build 2020-01-17 14:46:29 +01:00
kjwang
49a907b477 feat(TUP-25786): Enable online help (#4339)
* Fix TUP-25786 Enable online help
https://jira.talendforge.org/browse/TUP-25786

* Fix TUP-25786 Enable online help
https://jira.talendforge.org/browse/TUP-25786
2020-01-17 18:10:42 +08:00
vdrokov
68758eadc7 fix(TESB-27713): Fix by checking if the child job is of type routines.system.api.TalendESBJob since method setEndpointRegistry exists only in routines.system.api.TalendESBJob interface and sometimes the job is not of that type. (#4334) 2020-01-17 11:57:10 +02:00
jiezhang-tlnd
d9f8f4edcd fix(TUP-25427)active or deactivate log4j2 need to reopen the child job (#4323)
https://jira.talendforge.org/browse/TUP-25427
2020-01-17 17:22:13 +08:00
hcyi
a5c06b4267 fix(TUP-25649):ERROR when Job in MDM project calling Job in Reference DI (#4331)
project:import routines.DataQuality cannot be resolved.
2020-01-17 16:59:50 +08:00
vyu-talend
b2fd2e2a60 fix(TDI-42920):update tAmazonEMRManage. (#4317) 2020-01-17 15:45:18 +08:00
Jane Ding
f5bbcdb213 fix(TUP-25862)Change client ip of SDK server from localhost to 127.0.0.1 (#4328)
https://jira.talendforge.org/browse/TUP-25862
2020-01-17 14:47:24 +08:00
kjwang
95e561d5ff feat(TUP-25241): Online Help in studio - Setup online Help (#4264)
* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help (Improve the
file's format)
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241

* Fix TUP-25241 Online Help in studio - Setup online Help
https://jira.talendforge.org/browse/TUP-25241
2020-01-17 13:46:23 +08:00
Dmytro Sylaiev
bf8d944ed1 fix(TDI-43222): Lock write operation by filename in parallel (#4256)
* fix(TDI-43222): Lock write operation by filename in parallel

* fix(TDI-43222): Apply same changes for tFileOutputPositional
2020-01-16 12:38:00 +02:00
wang wei
1f1bceca93 fix(TDI-43509): Upgrade redshift driver for 7.3.1 (#4302) 2020-01-16 17:41:25 +08:00
jzhao
6f5a6e0443 feat(TDI-43516):Salesforce api update to 47.0 (#4304)
* feat(TDI-43516):Salesforce api update to 47.0

* fix(TDI-43516):tSalesforceEinsteinBulkExec default endpoint not upgrade
2020-01-16 16:57:56 +08:00
Chao MENG
9892b271ea Cmeng/feat/tup 25554 add back share component (#4313)
* feat(TUP-25554): Add back the share component part of component manager

https://jira.talendforge.org/browse/TUP-25554

* feat(TUP-25554): Add back the share component part of component manager

https://jira.talendforge.org/browse/TUP-25554

* feat(TUP-25554): Add back the share component part of component manager

https://jira.talendforge.org/browse/TUP-25554
2020-01-16 10:51:55 +08:00
chmyga
3ff14ecbb5 Dchmyga/tdi 42591 bigquery dynamic schema (#4196)
* feat(TDI-42591): bigquery dynamic column support

* Add helpers for bigquery input components

* feat(TDI-42591): bigquery dynamic column support

* Add dynamic column support to bigquery input component

* feat(TDI-42591): bigquery dynamic column support

* Add dynamic column support for bigquery output component

* feat(TDI-42591): bigquery dynamic column support

* Fix PR comments

* feat(TDI-42591): bigquery dynamic column support

* Fix fixed component name issue

* feat(TDI-42591): bigquery dynamic column support

* Add support of geography type

* Add support of numeric type

* Fix bytes type support

* feat(TDI-42591): bigquery dynamic column support

* Fix OAuth null objects issue

* Fix byte array service account issue

* feat(TDI-42591): bigquery dynamic column support

* fix compilation error

* feat(TDI-42591): bigquery dynamic column support

* Exclude record type from table creation for OAuth
2020-01-15 16:57:07 +02:00
vyu-talend
cd7f09fc90 fix(TDI-43555):assign name for parameters in Outline. (#4322) 2020-01-15 17:29:42 +08:00
hcyi
10d4f8dcb6 fix(TUP-25727):Dataviewer for hdfs schema failed if it's checked webHDFS (#4311)
encryption
2020-01-14 14:41:58 +08:00
Jane Ding
1c58501af2 fix(TUP-25820)TMC: error show in error log when logon TMC project (#4318)
https://jira.talendforge.org/browse/TUP-25820
2020-01-14 14:28:12 +08:00
jiezhang-tlnd
a2f0a61607 change back log4j version to 2.12.1 (#4321) 2020-01-14 11:25:33 +08:00
bhe-talendbj
0095b468f9 bugfix(TUP-25707) Set correct text direction for destination combo (#4312)
* bugfix(TUP-25707) Set correct text direction for destination combo

* bugfix(TUP-25707) Fix test direction
2020-01-14 09:10:24 +08:00
Richard Lecomte
7bfad56c30 TDI-43421 : google cloud bigquery api update (#4316) 2020-01-13 11:58:10 +01:00
Jane Ding
efbf56f6d0 fix(TUP-25650)On enabling Parallelization on tDBInput, the data is (#4306)
getting inserted as NULLs
https://jira.talendforge.org/browse/TUP-25650
2020-01-13 18:33:17 +08:00
Dmytro Sylaiev
c29f483f2c fix(TDI-42875): Add missing log4j to component deps (#4315) 2020-01-13 11:17:35 +02:00
vdrokov
c05d88ee3d fix(TESB-27740): Fix error by downgrading log4j library from version 2.12.1 to version 2.11.1. 2.11.1 version is compatible with spring boot 2.1.2.RELEASE so no error is observed after downgrade. Replace strings with constants. (#4289) 2020-01-13 10:41:09 +02:00
hwang-talend
ecdf7a18b0 bugfix(TUP-24413):The tab name of resource is changed (#4298) 2020-01-13 10:26:15 +08:00
jzhao
6eddb2cd20 feat(TDI-43395):update dom4j to 2.1.1 (#4296)
* feat(TDI-43395):update dom4j

* feat(TDI-43395):update dom4j depends jaxen

* feat(TDI-43395):dom4j api changed

* fix compiler error
* remove useless code of txmlmap

* feat(TDI-43395):add migration task for bonita components after dom4j
update
2020-01-10 17:18:26 +08:00
Dmytro Sylaiev
0bd016b1c5 Dsylaiev/tdi 42875 upgrade t google dataproc crm (#4212)
* feat(TDI-42875): Upgrade google dataproc

* Add region enum

* Update zones

* Make file property use file widget

* feat(TDI-42875): Implement generated code for feature

* feat(TDI-42875): Remore IF in required operator

* feat(TDI-42875): Use region for stopping cluster either

* feat(TDI-42875): Add commons-codec in dependencies
2020-01-09 13:21:31 +02:00
hwang-talend
93b97ab0ed bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4307)
bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2020-01-09 17:11:18 +08:00
hwang-talend
0e08ab1a54 bugfix(TUP-24510):Studio can't use JDBC connection to trace job (#4273)
bugfix(TUP-24510):Studio can't use JDBC connection to trace job stats&logs
2020-01-09 10:50:17 +08:00
Mike Yan
c3f79cd79e feat(TESB-25283)[CI] Support Maven based Versioning for ESB Use cases (#4057)
* feat(TESB-25283)[CI] Support Maven based Versioning for ESB Use cases

* fix(TESB-25283):update osgi pom template to skip deploy
2020-01-09 10:48:24 +08:00
jiezhang-tlnd
29c1c47d97 fix(TUP-21660)add slf4j-simple.jar to Bundle-ClassPath in the plugin (#4303)
org.talend.sdk.component.studio-integration
2020-01-09 10:01:33 +08:00
sponomarova
ccdaa1af1c fix(TBD-9317): clean up components from org.talend.libraries.pig dependency (#4012)
* fix(TBD-9317): clean up components from org.talend.libraries.pig
2020-01-08 12:58:55 +02:00
hcyi
25857d0c10 fix(TUP-25229):Some TCOMPV0's components should not has "Use this (#4299)
Component" in Connection Component
2020-01-07 18:12:15 +08:00
jiezhang-tlnd
0f3a3dda19 fix(TUP-25702)TOS:Log4j:cannot print info on console when run job (#4301)
* fix(TUP-25702)TOS:Log4j:cannot print info on console when run job
https://jira.talendforge.org/browse/TUP-25720

* fix(TUP-25702)TOS:Log4j:cannot print info on console when run job
2020-01-07 16:21:14 +08:00
zshen-talend
943dbe61b2 fix(TDQ-17709): create number limit text controller (#4278)
* fix(TDQ-17709): create number limit text controller

* fix(TDQ-17709): improve code and add some lost modify

* fix(TDQ-17709): improve the code about the limit

* fix(TDQ-17709): remove unused code
2020-01-07 11:27:30 +08:00
vyu-talend
2c7dca1458 Vyu/tdi 43446 fix compilation error for tmap (#4291)
* Fix(TDI-43446):fix compilation error.

* Fix(TDI-43446):fix issues.

* Fix(TDI-43446):fix a bug written by myself.

* Fix(TDI-43446):fix an issue.

* Fix(TDI-43446):fix the exception problem.

* Fix(TDI-43446):fix issue.
2020-01-06 17:36:59 +08:00
jiezhang-tlnd
06a4ca938c fix(TUP-21660)SLF4J: Failed to load class (#4300)
"org.slf4j.impl.StaticLoggerBinder"
2020-01-06 09:59:15 +08:00
Zhiwei Xue
217360420f feat(TUP-24722):Force each studio on the same project to install the same patch (#4270)
* feat(TUP-24722):Force each studio on the same project to install the
same patch

* feat(TUP-24722):refactor util class.
2020-01-03 18:14:51 +08:00
hwang-talend
9bf91aca62 bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4281)
* bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2020-01-03 17:30:21 +08:00
jiezhang-tlnd
155973785f fix(TUP-25572) improve the performance of import job (#4295)
* fix(TUP-25572) improve the performance of import job

* fix(TUP-25572) improve the performance of import job
2020-01-03 16:43:45 +08:00
jiezhang-tlnd
5e0f8e9194 fix(TUP-25338)Run job use sap miss jar slf4j-api-1.7.16 and there is no (#4292)
* fix(TUP-25338)Run job use sap miss jar slf4j-api-1.7.16 and there is no
pop up let me download or browse in.
https://jira.talendforge.org/browse/TUP-25338

* fix(TUP-25338) avoid multiple versions

* avoid multiple module versions

* remove the same version jar only about slf4j-api

* remove the same version jar only about slf4j-api

* avoid multiple versions of slf4j-api
2020-01-03 16:36:29 +08:00
Jane Ding
64b049164c feat(TUP-24057)There is no error message when studio can't connect to (#4297)
customer artifact repository
https://jira.talendforge.org/browse/TUP-24057
2020-01-03 11:25:19 +08:00
Emmanuel GALLOIS
1be6c8fdd4 chore(TDI-40449): removed unused jars (#2815)
* chore(TDI-40449): removed unused jars

* chore(TDI-40449): add version to jars w/o version

* revert(TDI-40449): remove external_sort jar

* Revert: chore(TDI-40449): removed external_sort.jar

* chore(TDI-40449): remove external_sort

* chore(TDI-40449): remove jnp-client and talend-bigdata-launcher

* chore(TDI-40449): fix maven paths

* chore(TDI-40449): restore boxjavalibv2-v3.0.10.jar

* chore(TDI-40449): restore talend-webservice-input-1.0.0-20170301.jar

* chore(TDI-40449): restore geopip

* chore(TDI-40449): restore talend-bigdata-launcher-1.1.0-20160405.jar

* chore(TDI-40449): bump ws-commons-util

* chore(TDI-40449): restore tMom jars needed

* fix(TDI-40449) : prepare servicenow for publishing
- force maven-deploy-plugin to 2.8+
- bump version to 1.0.1
- reformat code to Talend standard
- add license header
- remove main method containing credentials(not critical)

* chore(TDI-40449): fixup some components

* chore(TDI-40449): mavenized google-storage-utils

* chore(TDI-40449): cleaning talendssl ant file

* chore(TDI-40449): reverting changes
- SugarCRM is deprecated and will be removed in this release

* chore(TDI-40449): fix version typo/group for asm

* chore(TDI-40449): rename google storage utils

* chore(TDI-40449): fix dependencies to the future google-storage-utils
- will be built during https://jira.talendforge.org/browse/TDI-43489

* chore(TDI-40449): fix javassist GAV dependency

* chore(TDI-40449): fix GAV for tMOM/tWebService
- revert change on tWebService_java.xml
- fix GAV for accuracy on tMOM

* chore(TDI-40449): remove last jars
- See https://jira.talendforge.org/browse/TDI-43490

* chore(TDI-40449): fix url

* chore(TDI-40449): distribution management needed for google storage utils on ci-common

* chore(TDI-40449): cleaning xml files

* chore(TDI-40449): update copyright

* chore(TDI-40449): last clean
- i swear

Co-authored-by: Pierre Teyssier <pteyssier@talend.com>
2020-01-02 15:56:13 +08:00
jiezhang-tlnd
73e7d360d0 fix(TUP-25524)issue for TOS Log4j (#4294) 2019-12-31 15:54:13 +08:00
hcyi
ec490c589e fix(TUP-25382):More than one dialog pop up when quickly click the button more than once. (#4269)
* fix(TUP-25382):More than one dialog pop up when quickly click the button
more than once.

* fix(TUP-25382):More than one dialog pop up when quickly click the button
more than once.
2019-12-31 11:41:46 +08:00
hwang-talend
2a67448f58 bugfix(TUP-24029):[FTP] Connection wizard add timeout property (#3864) (#3982)
* bugfix(TUP-24029):[FTP] Connection wizard add timeout property (#3864)

bugfix(TUP-24029):[FTP] Connection wizard add timeout property

* feature(TUP-24029): [FTP] Connection wizard add timeout property

* feature(TUP-24768):[bug] no unit for timeout parameter for ftp wizard.

* feature(TUP-24768):[bug] timeout value support to type non-digit.

* feature(TUP-24768):[bug] timeout value support to type non-digit.
2019-12-30 18:08:49 +08:00
Jane Ding
617095c83b feat(TUP-24057)There is no error message when studio can't connect to (#4290)
customer artifact repository
https://jira.talendforge.org/browse/TUP-24057
2019-12-30 17:49:20 +08:00
mbasiuk-talend
1c857a778a Mbasiuk/tdi 43095 improve set section (#4241)
* feat(TDI-43095): improve set section for tELTTeradataOutput

* feat(TDI-43095): improve SET section for tELTOutput generic

* feat(TDI-43095): extend logic for USE_UPDATE_STATEMENT, fix typo

* feat(TDI-43095): make changes according to PR comments
2019-12-30 11:41:34 +02:00
hcyi
568c60df0f fix(TUP-23672):Check connection button for tacokit integration to Studio (#4271) 2019-12-30 10:43:01 +08:00
hwang-talend
425dce5d7a bugfix(TUP-21691):Run job fail after drag it out from a folder. (#4265)
bugfix(TUP-21691):Run job fail after drag it out from a folder.
2019-12-30 10:09:50 +08:00
kjwang
30b9a6af9d Fix TUP-24523 Online Help in studio - refactor help features (#4227)
* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523

* Fix TUP-24523 Online Help in studio - refactor help features
https://jira.talendforge.org/browse/TUP-24523
2019-12-27 18:20:39 +08:00
hcyi
c782cdc070 fix(TUP-24405):stats/logs database ui is not right when choose teradata (#4277)
database
2019-12-27 17:36:37 +08:00
hcyi
7bd4d48476 fix(TUP-24727):Use shortcut F1 to search help document on component then (#4285)
throw errors
2019-12-27 17:25:22 +08:00
Zhiwei Xue
99ad6fb48b fix(TUP-25223):Child job maven version is incorrect if it is opened by (#4280)
tRunjob
2019-12-27 17:25:13 +08:00
jiezhang-tlnd
92c5a956b8 Jzhang/bugfix/tup 25572 avoid loop method on log4j (#4259)
* avoid loop method

* fix(TUP-25572)avoid loop method to improve performance

* avoid loop method to improve performance

* update junits

* fix jar start with "talend-bigdata"

* fix(TUP-25541)bridge jar not included in class path for log4j1 jar
needed in child process but not in father's.

* update class path order of log4j

* fix dateviewer error on oracle db

* fix data viewer, and conflict spark-assembly jar and log4j jar

* update class path order for spark-assembly

* avoid repeating for child module and refactor code

* refactor code for sort class path

* add junits

* fix tuj tMap_04_persistence_04_n_lookups
2019-12-27 17:15:41 +08:00
Zhiwei Xue
a3daac36e9 fix(TUP-25328): When move folder out of folder will have error :Resource (#4274)
*** not exist
2019-12-27 10:59:42 +08:00
hcyi
b7c5bb914c fix(TUP-24625):implement lazy migration. (#4174)
* fix(TUP-24625):Implement jobs lazy migration.

* fix(TUP-24625):When/If lazy migration will be done following fix won't
be needed anymore on TUP-21055 .

* fix(TUP-24625):remove job migration code when start-up if implement lazy
migration.

* fix(TUP-24625):remove migration code when start-up if implement lazy
migration.

* fix(TUP-24625):improve to implement lazy migration.
2019-12-26 16:59:49 +08:00
Chao MENG
3f9f37bf90 fix(TUP-24905): Studio automatically set password when set up multiple (#4272)
fix(TUP-24905): Studio automatically set password when set up multiple login connections
https://jira.talendforge.org/browse/TUP-24905
2019-12-26 14:55:48 +08:00
hzhao-talendbj
24471c3ed3 feat(TUP-25120)Merge nexus proxy setup (#4211)
* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup

* feat(TUP-25120)Merge nexus proxy setup
2019-12-26 11:16:49 +08:00
bhe-talendbj
e592a52dfa bugfix(TUP-25409) Improve building speed of CI builder (#4253)
* bugfix(TUP-25409) Generate source code for main job only

* bugfix(TUP-25409) Resolve comments
2019-12-25 14:51:13 +08:00
hzhao-talendbj
b0d4990091 bugfix(TUP-25035)Implicit tContextLoad feature (with selection from (#4268)
file) uses a hardcoded encoding of ISO-8859-15
2019-12-25 11:40:29 +08:00
Zhiwei Xue
4766e40047 fix(TUP-23331):Studio title change to 'Setup Traces' after I click Setup (#4262)
Traces
2019-12-24 18:15:46 +08:00
vyu-talend
78bbfc61fb Fix(TDI-36397):Reset default value for 'Batch mode' and migration. (#4267)
* Fix(TDI-36397):Reset default value for 'Batch mode' and migration.

* Fix(TDI-36397):register migration task to plugin.xml.
2019-12-24 17:26:54 +08:00
Zhiwei Xue
f3bd46488e fix(TUP-25206):NPE thrown in error log when close job (#4263) 2019-12-24 16:45:58 +08:00
chmyga
36f98068e5 fix(TDI-43319): CDC Oracle wrong settings (#4219)
* Add migration task for old jobs with OracleCDC component
2019-12-23 15:35:50 +02:00
hwang-talend
41cbeeb140 bugfix(TUP-22500):jars already existed on nexus still try to upload (#4152)
bugfix(TUP-22500):jars already existed on nexus still try to upload again
2019-12-23 10:17:32 +08:00
Chao MENG
c96bae71da fix(TUP-25502): import dialog i18n error (#4260)
fix(TUP-25502): import dialog i18n error
https://jira.talendforge.org/browse/TUP-25502
2019-12-23 10:11:58 +08:00
vyu-talend
eedf273635 Fix(TDI-43323):Fix issue of duplicated variables. (#4233) 2019-12-23 10:00:57 +08:00
hcyi
d047644d8b fix(TUP-25575):Upgrade tacokit to 1.1.15. (#4258) 2019-12-20 17:28:10 +08:00
vyu-talend
bd04ae1b54 Fix(TDI-42408):rename for tGPGDecrypt. (#4254) 2019-12-20 15:40:59 +08:00
chmyga
8641ad63d1 feat(TDI-40351): Add alternative schema name (#4007)
*  feat(TDI-40351): Add alternative schema name

 * Add "use alternative schema" property for db output components

* feat(TDI-40351): Add alternative schema name

* Move alternate schema to advanced settings

* Rename alternate schema properties

* feat(TDI-40351): Add alternative schema name

* fix problems for ParAccel, PostgreSql, PostgresPlus, Redshift

* feat(TDI-40351): Add alternative schema name

* Remove schema field when use existing connection is checked
2019-12-19 14:18:07 +02:00
chmyga
9f80f3f7ff feat(TUP-25053): use alternate schema in DB Output (#4090)
* Add usage of alternate schema during table selection
2019-12-19 14:17:13 +02:00
wang wei
bbb2bddea5 fix(TDI-43396): Compile error for S3 and tFileFetch TUJs when log4j is not enabled at default(#4238) 2019-12-19 11:35:10 +08:00
jiezhang-tlnd
ad193ddfd8 fix(TUP-25483)color default setting not work on ubuntu (#4251)
https://jira.talendforge.org/browse/TUP-25483
2019-12-18 17:57:27 +08:00
jiezhang-tlnd
8ee5787201 fix(TUP-25541)tMap TUJs are broken for the ClassNotFound (#4249)
* fix(TUP-25541)tMap TUJs are broken for the ClassNotFound

* fix(TUP-25541)tMap TUJs are broken for the ClassNotFound
2019-12-18 17:26:44 +08:00
hcyi
7f91106ec7 fix(TUP-25411):Table name and field should in double quotes in (#4242)
tELTPostgresqlMap.
2019-12-18 14:53:09 +08:00
hwang-talend
f7607f6e8a bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4221)
bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2019-12-18 14:43:22 +08:00
hwang-talend
5027f0fd3b bugfix(TUP-25278):Issues about delete columns in tELTMap with alias (#4220)
bugfix(TUP-25278):Issues about delete columns in tELTMap with alias
2019-12-18 14:41:51 +08:00
jiezhang-tlnd
504a5477bc fix(TUP-25530)Log4j error when execute a job on datatabricks (#4243)
* fix(TUP-25530)Log4j error when execute a job on datatabricks

* Log4j error when execute a job on datatabricks
2019-12-17 19:17:09 +08:00
jiezhang-tlnd
196bec65e5 fix(TUP-25504)Active log4j by default (#4237)
* fix(TUP-25504)Active log4j by default
https://jira.talendforge.org/browse/TUP-25504

* fix tos log4j default
2019-12-16 16:27:10 +08:00
jiezhang-tlnd
c6fdf4da5b fix log4j failed junits (#4239) 2019-12-16 15:09:39 +08:00
jiezhang-tlnd
f52c9c2584 fix(TBD-9677)under the log4j2, Run some job with spark 2.4 local get (#4235) 2019-12-13 14:27:47 +01:00
jiezhang-tlnd
0fb3315eee feat(TUP-24651)Add choice between log4j and log4j2 in project settings (#4173)
* feat(TUP-24651)Add choice between log4j and log4j2 in project settings

* add warnings

* fix(TUP-25319): When we deactivate the log4j option, DI need use default
jars log4j2 needed

* remove system property and enable the feature by defult

* fix(TUP-25222)NoClassDefFoundError is thrown when switch between 2 log4j
versions for tRunJob if Use independent Process is checked

* add dialog for log4j version changed

* issue of tRunjob with unactivate log4j

* fix junit for log4j

* fix issue:get error when config match rules on TDM component
"tMatchGroup"

* fix log4j default issue for new project
2019-12-13 17:22:43 +08:00
Jane Ding
ba12b6577a Fix junits failure (#4229)
caused by org.eclipse.emf.ecore.resource.Resource$IOWrappedException:
The object 'testauto@talend.com' is not contained in a resource
2019-12-13 16:50:11 +08:00
hwang-talend
ce7bfa060f bugfix(TUP-25106):when creating an onsubjobOk link with tELTMap, this (#4224)
bugfix(TUP-25106):when creating an onsubjobOk link with tELTMap, this generate an incorrect dsiplay
2019-12-13 15:29:07 +08:00
Mike Yan
a8155b659c fix(TESB-27266):Exclude tdm-lib-di from REST DS, but keep package info (#4230) 2019-12-13 14:20:07 +08:00
Zhiwei Xue
f0c4517bbc fix(TUP-24719): improve the exception handling of signature failure. (#4226) 2019-12-13 10:07:20 +08:00
bhe-talendbj
247fc84baa bugfix(TUP-25237) Need to clear modules per job cache before generate pom (#4216) 2019-12-12 15:26:18 +08:00
Zhiwei Xue
1b8fe81314 fix(TUP-24719): fix possible NPE. (#4225) 2019-12-11 19:13:04 +08:00
Zhiwei Xue
5d32586b46 feat(TUP-24719):Studio updates need to install new components by (#4171)
project/license type
2019-12-11 16:05:36 +08:00
jiezhang-tlnd
4596f2a3a8 fix(TUP-25457)Harden RunTrace deserialization (#4223)
https://jira.talendforge.org/browse/TUP-25457
2019-12-11 14:55:32 +08:00
kjwang
f5001bcddf Kjwang/ tup 25343 database can not dataview (#4215)
* TUP-25343 [BUG] database can't dataview after set
routine.encryption.key.v2
https://jira.talendforge.org/browse/TUP-25343

* TUP-25343 [BUG] database can't dataview after set
routine.encryption.key.v2
https://jira.talendforge.org/browse/TUP-25343
2019-12-11 11:25:26 +08:00
jiezhang-tlnd
15f87a595f Jzhang/tup 25461 log4j2 default setting for level (#4222)
* log4j2 setting default on log level

* log4j2 setting default on log level
2019-12-11 11:00:36 +08:00
jiezhang-tlnd
74d0a5fc4d fix(TUP-25453)color setting for log4j2 (#4214)
* fix(TUP-25453)color setting for log4j2

* color setting for log4j2
2019-12-10 16:28:07 +08:00
wang wei
9a7f158ea8 fix(TDI-43101): revert it as only a poc research topic, should not merge (#4213) 2019-12-10 15:12:09 +08:00
Liu Xinquan
76aff58dc9 TDQ-17742 change the dialog title to Enter a new license key (#4195)
* TDQ-17742 change the dialog title to Enter a new license key

* TDQ-17742 change the dialog title to Enter a new license key
2019-12-09 15:56:39 +08:00
jiezhang-tlnd
ab1c2ac982 fix(TUP-25460)schema hardcoded in guess query in tDBInput component (#4204)
https://jira.talendforge.org/browse/TUP-25406
2019-12-09 14:21:17 +08:00
hcyi
856a9343b8 fix(TUP-25218):tELTXXXMap: Output table : Expression & Column set to (#4200)
blank on Mac OS X。
2019-12-09 11:06:01 +08:00
sponomarova
29df704fae fix(TBD-9317): Merge branch 'master' into fix/TBD-9317 2019-12-06 17:26:09 +02:00
hzhao-talendbj
1db0ec3664 feat(TUP-24382)Enable HTTPS with authentication (#4168)
* feat(TUP-24382)Enable HTTPS with authentication

* feat(TUP-24382)Enable HTTPS with authentication

* feat(TUP-24382)Enable HTTPS with authentication

* feat(TUP-24382)Enable HTTPS with authentication
2019-12-06 17:46:27 +08:00
Dmytro Sylaiev
b39ca1b28e fix(TDI-43085): Bump jackson version to 2.10.1 (#4192)
* Bump http-utils lib version
2019-12-06 11:24:03 +02:00
jiezhang-tlnd
54e445bdd2 bugfix(TUP-25350)Show if for opened list: active values are not updated (#4199)
until you deselect component
https://jira.talendforge.org/browse/TUP-25350
2019-12-06 11:38:08 +08:00
Jane Ding
9cbe98d4ef feat(TUP-24849)Remove jars from the studio (#4107)
https://jira.talendforge.org/browse/TUP-24849
2019-12-05 12:13:51 +08:00
vyu-talend
26e88b26be Fix(TDI-42424):Improve for tELTPostgresqlOutput generate wrong update (#3615)
* Fix(TDI-42424):Improve for tELTPostgresqlOutput generate wrong update
query

* Fix(TDI-42424)

* Fix(TDI-42424)

* Fix(TDI-42424)

* Fix(TDI-42424):make modifications for new suquery.

* Fix(TDI-42424):fix some issues.

* Fix(TDI-42424):fix bug.

* Fix(TDI-42424):fix issue.
2019-12-05 10:43:25 +08:00
vyu-talend
6217c3042d Vyu/tdi 40599 enable use hint in merge (#4177)
* Fix(TDI-40599):A demo for current new feature.

* Fix(TDI-40599):rename 'MERGE'
2019-12-05 10:09:23 +08:00
wang wei
2c8df34e97 fix(TDI-43266): Studio returns exit code 0 when OutOfMemoryError in routine occurs (#4178)
* fix(TDI-43266): Studio returns exit code 0 when OutOfMemoryError in
routine occurs

* fix(TDI-43266): main thread should stop at once when subthread throw
outofmemory error
2019-12-05 09:31:09 +08:00
vyu-talend
86b8e75e73 Fix(TDI-43264):Fix issues. (#4182) 2019-12-03 16:33:40 +08:00
wang wei
f2bcb6d71f fix(TDI-43091): Mysql8:SSLException when run job which contain ELT component(#4181) 2019-12-03 11:41:27 +08:00
hcyi
0403d4d7d4 fix(TUP-25227):Changing the joblet name is not reflected properly. (#4188) 2019-12-02 16:07:51 +08:00
Chao MENG
4198afd1d2 fix(TUP-25344): can't logon project after change workspace if set log4j2 (#4183)
fix(TUP-25344): can't logon project after change workspace if set log4j2
https://jira.talendforge.org/browse/TUP-25344
2019-12-02 10:29:53 +08:00
wang wei
a971215151 fix(TDI-43296): Generate a sample logging file for runtime lineage cause compile error(#4185) 2019-11-29 12:03:14 +08:00
Chao MENG
14c59926c6 fix(TUP-24749): [7.2.1] issue to download component jar via https proxy (#4167)
fix(TUP-24749): [7.2.1] issue to download component jar via https proxy
https://jira.talendforge.org/browse/TUP-24749
2019-11-29 10:22:52 +08:00
bhe-talendbj
374903d266 bugfix(TUP-25257) Fix missing jobid for trunjob (#4166)
* bugfix(TUP-25257) Fix missing jobid for trunjob

* bugfix(TUP-25257) Fix missing jobid for trunjob

* bugfix(TUP-25257) Refactor

* bugfix(TUP-25257) Refactor
2019-11-28 15:06:33 +08:00
bhe-talendbj
abab035072 bugfix(TUP-25186) Revert previous pr (#4175) 2019-11-28 15:04:01 +08:00
hwang-talend
03c3e264e6 bugfix(TUP-24937):Propagate changes in tELTInput schema to tELTMap (#4165) 2019-11-28 14:34:01 +08:00
jiezhang-tlnd
948eb33247 fix(TUP-25119)add jar needed and (#4122)
* fix(TUP-25119)add jar needed and
log4j.xml
https://jira.talendforge.org/browse/TUP-25119

* fix(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119

* fix(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119

* fix(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119

* feat(TUP-25119)Studio Create Json file failed
https://jira.talendforge.org/browse/TUP-25119
2019-11-28 10:11:26 +08:00
wang wei
70b86c3519 fix(TDI-43101) : support to generate a sample logging file for runtime lineage (#4164)
* fix(TDI-33262): Support for Apache Log4j2

* use release version

* fix(TDI-43101): [Research] Generate a sample logging file for runtime
lineage

* fix(TDI-43101): fix it

* fi something

* add a system property to disable this feature
2019-11-26 18:11:43 +08:00
Dmytro Sylaiev
57ef3ab3f6 fix(TDI-41651): Add where clause support for update query (#4172) 2019-11-26 10:50:30 +02:00
Jane Ding
bb0dc580a2 fix(TUP-24732)Studio Code tab doesn't generate code/show errors in Job (#4148)
https://jira.talendforge.org/browse/TUP-24732
2019-11-26 10:44:05 +08:00
Dmytro Sylaiev
2f1029eba9 fix(TDI-41651): Provide possibility to specify alias for output table… (#3645)
* fix(TDI-41651): Provide possibility to specify alias for output table ELT

* Do the same fix as mssqlELT

* Small refactoring for query codeGen

* fix(TDI-41651): Fix issues

* Correct notion in javaxml file

* Rename cleanUp method

* Use stringUtils in alias for input table

* fix(TDI-41651): Add getters for inner class

* fix(TDI-41651): Add missing property in pom.xml

* fix(TDI-41651): Add missing i18n label

* fix(TDI-41651): Remove use alias for output field

* fix(TDI-41651): Fix conflicts
2019-11-25 14:59:46 +02:00
Dmytro Sylaiev
fff59f3b2f feat(TDI-43088): Update cxf deps for msCRM and webservice components (#4131)
* feat(TDI-43088): Update cxf deps for msCRM and webservice components

* feat(TDI-43088): Update talend-ws usage of CXF
2019-11-25 13:03:45 +02:00
Chao MENG
712db2045d feat(TUP-24720): Remove component manager in the studio (#4108)
* feat(TUP-24720): Remove component manager in the studio

https://jira.talendforge.org/browse/TUP-24720

* feat(TUP-24720): Remove component manager in the studio

https://jira.talendforge.org/browse/TUP-24720
2019-11-25 17:08:52 +08:00
mbasiuk-talend
96bc7ed4c5 fix(TDI-43212): improve code generation for query string (#4149) 2019-11-25 08:49:40 +02:00
Roman
f749ac3e2a feat(TDI-42766): Redshift SSO support with Ping Federate
* feat(TDI-42766): remove code related to SSO without login/pass

* feat(TDI-42766): add migration task
2019-11-22 12:54:03 +02:00
jiezhang-tlnd
91e83eedf0 Jzhang/feat/tup 24894 automatically review changes slf4j log4j dependencies5 (#4140)
* feat(TUP-24894)add some bridge jars mostly for DB

* feat(TUP-24894)add library

* feat(TUP-24894)add some jar mostly for TBD needs

* feat(TUP-24894)add some jar mostly for DB needs

* feat(TUP-24894)add some jar mostly for DB needs

* feat(TUP-24894)add junits
2019-11-22 11:42:02 +08:00
wang wei
4d5844376e fix(TDI-43211): SharedDBConnectionLog4j use log4j1 api, should use slf4j (#4141) 2019-11-22 11:01:33 +08:00
Dmytro Sylaiev
e56ccf3f62 fix(TDI-43036): Wait for threads finish for OnComponentOK and RunIf for multithead iterate execution (#4075) 2019-11-21 12:08:15 +02:00
hzhao-talendbj
e04ef82e30 Hengzhao/bugfix/tup 25035 (#4157)
* feat(TUP-25035)Implicit tContextLoad feature (with selection from file)
uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15

* bugfix(TUP-25035)Implicit tContextLoad feature (with selection from
file) uses a hardcoded encoding of ISO-8859-15
2019-11-21 14:59:30 +08:00
Zhiwei Xue
273a5025c7 fix(TUP-25186):tRunJob broken after migrating project from 6.3.1 (#4145) 2019-11-21 14:41:05 +08:00
wang wei
612e5a5c27 fix(TDI-43189): Some tJDBC components use tCompV0, but they still keep the folder in localprovider(#4139) 2019-11-20 15:38:14 +08:00
vyu-talend
38f544be0a Fix(TDI-34953):Show SQL query in debug log for Db components (#4084)
* Fix(TDI-34953):issues fixed.

* Fix(TDI-34953):Fix the reopen issues(1,2,6).

* Fix(TDI-34953):Fix printing order.
2019-11-20 14:09:35 +08:00
wang wei
3b872b62e5 fix(TDI-43174): tMSSqlinput process datetimeoffset data type failed when use DST timezone as localtime (#4138) 2019-11-20 13:50:19 +08:00
wang wei
1910ed40e5 fix(TDI-39084): fail to get sap table parameter for tsapbapi and throw NPE (#4062)
* fix(TDI-39084): fail to get sap table parameter for tsapbapi and throw
NPE

Conflicts:
	main/plugins/org.talend.repository/plugin.xml

* correct the version after merge origin/master

* fix migration
2019-11-20 11:07:55 +08:00
AlixMetivier
d0912599ec csv option were not called (#4150) 2019-11-18 15:27:30 +01:00
hcyi
ee20cf02ca fix(TUP-25218):tELTXXXMap: Output table : Expression & Column set to (#4142)
blank if you add a Column without Expression on Mac OS X.
2019-11-17 22:12:43 +08:00
hcyi
5427bc53f1 fix(TUP-25099):NPE when using joblets from reference Projects. (#4132) 2019-11-17 22:07:05 +08:00
AlixMetivier
9b4bfe85be feat(TBD-9231): dataset can be used in left outer and inner (#4137)
* dataset can be used in left outer and inner

* refacto

* refacto

* wip

* Update MapperComponent.java
2019-11-15 10:59:33 +01:00
hwang-talend
b5916d9122 bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job (#4126)
bugfix(TUP-24884):Spark Joblets cannot be built within Parent DI job
2019-11-15 17:20:46 +08:00
bhe-talendbj
5e9f08c9ff bugfix(TUP-25109) redshift guess schema by executing query. (#4129) 2019-11-15 16:23:21 +08:00
Chao MENG
8e977f5110 fix(TUP-25052): File json miss source schema when after do export to (#4123)
fix(TUP-25052): File json miss source schema when after do export to context.
https://jira.talendforge.org/browse/TUP-25052
2019-11-15 15:40:47 +08:00
mbasiuk-talend
90c7f820b9 feat(TDI-41954): add commit and rollback to unified (#4103) 2019-11-15 08:39:25 +02:00
kjwang
1d5f0bacc6 Fix TUP-24842 Switch jobscript and run job get error : NoClassDefFoundError: org/talend/daikon/crypto/Encryption (#4085)
* Fix TUP-24842 Switch jobscript and run job get error :
NoClassDefFoundError: org/talend/daikon/crypto/Encryption
https://jira.talendforge.org/browse/TUP-24842

* Fix TUP-24842 Switch jobscript and run job get error :
NoClassDefFoundError: org/talend/daikon/crypto/Encryption
https://jira.talendforge.org/browse/TUP-24842
2019-11-15 14:28:54 +08:00
sponomarova
7726effcbb fix(TBD-9317): correct plugin.xml 2019-10-11 17:14:00 +03:00
sponomarova
e3743b26c1 fix(TBD-9317): clean up components from org.talend.libraries.pig 2019-10-11 17:09:37 +03:00
543 changed files with 8484 additions and 6222 deletions

View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>org.talend.help.tos.components.feature</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.eclipse.pde.FeatureBuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.eclipse.pde.FeatureNature</nature>
</natures>
</projectDescription>

View File

@@ -0,0 +1 @@
bin.includes = feature.xml

View File

@@ -0,0 +1,94 @@
<?xml version="1.0" encoding="UTF-8"?>
<feature
id="org.talend.help.tos.components.feature"
label="Talend COMPONENTS help feature"
version="7.3.1.qualifier"
provider-name=".Talend SA.">
<description url="http://www.example.com/description">
[Enter Feature Description here.]
</description>
<copyright url="http://www.example.com/copyright">
[Enter Copyright Description here.]
</copyright>
<license url="http://www.example.com/license">
[Enter License Description here.]
</license>
<url>
<update label="CORE Talend update site for milestones" url="http://talendforge.org/core/updatesite/testing/"/>
</url>
<plugin
id="org.talend.help.azurestorage"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.filedelimited"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.filterrow"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.googledrive"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.jdbc"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.jira"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.marketo"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.salesforce"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.snowflake"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
<plugin
id="org.talend.help.splunk"
download-size="0"
install-size="0"
version="0.0.0"
unpack="false"/>
</feature>

View File

@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.talend.studio</groupId>
<artifactId>tdi-studio-se</artifactId>
<version>7.3.1-SNAPSHOT</version>
<relativePath>../../../</relativePath>
</parent>
<artifactId>org.talend.help.tos.components.feature</artifactId>
<packaging>eclipse-feature</packaging>
</project>

View File

@@ -9,6 +9,7 @@
</url>
<requires>
<import feature="org.talend.tos.feature" version="0.0.0" match="greaterOrEqual"/>
<import feature="org.talend.help.di.feature" version="0.0.0" match="greaterOrEqual"/>
</requires>
<plugin id="org.talend.designer.maven.tos" download-size="0" install-size="0" version="0.0.0" fragment="true"/>
<plugin id="org.talend.presentation.onboarding" download-size="0" install-size="0" version="0.0.0" unpack="false"/>

View File

@@ -53,16 +53,6 @@
<plugin id="org.talend.designer.xmlmap.nl" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="org.talend.expressionbuilder" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.expressionbuilder.nl" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="org.talend.help.azurestorage" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.filedelimited" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.filterrow" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.googledrive" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.jdbc" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.jira" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.marketo" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.salesforce" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.snowflake" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.help.splunk" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.repository.generic" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.sdk.component.studio-integration" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
</feature>

View File

@@ -23,7 +23,8 @@ Require-Bundle: org.talend.core.repository,
org.talend.components.api,
org.talend.daikon,
org.talend.daikon.crypto.utils,
org.talend.designer.core.generic
org.talend.designer.core.generic,
org.talend.updates.runtime
Eclipse-LazyStart: true
Bundle-ClassPath: .
Export-Package: org.talend.designer.codegen,

View File

@@ -329,14 +329,26 @@ end_Hash.put("<%=node.getUniqueName() %>", System.currentTimeMillis());
}
// Else, that means the component is a ON_COMPONENT_OK virtual component typed.
}
if(generateMethodCall) {
/*check if parralel iterate call not finished from this component */
boolean parallelIterate = false;
for (IConnection iterateConn : iterateConnSet) {
parallelIterate = "true".equals(ElementParameterParser.getValue(iterateConn, "__ENABLE_PARALLEL__"));
if (parallelIterate) {
if (codePart.equals(ECodePart.END)) {
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
%>
mtp_<%=iterateTargetNodeName %>.waitForEndOfQueue();
<%
}
}
}
%>
<%=outgoingConn.getTarget().getUniqueName() %>Process(globalMap);
<%
}
}
}
if (outgoingConn.getLineStyle().equals(EConnectionType.RUN_IF)) {
%>
@@ -344,12 +356,27 @@ end_Hash.put("<%=node.getUniqueName() %>", System.currentTimeMillis());
<%//send the true status to socket
if(stat){
%>
if(execStat){
if(execStat){
runStat.updateStatOnConnection("<%=outgoingConn.getUniqueName() %>", 0, "true");
}
<%}%>
<%=outgoingConn.getTarget().getUniqueName() %>Process(globalMap);
}
<%
}
/*check if parralel iterate call not finished from this component */
boolean parallelIterate = false;
for (IConnection iterateConn : iterateConnSet) {
parallelIterate = "true".equals(ElementParameterParser.getValue(iterateConn, "__ENABLE_PARALLEL__"));
if (parallelIterate) {
if (codePart.equals(ECodePart.END)) {
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
%>
mtp_<%=iterateTargetNodeName %>.waitForEndOfQueue();
<%
}
}
}
%>
<%=outgoingConn.getTarget().getUniqueName() %>Process(globalMap);
}
<%
//send the false status to socket
@@ -376,7 +403,7 @@ end_Hash.put("<%=node.getUniqueName() %>", System.currentTimeMillis());
parallelIterate = "true".equals(ElementParameterParser.getValue(iterateConn, "__ENABLE_PARALLEL__"));
if (parallelIterate) {
if (codePart.equals(ECodePart.END)) {
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
String iterateTargetNodeName = iterateConn.getTarget().getUniqueName();
%>
mtp_<%=iterateTargetNodeName %>.waitForEndOfQueue();

View File

@@ -522,16 +522,12 @@
.filter(it -> it.startsWith("monitoring.audit.logger.properties."))
.forEach(key -> properties_<%=jobCatcherNode.getUniqueName()%>.setProperty(key.substring("monitoring.audit.logger.properties.".length()), System.getProperty(key)));
<%if(isLog4jEnabled) {%>
<%if(isLog4j1Enabled) {%>
org.apache.log4j.Logger.getLogger("audit").setLevel(org.apache.log4j.Level.DEBUG);
<%}%>
<%if(isLog4j2Enabled) {%>
org.apache.logging.log4j.core.config.Configurator.setLevel("audit", org.apache.logging.log4j.Level.DEBUG);
<%}%>
<%} else {%>
org.apache.log4j.Logger.getLogger("audit").setLevel(org.apache.log4j.Level.DEBUG);
<%if(isLog4j1Enabled) {%>
org.apache.log4j.Logger.getLogger("audit").setLevel(org.apache.log4j.Level.DEBUG);
<%}%>
<%if(isLog4j2Enabled) {%>
org.apache.logging.log4j.core.config.Configurator.setLevel("audit", org.apache.logging.log4j.Level.DEBUG);
<%}%>
auditLogger_<%=jobCatcherNode.getUniqueName()%> = org.talend.job.audit.JobEventAuditLoggerFactory.createJobAuditLogger(properties_<%=jobCatcherNode.getUniqueName()%>);
}

View File

@@ -66,6 +66,11 @@ ComponentJobletPreferencePage.label=Don't show corresponding job after double cl
ComponentsPreferencePage_SuccessTitle=Success
ComponentsPreferencePage_SuccessMessage1=Need restart studio to load the new components. Else, will be loaded in next time when start.
ComponentsPreferencePage_SuccessMessage2=Do you want to restart right now?
ComponentsPreferencePage.enableOnLineHelp=Enable online help for components
ComponentsPreferencePage.titleInstallHelp=Install offline help feature?
ComponentsPreferencePage.msgInstallHelp=This will open additional packages dialog to install offline help feature, do you want to continue?
ComponentsPreferencePage.grpHelp=Help
ComponenttRunJobPreferencePage.showCorrespondingJob=Show corresponding job after double click on a tRunJob Component.
XsdValidationCacheManager.unableLoadxsd=unable to load xsd validation cache file
XsdValidationCacheManager.unableSavexsd=unable to save xsd validation cache file

View File

@@ -14,6 +14,8 @@ package org.talend.designer.codegen.components.ui;
import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.util.HashSet;
import java.util.Set;
import org.apache.commons.lang.StringUtils;
import org.eclipse.core.runtime.IProgressMonitor;
@@ -30,6 +32,7 @@ import org.eclipse.swt.events.ModifyEvent;
import org.eclipse.swt.events.ModifyListener;
import org.eclipse.swt.layout.GridData;
import org.eclipse.swt.layout.GridLayout;
import org.eclipse.swt.widgets.Button;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Group;
@@ -52,6 +55,12 @@ import org.talend.designer.codegen.i18n.Messages;
import org.talend.designer.core.DesignerPlugin;
import org.talend.designer.core.assist.TalendEditorComponentCreationUtil;
import org.talend.designer.core.ui.preferences.TalendDesignerPrefConstants;
import org.talend.designer.core.utils.ComponentsHelpUtil;
import org.talend.updates.runtime.InstallFeatureObserver;
import org.talend.updates.runtime.engine.ExtraFeaturesUpdatesFactory;
import org.talend.updates.runtime.model.AbstractExtraFeature;
import org.talend.updates.runtime.model.ExtraFeature;
import org.talend.updates.runtime.ui.ShowWizardHandler;
/**
* This class represents a preference page that is contributed to the Preferences dialog. By subclassing
@@ -71,6 +80,8 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
private DirectoryFieldEditor filePathTemp;
private CheckBoxFieldEditor enableOnLineHelpField;
private final String dataViewer = "Data Viewer"; //$NON-NLS-1$
private final String mapper = "Mapper "; //$NON-NLS-1$
@@ -81,6 +92,8 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
private final String assist = "Component Assist"; //$NON-NLS-1$
private final String HELP_FEATURE_NAME = "Talend Help"; //$NON-NLS-1$
private static String oldPath = null;
/**
@@ -209,6 +222,16 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
return group;
}
protected Composite createForHelpType(Composite parent) {
Group group = createGroup(parent);
group.setText(Messages.getString("ComponentsPreferencePage.grpHelp"));
Composite composite = createComposite(group);
addHelpTypeFiled(composite);
GridLayout layout = createLayout();
composite.setLayout(layout);
return group;
}
protected Group createGroup(Composite parent) {
Group group = new Group(parent, SWT.NONE);
GridData layoutData = new GridData(GridData.FILL_HORIZONTAL);
@@ -271,6 +294,46 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
addField(enableComponentAssistCheckBoxField);
}
protected void addHelpTypeFiled(Composite composite) {
enableOnLineHelpField = new CheckBoxFieldEditor(TalendDesignerPrefConstants.HELP_OFFLINE,
Messages.getString("ComponentsPreferencePage.enableOnLineHelp"), composite) {
@Override
protected void doLoad() {
boolean isOffLine = DesignerPlugin.getDefault().getPreferenceStore().getBoolean(getPreferenceName());
Button checkBox = getButton();
if (checkBox != null) {
if (isOffLine && !ComponentsHelpUtil.isHelpInstalled()) {
checkBox.setSelection(true);
doStore();
} else {
checkBox.setSelection(!isOffLine);
}
}
}
@Override
protected void doStore() {
Button checkBox = getButton();
if (checkBox != null) {
DesignerPlugin.getDefault().getPreferenceStore().setValue(getPreferenceName(), !checkBox.getSelection());
}
}
@Override
protected void doLoadDefault() {
Button checkBox = getButton();
if (checkBox != null) {
checkBox.setSelection(true);
}
}
}; // $NON-NLS-1$
addField(enableOnLineHelpField);
}
@Override
public void createFieldEditors() {
final Composite parent = getFieldEditorParent();
@@ -316,6 +379,8 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
createForJoblet(parent);
}
createForComponentAssist(parent);
createForHelpType(parent);
parent.pack();
}
@@ -382,8 +447,8 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
if ("".equals(newPath)) { //$NON-NLS-1$
newPath = null;
}
DesignerPlugin.getDefault().getPreferenceStore()
.setValue(TalendDesignerPrefConstants.COMPONENT_ASSIST, enableComponentAssistCheckBoxField.getBooleanValue());
DesignerPlugin.getDefault().getPreferenceStore().setValue(TalendDesignerPrefConstants.COMPONENT_ASSIST,
enableComponentAssistCheckBoxField.getBooleanValue());
TalendEditorComponentCreationUtil.updateAssistListener();
if (this.oldPath != newPath) {
@@ -432,8 +497,9 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
}
}
if (StringUtils.isNotEmpty(component.getFailureMessage())) {
MessageDialog.openError(getShell(), Messages
.getString("ComponentsPreferencePage.installComponentsFailure"),//$NON-NLS-1$
MessageDialog.openError(getShell(),
Messages.getString(
"ComponentsPreferencePage.installComponentsFailure"), //$NON-NLS-1$
component.getFailureMessage());
}
} finally {
@@ -472,6 +538,51 @@ public class ComponentsPreferencePage extends FieldEditorPreferencePage implemen
this.oldPath = newPath;
}
if (enableOnLineHelpField != null && !enableOnLineHelpField.getBooleanValue()
&& !ComponentsHelpUtil.isHelpInstalled()
&& InstallFeatureObserver.getInstance().isNeedLanuchInstallWizard(HELP_FEATURE_NAME)) {
if (MessageDialog.openConfirm(getShell(), Messages.getString("ComponentsPreferencePage.titleInstallHelp"),
Messages.getString("ComponentsPreferencePage.msgInstallHelp"))) {
installHelpFeature();
flag = true;
} else {
flag = false;
}
}
return flag;
}
private void installHelpFeature() {
final Set<ExtraFeature> installSet = new HashSet<ExtraFeature>();
final IRunnableWithProgress runnable = new IRunnableWithProgress() {
@Override
public void run(final IProgressMonitor monitor) throws InvocationTargetException, InterruptedException {
ExtraFeaturesUpdatesFactory extraFeaturesFactory = new ExtraFeaturesUpdatesFactory(false);
final Set<ExtraFeature> uninstalledExtraFeatures = new HashSet<ExtraFeature>();
extraFeaturesFactory.retrieveUninstalledExtraFeatures(monitor, uninstalledExtraFeatures, false);
for (ExtraFeature feature : uninstalledExtraFeatures) {
if (HELP_FEATURE_NAME.equalsIgnoreCase(feature.getName())) {
if (feature instanceof AbstractExtraFeature) {
((AbstractExtraFeature) feature).setMustBeInstalled(true);
}
installSet.add(feature);
}
}
}
};
final ProgressMonitorDialog dialog = new ProgressMonitorDialog(null);
try {
dialog.run(true, true, runnable);
} catch (InvocationTargetException e) {
return;
} catch (InterruptedException e) {
return;
}
synchronized (ShowWizardHandler.showWizardLock) {
new ShowWizardHandler().showUpdateWizard(PlatformUI.getWorkbench().getActiveWorkbenchWindow().getShell(), installSet);
}
}
}

View File

@@ -1,88 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.bigdata/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="GoogleStorageUtils.jar" />
<property name="component.name" value="tGSPut" />
<property name="author.name" value="jzhao" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
<pathelement location="${component.plugin.home}/tBigQueryBulkExec/jets3t-0.9.0.jar" />
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -1,127 +0,0 @@
package org.talend.gs.util;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Set;
import org.jets3t.service.model.GSObject;
import org.jets3t.service.utils.Mimetypes;
public class GSObjectUtil {
public List<GSObject> genGSObjectList(List<GSObject> objects, File file,
String keyParent, boolean isGenFileObject, boolean isGenFolderObject)
throws NoSuchAlgorithmException, IOException {
if (file.isDirectory()) {
if (keyParent != null && !"".equals(keyParent)) {
if (keyParent.trim().lastIndexOf("/") != keyParent.trim()
.length() - 1) {
keyParent = keyParent + "/";
}
if(isGenFolderObject){
objects.add(new GSObject(keyParent));
}
}
File[] files = file.listFiles();
for (File f : files) {
if (f.isDirectory()) {
objects = genGSObjectList(objects, f,
keyParent + f.getName() + "/", isGenFileObject,
isGenFolderObject);
} else {
objects = genGSObjectList(objects, f,
keyParent + f.getName(), isGenFileObject,
isGenFolderObject);
}
}
} else {
if (isGenFileObject) {
GSObject obj = new GSObject(file);
obj.setKey(keyParent);
objects.add(obj);
}
}
return objects;
}
public java.util.Map<String,String> genFileFilterList(java.util.List<java.util.Map<String,String>> list,String localdir,String remotedir){
if (remotedir != null && !"".equals(remotedir)) {
if (remotedir.trim().lastIndexOf("/") != remotedir.trim()
.length() - 1) {
remotedir = remotedir + "/";
}
}
java.util.Map<String,String> fileMap=new HashMap<String,String>();
for (java.util.Map<String, String> map : list) {
java.util.Set<String> keySet = map.keySet();
for (String key : keySet){
String tempdir = localdir;
String filemask = key;
String dir = null;
String mask = filemask.replaceAll("\\\\", "/") ;
int i = mask.lastIndexOf('/');
if (i!=-1){
dir = mask.substring(0, i);
mask = mask.substring(i+1);
}
if (dir!=null && !"".equals(dir)) tempdir = tempdir + "/" + dir;
mask = mask.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*");
final String finalMask = mask;
java.io.File[] listings = null;
java.io.File file = new java.io.File(tempdir);
if (file.isDirectory()) {
listings = file.listFiles(new java.io.FileFilter() {
public boolean accept(java.io.File pathname) {
boolean result = false;
if (pathname != null && pathname.isFile()) {
result = java.util.regex.Pattern.compile(finalMask).matcher(pathname.getName()).find();
}
return result;
}
});
}
if(listings == null || listings.length <= 0){
System.err.println("No match file("+key+") exist!");
}else{
String localFilePath = "";
String newObjectKey = "";
for (int m = 0; m < listings.length; m++){
if (listings[m].getName().matches(mask)){
localFilePath = listings[m].getAbsolutePath();
if(map.get(key)!=null && map.get(key).length()>0){
newObjectKey = remotedir+map.get(key);
}else{
newObjectKey = remotedir+listings[m].getName();
}
fileMap.put(localFilePath, newObjectKey);
}
}
}
}
}
return fileMap;
}
public List<GSObject> genObjectByFileMap(java.util.Map<String,String> fileMap) throws Exception, IOException{
Set<String> localFiles=fileMap.keySet();
List<GSObject> objects=new ArrayList<GSObject>();
for(String localFilePath:localFiles){
GSObject object=new GSObject(new File(localFilePath));
object.setKey(fileMap.get(localFilePath));
objects.add(object);
}
return objects;
}
public void initMimeTypes() throws IOException {
InputStream mimetypesFile = this.getClass().getResourceAsStream(
"/resource/mime.types");
Mimetypes.getInstance().loadAndReplaceMimetypes(mimetypesFile);
}
}

View File

@@ -1,88 +1,88 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="external_sort.jar" />
<property name="component.name" value="tSortOut" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="external_sort.jar" />
<property name="component.name" value="tSortOut" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -0,0 +1,88 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (C) 2006-2020 Talend Inc. - www.talend.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components</groupId>
<artifactId>google-storage-utils</artifactId>
<version>1.0.0</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
</properties>
<name>Google Storage Utils</name>
<description>Talend Helper library for Google Storage</description>
<url>
https://github.com/Talend/tdi-studio-se/
</url>
<licenses>
<license>
<name>Apache-2.0</name>
<url>
http://www.talendforge.org/modules/licenses/APACHE_v2.txt
</url>
</license>
</licenses>
<dependencies>
<dependency>
<groupId>net.java.dev.jets3t</groupId>
<artifactId>jets3t</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
</repository>
</distributionManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
<resource>
<directory>src/main/resources</directory>
<targetPath>resource</targetPath>
</resource>
</resources>
</build>
</project>

View File

@@ -0,0 +1,123 @@
package org.talend.gs.util;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Set;
import org.jets3t.service.model.GSObject;
import org.jets3t.service.utils.Mimetypes;
public class GSObjectUtil {
public List<GSObject> genGSObjectList(List<GSObject> objects, File file, String keyParent, boolean isGenFileObject,
boolean isGenFolderObject) throws NoSuchAlgorithmException, IOException {
if (file.isDirectory()) {
if (keyParent != null && !"".equals(keyParent)) {
if (keyParent.trim().lastIndexOf("/") != keyParent.trim().length() - 1) {
keyParent = keyParent + "/";
}
if (isGenFolderObject) {
objects.add(new GSObject(keyParent));
}
}
File[] files = file.listFiles();
for (File f : files) {
if (f.isDirectory()) {
objects = genGSObjectList(objects, f, keyParent + f.getName() + "/", isGenFileObject,
isGenFolderObject);
} else {
objects = genGSObjectList(objects, f, keyParent + f.getName(), isGenFileObject, isGenFolderObject);
}
}
} else {
if (isGenFileObject) {
GSObject obj = new GSObject(file);
obj.setKey(keyParent);
objects.add(obj);
}
}
return objects;
}
public java.util.Map<String, String> genFileFilterList(java.util.List<java.util.Map<String, String>> list,
String localdir, String remotedir) {
if (remotedir != null && !"".equals(remotedir)) {
if (remotedir.trim().lastIndexOf("/") != remotedir.trim().length() - 1) {
remotedir = remotedir + "/";
}
}
java.util.Map<String, String> fileMap = new HashMap<String, String>();
for (java.util.Map<String, String> map : list) {
java.util.Set<String> keySet = map.keySet();
for (String key : keySet) {
String tempdir = localdir;
String filemask = key;
String dir = null;
String mask = filemask.replaceAll("\\\\", "/");
int i = mask.lastIndexOf('/');
if (i != -1) {
dir = mask.substring(0, i);
mask = mask.substring(i + 1);
}
if (dir != null && !"".equals(dir))
tempdir = tempdir + "/" + dir;
mask = mask.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*");
final String finalMask = mask;
java.io.File[] listings = null;
java.io.File file = new java.io.File(tempdir);
if (file.isDirectory()) {
listings = file.listFiles(new java.io.FileFilter() {
public boolean accept(java.io.File pathname) {
boolean result = false;
if (pathname != null && pathname.isFile()) {
result = java.util.regex.Pattern.compile(finalMask).matcher(pathname.getName()).find();
}
return result;
}
});
}
if (listings == null || listings.length <= 0) {
System.err.println("No match file(" + key + ") exist!");
} else {
String localFilePath = "";
String newObjectKey = "";
for (int m = 0; m < listings.length; m++) {
if (listings[m].getName().matches(mask)) {
localFilePath = listings[m].getAbsolutePath();
if (map.get(key) != null && map.get(key).length() > 0) {
newObjectKey = remotedir + map.get(key);
} else {
newObjectKey = remotedir + listings[m].getName();
}
fileMap.put(localFilePath, newObjectKey);
}
}
}
}
}
return fileMap;
}
public List<GSObject> genObjectByFileMap(java.util.Map<String, String> fileMap) throws Exception, IOException {
Set<String> localFiles = fileMap.keySet();
List<GSObject> objects = new ArrayList<GSObject>();
for (String localFilePath : localFiles) {
GSObject object = new GSObject(new File(localFilePath));
object.setKey(fileMap.get(localFilePath));
objects.add(object);
}
return objects;
}
public void initMimeTypes() throws IOException {
InputStream mimetypesFile = this.getClass().getResourceAsStream("/resource/mime.types");
Mimetypes.getInstance().loadAndReplaceMimetypes(mimetypesFile);
}
}

View File

@@ -1,95 +1,125 @@
<!--
Copyright (C) 2006-2020 Talend Inc. - www.talend.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components</groupId>
<artifactId>components-servicenow</artifactId>
<name>servicenow</name>
<version>1.0.0</version>
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.components</groupId>
<artifactId>components-servicenow</artifactId>
<name>servicenow</name>
<version>1.0.1</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
</properties>
<description>Talend Helper library for ServiceNow components</description>
<url>
https://github.com/Talend/tdi-studio-se/
</url>
<licenses>
<license>
<name>Apache-2.0</name>
<url>
http://www.talendforge.org/modules/licenses/APACHE_v2.txt
</url>
</license>
</licenses>
<dependencies>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.6</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.3</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient-cache</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20150729</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.0</version>
<type>jar</type>
<optional>true</optional>
</dependency>
</dependencies>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
</repository>
</distributionManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
</plugins>
<dependencies>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.6</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.3</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient-cache</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20150729</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.0</version>
<type>jar</type>
<optional>true</optional>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
</resources>
<!-- TODO
<testResources>
<testResource>
<directory>src/test/java</directory>
</testResource>
</testResources>
-->
</build>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
</resources>
</build>
</project>

View File

@@ -1,274 +1,269 @@
package org.talend.servicenow;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.apache.http.HttpResponse;
import org.apache.http.ParseException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
public class Util {
private HttpClient client;
private String baseurl;
public Util(HttpClient client, String baseurl) {
this.client = client;
this.baseurl = baseurl;
}
@SuppressWarnings("rawtypes")
public static void main(String[] args) throws ClientProtocolException, IOException {
boolean exception = true;
org.apache.http.impl.client.CloseableHttpClient client = null;
while(exception) {
try {
//login
org.apache.http.client.CredentialsProvider credsProvider = new org.apache.http.impl.client.BasicCredentialsProvider();
credsProvider.setCredentials(
new org.apache.http.auth.AuthScope(new org.apache.http.HttpHost("sandbox.service-now.com")), new org.apache.http.auth.UsernamePasswordCredentials("talend", "talend")
);
client = org.apache.http.impl.client.HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();
//get metadata
Util util = new Util(client,"https://sandbox.service-now.com");
Object result = util.getMetadata("incident");
System.out.println(result);
System.out.println(((Map)result).size());
} catch(Exception e) {
System.out.println(e.getMessage());
continue;
} finally {
client.close();
}
exception = false;
}
}
private Map<String,String> typeMapping = new HashMap<String,String>();
{
typeMapping.put("string", "id_String");
typeMapping.put("boolean", "id_Boolean");
typeMapping.put("integer", "id_Integer");
typeMapping.put("decimal", "id_BigDecimal");
typeMapping.put("float", "id_Float");
typeMapping.put("glide_date_time", "id_Date");
typeMapping.put("glide_date", "id_Date");
typeMapping.put("glide_time", "id_Date");
}
public static class ColumnMetadata {
private String name;
private String type;
private int maxlength;
private String pattern;
public ColumnMetadata(String name, String type, int maxlength, String pattern) {
this.name = name;
this.type = type;
this.maxlength = maxlength;
this.pattern = pattern;
}
public String getName() {
return name;
}
public String getType() {
return type;
}
public int getMaxLength() {
return maxlength;
}
public String getPattern() {
return pattern;
}
public String toString() {
return name + ":" + type + ":" + maxlength;
}
}
public Map<String,ColumnMetadata> getMetadata(String tablename) throws ClientProtocolException, IOException {
Map<String,ColumnMetadata> result = new HashMap<String,ColumnMetadata>();
List<String> relation = getRelationship(tablename);
StringBuilder sb = new StringBuilder();
for(String tname : relation) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_dictionary");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tname);
sb.append("&sysparm_fields=element,internal_type,max_length,active");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
List<Map<String, String>> info = extractResponse4MultiRowFromArray(response);
for(Map<String, String> row : info) {
String element = row.get("element");
boolean active = Boolean.parseBoolean(row.get("active"));
if(element!=null && !element.isEmpty() && active) {
String talend_type = null;
String pattern = null;
String servicenow_type = (String)row.get("internal_type");
if(servicenow_type!=null && !servicenow_type.isEmpty()) {
talend_type = typeMapping.get(servicenow_type);
//need date pattern
if("glide_date_time".equals(servicenow_type)) {
pattern = "yyyy-MM-dd HH:mm:ss";
} else if("glide_date".equals(servicenow_type)) {
pattern = "yyyy-MM-dd";
} else if("glide_time".equals(servicenow_type)) {
pattern = "HH:mm:ss";
}
}
if(talend_type == null) {
talend_type = "id_String";
}
int mlength = 64;
String maxlength = (String)row.get("max_length");
if(maxlength!=null && !maxlength.isEmpty()) {
mlength = Integer.parseInt(maxlength);
}
ColumnMetadata column = new ColumnMetadata(element,talend_type,mlength,pattern);
result.put(element, column);
}
}
}
return result;
}
private List<String> getRelationship(String tablename) throws ClientProtocolException, IOException {
LinkedList<String> result = new LinkedList<String>();
StringBuilder sb = new StringBuilder();
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tablename);
sb.append("&sysparm_fields=name,super_class");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
Map<String,String> info = extractResponse4OneRowFromArray(response);
result.add(info.get("name"));
String superclass = info.get("super_class");
while(superclass!=null && !superclass.isEmpty()) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object/");
sb.append(superclass);
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_fields=name,super_class");
httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
response = this.client.execute(httpget);
info = extractResponse4OneRowFromObject(response);
result.add(info.get("name"));
superclass = info.get("super_class");
}
Collections.reverse(result);
return result;
}
private Map<String,String> extractResponse4OneRowFromArray(HttpResponse response) throws ParseException, IOException {
validateResponse(response);
Map<String,String> result = new HashMap<String,String>();
org.json.JSONArray array = (org.json.JSONArray)getResult(response);
for(int i=0;i<array.length();i++) {
org.json.JSONObject row = (org.json.JSONObject)array.get(i);
for(String key : row.keySet()) {
result.put(key, (String)row.get(key));
}
}
return result;
}
private Map<String,String> extractResponse4OneRowFromObject(HttpResponse response) throws ParseException, IOException {
validateResponse(response);
Map<String,String> result = new HashMap<String,String>();
org.json.JSONObject object = (org.json.JSONObject)getResult(response);
for(String key : object.keySet()) {
result.put(key, (String)object.get(key));
}
return result;
}
private List<Map<String, String>> extractResponse4MultiRowFromArray(HttpResponse response) throws ParseException, IOException {
validateResponse(response);
List<Map<String, String>> result = new ArrayList<Map<String,String>>();
org.json.JSONArray array = (org.json.JSONArray)getResult(response);
for(int i=0;i<array.length();i++) {
Map<String,String> element = new HashMap<String,String>();
org.json.JSONObject row = (org.json.JSONObject)array.get(i);
for(String key : row.keySet()) {
element.put(key, (String)row.get(key));
}
result.add(element);
}
return result;
}
private void validateResponse(HttpResponse response) {
if(response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Fail to get the table metadata. The operation has returned the code : " + response.getStatusLine() + ".");
}
}
private Object getResult(HttpResponse response) throws ParseException, IOException {
String responseBody = org.apache.http.util.EntityUtils.toString(response.getEntity());
org.json.JSONObject json = new org.json.JSONObject(responseBody);
return json.get("result");
}
/**
* Copyright (C) 2006-2019 Talend Inc. - www.talend.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.talend.servicenow;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.apache.http.HttpResponse;
import org.apache.http.ParseException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
public class Util {
private HttpClient client;
private String baseurl;
public Util(HttpClient client, String baseurl) {
this.client = client;
this.baseurl = baseurl;
}
private Map<String, String> typeMapping = new HashMap<String, String>();
{
typeMapping.put("string", "id_String");
typeMapping.put("boolean", "id_Boolean");
typeMapping.put("integer", "id_Integer");
typeMapping.put("decimal", "id_BigDecimal");
typeMapping.put("float", "id_Float");
typeMapping.put("glide_date_time", "id_Date");
typeMapping.put("glide_date", "id_Date");
typeMapping.put("glide_time", "id_Date");
}
public static class ColumnMetadata {
private String name;
private String type;
private int maxlength;
private String pattern;
public ColumnMetadata(String name, String type, int maxlength, String pattern) {
this.name = name;
this.type = type;
this.maxlength = maxlength;
this.pattern = pattern;
}
public String getName() {
return name;
}
public String getType() {
return type;
}
public int getMaxLength() {
return maxlength;
}
public String getPattern() {
return pattern;
}
public String toString() {
return name + ":" + type + ":" + maxlength;
}
}
public Map<String, ColumnMetadata> getMetadata(String tablename) throws ClientProtocolException, IOException {
Map<String, ColumnMetadata> result = new HashMap<String, ColumnMetadata>();
List<String> relation = getRelationship(tablename);
StringBuilder sb = new StringBuilder();
for (String tname : relation) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_dictionary");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tname);
sb.append("&sysparm_fields=element,internal_type,max_length,active");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
List<Map<String, String>> info = extractResponse4MultiRowFromArray(response);
for (Map<String, String> row : info) {
String element = row.get("element");
boolean active = Boolean.parseBoolean(row.get("active"));
if (element != null && !element.isEmpty() && active) {
String talend_type = null;
String pattern = null;
String servicenow_type = (String) row.get("internal_type");
if (servicenow_type != null && !servicenow_type.isEmpty()) {
talend_type = typeMapping.get(servicenow_type);
//need date pattern
if ("glide_date_time".equals(servicenow_type)) {
pattern = "yyyy-MM-dd HH:mm:ss";
} else if ("glide_date".equals(servicenow_type)) {
pattern = "yyyy-MM-dd";
} else if ("glide_time".equals(servicenow_type)) {
pattern = "HH:mm:ss";
}
}
if (talend_type == null) {
talend_type = "id_String";
}
int mlength = 64;
String maxlength = (String) row.get("max_length");
if (maxlength != null && !maxlength.isEmpty()) {
mlength = Integer.parseInt(maxlength);
}
ColumnMetadata column = new ColumnMetadata(element, talend_type, mlength, pattern);
result.put(element, column);
}
}
}
return result;
}
private List<String> getRelationship(String tablename) throws ClientProtocolException, IOException {
LinkedList<String> result = new LinkedList<String>();
StringBuilder sb = new StringBuilder();
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object");
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_query=name=");
sb.append(tablename);
sb.append("&sysparm_fields=name,super_class");
HttpGet httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
HttpResponse response = this.client.execute(httpget);
Map<String, String> info = extractResponse4OneRowFromArray(response);
result.add(info.get("name"));
String superclass = info.get("super_class");
while (superclass != null && !superclass.isEmpty()) {
sb.setLength(0);
sb.append(this.baseurl);
sb.append("/api/now/table/");
sb.append("sys_db_object/");
sb.append(superclass);
sb.append("?sysparm_exclude_reference_link=true");
sb.append("&sysparm_fields=name,super_class");
httpget = new HttpGet(sb.toString());
httpget.setHeader("Accept", "application/json");
response = this.client.execute(httpget);
info = extractResponse4OneRowFromObject(response);
result.add(info.get("name"));
superclass = info.get("super_class");
}
Collections.reverse(result);
return result;
}
private Map<String, String> extractResponse4OneRowFromArray(HttpResponse response)
throws ParseException, IOException {
validateResponse(response);
Map<String, String> result = new HashMap<String, String>();
org.json.JSONArray array = (org.json.JSONArray) getResult(response);
for (int i = 0; i < array.length(); i++) {
org.json.JSONObject row = (org.json.JSONObject) array.get(i);
for (String key : row.keySet()) {
result.put(key, (String) row.get(key));
}
}
return result;
}
private Map<String, String> extractResponse4OneRowFromObject(HttpResponse response)
throws ParseException, IOException {
validateResponse(response);
Map<String, String> result = new HashMap<String, String>();
org.json.JSONObject object = (org.json.JSONObject) getResult(response);
for (String key : object.keySet()) {
result.put(key, (String) object.get(key));
}
return result;
}
private List<Map<String, String>> extractResponse4MultiRowFromArray(HttpResponse response)
throws ParseException, IOException {
validateResponse(response);
List<Map<String, String>> result = new ArrayList<Map<String, String>>();
org.json.JSONArray array = (org.json.JSONArray) getResult(response);
for (int i = 0; i < array.length(); i++) {
Map<String, String> element = new HashMap<String, String>();
org.json.JSONObject row = (org.json.JSONObject) array.get(i);
for (String key : row.keySet()) {
element.put(key, (String) row.get(key));
}
result.add(element);
}
return result;
}
private void validateResponse(HttpResponse response) {
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException(
"Fail to get the table metadata. The operation has returned the code : " + response.getStatusLine()
+ ".");
}
}
private Object getResult(HttpResponse response) throws ParseException, IOException {
String responseBody = org.apache.http.util.EntityUtils.toString(response.getEntity());
org.json.JSONObject json = new org.json.JSONObject(responseBody);
return json.get("result");
}
}

View File

@@ -1,95 +1,95 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="sugarCRMManagement.jar" />
<property name="component.name" value="tSugarCRMInput" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
<pathelement location="${component.plugin.home}/tWebServiceInput/axis.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/commons-discovery-0.2.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/wsdl4j-1.5.1.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/saaj.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/jaxrpc.jar" />
<pathelement location="${component.plugin.home}/tXMLRPCInput/commons-logging-1.1.jar" />
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}" />
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="sugarCRMManagement.jar" />
<property name="component.name" value="tSugarCRMInput" />
<property name="author.name" value="wyang" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
<pathelement location="${component.plugin.home}/tWebServiceInput/axis.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/commons-discovery-0.2.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/wsdl4j-1.5.1.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/saaj.jar" />
<pathelement location="${component.plugin.home}/tWebServiceInput/jaxrpc.jar" />
<pathelement location="${component.plugin.home}/tXMLRPCInput/commons-logging-1.1.jar" />
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}" />
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -4,7 +4,7 @@
<groupId>org.talend</groupId>
<artifactId>talend-httputil</artifactId>
<name>talend-httputil</name>
<version>1.0.4</version>
<version>1.0.5</version>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
@@ -20,7 +20,7 @@
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.9</version>
<version>2.10.1</version>
</dependency>
<dependency>

View File

@@ -4,12 +4,12 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend-ws</artifactId>
<version>1.0.1-20190204</version>
<version>1.0.1-20191112</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<cxf.version>3.1.1</cxf.version>
<cxf.version>3.3.4</cxf.version>
<odata.version>4.3.0</odata.version>
<slf4j.version>1.7.12</slf4j.version>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
@@ -90,42 +90,42 @@
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-core</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-simple</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-tools-common</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-wsdl</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-bindings-soap</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-databinding-jaxb</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http-jetty</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
<version>3.2.6</version>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.talend.libraries</groupId>

View File

@@ -2,9 +2,9 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend_DB_mssqlUtil-1.3-20190523</artifactId>
<version>6.0.0</version>
<groupId>org.talend.components.lib</groupId>
<artifactId>talend_DB_mssqlUtil</artifactId>
<version>1.4</version>
<packaging>jar</packaging>
<name>talend_DB_mssqlUtil</name>

View File

@@ -16,6 +16,8 @@ import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import java.util.TimeZone;
import java.text.SimpleDateFormat;
import java.text.ParseException;
/**
* DOC Administrator class global comment. Detailled comment
@@ -117,10 +119,15 @@ public class MSSqlGenerateTimestampUtil {
return null;
}
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
{
sdf.setTimeZone(TimeZone.getTimeZone("GMT"));
}
/**
* parse datetimeoffset string to date.
* datetimeoffset string show as YYYY-MM-DD hh:mm:ss[.nnnnnnn] [+|-]hh:mm
* and no enough infomation for the DST decision,so no DST consider[not sure if 100% support for DST].
*
* @param datetimeOffsetString
* @return
@@ -131,13 +138,17 @@ public class MSSqlGenerateTimestampUtil {
String offsetString = datetimeOffsetString.substring(idx + 1);
int offset = TimeZone.getTimeZone("GMT" + offsetString).getRawOffset();
// get local timezone, also consider the DST
TimeZone local = TimeZone.getDefault();
int localOffset = local.getOffset(new java.util.Date().getTime());
long milliseconds = java.sql.Timestamp.valueOf(datetimeString).getTime();
long millisecondsToGMT1970_01_01 = milliseconds - offset + localOffset;
return new java.util.Date(millisecondsToGMT1970_01_01);
long millisecondsWithOffset = 0l;
try {
millisecondsWithOffset = sdf.parse(datetimeString).getTime();
} catch(ParseException e) {
throw new RuntimeException(e.getMessage());
}
//get GMT's millisecond number since GMT 1970_01_01
long milliseconds4GMT_to_GMT_1970_01_01 = millisecondsWithOffset - offset;
return new java.util.Date(milliseconds4GMT_to_GMT_1970_01_01);
}
}

View File

@@ -1,10 +1,10 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<groupId>org.talend.components.lib</groupId>
<artifactId>talendsap</artifactId>
<name>talendsap</name>
<version>1.0.1</version>
<version>1.0.2</version>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>

View File

@@ -90,6 +90,7 @@ public class DocumentExtractor {
List<Element> tablesAndChangingElements = new ArrayList<Element>(3);
tablesAndChangingElements.add(functionElement.element("TABLES"));
tablesAndChangingElements.add(functionElement.element("CHANGING"));
tablesAndChangingElements.add(functionElement.element("OUTPUT"));
for(Element tablesOrChangingElement : tablesAndChangingElements) {
if (tablesOrChangingElement == null) {
@@ -121,6 +122,10 @@ public class DocumentExtractor {
}
result.add(row);
}
if(!result.isEmpty()) {
return result;
}
}
return result;

View File

@@ -49,11 +49,11 @@ public class DocumentHelper {
}
}
public void addSingleParameter(String name, String value, boolean isChanging) {
public void addSingleParameter(String name, String value, SAPParameterType parameter_type) {
if(value == null) {
value = "";
}
if (isChanging) {
if (parameter_type == SAPParameterType.CHANGING) {
correctChanging();
changing.addElement(name).setText(value);
} else {
@@ -62,8 +62,8 @@ public class DocumentHelper {
}
}
public void addStructParameter(String name, boolean isChanging) {
if (isChanging) {
public void addStructParameter(String name, SAPParameterType parameter_type) {
if (parameter_type == SAPParameterType.CHANGING) {
correctChanging();
currentStruct = changing.addElement(name);
} else {
@@ -79,13 +79,16 @@ public class DocumentHelper {
currentStruct.addElement(name).setText(value);
}
public void addTableParameter(String name, boolean isChanging) {
if (isChanging) {
public void addTableParameter(String name, SAPParameterType parameter_type) {
if (parameter_type == SAPParameterType.CHANGING) {
correctChanging();
currentTable = changing.addElement(name);
} else {
} else if(parameter_type == SAPParameterType.TABLES) {
correctTables();
currentTable = tables.addElement(name);
} else {
correctInput();
currentTable = input.addElement(name);
}
}
@@ -108,18 +111,18 @@ public class DocumentHelper {
DocumentHelper helper = new DocumentHelper();
helper.setFunctionName("READ_TABLE_FUNCTION");
helper.addSingleParameter("ID", "1", true);
helper.addSingleParameter("NAME", "gaoyan", false);
helper.addSingleParameter("ID", "1", SAPParameterType.CHANGING);
helper.addSingleParameter("NAME", "gaoyan", SAPParameterType.IMPORT);
helper.addStructParameter("INFO", true);
helper.addStructParameter("INFO", SAPParameterType.CHANGING);
helper.addStructChildParameter("ID", "2");
helper.addStructChildParameter("NAME", "wangwei");
helper.addStructParameter("INFO1", false);
helper.addStructParameter("INFO1", SAPParameterType.IMPORT);
helper.addStructChildParameter("ID1", "4");
helper.addStructChildParameter("NAME1", "momo");
helper.addTableParameter("TABLE1", false);
helper.addTableParameter("TABLE1", SAPParameterType.TABLES);
for (int i = 0; i < 200000; i++) {
helper.addTableRow();
helper.addTableRowChildParameter("c1", i + "");
@@ -132,7 +135,7 @@ public class DocumentHelper {
helper.addTableRowChildParameter("c8", "wangwei" + i);
}
helper.addTableParameter("TABLE2", false);
helper.addTableParameter("TABLE2", SAPParameterType.TABLES);
for (int i = 0; i < 2; i++) {
helper.addTableRow();
helper.addTableRowChildParameter("ID4", i + "");

View File

@@ -0,0 +1,8 @@
package org.talend.sap;
public enum SAPParameterType {
IMPORT,
CHANGING,
TABLES,
EXPORT
}

View File

@@ -1,88 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project name="org.talend.designer.components.libs" default="process" basedir=".">
<property name="component.plugin.home" value="../../../org.talend.designer.components.localprovider/components" />
<!-- #################################################### -->
<!-- modification 1: config -->
<property name="jar.name" value="talendssl.jar" />
<property name="component.name" value="tLDAPInput" />
<property name="author.name" value="wliu" />
<!-- modification 2: compile classpath -->
<path id="compile.classpath">
</path>
<!-- #################################################### -->
<!-- sourcecode and final jar path -->
<property name="source.home" value="." />
<property name="jar.home" value="${component.plugin.home}/${component.name}/${jar.name}" />
<!-- temp dir for clasee files -->
<property name="build.dir" value="../../build" />
<!-- compile option -->
<property name="compile.debug" value="true" />
<property name="compile.deprecation" value="false" />
<property name="compile.optimize" value="true" />
<target name="process" description="prepare a temp dir">
<antcall target="prepare" />
<antcall target="compile" />
<antcall target="clean" />
</target>
<target name="prepare" description="prepare a temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
<mkdir dir="${build.dir}/classes" />
</target>
<target name="compile" description="Compile Java sources">
<!-- compile -->
<javac srcdir="${source.home}" destdir="${build.dir}/classes" debug="${compile.debug}" deprecation="${compile.deprecation}" optimize="${compile.optimize}">
<classpath refid="compile.classpath" />
</javac>
<!-- include source code -->
<copy todir="${build.dir}/classes">
<fileset dir="${source.home}">
<exclude name="build.xml" />
</fileset>
</copy>
<!-- make jar -->
<tstamp>
<format property="date" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<jar destfile="${build.dir}/${jar.name}" basedir="${build.dir}/classes">
<manifest>
<!-- who -->
<attribute name="Built-By" value="${author.name}" />
<!-- when -->
<attribute name="Built-Date" value="${date}"/>
<!-- JDK version -->
<attribute name="Created-By" value="${java.version} (${java.vendor})" />
<!-- Information about the program itself -->
<attribute name="Implementation-Vendor" value="Talend SA" />
<attribute name="Implementation-Title" value="${jar.name}" />
<attribute name="Implementation-Version" value="1.0" />
</manifest>
</jar>
<!-- move jar -->
<move file="${build.dir}/${jar.name}" tofile="${jar.home}" />
</target>
<target name="clean" description="clean the temp dir">
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>
</project>

View File

@@ -149,6 +149,8 @@
<IMPORT NAME="Driver-V5R3_V6R1" MODULE="jt400_V5R3.jar" MVN="mvn:org.talend.libraries/jt400_V5R3/6.0.0" REQUIRED_IF="DB_VERSION == 'jt400_V5R3.jar'" />
<IMPORT NAME="Driver-V6R1_V7R2" MODULE="jt400_V6R1.jar" MVN="mvn:org.talend.libraries/jt400_V6R1/6.0.0" REQUIRED_IF="DB_VERSION == 'jt400_V6R1.jar'" />
<IMPORT NAME="Driver-V7R1_V7R3" MODULE="jt400-9.8.jar" MVN="mvn:net.sf.jt400/jt400/9.8" REQUIRED_IF="DB_VERSION == 'jt400-9.8.jar'" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -301,6 +301,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
@@ -308,6 +309,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.sql.SQLException e) {
@@ -431,6 +433,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
@@ -438,6 +441,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().updating(dbLog.var("nb_line"));%>
} catch(java.sql.SQLException e) {
@@ -584,6 +588,7 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -659,6 +664,7 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -833,11 +839,13 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
}else{
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().updating(dbLog.var("nb_line"));%>
}
} catch(java.sql.SQLException e) {
@@ -914,12 +922,14 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
}else {
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>

View File

@@ -58,7 +58,7 @@ USER.NAME=Username
USE_COMMIT_CONTROL.NAME=Use commit control
USE_EXISTING_CONNECTION.NAME=Using existing connection
USE_FIELD_OPTIONS.NAME=Use field options
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.V5R2_V5R4=V5R2 to V5R4 (Deprecated)
DB_VERSION.ITEM.V5R3_V6R1=V5R3 to V6R1 (Deprecated)

View File

@@ -123,6 +123,9 @@
<IMPORT NAME="common2.6" MODULE="commons-lang-2.6.jar" MVN="mvn:commons-lang/commons-lang/2.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-lang-2.6.jar" REQUIRED="true"/>
<IMPORT NAME="common-logging1.1.3" MODULE="commons-logging-1.1.3.jar" MVN="mvn:commons-logging/commons-logging/1.1.3" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-logging-1.1.3.jar" REQUIRED="true"/>
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -139,6 +139,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -228,6 +229,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -345,6 +347,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -406,6 +409,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try{
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -501,6 +505,7 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updateFlag_<%=cid%>=pstmtUpdate_<%=cid %>.executeUpdate();
if(updateFlag_<%=cid%> != 0) {
nb_line_<%=cid%>++;
@@ -567,6 +572,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -645,6 +651,7 @@ skeleton="../templates/db_output_bulk.skeleton"
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>

View File

@@ -52,7 +52,7 @@ FIELD_OPTIONS.ITEM.UPDATE_KEY=Key in update
FIELD_OPTIONS.ITEM.DELETE_KEY=Key in delete
FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
NB_LINE.NAME=Number of line
NB_LINE_UPDATED.NAME= NB Line Updated

View File

@@ -1,240 +0,0 @@
MaxMind GeoIP License version 1.6, March 23rd, 2007
MAXMIND END-USER LICENSE AGREEMENT
By clicking on the words "I Agree" below, you agree that your use of the
MaxMind GeoIP database products and services is subject to the terms and
conditions set forth in this MaxMind End User License Agreement (this
"Agreement").
MaxMind, Inc. ("MaxMind") provides a line of database services and products
that provide the geographic information and other data associated with
specific Internet protocol addresses. These database services and products
are referred to in this Agreement individually as the "GeoIP Database" and
collectively as the "GeoIP Databases". The data available through the GeoIP
Databases is referred to in this Agreement as the "GeoIP Data". Some
products in the MaxMind GeoIP Line may include MaxMind's proprietary computer
programs used to query the GeoIP Databases (the "GeoIP Programs").
ADDITIONAL POLICIES.
The following policies are incorporated into this End User Agreement by
reference and provide additional terms and conditions related to specific
services and products provided by MaxMind:
minFraud Service Terms of Use
Each of these policies may be amended at any time and such amendments
shall be binding and effective as indicated below in the section on
Changes To The Agreement.
LIMITED GRANT OF RIGHTS.
In accordance with the terms of this Agreement, MaxMind hereby grants you a
non-exclusive, non-transferable limited license to access and use the GeoIP
Databases and GeoIP Data for your own internal Restricted Business
purposes. Restricted Business purposes are limited to customizing website
content, fraud prevention, geographic reporting, and similar business
purposes. You agree to use the GeoIP Data and GeoIP Databases only in a
manner that is consistent with applicable laws.
RESTRICTIONS ON USE.
Except as expressly permitted in this Agreement, you may not, nor may you
permit others to:
(a) copy any portion of the GeoIP Databases except as reasonably required
for using the GeoIP Database as permitted hereunder,
(b) allow anyone other than yourself or your employees to access the GeoIP
Databases, or any portion thereof, without MaxMind's express written
permission,
(c) use the GeoIP Databases to develop a database, infobase, online or
similar database service, or other information resource in any media (print,
electronic or otherwise, now existing or developed in the future) for sale
to, distribution to, display to or use by others,
(d) create compilations or derivative works of the GeoIP Databases,
(e) use the GeoIP Databases in any fashion that may infringe any copyright,
intellectual property right, contractual right, or proprietary or property
right or interest held by MaxMind,
(f) store in a retrieval system accessible to the public, transfer,
publish, distribute, display to others, broadcast, sell, or sublicense the
GeoIP Databases, or any portion thereof,
(g) remove or obscure any copyright notice or other notice or terms of use
contained in the GeoIP Databases,
(h) use the GeoIP Data to create or otherwise support the transmission of
unsolicited, commercial email,
(i) remove, disable, avoid, circumvent, or defeat any functionality in the
GeoIP Databases or GeoIP Programs designed to limit or control access to or
use of the GeoIP Databases or GeoIP Data,
(j) use, copy or otherwise access any portion of the GeoIP Data for which
you have not made payment to MaxMind. If for any reason, you access such
GeoIP Data, these terms and conditions apply to your use of such data and
you agree to pay all applicable charges, or
(k) copy, reverse engineer, decompile, disassemble, derive source code,
modify or prepare derivative works of the GeoIP Programs.
OWNERSHIP AND INTELLECTUAL PROPERTY RIGHTS.
(a) Trade Secrets and Confidential Information. You acknowledge and agree
that any and all confidential information and materials from which MaxMind
derives actual or potential economic value constitutes MaxMind's
confidential and proprietary trade secrets (collectively, "Trade Secrets").
You further acknowledge and agree that MaxMind's Trade Secrets include, but
are not limited to, the GeoIP Data contained in the GeoIP Databases and the
technology used in the GeoIP Programs. You shall maintain any information
learned about MaxMind's Trade Secrets as confidential and shall not disclose
such information or permit such information to be disclosed to any person or
entity. With respect to all such information, you shall exercise the same
degree of care to protect MaxMind's Trade Secrets that you exercise with
respect to protecting your own confidential information, and in no event
less than reasonable care.
(b) Ownership. All intellectual property rights including copyrights, moral
rights, trademarks, trade secrets, proprietary rights to the GeoIP
Databases, GeoIP Data and GeoIP Programs are exclusively owned by MaxMind.
You acknowledge and agree that you obtain no right, title or interest
therein. You hereby assign to MaxMind all copyrights, intellectual property
rights, and any other proprietary or property rights or interests in and to
any work created in violation of this Agreement.
FEES.
MaxMind's current fee schedule for using the GeoIP Databases and related
services is posted on the MaxMind website (url: http://www.maxmind.com).
You are responsible for paying all fees associated with the use of the GEOIP
Databases.
CHANGES TO THE AGREEMENT.
MaxMind may amend this Agreement at any time. Any such amendment(s) shall
be binding and effective upon the earlier of (i) the date that is thirty
(30) days after posting of the amended Agreement on MaxMind's web site, or
(ii) the date that MaxMind provides notice to you of the amended Agreement
pursuant to the notice provisions in this Agreement; except that changes to
charges and payment terms may be made only upon 30 days' prior written
notice to you. You may immediately terminate this Agreement upon notice to
MaxMind if a change is unacceptable to you. Your continued use of the GeoIP
Data, GeoIP Databases or GeoIP Programs following notice to you of a change
shall constitute your acceptance of the change.
LIMITATION ON LIABILITY.
MAXMIND'S MAXIMUM TOTAL LIABILITY FOR ALL OCCURRENCES (IF ANY), TAKING PLACE
DURING ANY TWELVE-MONTH PERIOD (OR A PORTION THEREOF, IF THIS AGREEMENT IS NOT
IN EFFECT FOR TWELVE MONTHS), ARISING OUT OF OR IN ANY WAY RELATED TO THE
AUTHORIZED OR UNAUTHORIZED ACTS OF MAXMIND'S EMPLOYEES OR MAXMIND'S PERFORMANCE
OR NONPERFORMANCE OF THE SERVICES PROVIDED HEREIN, INCLUDING (BUT NOT LIMITED
TO) ERRORS OF DESIGN OR ERRORS WHICH ARE DUE SOLELY TO MALFUNCTION OF
MAXMIND-CONTROLLED MACHINES OR FAILURES OF MAXMIND OPERATORS, MAXMIND
PROGRAMMERS OR MAXMIND-DEVELOPED PROGRAMS, SHALL BE LIMITED TO GENERAL MONEY
DAMAGES IN AN AMOUNT NOT TO EXCEED THE TOTAL AMOUNT PAID BY YOU FOR SERVICES
PROVIDED BY MAXMIND UNDER THIS AGREEMENT DURING SAID TWELVE-MONTH PERIOD (OR
DURING SUCH SHORTER PERIOD THAT THIS AGREEMENT IS IN EFFECT). YOU AGREE THAT THE
FOREGOING SHALL CONSTITUTE YOUR EXCLUSIVE REMEDY. YOU HEREBY RELEASE MAXMIND,
ITS OFFICERS, EMPLOYEES AND AFFILIATES FROM ANY AND ALL OBLIGATIONS, LIABILITIES
AND CLAIMS IN EXCESS OF THIS LIMITATION.
NO CONSEQUENTIAL DAMAGES.
UNDER NO CIRCUMSTANCES INCLUDING NEGLIGENCE SHALL MAXMIND OR ANY RELATED PARTY OR SUPPLIER BE
LIABLE FOR INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR PUNITIVE DAMAGES;
OR FOR LOSS OF PROFITS, REVENUE, OR DATA; THAT ARE DIRECTLY OR INDIRECTLY
RELATED TO THE USE OF, OR THE INABILITY TO USE THE SITE OR SERVICES, WHETHER
IN AN ACTION IN CONTRACT, TORT, PRODUCT LIABILITY, STRICT LIABILITY, STATUTE
OR OTHERWISE EVEN IF MAXMIND HAS BEEN ADVISED OF THE POSSIBILITY OF THOSE
DAMAGES.
NO WARRANTIES.
THE GEOIP DATABASES, GEOIP DATA AND GEOIP PROGRAMS ARE FURNISHED ON AN "AS
IS", AS-AVAILABLE BASIS. MAXMIND MAKES NO WARRANTY, EXPRESS OR IMPLIED,
WITH RESPECT TO THE CAPABILITY OF THE GEOIP DATABASES, GEOIP DATA AND GEOIP
PROGRAMS OR THE ACCURACY OR THE COMPLETENESS OF THE GEOIP DATA. ALL
WARRANTIES OF ANY TYPE, EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT OF
THIRD PARTY RIGHTS ARE EXPRESSLY DISCLAIMED. MAXMIND DOES NOT WARRANT THAT
THE SERVICES WILL MEET ANY PARTICULAR CRITERIA OF PERFORMANCE OR QUALITY, OR
THAT THE SITE IS FREE OF OTHER HARMFUL COMPONENTS. NEVERTHELESS, MAXMIND
SHALL MAKE COMMERCIALLY REASONABLE EFFORTS TO MAINTAIN THE SITE FREE OF
VIRUSES AND MALICIOUS CODE.
GOVERNING LAW.
This Agreement shall be governed and interpreted pursuant to the laws of the
Commonwealth of Massachusetts, applicable to contracts made and to be
performed wholly in Massachusetts, without regard to principles of conflicts
of laws.
You specifically consent to personal jurisdiction in Massachusetts in connection
with any dispute between you and MaxMind arising out of this Agreement or
pertaining to the subject matter hereof. You agree that the exclusive venue for
any dispute hereunder shall be in the state and federal courts in Boston,
Massachusetts.
NOTICES
Notices given under this Agreement shall be in writing and sent by
facsimile, e-mail, or by first class mail or equivalent. MaxMind shall
direct notice to you at the facsimile number, e-mail address, or physical
mailing address (collectively, "Address") you provided in the registration
process. You shall direct notice to MaxMind at the following address:
MaxMind, Inc.
PO Box 230074
Boston MA 02123-0074
Email: legal@maxmind.com
Fax: (815) 301-8737
Either party may change its Address for notice at anytime by giving notice
of the new Address as provided in this section.
COMPLETE AGREEMENT.
This Agreement represents
the entire agreement between you and MaxMind with respect to the subject
matter hereof and supersedes all previous representations, understandings or
agreements, oral or written, between the parties regarding the subject
matter hereof.
APPLICABLE LAWS.
You agree to use the GeoIP Data and GeoIP Databases only in a
manner that is consistent with applicable laws.
ASSIGNMENT.
You may not assign your rights in this Agreement without MaxMind's prior
written consent.
SEVERABILITY.
Should any provision of this Agreement be held void, invalid or inoperative,
such decision shall not affect any other provision hereof, and the remainder
of this Agreement shall be effective as though such void, invalid or
inoperative provision had not been contained herein.
FAILURE TO ENFORCE.
The failure of MaxMind to enforce any provision of these terms and
conditions shall not constitute or be construed as a waiver of such
provision or of the right to enforce it at a later time.
FACSIMILES.
If you submit a document which includes your facsimile signature, MaxMind
may treat the facsimile signature as an original of your signature.
CAPTIONS.
The section headings used herein, are for convenience only and shall have no
force or effect upon the construction or interpretation of any provision
hereof.

View File

@@ -211,10 +211,8 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java_DOM4J1.6" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar"
REQUIRED="true" BundleID="" />
<IMPORT NAME="Java_JAXEN1.1" MODULE="jaxen-1.1.1.jar" MVN="mvn:org.talend.libraries/jaxen-1.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/jaxen-1.1.1.jar"
REQUIRED="true" BundleID="" />
<IMPORT NAME="Java_DOM4J_2.1.1" MODULE="dom4j-2.1.1.jar" MVN="mvn:org.dom4j/dom4j/2.1.1" REQUIRED="true" BundleID="" />
<IMPORT NAME="Java_JAXEN_1.1.6" MODULE="jaxen-1.1.6.jar" MVN="mvn:jaxen/jaxen/1.1.6" REQUIRED="true" BundleID="" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -215,6 +215,8 @@
</TEMPLATES>
<IMPORTS>
<IMPORT NAME="Driver-MysqlConnector" MODULE="mysql-connector-java-5.1.30-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.30-bin/6.0.0" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="(USE_SHARED_CONNECTION == 'true' AND SPECIFY_DATASOURCE_ALIAS=='false')" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="(USE_SHARED_CONNECTION == 'true' AND SPECIFY_DATASOURCE_ALIAS=='false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS/>

View File

@@ -73,7 +73,7 @@ FIELD_OPTIONS.ITEM.DELETE_KEY=Key in delete
FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
PROPERTIES.NAME=Additional JDBC Parameters
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
NB_LINE_UPDATED.NAME=Number Of Updated Lines
NB_LINE_INSERTED.NAME=Number Of Inserted Lines

View File

@@ -121,14 +121,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -5,7 +5,7 @@ if(!use_customize_version){
%>String[] applications_<%=cid%> = <%
String application = ElementParameterParser.getValue(node, "__APPLICATION__");
if("ALL_APPLICATIONS".equals(application)) {
if("\"emr-4.6.0\",\"emr-4.9.2\",\"emr-5.11.0\"".contains(cluster_version)) {
if("\"emr-4.6.0\",\"emr-4.9.2\",\"emr-5.11.0\",\"emr-5.15.0\"".contains(cluster_version)) {
%>
{}
<%
@@ -13,21 +13,9 @@ if(!use_customize_version){
%>
{"Ganglia","Hadoop","Hive","Hue","Mahout","Pig","Spark"}
<%
} else if("\"emr-4.2.0\"".equals(cluster_version)) {
%>
{"Ganglia","Hadoop","Hive","Hue","Mahout","Pig","Spark"}
<%
} else if("\"emr-4.1.0\"".equals(cluster_version)) {
%>
{"Hadoop","Hive","Hue","Mahout","Pig","Spark"}
<%
} else if("\"emr-4.0.0\"".equals(cluster_version)) {
%>
{"Hadoop","Hive","Mahout","Pig","Spark"}
<%
}
} else if("CORE_HADOOP".equals(application)) {
if("\"emr-5.11.0\"".equals(cluster_version)){%>
if("\"emr-5.11.0\"".equals(cluster_version) || "\"emr-5.15.0\"".equals(cluster_version)){%>
{"Ganglia","Hadoop","Hive","Hue","Mahout","Pig","Tez"}
<%
}else if("\"emr-4.6.0\"".equals(cluster_version) || "\"emr-4.9.2\"".equals(cluster_version)) {
@@ -38,18 +26,6 @@ if(!use_customize_version){
%>
{"Ganglia","Hadoop","Hive","Pig"}
<%
} else if("\"emr-4.2.0\"".equals(cluster_version)) {
%>
{"Ganglia","Hadoop","Hive","Pig"}
<%
} else if("\"emr-4.1.0\"".equals(cluster_version)) {
%>
{"Hadoop","Hive","Pig"}
<%
} else if("\"emr-4.0.0\"".equals(cluster_version)) {
%>
{"Hadoop","Hive","Pig"}
<%
}
} else if("PRESTO_SANDBOX".equals(application)) {
if("\"emr-4.6.0\"".equals(cluster_version) || "\"emr-4.9.2\"".equals(cluster_version)) {
@@ -60,17 +36,9 @@ if(!use_customize_version){
%>
{"Hadoop","Hive","Presto-Sandbox"}
<%
} else if("\"emr-4.2.0\"".equals(cluster_version)) {
%>
{"Hadoop","Hive","Presto-Sandbox"}
<%
} else if("\"emr-4.1.0\"".equals(cluster_version)) {
%>
{"Hadoop","Hive","Presto-Sandbox"}
<%
}
} else if("PRESTO".equals(application)) {
if("\"emr-5.11.0\"".equals(cluster_version)){%>
if("\"emr-5.11.0\"".equals(cluster_version) || "\"emr-5.15.0\"".equals(cluster_version)){%>
{"Hadoop","Hive","Presto"}
<% } else { %>
if(true) {
@@ -79,7 +47,7 @@ if(!use_customize_version){
<%
}
} else if("SPARK".equals(application)) {
if("\"emr-5.11.0\"".equals(cluster_version)){%>
if("\"emr-5.11.0\"".equals(cluster_version) || "\"emr-5.15.0\"".equals(cluster_version)){%>
{"Ganglia","Hadoop","Spark","Zeppelin"}
<%
} else if("\"emr-4.6.0\"".equals(cluster_version) || "\"emr-4.9.2\"".equals(cluster_version)) {
@@ -90,21 +58,9 @@ if(!use_customize_version){
%>
{"Ganglia","Hadoop","Spark"}
<%
} else if("\"emr-4.2.0\"".equals(cluster_version)) {
%>
{"Ganglia","Hadoop","Spark"}
<%
} else if("\"emr-4.1.0\"".equals(cluster_version)) {
%>
{"Hadoop","Spark"}
<%
} else if("\"emr-4.0.0\"".equals(cluster_version)) {
%>
{"Hadoop","Spark"}
<%
}
} else if("HBASE".equals(application)) {
if("\"emr-5.11.0\"".equals(cluster_version) || "\"emr-4.9.2\"".equals(cluster_version)){%>
if("\"emr-5.11.0\"".equals(cluster_version) || "\"emr-4.9.2\"".equals(cluster_version) || "\"emr-5.15.0\"".equals(cluster_version)){%>
{"HBase","Ganglia","Hadoop","Hive","Hue","Phoenix","ZooKeeper"}
<%
} else if("\"emr-4.6.0\"".equals(cluster_version)) {

View File

@@ -97,14 +97,12 @@
<PARAMETER NAME="CLUSTER_VERSION" FIELD="OPENED_LIST"
NUM_ROW="32" SHOW_IF="(ACTION=='START') AND (USE_CUSTOMIZE_VERSION == 'false')" GROUP="CONFIGURATION">
<ITEMS DEFAULT="EMR5_11_0">
<ITEMS DEFAULT="EMR5_15_0">
<ITEM NAME="EMR5_15_0" VALUE="&quot;emr-5.15.0&quot;" />
<ITEM NAME="EMR5_11_0" VALUE="&quot;emr-5.11.0&quot;" />
<ITEM NAME="EMR4_9_2" VALUE="&quot;emr-4.9.2&quot;" />
<ITEM NAME="EMR4_6_0" VALUE="&quot;emr-4.6.0&quot;" />
<ITEM NAME="EMR4_5_0" VALUE="&quot;emr-4.5.0&quot;" />
<ITEM NAME="EMR4_2_0" VALUE="&quot;emr-4.2.0&quot;" />
<ITEM NAME="EMR4_1_0" VALUE="&quot;emr-4.1.0&quot;" />
<ITEM NAME="EMR4_0_0" VALUE="&quot;emr-4.0.0&quot;" />
<ITEM NAME="AMI3_10_0" VALUE="&quot;3.10.0&quot;" />
<ITEM NAME="AMI3_9_0" VALUE="&quot;3.9.0&quot;" />
<ITEM NAME="AMI3_8_0" VALUE="&quot;3.8.0&quot;" />
@@ -114,14 +112,14 @@
</PARAMETER>
<PARAMETER NAME="APPLICATION" FIELD="CLOSED_LIST" NUM_ROW="32"
SHOW_IF="(ACTION=='START') AND (USE_CUSTOMIZE_VERSION == 'false') AND ((CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR4_9_2') OR (CLUSTER_VERSION=='EMR4_6_0') OR (CLUSTER_VERSION=='EMR4_5_0') OR (CLUSTER_VERSION=='EMR4_2_0') OR (CLUSTER_VERSION=='EMR4_1_0') OR (CLUSTER_VERSION=='EMR4_0_0'))" GROUP="CONFIGURATION">
SHOW_IF="(ACTION=='START') AND (USE_CUSTOMIZE_VERSION == 'false') AND ((CLUSTER_VERSION=='EMR5_15_0') OR (CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR4_9_2') OR (CLUSTER_VERSION=='EMR4_6_0') OR (CLUSTER_VERSION=='EMR4_5_0'))" GROUP="CONFIGURATION">
<ITEMS DEFAULT="CORE_HADOOP">
<ITEM NAME="ALL_APPLICATIONS" VALUE="ALL_APPLICATIONS" NOT_SHOW_IF="(CLUSTER_VERSION=='EMR4_6_0') OR (CLUSTER_VERSION=='EMR4_9_2') OR (CLUSTER_VERSION=='EMR5_11_0')"/>
<ITEM NAME="ALL_APPLICATIONS" VALUE="ALL_APPLICATIONS" NOT_SHOW_IF="(CLUSTER_VERSION=='EMR4_6_0') OR (CLUSTER_VERSION=='EMR4_9_2') OR (CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR5_15_0')"/>
<ITEM NAME="CORE_HADOOP" VALUE="CORE_HADOOP" />
<ITEM NAME="HBASE" VALUE="HBASE" SHOW_IF="(CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR4_9_2') OR (CLUSTER_VERSION=='EMR4_6_0')"/>
<ITEM NAME="PRESTO_SANDBOX" VALUE="PRESTO_SANDBOX" NOT_SHOW_IF="(CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR4_0_0')"/>
<ITEM NAME="PRESTO" VALUE="PRESTO" SHOW_IF="(CLUSTER_VERSION=='EMR5_11_0')"/>
<ITEM NAME="HBASE" VALUE="HBASE" SHOW_IF="(CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR4_9_2') OR (CLUSTER_VERSION=='EMR4_6_0') OR (CLUSTER_VERSION=='EMR5_15_0')"/>
<ITEM NAME="PRESTO_SANDBOX" VALUE="PRESTO_SANDBOX" NOT_SHOW_IF="(CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR5_15_0')"/>
<ITEM NAME="PRESTO" VALUE="PRESTO" SHOW_IF="(CLUSTER_VERSION=='EMR5_11_0') OR (CLUSTER_VERSION=='EMR5_15_0')"/>
<ITEM NAME="SPARK" VALUE="SPARK" />
</ITEMS>
</PARAMETER>
@@ -298,7 +296,7 @@
<PARAMETER NAME="CUSTOMIZE_VERSION" FIELD="TEXT"
NUM_ROW="12" SHOW_IF="(ACTION=='START') AND (USE_CUSTOMIZE_VERSION =='true')" GROUP="CUSTOMIZE_APPLICATION_GROUP">
<DEFAULT>"emr-5.11.0"</DEFAULT>
<DEFAULT>"emr-5.15.0"</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CUSTOMIZE_APPLICATION_TABLE" FIELD="TABLE" NUM_ROW="13" NB_LINES="5" GROUP="CUSTOMIZE_APPLICATION_GROUP"
@@ -427,14 +425,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -29,13 +29,11 @@ REGION.ITEM.SA_EAST_1=South America (Sao Paulo)
CLUSTER_NAME.NAME=Cluster name
CLUSTER_VERSION.NAME=Cluster version
CLUSTER_VERSION.ITEM.EMR5_15_0=EMR 5.15.0
CLUSTER_VERSION.ITEM.EMR5_11_0=EMR 5.11.0
CLUSTER_VERSION.ITEM.EMR4_9_2=EMR 4.9.2
CLUSTER_VERSION.ITEM.EMR4_6_0=EMR 4.6.0
CLUSTER_VERSION.ITEM.EMR4_5_0=EMR 4.5.0
CLUSTER_VERSION.ITEM.EMR4_2_0=EMR 4.2.0
CLUSTER_VERSION.ITEM.EMR4_1_0=EMR 4.1.0
CLUSTER_VERSION.ITEM.EMR4_0_0=EMR 4.0.0
CLUSTER_VERSION.ITEM.AMI3_10_0=AMI 3.10.0
CLUSTER_VERSION.ITEM.AMI3_9_0=AMI 3.9.0
CLUSTER_VERSION.ITEM.AMI3_8_0=AMI 3.8.0

View File

@@ -178,14 +178,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -164,6 +164,8 @@
<IMPORTS>
<IMPORT NAME="Driver-MysqlConnector" MODULE="mysql-connector-java-5.1.30-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.30-bin/6.0.0" REQUIRED_IF="DB_VERSION == 'MYSQL_5'" />
<IMPORT NAME="Driver-MysqlConnector_Old" MODULE="mysql-connector-java-5.1.0-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.0-bin/6.0.0" REQUIRED="false" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS/>

View File

@@ -763,6 +763,7 @@ if(columnList != null && columnList.size() > 0) {
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
<%
if (!("").equals(numPerInsert ) && !("0").equals(numPerInsert )) {
@@ -776,6 +777,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -853,6 +855,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -979,6 +982,7 @@ if(columnList != null && columnList.size() > 0) {
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
@@ -986,6 +990,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().updating(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -1137,6 +1142,7 @@ if(columnList != null && columnList.size() > 0) {
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -1212,6 +1218,7 @@ if(columnList != null && columnList.size() > 0) {
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -1389,11 +1396,13 @@ if(columnList != null && columnList.size() > 0) {
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
}else{
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().updating(dbLog.var("nb_line"));%>
}
} catch(java.lang.Exception e) {
@@ -1472,12 +1481,14 @@ if(columnList != null && columnList.size() > 0) {
%>
pstmt_<%=cid %>.addBatch();
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().addingToBatch(dbLog.var("nb_line"), dbLog.str(dataAction));%>
batchSizeCounter_<%=cid%>++;
<%
}else {
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>
@@ -1625,6 +1636,7 @@ if(columnList != null && columnList.size() > 0) {
%>
int count_on_duplicate_key_<%=cid%> = 0;
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
count_on_duplicate_key_<%=cid%> = pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().insertingOnDuplicateKeyUpdating(null);%>
} catch(java.lang.Exception e) {
@@ -1690,6 +1702,7 @@ if(columnList != null && columnList.size() > 0) {
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().insertingOnDuplicateKeyUpdating(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {

View File

@@ -82,7 +82,7 @@ FIELD_OPTIONS.ITEM.DELETE_KEY=Key in delete
FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
PROPERTIES.NAME=Additional JDBC Parameters
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
NB_LINE_UPDATED.NAME=Number Of Updated Lines
NB_LINE_INSERTED.NAME=Number Of Inserted Lines

View File

@@ -56,7 +56,7 @@ imports="
if(isLog4jEnabled){
shareDBClass = "SharedDBConnectionLog4j";
%>
SharedDBConnectionLog4j.initLogger(log,"<%=cid%>");
SharedDBConnectionLog4j.initLogger(log.getName(),"<%=cid%>");
<%
}
%>

View File

@@ -243,6 +243,8 @@
<IMPORT NAME="Driver-Ojdbc8" MODULE="ojdbc8-12.2.0.1.jar" MVN="mvn:com.oracle.jdbc/ojdbc8/12.2.0.1" REQUIRED_IF="(DB_VERSION == 'ORACLE_18')" />
<IMPORT NAME="Driver-Ojdbc7" MODULE="ojdbc7.jar" MVN="mvn:org.talend.libraries/ojdbc7/6.0.0" REQUIRED_IF="(DB_VERSION == 'ORACLE_12')" />
<IMPORT NAME="Driver-Ojdbc6" MODULE="ojdbc6.jar" MVN="mvn:org.talend.libraries/ojdbc6/6.0.0" REQUIRED_IF="(DB_VERSION == 'ORACLE_11')" />
<IMPORT NAME="slf4j-api-1.7.25.jar" MODULE="slf4j-api-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-api/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
<IMPORT NAME="slf4j-log4j12-1.7.25.jar" MODULE="slf4j-log4j12-1.7.25.jar" MVN="mvn:org.slf4j/slf4j-log4j12/1.7.25" REQUIRED_IF="USE_SHARED_CONNECTION =='true'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -46,6 +46,10 @@ String batchSize=ElementParameterParser.getValue(node,"__BATCH_SIZE__");
boolean useExistingConnection = "true".equals(ElementParameterParser.getValue(node,"__USE_EXISTING_CONNECTION__"));
boolean useAlternateSchema = "true".equals(ElementParameterParser.getValue(node, "__USE_ALTERNATE_SCHEMA__"));
String alternateSchema = ElementParameterParser.getValue(node, "__ALTERNATE_SCHEMA__");
String rejectConnName = null;
List<? extends IConnection> rejectConns = node.getOutgoingConnections("REJECT");
if(rejectConns != null && rejectConns.size() > 0) {
@@ -198,8 +202,16 @@ if(useExistingConnection) {
String connection = ElementParameterParser.getValue(node,"__CONNECTION__");
String conn = "conn_" + connection;
String schema = "dbschema_" + connection;
if(useAlternateSchema) {
%>
dbschema_<%=cid%> = <%=alternateSchema%>;
<%
} else {
%>
dbschema_<%=cid%> = (String)globalMap.get("<%=schema%>");
<%
}
%>
dbschema_<%=cid%> = (String)globalMap.get("<%=schema%>");
conn_<%=cid%> = (java.sql.Connection)globalMap.get("<%=conn%>");
<%dbLog.conn().useExistConn("conn_"+cid+".getMetaData().getURL()", "conn_"+cid+".getMetaData().getUserName()");%>
<%

View File

@@ -145,7 +145,7 @@
NUM_ROW="50"
REPOSITORY_VALUE="SCHEMA"
REQUIRED="false"
SHOW_IF="((USE_EXISTING_CONNECTION == 'false') "
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') "
>
<DEFAULT>""</DEFAULT>
</PARAMETER>
@@ -266,7 +266,6 @@
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') and (CONNECTION_TYPE != 'ORACLE_WALLET')">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="NOTE"
FIELD="LABEL"
@@ -274,6 +273,26 @@
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') and (CONNECTION_TYPE != 'ORACLE_WALLET')">
<DEFAULT>*Note: Example for Additional JDBC Parameters: "parameterName1=value1&amp;&amp;parameterName2=value2"</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="USE_ALTERNATE_SCHEMA"
FIELD="CHECK"
NUM_ROW="15"
SHOW_IF="USE_EXISTING_CONNECTION == 'true'"
>
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="ALTERNATE_SCHEMA"
FIELD="TEXT"
NUM_ROW="20"
REQUIRED="false"
SHOW_IF="(USE_EXISTING_CONNECTION == 'true') and (USE_ALTERNATE_SCHEMA == 'true')"
>
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="ENCODING"
FIELD="ENCODING_TYPE"

View File

@@ -453,6 +453,7 @@ IMetadataColumn talendDynCol = null;
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -594,6 +595,7 @@ IMetadataColumn talendDynCol = null;
%>
try {
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
<%dbLog.data().updating(dbLog.var("nb_line"));%>
} catch(java.lang.Exception e) {
@@ -758,6 +760,7 @@ IMetadataColumn talendDynCol = null;
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
updatedCount_<%=cid%> = updatedCount_<%=cid%> + pstmtUpdate_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().updating(dbLog.var("nb_line"));%>
@@ -838,6 +841,7 @@ IMetadataColumn talendDynCol = null;
}
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
@@ -1023,11 +1027,13 @@ IMetadataColumn talendDynCol = null;
<%
}
%>
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
insertedCount_<%=cid%> = insertedCount_<%=cid%> + pstmtInsert_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().inserting(dbLog.var("nb_line"));%>
}else{
nb_line_<%=cid%>++;
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
<%dbLog.data().updating(dbLog.var("nb_line"));%>
}
} catch(java.lang.Exception e) {
@@ -1116,6 +1122,7 @@ IMetadataColumn talendDynCol = null;
}else {
%>
try {
<%if(isEnableDebug){dbLog.data().sqlDebugPrint("globalMap.get(\""+cid+"_QUERY\")");}%>
deletedCount_<%=cid%> = deletedCount_<%=cid%> + pstmt_<%=cid %>.executeUpdate();
nb_line_<%=cid%>++;
<%dbLog.data().deleting(dbLog.var("nb_line"));%>

View File

@@ -77,7 +77,7 @@ FIELD_OPTIONS.ITEM.UPDATABLE=Updatable
FIELD_OPTIONS.ITEM.INSERTABLE=Insertable
LOCAL_SERVICE_NAME.NAME=Local Service Name
CONVERT_COLUMN_TABLE_TO_UPPERCASE.NAME=Convert columns and table to uppercase
ENABLE_DEBUG_MODE.NAME=Enable debug mode
ENABLE_DEBUG_MODE.NAME=Debug query mode
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.ORACLE_18=Oracle 18
DB_VERSION.ITEM.ORACLE_12=Oracle 12-7
@@ -118,3 +118,5 @@ SUPPORT_NULL_WHERE.NAME=Support null in "SQL WHERE" statement
USE_TIMESTAMP_FOR_DATE_TYPE.NAME = Use Timestamp for Date type
NB_LINE_REJECTED.NAME=Number Of Rejected Lines
USE_ALTERNATE_SCHEMA.NAME=Use alternate schema
ALTERNATE_SCHEMA.NAME=Schema

View File

@@ -234,14 +234,14 @@
<IMPORT NAME="aws-java-sdk-1.11.406.jar" MODULE="aws-java-sdk-1.11.406.jar"
MVN="mvn:com.amazonaws/aws-java-sdk/1.11.406"
REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind-2.9.9.jar" MODULE="jackson-databind-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
<IMPORT NAME="jackson-databind-2.10.1.jar" MODULE="jackson-databind-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations-2.9.0.jar" MODULE="jackson-annotations-2.9.0.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
<IMPORT NAME="jackson-annotations-2.10.1.jar" MODULE="jackson-annotations-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar"
MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9"

View File

@@ -5,6 +5,7 @@
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.process.IConnection
java.util.List
"
%>
@@ -36,6 +37,18 @@
String passwordFieldName = "";
IConnection incomingConnection = null;
if(node.getUniqueName().startsWith("tBigQueryOutput_")) {
List< ? extends IConnection> conns = node.getIncomingConnections();
IConnection virtConnection = null;
if(conns!=null && conns.size()>0){
virtConnection = node.getIncomingConnections().get(0);
}
if(virtConnection != null && virtConnection.getSource() != null && virtConnection.getSource().getIncomingConnections() != null && virtConnection.getSource().getIncomingConnections().size() > 0) {
incomingConnection = virtConnection.getSource().getIncomingConnections().get(0);
}
}
if (authMode.equals("OAUTH")) {
%>
/* ----START-CREATING-CLIENT (OAuth 2.0)---- */
@@ -247,6 +260,11 @@
%>
<%
if(isLog4jEnabled){
%>
int currIndex_<%=cid%> = 0;
<%
}
if (authMode.equals("OAUTH")) {
if(isLog4jEnabled){
%>
@@ -309,41 +327,82 @@
if (metadata != null) {
List<IMetadataColumn> columns = metadata.getListColumns();
int nbColumns = columns.size();
int dynamicColumnSize = 0;
for (int i = 0; i < nbColumns; i++ ) {
IMetadataColumn column = columns.get(i);
String columnName = column.getLabel();
String typeToGenerate = "string";
if("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "float";
}else if("id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType()) || "id_Short".equals(column.getTalendType())) {
typeToGenerate = "integer";
} else if("id_Character".equals(column.getTalendType())) {
typeToGenerate = "string";
} else if("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "numeric";
} else if("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "boolean";
} else if("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "date";
}else if(pattern.length() > 12){
typeToGenerate = "timestamp";
}else{
typeToGenerate = "string";
}
}
String modeType = (!column.isNullable()) ? "REQUIRED" : "NULLABLE";
if(!"id_Dynamic".equals(column.getTalendType())) {
String typeToGenerate = "string";
if("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "float";
}else if("id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType()) || "id_Short".equals(column.getTalendType())) {
typeToGenerate = "integer";
} else if("id_Character".equals(column.getTalendType())) {
typeToGenerate = "string";
} else if("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "numeric";
} else if("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "boolean";
} else if("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "date";
}else if(pattern.length() > 12){
typeToGenerate = "timestamp";
}else{
typeToGenerate = "string";
}
}
String modeType = (!column.isNullable()) ? "REQUIRED" : "NULLABLE";
%>
com.google.api.services.bigquery.model.TableFieldSchema <%=columnName%>_<%=cid%> = new com.google.api.services.bigquery.model.TableFieldSchema();
<%=columnName%>_<%=cid%>.setName("<%=columnName%>");
<%=columnName%>_<%=cid%>.setType("<%=typeToGenerate%>");
<%=columnName%>_<%=cid%>.setMode("<%=modeType%>");
fields_<%=cid%>.add(<%=columnName%>_<%=cid%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[" + currIndex_<%=cid%> + "] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\",\"mode\":\"<%=modeType%>\"}");
currIndex_<%=cid%>++;
<%
}
} else if(incomingConnection != null) {
%>
com.google.api.services.bigquery.model.TableFieldSchema <%=columnName%>_<%=cid%> = new com.google.api.services.bigquery.model.TableFieldSchema();
<%=columnName%>_<%=cid%>.setName("<%=columnName%>");
<%=columnName%>_<%=cid%>.setType("<%=typeToGenerate%>");
<%=columnName%>_<%=cid%>.setMode("<%=modeType%>");
fields_<%=cid%>.add(<%=columnName%>_<%=cid%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[<%=i%>] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\",\"mode\":\"<%=modeType%>\"}");
routines.system.Dynamic column_<%=cid%> = (routines.system.Dynamic)<%=incomingConnection.getName()%>.<%=column.getLabel()%>;
for (int i = 0; i < column_<%=cid%>.getColumnCount(); i++) {
routines.system.DynamicMetadata metadata = column_<%=cid%>.getColumnMetadata(i);
String dataType = routines.system.MetadataTalendType.getDefaultSelectedDbType("bigquery_id", metadata.getType(), -1, -1);
String orignalDBType = metadata.getDbType();
java.util.List<String> outputDBTypelist = routines.system.MetadataTalendType.getTalendToDBList("bigquery_id", metadata.getType());
if (outputDBTypelist != null && outputDBTypelist.size() > 0) {
for (String dbType : outputDBTypelist) {
if (orignalDBType.equalsIgnoreCase(dbType)) {
dataType = dbType;
break;
}
}
}
if (dataType == null || ("").equals(dataType)) {
dataType = orignalDBType;
} else if(dataType.equalsIgnoreCase("RECORD") || dataType.equalsIgnoreCase("STRUCT")) {
dataType = "STRING";
}
String modeType = (!metadata.isNullable()) ? "REQUIRED" : "NULLABLE";
com.google.api.services.bigquery.model.TableFieldSchema <%=columnName%>_<%=cid%> = new com.google.api.services.bigquery.model.TableFieldSchema();
<%=columnName%>_<%=cid%>.setName(metadata.getName());
<%=columnName%>_<%=cid%>.setType(dataType);
<%=columnName%>_<%=cid%>.setMode(modeType);
fields_<%=cid%>.add(<%=columnName%>_<%=cid%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[" + currIndex_<%=cid%> + "] {\"name\":\"" + metadata.getName() + "\",\"type\":\"" + dataType + "\",\"mode\":\"" + modeType + "\"}");
currIndex_<%=cid%>++;
<%
}
%>
}
<%
}
}
@@ -529,38 +588,117 @@
for (int i = 0; i < nbColumns; i++ ) {
IMetadataColumn column = columns.get(i);
String columnName = column.getLabel();
String typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
if("id_String".equals(column.getTalendType()) || "id_Character".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
} else if ("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.FLOAT";
} else if ("id_Short".equals(column.getTalendType()) || "id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.INTEGER";
} else if ("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.NUMERIC";
} else if ("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.BOOLEAN";
} else if ("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.DATE";
}else if(pattern.length() > 12){
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.TIMESTAMP";
}else{
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
}
}
String modeType = (!column.isNullable()) ? "REQUIRED" : "NULLABLE";
%>
com.google.cloud.bigquery.Field field_<%=i%> = com.google.cloud.bigquery.Field.newBuilder("<%=columnName%>", <%=typeToGenerate%>)
.setMode(com.google.cloud.bigquery.Field.Mode.valueOf("<%=modeType%>"))
.build();
fields_<%=cid%>.add(field_<%=i%>);
<%
if(isLog4jEnabled){
if(!"id_Dynamic".equals(column.getTalendType())) {
String typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
if("id_String".equals(column.getTalendType()) || "id_Character".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
} else if ("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.FLOAT";
} else if ("id_Short".equals(column.getTalendType()) || "id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.INTEGER";
} else if ("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.NUMERIC";
} else if ("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.BOOLEAN";
} else if ("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.DATE";
}else if(pattern.length() > 12){
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.TIMESTAMP";
}else{
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
}
}
String modeType = (!column.isNullable()) ? "REQUIRED" : "NULLABLE";
%>
log.debug("<%=cid%> - Field index[<%=i%>] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\"}");
com.google.cloud.bigquery.Field field_<%=i%> = com.google.cloud.bigquery.Field.newBuilder("<%=columnName%>", <%=typeToGenerate%>)
.setMode(com.google.cloud.bigquery.Field.Mode.valueOf("<%=modeType%>"))
.build();
fields_<%=cid%>.add(field_<%=i%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[" + currIndex_<%=cid%> + "] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\"}");
currIndex_<%=cid%>++;
<%
}
} else if(incomingConnection != null) {
%>
routines.system.Dynamic column_<%=cid%> = (routines.system.Dynamic)<%=incomingConnection.getName()%>.<%=column.getLabel()%>;
for (int i = 0; i < column_<%=cid%>.getColumnCount(); i++) {
com.google.cloud.bigquery.LegacySQLTypeName typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.STRING;
routines.system.DynamicMetadata metadata = column_<%=cid%>.getColumnMetadata(i);
String dataType = routines.system.MetadataTalendType.getDefaultSelectedDbType("bigquery_id", metadata.getType(), -1, -1);
String orignalDBType = metadata.getDbType();
java.util.List<String> outputDBTypelist = routines.system.MetadataTalendType.getTalendToDBList("bigquery_id", metadata.getType());
if (outputDBTypelist != null && outputDBTypelist.size() > 0) {
for (String dbType : outputDBTypelist) {
if (orignalDBType.equalsIgnoreCase(dbType)) {
dataType = dbType;
break;
}
}
}
if (dataType == null || ("").equals(dataType)) {
dataType = orignalDBType;
}
String modeType = (!metadata.isNullable()) ? "REQUIRED" : "NULLABLE";
switch(dataType) {
case "BOOL":
case "BOOLEAN":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.BOOLEAN;
break;
case "INT64":
case "INTEGER":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.INTEGER;
break;
case "FLOAT64":
case "FLOAT":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.FLOAT;
break;
case "STRING":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.STRING;
break;
case "BYTES":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.BYTES;
break;
case "TIMESTAMP":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.TIMESTAMP;
break;
case "DATE":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.DATE;
break;
case "TIME":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.TIME;
break;
case "DATETIME":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.DATETIME;
break;
case "NUMERIC":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.NUMERIC;
break;
case "GEOGRAPHY":
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.valueOf("GEOGRAPHY");
break;
default:
typeToGenerate = com.google.cloud.bigquery.LegacySQLTypeName.STRING;
break;
}
com.google.cloud.bigquery.Field field_<%=i%> = com.google.cloud.bigquery.Field.newBuilder(metadata.getName(), typeToGenerate)
.setMode(com.google.cloud.bigquery.Field.Mode.valueOf(modeType))
.build();
fields_<%=cid%>.add(field_<%=i%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[" + currIndex_<%=cid%> + "] {\"name\":\"" + metadata.getName() + "\",\"type\":\"" + typeToGenerate + "\",\"mode\":\"" + modeType + "\"}");
currIndex_<%=cid%>++;
<%
}
%>
}
<%
}
}

View File

@@ -183,14 +183,14 @@
</ADVANCED_PARAMETERS>
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="google-api-client-1.19.0.jar" MODULE="google-api-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-api-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-api-services-oauth2-v2-rev78-1.19.0.jar" MODULE="google-api-services-oauth2-v2-rev78-1.19.0.jar" MVN="mvn:org.talend.libraries/google-api-services-oauth2-v2-rev78-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-services-oauth2-v2-rev78-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-api-services-bigquery-v2-rev391-1.21.0.jar" MODULE="google-api-services-bigquery-v2-rev391-1.21.0.jar" MVN="mvn:org.talend.libraries/google-api-services-bigquery-v2-rev391-1.21.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-services-bigquery-v2-rev391-1.21.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-1.19.0.jar" MODULE="google-http-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-oauth-client-1.19.0.jar" MODULE="google-oauth-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-oauth-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-oauth-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.19.0.jar" MODULE="google-http-client-jackson2-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-jackson2-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-jackson2-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-api-client-1.25.0.jar" MODULE="google-api-client-1.25.0.jar" MVN="mvn:com.google.api-client/google-api-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-api-services-oauth2-v2-rev151-1.25.0.jar" MODULE="google-api-services-oauth2-v2-rev151-1.25.0.jar" MVN="mvn:com.google.apis/google-api-services-oauth2/v2-rev151-1.25.0" REQUIRED="true" />
<IMPORT NAME="google-api-services-bigquery-v2-rev454-1.25.0.jar" MODULE="google-api-services-bigquery-v2-rev454-1.25.0.jar" MVN="mvn:com.google.apis/google-api-services-bigquery/v2-rev454-1.25.0" REQUIRED="true" />
<IMPORT NAME="google-http-client-1.25.0.jar" MODULE="google-http-client-1.25.0.jar" MVN="mvn:com.google.http-client/google-http-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-oauth-client-1.25.0.jar" MODULE="google-oauth-client-1.25.0.jar" MVN="mvn:com.google.oauth-client/google-oauth-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.25.0.jar" MODULE="google-http-client-jackson2-1.25.0.jar" MVN="mvn:com.google.http-client/google-http-client-jackson2/1.25.0" REQUIRED="true" />
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar" MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1" REQUIRED="true" />
<!-- REQUIRED FOR GOOGLE STORAGE -->
<IMPORT NAME="jets3t-0.9.1" MODULE="jets3t-0.9.1.jar" MVN="mvn:org.talend.libraries/jets3t-0.9.1/6.0.0" REQUIRED="true" />
<IMPORT NAME="commons-logging-1.1.1" MODULE="commons-logging-1.1.1.jar" MVN="mvn:org.talend.libraries/commons-logging-1.1.1/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.1.1.v201101211721.jar" REQUIRED="true" />

View File

@@ -32,12 +32,37 @@
String basePackage = "";
boolean isCustomTemporaryName = ElementParameterParser.getBooleanValue(node,"__USE_CUSTOM_TEMPORARY_DATASET__");
String tempDataset = ElementParameterParser.getValue(node,"__TEMPORARY_DATASET_NAME__");
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
String advancedSeparatorStr = ElementParameterParser.getValue(node, "__ADVANCED_SEPARATOR__");
boolean advancedSeparator = (advancedSeparatorStr!=null&&!("").equals(advancedSeparatorStr))?("true").equals(advancedSeparatorStr):false;
String thousandsSeparator = ElementParameterParser.getValueWithJavaType(node, "__THOUSANDS_SEPARATOR__", JavaTypesManager.CHARACTER);
String decimalSeparator = ElementParameterParser.getValueWithJavaType(node, "__DECIMAL_SEPARATOR__", JavaTypesManager.CHARACTER);
query = query.replaceAll("\n"," ");
query = query.replaceAll("\r"," ");
String tokenFile = ElementParameterParser.getValue(node,"__TOKEN_NAME__");
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
//Dynamic start
List<IMetadataTable> metadatas = node.getMetadataList();
int sizeListColumns = 0;
List<IMetadataColumn> columnList = null;
boolean isDynamic = false;
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
isDynamic = metadata.isDynamicSchema();
if (metadata!=null) {
columnList = metadata.getListColumns();
sizeListColumns = columnList.size();
}
}
int dynamic_index=-1;
//Dynamic end
if (authMode.equals("OAUTH")) {
%>
final String CLIENT_ID_<%=cid %> = <%=clientId %>;
@@ -214,6 +239,43 @@
}
%>
String pageToken_<%=cid%> = null;
//Dynamic start
<%
if(isDynamic){
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/DB/Input/BigQuery/BigQueryHelper_Oauth.javajet"%>
BigQueryHelper_<%=cid%> helper_<%=cid%> = new BigQueryHelper_<%=cid%>();
com.google.api.services.bigquery.model.TableSchema schema_<%=cid%> = bigqueryclient_<%=cid%>
.jobs().getQueryResults(PROJECT_ID_<%=cid%>, insert_<%=cid%>.getJobReference().getJobId()).execute().getSchema();
routines.system.Dynamic dcg_<%=cid%> = new routines.system.Dynamic();
<%
for(int i=0; i< columnList.size(); i++) {
if(columnList.get(i).getTalendType().equals("id_Dynamic")){
dynamic_index = i+1;
}
}
%>
//
int colsLen_<%=cid %> = schema_<%=cid%>.getFields().size();
for (int i = <%=dynamic_index-1%>; i < colsLen_<%=cid %>-<%=columnList.size()-dynamic_index%>; i++) {
routines.system.DynamicMetadata dynamicMetadata_<%=cid%> = new routines.system.DynamicMetadata();
dynamicMetadata_<%=cid%>.setName(schema_<%=cid%>.getFields().get(i).getName());
dynamicMetadata_<%=cid%>.setDbName(dynamicMetadata_<%=cid%>.getName());
dynamicMetadata_<%=cid%>.setType(routines.system.Dynamic.getTalendTypeFromDBType("bigquery_id", schema_<%=cid%>.getFields().get(i).getType(), 0, 0));
dynamicMetadata_<%=cid%>.setDbType(schema_<%=cid%>.getFields().get(i).getType());
dynamicMetadata_<%=cid%>.setNullable("NULLABLE".equals(schema_<%=cid%>.getFields().get(i).getMode()));
dynamicMetadata_<%=cid%>.setColumnPosition(i);
dcg_<%=cid%>.metadatas.add(dynamicMetadata_<%=cid%>);
}
<%
}
%>
//Dynamic end
<%
} else if (authMode.equals("SERVICEACCOUNT")) {
%>
@@ -310,6 +372,44 @@
log.info("<%=cid%> - Job " + jobId_<%=cid%> + " finished successfully.");
<% } %>
com.google.cloud.bigquery.TableResult result_<%=cid%> = job_<%=cid%>.getQueryResults();
//Dynamic start
<%
if(isDynamic){
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/DB/Input/BigQuery/BigQueryHelper_ServiceAccount.javajet"%>
BigQueryHelper_<%=cid%> helper_<%=cid%> = new BigQueryHelper_<%=cid%>();
com.google.cloud.bigquery.Schema schema_<%=cid%> = result_<%=cid%>.getSchema();
routines.system.Dynamic dcg_<%=cid%> = new routines.system.Dynamic();
<%
for(int i=0; i< columnList.size(); i++) {
if(columnList.get(i).getTalendType().equals("id_Dynamic")){
dynamic_index = i+1;
}
}
%>
//
int colsLen_<%=cid %> = schema_<%=cid%>.getFields().size();
for (int i = <%=dynamic_index-1%>; i < colsLen_<%=cid %>-<%=columnList.size()-dynamic_index%>; i++) {
routines.system.DynamicMetadata dynamicMetadata_<%=cid%> = new routines.system.DynamicMetadata();
dynamicMetadata_<%=cid%>.setName(schema_<%=cid%>.getFields().get(i).getName());
dynamicMetadata_<%=cid%>.setDbName(dynamicMetadata_<%=cid%>.getName());
com.google.cloud.bigquery.StandardSQLTypeName typeName_<%=cid%> = schema_<%=cid%>.getFields().get(i).getType().getStandardType();
String typeNameString_<%=cid%> = (typeName_<%=cid%> != null) ? typeName_<%=cid%>.name() : schema_<%=cid%>.getFields().get(i).getType().name();
dynamicMetadata_<%=cid%>.setType(routines.system.Dynamic.getTalendTypeFromDBType("bigquery_id", typeNameString_<%=cid%>, 0, 0));
dynamicMetadata_<%=cid%>.setDbType(typeNameString_<%=cid%>);
dynamicMetadata_<%=cid%>.setNullable(schema_<%=cid%>.getFields().get(i).getMode() == com.google.cloud.bigquery.Field.Mode.NULLABLE);
dynamicMetadata_<%=cid%>.setColumnPosition(i);
dcg_<%=cid%>.metadatas.add(dynamicMetadata_<%=cid%>);
}
<%
}
%>
//Dynamic end
long nb_line_<%=cid%> = 0;
<%
} else {
@@ -349,21 +449,23 @@
} else {
throw new IllegalArgumentException("authentication mode should be either \"SERVICEACCOUNT\" or \"OAUTH\", but it is " + authMode);
}
%>
int fieldsCount_<%=cid %> = field_<%=cid %>.size();
int column_index_<%=cid%> =0;
<%if(isDynamic){%>
int fixedColumnCount_<%=cid%> = <%=dynamic_index-1%>;
<%}
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
if (conns != null){
if (conns.size()>0){
IConnection conn =conns.get(0);
String connName = conn.getName();
if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)) {
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null) && (metadatas.size() > 0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata != null) {
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
String advancedSeparatorStr = ElementParameterParser.getValue(node, "__ADVANCED_SEPARATOR__");
boolean advancedSeparator = (advancedSeparatorStr!=null&&!("").equals(advancedSeparatorStr))?("true").equals(advancedSeparatorStr):false;
String thousandsSeparator = ElementParameterParser.getValueWithJavaType(node, "__THOUSANDS_SEPARATOR__", JavaTypesManager.CHARACTER);
String decimalSeparator = ElementParameterParser.getValueWithJavaType(node, "__DECIMAL_SEPARATOR__", JavaTypesManager.CHARACTER);
List<IMetadataColumn> columns = metadata.getListColumns();
int nbColumns = columns.size();
@@ -373,14 +475,33 @@
String typeToGenerate = JavaTypesManager.getTypeToGenerate(column.getTalendType(), column.isNullable());
JavaType javaType = JavaTypesManager.getJavaTypeFromId(column.getTalendType());
String defVal = JavaTypesManager.getDefaultValueFromJavaType(typeToGenerate);
if(isDynamic && !("Dynamic").equals(typeToGenerate) && dynamic_index - 1 < i) {
%>
column_index_<%=cid%> = <%=i-1%> + dcg_<%=cid%>.getColumnCount();
<%}else{%>
column_index_<%=cid%> = <%=i%>;
<%
}
String patternValue = column.getPattern() == null || column.getPattern().trim().length() == 0 ? null : column.getPattern();
if(("Dynamic").equals(typeToGenerate)) {
%>
<%=connName%>.<%=columnName%> = dcg_<%=cid%>;
helper_<%=cid%>.readColumnsFromBigQuery(<%=connName%>.<%=columnName%>, field_<%=cid %>, fixedColumnCount_<%=cid%>, true);
<%
} else {
%>
if(fieldsCount_<%=cid %> <= column_index_<%=cid%>) {
<%=connName%>.<%=column.getLabel()%> = <%=defVal%>;
} else {
<%
if (authMode.equals("OAUTH")) {
%>
value_<%=cid%> = field_<%=cid %>.get(<%=i%>).getV();
value_<%=cid%> = field_<%=cid %>.get(column_index_<%=cid%>).getV();
<%
} else if (authMode.equals("SERVICEACCOUNT")) {
%>
value_<%=cid%> = field_<%=cid %>.get(<%=i%>).getValue();
value_<%=cid%> = field_<%=cid %>.get(column_index_<%=cid%>).getValue();
<%
} else {
throw new IllegalArgumentException("authentication mode should be either \"SERVICEACCOUNT\" or \"OAUTH\", but it is " + authMode);
@@ -426,7 +547,9 @@
}else{
<%=connName%>.<%=columnName%> = <%=JavaTypesManager.getDefaultValueFromJavaType(typeToGenerate, column.getDefault())%>;
}
}
<%
}
}
log4jFileUtil.debugRetriveData(node);
}

View File

@@ -29,6 +29,14 @@
<CONNECTOR CTYPE="RUN_IF"/>
</CONNECTORS>
<PARAMETERS>
<PARAMETER
NAME="MAPPING"
FIELD="MAPPING_TYPE"
NUM_ROW="120"
SHOW="false"
>
<DEFAULT>bigquery_id</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="SCHEMA"
FIELD="SCHEMA_TYPE"
@@ -171,14 +179,14 @@
</ADVANCED_PARAMETERS>
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="google-api-client-1.19.0.jar" MODULE="google-api-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-api-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-api-services-oauth2-v2-rev78-1.19.0.jar" MODULE="google-api-services-oauth2-v2-rev78-1.19.0.jar" MVN="mvn:org.talend.libraries/google-api-services-oauth2-v2-rev78-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-services-oauth2-v2-rev78-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-api-services-bigquery-v2-rev391-1.21.0.jar" MODULE="google-api-services-bigquery-v2-rev391-1.21.0.jar" MVN="mvn:org.talend.libraries/google-api-services-bigquery-v2-rev391-1.21.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-services-bigquery-v2-rev391-1.21.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-1.19.0.jar" MODULE="google-http-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-oauth-client-1.19.0.jar" MODULE="google-oauth-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-oauth-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-oauth-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.19.0.jar" MODULE="google-http-client-jackson2-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-jackson2-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-jackson2-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-api-client-1.25.0.jar" MODULE="google-api-client-1.25.0.jar" MVN="mvn:com.google.api-client/google-api-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-api-services-oauth2-v2-rev151-1.25.0.jar" MODULE="google-api-services-oauth2-v2-rev151-1.25.0.jar" MVN="mvn:com.google.apis/google-api-services-oauth2/v2-rev151-1.25.0" REQUIRED="true" />
<IMPORT NAME="google-api-services-bigquery-v2-rev454-1.25.0.jar" MODULE="google-api-services-bigquery-v2-rev454-1.25.0.jar" MVN="mvn:com.google.apis/google-api-services-bigquery/v2-rev454-1.25.0" REQUIRED="true" />
<IMPORT NAME="google-http-client-1.25.0.jar" MODULE="google-http-client-1.25.0.jar" MVN="mvn:com.google.http-client/google-http-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-oauth-client-1.25.0.jar" MODULE="google-oauth-client-1.25.0.jar" MVN="mvn:com.google.oauth-client/google-oauth-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.25.0.jar" MODULE="google-http-client-jackson2-1.25.0.jar" MVN="mvn:com.google.http-client/google-http-client-jackson2/1.25.0" REQUIRED="true" />
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar" MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1" REQUIRED="true" />
<IMPORT NAME="google-cloud-bigquery-1.32.0.jar" MODULE="google-cloud-bigquery-1.32.0.jar" MVN="mvn:com.google.cloud/google-cloud-bigquery/1.32.0" REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'" />
<IMPORT NAME="google-http-client-jackson-1.23.0.jar" MODULE="google-http-client-jackson-1.23.0.jar" MVN="mvn:com.google.http-client/google-http-client-jackson/1.23.0" REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'" />
<IMPORT NAME="threetenbp-1.3.3.jar" MODULE="threetenbp-1.3.3.jar" MVN="mvn:org.threeten/threetenbp/1.3.3" REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'" />

View File

@@ -33,6 +33,14 @@
</CONNECTORS>
<PARAMETERS>
<PARAMETER
NAME="MAPPING"
FIELD="MAPPING_TYPE"
NUM_ROW="120"
SHOW="false"
>
<DEFAULT>bigquery_id</DEFAULT>
</PARAMETER>
<PARAMETER NAME="PROPERTY" FIELD="PROPERTY_TYPE" SHOW="true" NUM_ROW="10" REPOSITORY_VALUE="DELIMITED" />
<PARAMETER NAME="FILENAME" FIELD="FILE" NUM_ROW="11" REQUIRED="true" REPOSITORY_VALUE="FILE_PATH">

View File

@@ -16,6 +16,63 @@ imports="
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata!=null) {
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
if(metadata.isDynamicSchema()) {
%>
class BigQueryOutputHelper_<%=cid%> {
public void writeData(routines.system.Dynamic column, String[] row, int offset) {
for (int i = 0; i < column.getColumnCount(); i++) {
if (column.getColumnValue(i) != null) {
routines.system.DynamicMetadata metadata = column.getColumnMetadata(i);
String dataType = routines.system.MetadataTalendType.getDefaultSelectedDbType("bigquery_id", metadata.getType(), -1, -1);
String orignalDBType = metadata.getDbType();
java.util.List<String> outputDBTypelist = routines.system.MetadataTalendType.getTalendToDBList("bigquery_id", metadata.getType());
if (outputDBTypelist != null && outputDBTypelist.size() > 0) {
for (String dbType : outputDBTypelist) {
if (orignalDBType.equalsIgnoreCase(dbType)) {
dataType = dbType;
break;
}
}
}
if (dataType == null || ("").equals(dataType)) {
dataType = orignalDBType;
}
if ("id_Date".equals(metadata.getType())) {
String pattern = getPattern(dataType);
row[offset + i] = routines.system.DynamicUtils.formatDate(column.getColumnValue(i), pattern);
} else if ("id_byte[]".equals(metadata.getType())) {
row[offset + i] = java.nio.charset.Charset.forName(<%=encoding %>).decode(java.nio.ByteBuffer.wrap((byte[])column.getColumnValue(i))).toString();
} else {
row[offset + i] = String.valueOf(column.getColumnValue(i));
}
}
}
}
public String getPattern(String typeName) {
if(typeName == null) {
return "yyyy-MM-dd hh:mm:ss.SSS";
}
switch(typeName.toUpperCase()) {
case "TIMESTAMP":
return "yyyy-MM-dd hh:mm:ss.SSSXXX";
case "DATE":
return "yyyy-MM-dd";
case "TIME":
return "hh:mm:ss.SSS";
case "DATETIME":
return "yyyy-MM-dd hh:mm:ss.SSS";
default:
return "yyyy-MM-dd hh:mm:ss.SSS";
}
}
}
BigQueryOutputHelper_<%=cid%> bigQueryHelper_<%=cid%> = new BigQueryOutputHelper_<%=cid%>();
<%
}
String filename = ElementParameterParser.getValue(node,"__FILENAME__");
String rowSeparator = ElementParameterParser.getValue(node,"__ROWSEPARATOR__");
@@ -26,8 +83,6 @@ imports="
String textEnclosure = ElementParameterParser.getValue(node, "__TEXT_ENCLOSURE__");
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
boolean checkDiskSpace = ("true").equals(ElementParameterParser.getValue(node,"__CHECK_DISK_SPACE__"));
boolean isAppend = ("true").equals(ElementParameterParser.getValue(node,"__APPEND__"));

View File

@@ -21,7 +21,12 @@
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata!=null) {
boolean hasDynamic = metadata.isDynamicSchema();
if(hasDynamic) {
%>
int dynamic_column_count_<%=cid%> = 1;
<%
}
boolean checkDiskSpace = ("true").equals(ElementParameterParser.getValue(node,"__CHECK_DISK_SPACE__"));
boolean flushOnRow = ("true").equals(ElementParameterParser.getValue(node, "__FLUSHONROW__"));
String flushMod = ElementParameterParser.getValue(node, "__FLUSHONROW_NUM__");
@@ -35,72 +40,95 @@
if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)) {
List<IMetadataColumn> columns = metadata.getListColumns();
int sizeColumns = columns.size();
%>
String[] row_<%=cid%>=new String[]{<%for(int j=0;j<sizeColumns;j++){%>"\\N",<%}%>}; //empty value must be NULL('\N' in bulk file)
IMetadataColumn dynamicCol = metadata.getDynamicColumn();
%>
String[] row_<%=cid%>=new String[<%=sizeColumns%><%=hasDynamic?"+"+conn.getName()+".":"" %><%=dynamicCol==null?"":dynamicCol.getLabel()+".getColumnCount()-1" %>];
java.util.Arrays.fill(row_<%=cid%>, "\\N");
<%
int dynamicColIndex = -1;
%>
int curColumnIndex_<%=cid%> = 0;
<%
for (int i = 0; i < sizeColumns; i++) {
IMetadataColumn column = columns.get(i);
JavaType javaType = JavaTypesManager.getJavaTypeFromId(column.getTalendType());
String pattern = column.getPattern() == null || column.getPattern().trim().length() == 0 ? null : column.getPattern();
if(JavaTypesManager.isJavaPrimitiveType( column.getTalendType(), column.isNullable())){
if(dynamicColIndex != -1 && i > dynamicColIndex) {
%>
curColumnIndex_<%=cid%> = <%=i-1%> + dynamic_column_count_<%=cid%>;
<%
} else {
%>
curColumnIndex_<%=cid%> = <%=i%>;
<%
}
if(("id_Dynamic").equals(column.getTalendType())) {
dynamicColIndex = i;
%>
dynamic_column_count_<%=cid%> = <%=conn.getName()%>.<%=column.getLabel()%>.getColumnCount();
if (<%=conn.getName()%>.<%=column.getLabel()%> != null) {
bigQueryHelper_<%=cid%>.writeData(<%=conn.getName()%>.<%=column.getLabel()%>, row_<%=cid%>, <%=dynamicColIndex%>);
}
<%
} else if(JavaTypesManager.isJavaPrimitiveType( column.getTalendType(), column.isNullable())){
if(javaType == JavaTypesManager.BOOLEAN ){
if(javaType == JavaTypesManager.BOOLEAN ){
%>
row_<%=cid%>[<%=i%>] = String.valueOf(
row_<%=cid%>[curColumnIndex_<%=cid%>] = String.valueOf(
true == <%=conn.getName() %>.<%=column.getLabel() %> ?"1":"0"
);
<%
} else {
} else {
%>
row_<%=cid%>[<%=i%>] = String.valueOf(<%=conn.getName() %>.<%=column.getLabel() %>);
row_<%=cid%>[curColumnIndex_<%=cid%>] = String.valueOf(<%=conn.getName() %>.<%=column.getLabel() %>);
<%
}
}else {
} else {
%>
if(<%=conn.getName() %>.<%=column.getLabel() %> != null){
<%
if(javaType == JavaTypesManager.STRING ){
%>
row_<%=cid%>[<%=i%>] = <%=conn.getName() %>.<%=column.getLabel() %>;
row_<%=cid%>[curColumnIndex_<%=cid%>] = <%=conn.getName() %>.<%=column.getLabel() %>;
<%
}else if(javaType == JavaTypesManager.DATE && pattern == null){
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd");
row_<%=cid%>[curColumnIndex_<%=cid%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd");
<%
}else if(javaType == JavaTypesManager.DATE && pattern != null){
if(pattern.length() > 12){
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd HH:mm:ss");
row_<%=cid%>[curColumnIndex_<%=cid%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd HH:mm:ss");
<%
}else if(pattern.length() == 12 || "\"\"".equals(pattern)) {
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd");
row_<%=cid%>[curColumnIndex_<%=cid%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd");
<%
}else {
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, <%=pattern%>);
row_<%=cid%>[curColumnIndex_<%=cid%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, <%=pattern%>);
<%
}
}else if(javaType == JavaTypesManager.BYTE_ARRAY){
%>
row_<%=cid%>[<%=i%>] = java.nio.charset.Charset.forName(<%=encoding %>).decode(java.nio.ByteBuffer.wrap(<%=conn.getName() %>.<%=column.getLabel() %>)).toString();
row_<%=cid%>[curColumnIndex_<%=cid%>] = java.nio.charset.Charset.forName(<%=encoding %>).decode(java.nio.ByteBuffer.wrap(<%=conn.getName() %>.<%=column.getLabel() %>)).toString();
<%
}else if(javaType == JavaTypesManager.BOOLEAN ){
%>
row_<%=cid%>[<%=i%>] = String.valueOf(
row_<%=cid%>[curColumnIndex_<%=cid%>] = String.valueOf(
true == <%=conn.getName() %>.<%=column.getLabel() %> ?"1":"0"
);
<%
}
else{
%>
row_<%=cid%>[<%=i%>] = String.valueOf(<%=conn.getName() %>.<%=column.getLabel() %>);
row_<%=cid%>[curColumnIndex_<%=cid%>] = String.valueOf(<%=conn.getName() %>.<%=column.getLabel() %>);
<%
}
%>
} else {
row_<%=cid%>[<%=i%>] = "\\N";
row_<%=cid%>[curColumnIndex_<%=cid%>] = "\\N";
}
<%
}

View File

@@ -166,37 +166,17 @@
</ADVANCED_PARAMETERS>
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="google-api-client-1.19.0.jar" MODULE="google-api-client-1.19.0.jar"
MVN="mvn:org.talend.libraries/google-api-client-1.19.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-client-1.19.0.jar"
REQUIRED="true"/>
<IMPORT NAME="google-api-services-oauth2-v2-rev78-1.19.0.jar"
MODULE="google-api-services-oauth2-v2-rev78-1.19.0.jar"
MVN="mvn:org.talend.libraries/google-api-services-oauth2-v2-rev78-1.19.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-services-oauth2-v2-rev78-1.19.0.jar"
REQUIRED="true"/>
<IMPORT NAME="google-api-services-bigquery-v2-rev391-1.21.0.jar"
MODULE="google-api-services-bigquery-v2-rev391-1.21.0.jar"
MVN="mvn:org.talend.libraries/google-api-services-bigquery-v2-rev391-1.21.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-api-services-bigquery-v2-rev391-1.21.0.jar"
REQUIRED="true"/>
<IMPORT NAME="google-http-client-1.19.0.jar" MODULE="google-http-client-1.19.0.jar"
MVN="mvn:org.talend.libraries/google-http-client-1.19.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-1.19.0.jar"
REQUIRED="true"/>
<IMPORT NAME="google-oauth-client-1.19.0.jar" MODULE="google-oauth-client-1.19.0.jar"
MVN="mvn:org.talend.libraries/google-oauth-client-1.19.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-oauth-client-1.19.0.jar"
REQUIRED="true"/>
<IMPORT NAME="google-http-client-jackson2-1.19.0.jar" MODULE="google-http-client-jackson2-1.19.0.jar"
MVN="mvn:org.talend.libraries/google-http-client-jackson2-1.19.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-jackson2-1.19.0.jar"
REQUIRED="true"/>
<IMPORT NAME="google-api-client-1.25.0.jar" MODULE="google-api-client-1.25.0.jar" MVN="mvn:com.google.api-client/google-api-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-api-services-oauth2-v2-rev151-1.25.0.jar" MODULE="google-api-services-oauth2-v2-rev151-1.25.0.jar" MVN="mvn:com.google.apis/google-api-services-oauth2/v2-rev151-1.25.0" REQUIRED="true" />
<IMPORT NAME="google-api-services-bigquery-v2-rev454-1.25.0.jar" MODULE="google-api-services-bigquery-v2-rev454-1.25.0.jar" MVN="mvn:com.google.apis/google-api-services-bigquery/v2-rev454-1.25.0" REQUIRED="true" />
<IMPORT NAME="google-http-client-1.25.0.jar" MODULE="google-http-client-1.25.0.jar" MVN="mvn:com.google.http-client/google-http-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-oauth-client-1.25.0.jar" MODULE="google-oauth-client-1.25.0.jar" MVN="mvn:com.google.oauth-client/google-oauth-client/1.25.0" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.25.0.jar" MODULE="google-http-client-jackson2-1.25.0.jar" MVN="mvn:com.google.http-client/google-http-client-jackson2/1.25.0" REQUIRED="true" />
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar"
MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0"
UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true"/>
<IMPORT NAME="jackson-core-2.9.9.jar" MODULE="jackson-core-2.9.9.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9" REQUIRED="true"/>
<IMPORT NAME="jackson-core-2.10.1.jar" MODULE="jackson-core-2.10.1.jar"
MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1" REQUIRED="true"/>
<IMPORT NAME="google-cloud-bigquery-1.32.0.jar" MODULE="google-cloud-bigquery-1.32.0.jar"
MVN="mvn:com.google.cloud/google-cloud-bigquery/1.32.0"
REQUIRED_IF="AUTH_MODE == 'SERVICEACCOUNT'"/>

View File

@@ -37,8 +37,8 @@
<PARAMETER NAME="DB_VERSION" FIELD="CLOSED_LIST" NUM_ROW="1">
<ITEMS DEFAULT="BONITA_523">
<ITEM NAME="BONITA_523" VALUE="antlr-2.7.6.jar;asm-1.5.3.jar;asm-3.1.jar;asm-attrs-1.5.3.jar;cglib-nodep-2.1_3.jar;commons-collections-2.1.1.jar;commons-logging-1.0.4.jar;dom4j-1.6.1.jar;ehcache-1.2.3.jar;groovy-all-1.7.0.jar;h2-1.2.132.jar;hibernate-3.2.6.ga.jar;jline-0.9.94.jar;jta-1.1.jar;novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar" />
<ITEM NAME="BONITA_531" VALUE="novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar;hibernate-core-3.5.6-Final.jar;commons-collections-3.2.2.jar;ehcache-core-2.2.0.jar;hibernate-commons-annotations-3.2.0.Final.jar;hibernate-search-3.2.1.Final.jar;javassist-3.8.0.GA.jar;lucene-core-2.9.3.jar;slf4j-api-1.6.1.jar;dom4j-1.6.1.jar;h2-1.2.132.jar;jta-1.1.jar;antlr-2.7.6.jar;commons-logging-1.1.1.jar" />
<ITEM NAME="BONITA_523" VALUE="antlr-2.7.6.jar;asm-1.5.3.jar;asm-3.1.jar;asm-attrs-1.5.3.jar;cglib-nodep-2.1_3.jar;commons-collections-2.1.1.jar;commons-logging-1.0.4.jar;dom4j-2.1.1.jar;ehcache-1.2.3.jar;groovy-all-1.7.0.jar;h2-1.2.132.jar;hibernate-3.2.6.ga.jar;jline-0.9.94.jar;jta-1.1.jar;novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar" />
<ITEM NAME="BONITA_531" VALUE="novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar;hibernate-core-3.5.6-Final.jar;commons-collections-3.2.2.jar;ehcache-core-2.2.0.jar;hibernate-commons-annotations-3.2.0.Final.jar;hibernate-search-3.2.1.Final.jar;javassist-3.8.0.GA.jar;lucene-core-2.9.3.jar;slf4j-api-1.6.1.jar;dom4j-2.1.1.jar;h2-1.2.132.jar;jta-1.1.jar;antlr-2.7.6.jar;commons-logging-1.1.1.jar" />
<ITEM NAME="BONITA_561" VALUE="BONITA_561" />
<ITEM NAME="BONITA_5101" VALUE="BONITA_5101" />
<ITEM NAME="BONITA_652" VALUE="BONITA_652" />
@@ -177,7 +177,7 @@
<IMPORT NAME="cglib_nodep_213" MODULE="cglib-nodep-2.1_3.jar" MVN="mvn:org.talend.libraries/cglib-nodep-2.1_3/6.0.0" REQUIRED="false" />
<IMPORT NAME="commons_collections_211" MODULE="commons-collections-2.1.1.jar" MVN="mvn:org.talend.libraries/commons-collections-2.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-collections-2.1.1.jar" REQUIRED="false" />
<IMPORT NAME="commons_logging_104" MODULE="commons-logging-1.0.4.jar" MVN="mvn:org.talend.libraries/commons-logging-1.0.4/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.0.4.v201101211617.jar" REQUIRED="false" />
<IMPORT NAME="dom4j_161" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar" REQUIRED_IF="DB_VERSION=='BONITA_561'" BundleID="" />
<IMPORT NAME="dom4j_2.1.1" MODULE="dom4j-2.1.1.jar" MVN="mvn:org.dom4j/dom4j/2.1.1" REQUIRED_IF="DB_VERSION=='BONITA_561'" BundleID="" />
<IMPORT NAME="ehcache_123" MODULE="ehcache-1.2.3.jar" MVN="mvn:org.talend.libraries/ehcache-1.2.3/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ehcache/lib/ehcache-1.2.3.jar" REQUIRED="false" />
<IMPORT NAME="groovy_all_170" MODULE="groovy-all-1.7.0.jar" MVN="mvn:org.talend.libraries/groovy-all-1.7.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.groovy/lib/groovy-all-1.7.0.jar" REQUIRED="false" />

View File

@@ -50,8 +50,8 @@
<PARAMETER NAME="DB_VERSION" FIELD="CLOSED_LIST" NUM_ROW="2">
<ITEMS DEFAULT="BONITA_523">
<ITEM NAME="BONITA_523" VALUE="antlr-2.7.6.jar;asm-1.5.3.jar;asm-3.1.jar;asm-attrs-1.5.3.jar;cglib-nodep-2.1_3.jar;commons-collections-2.1.1.jar;commons-logging-1.0.4.jar;dom4j-1.6.1.jar;ehcache-1.2.3.jar;groovy-all-1.7.0.jar;h2-1.2.132.jar;hibernate-3.2.6.ga.jar;jline-0.9.94.jar;jta-1.1.jar;novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar" />
<ITEM NAME="BONITA_531" VALUE="novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar;hibernate-core-3.5.6-Final.jar;commons-collections-3.2.2.jar;ehcache-core-2.2.0.jar;hibernate-commons-annotations-3.2.0.Final.jar;hibernate-search-3.2.1.Final.jar;javassist-3.8.0.GA.jar;lucene-core-2.9.3.jar;slf4j-api-1.6.1.jar;dom4j-1.6.1.jar;h2-1.2.132.jar;jta-1.1.jar;antlr-2.7.6.jar;commons-logging-1.1.1.jar" />
<ITEM NAME="BONITA_523" VALUE="antlr-2.7.6.jar;asm-1.5.3.jar;asm-3.1.jar;asm-attrs-1.5.3.jar;cglib-nodep-2.1_3.jar;commons-collections-2.1.1.jar;commons-logging-1.0.4.jar;dom4j-2.1.1.jar;ehcache-1.2.3.jar;groovy-all-1.7.0.jar;h2-1.2.132.jar;hibernate-3.2.6.ga.jar;jline-0.9.94.jar;jta-1.1.jar;novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar" />
<ITEM NAME="BONITA_531" VALUE="novaBpmIdentity-1.0.jar;novaBpmPerf-1.0.jar;novaBpmUtil-1.0.jar;hibernate-core-3.5.6-Final.jar;commons-collections-3.2.2.jar;ehcache-core-2.2.0.jar;hibernate-commons-annotations-3.2.0.Final.jar;hibernate-search-3.2.1.Final.jar;javassist-3.8.0.GA.jar;lucene-core-2.9.3.jar;slf4j-api-1.6.1.jar;dom4j-2.1.1.jar;h2-1.2.132.jar;jta-1.1.jar;antlr-2.7.6.jar;commons-logging-1.1.1.jar" />
<ITEM NAME="BONITA_561" VALUE="BONITA_561" />
<ITEM NAME="BONITA_5101" VALUE="BONITA_5101" />
<ITEM NAME="BONITA_652" VALUE="BONITA_652" />
@@ -226,7 +226,7 @@
<IMPORT NAME="cglib_nodep_213" MODULE="cglib-nodep-2.1_3.jar" MVN="mvn:org.talend.libraries/cglib-nodep-2.1_3/6.0.0" REQUIRED="false" />
<IMPORT NAME="commons_collections_211" MODULE="commons-collections-2.1.1.jar" MVN="mvn:org.talend.libraries/commons-collections-2.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-collections-2.1.1.jar" REQUIRED="false" />
<IMPORT NAME="commons_logging_104" MODULE="commons-logging-1.0.4.jar" MVN="mvn:org.talend.libraries/commons-logging-1.0.4/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.0.4.v201101211617.jar" REQUIRED="false" />
<IMPORT NAME="dom4j_161" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar" REQUIRED_IF="(CLIENT_MODE=='JAVA_CLIENT') AND (DB_VERSION=='BONITA_561')" BundleID="" />
<IMPORT NAME="dom4j_2.1.1" MODULE="dom4j-2.1.1.jar" MVN="mvn:org.dom4j/dom4j/2.1.1" REQUIRED_IF="(CLIENT_MODE=='JAVA_CLIENT') AND (DB_VERSION=='BONITA_561')" BundleID="" />
<IMPORT NAME="ehcache_123" MODULE="ehcache-1.2.3.jar" MVN="mvn:org.talend.libraries/ehcache-1.2.3/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ehcache/lib/ehcache-1.2.3.jar" REQUIRED="false" />
<IMPORT NAME="groovy_all_170" MODULE="groovy-all-1.7.0.jar" MVN="mvn:org.talend.libraries/groovy-all-1.7.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.groovy/lib/groovy-all-1.7.0.jar" REQUIRED="false" />

View File

@@ -69,16 +69,16 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="box-java-sdk"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:org.talend.libraries/boxjavalibv2-v3.0.10/6.0.0"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:net.box/boxjavalibv2/v3.0.10"
REQUIRED="true" />
<IMPORT NAME="jackson-core"
MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-databind"
MODULE="jackson-databind-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
MODULE="jackson-databind-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED="true" />
<IMPORT NAME="jackson-annotations"
MODULE="jackson-annotations-2.9.0.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
MODULE="jackson-annotations-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED="true" />
<IMPORT NAME="apache-http-client"
MODULE="httpclient-4.2.5.jar" MVN="mvn:org.talend.libraries/httpclient-4.2.5/6.0.0"

View File

@@ -124,16 +124,16 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="box-java-sdk"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:org.talend.libraries/boxjavalibv2-v3.0.10/6.0.0"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:net.box/boxjavalibv2/v3.0.10"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-core"
MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-databind"
MODULE="jackson-databind-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
MODULE="jackson-databind-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-annotations"
MODULE="jackson-annotations-2.9.0.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
MODULE="jackson-annotations-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="apache-http-client"
MODULE="httpclient-4.2.5.jar" MVN="mvn:org.talend.libraries/httpclient-4.2.5/6.0.0"

View File

@@ -85,16 +85,16 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="box-java-sdk"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:org.talend.libraries/boxjavalibv2-v3.0.10/6.0.0"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:net.box/boxjavalibv2/v3.0.10"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-core"
MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-databind"
MODULE="jackson-databind-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
MODULE="jackson-databind-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-annotations"
MODULE="jackson-annotations-2.9.0.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
MODULE="jackson-annotations-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="apache-http-client"
MODULE="httpclient-4.2.5.jar" MVN="mvn:org.talend.libraries/httpclient-4.2.5/6.0.0"

View File

@@ -95,16 +95,16 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="box-java-sdk"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:org.talend.libraries/boxjavalibv2-v3.0.10/6.0.0"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:net.box/boxjavalibv2/v3.0.10"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-core"
MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-databind"
MODULE="jackson-databind-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
MODULE="jackson-databind-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-annotations"
MODULE="jackson-annotations-2.9.0.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
MODULE="jackson-annotations-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="apache-http-client"
MODULE="httpclient-4.2.5.jar" MVN="mvn:org.talend.libraries/httpclient-4.2.5/6.0.0"

View File

@@ -103,16 +103,16 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="box-java-sdk"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:org.talend.libraries/boxjavalibv2-v3.0.10/6.0.0"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:net.box/boxjavalibv2/v3.0.10"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-core"
MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-databind"
MODULE="jackson-databind-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
MODULE="jackson-databind-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-annotations"
MODULE="jackson-annotations-2.9.0.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
MODULE="jackson-annotations-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="apache-http-client"
MODULE="httpclient-4.2.5.jar" MVN="mvn:org.talend.libraries/httpclient-4.2.5/6.0.0"

View File

@@ -106,16 +106,16 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="box-java-sdk"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:org.talend.libraries/boxjavalibv2-v3.0.10/6.0.0"
MODULE="boxjavalibv2-v3.0.10.jar" MVN="mvn:net.box/boxjavalibv2/v3.0.10"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-core"
MODULE="jackson-core-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.9"
MODULE="jackson-core-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-databind"
MODULE="jackson-databind-2.9.9.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.9.9"
MODULE="jackson-databind-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-databind/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="jackson-annotations"
MODULE="jackson-annotations-2.9.0.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.9.0"
MODULE="jackson-annotations-2.10.1.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-annotations/2.10.1"
REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
<IMPORT NAME="apache-http-client"
MODULE="httpclient-4.2.5.jar" MVN="mvn:org.talend.libraries/httpclient-4.2.5/6.0.0"

Some files were not shown because too many files have changed in this diff Show More