Compare commits

...

95 Commits

Author SHA1 Message Date
Yves Piel
c4668b3e65 fix(TDI-42689) : Upgrade all to json-lib-2.4.2-talend 2019-08-19 14:52:45 +02:00
Yves Piel
3919231ca8 fix(TDI-42689) : double precision is usually enough 2019-08-19 14:52:45 +02:00
Yves Piel
3bb5b3cb50 fix(TDI-42689) : prefer BigDecimal for float&double 2019-08-19 14:52:45 +02:00
hwang-talend
534feea221 bugfix(TUP-23583):Unable to see all tables in AS400 Connection (#3757)
bugfix(TUP-23583):Unable to see all tables in AS400 Connection
2019-08-19 15:20:13 +08:00
jiezhang-tlnd
f4a45b3659 fix(TUP-23160)Error popup during build father job after modified sun job (#3826)
and do not open father job
https://jira.talendforge.org/browse/TUP-23160
2019-08-19 11:42:35 +08:00
vyu-talend
e3a55d8b63 Vyu/tdi 32477 always lowercase column's name of postgres when create table (#3737)
* Fix(TDI-32477):Always lowercase column's name of postgres when create
table

* Fix(TDI-32477):Always lowercase column's name and table in Postgresql

* Fix(TDI-32477):Enable when the type is Dynamic

* Fix(TDI-32477):Dynamic column includes the part of columns

* Fix(TDI-32477)

* Fix(TDI-32477):fix the two bugs in comment
2019-08-19 10:11:14 +08:00
Emmanuel GALLOIS
80fbfd17a8 feat(TDI-40339): add Snowflake to tSQLTemplate family (#3767)
* feat(TDI-40339): add Snowflake to tSQLTemplate family
* feat(TDI-40339): add Snowflake to Managers
- update db_output_bulk
* feat(TDI-40339): add Snowflake (tcompv0) connection handling in javajets
* feat(TDI-40339): correct style
* feat(TDI-40339): remove duplicated inherited methods
2019-08-16 11:23:42 +02:00
pyzhou
0652420c79 fix(TDI-41375):HTTPS proxy server with user authentication (#3762)
* fix(TDI-41375):HTTPS proxy server with user authentication

* fix(TDI-41368):add a check box

* feat(TDI-41368):add checkbox in advance setting
2019-08-16 14:18:09 +08:00
wang wei
23e645e8e6 fix(TDI-42522): avoid 65535 issue (#3636) 2019-08-16 11:59:52 +08:00
Dmytro Sylaiev
2dcba4cddb chore(TDI-42791): Refactor creation connection ftp (#3796)
* chore(TDI-42791): Refactor creation connection ftp

* chore(TDI-42791): Fix compile error after refactoring
2019-08-15 18:26:12 +03:00
vyu-talend
9e7e4c0971 Fix(TDI-42374):"None Proxy Hosts" rename to "No Proxy for" (#3816) 2019-08-15 16:09:25 +08:00
vyu-talend
d5f863477d Fix(TDI-40237):Fix the omitted components in tRedShiftXXX (#3815)
* Fix(TDI-40237):Fix the omitted components in tRedShiftXXX

* Fix(TDI-40237):add China (Ningxia)
2019-08-14 14:59:22 +08:00
Dmytro Sylaiev
b802bb6a72 fix(TDI-42836): Add milis elaboration to properties labels (#3819) 2019-08-13 16:50:04 +03:00
AlixMetivier
21b8a407de new migration task to change tMapROjaiOutput component dbVersion to corresponding one in 7.1.0+ (#3811) 2019-08-13 14:12:54 +02:00
dmytroshekera
0352e0556c feat(TDI-38420):Add support for AND/OR filter in tServiceNowInput (#3818) 2019-08-13 14:36:37 +03:00
pyzhou
9ce74b5a90 fix(TDI-42725):change the groupID for Exa jar (#3817) 2019-08-13 17:42:37 +08:00
Zhiwei Xue
b9c4102074 fix(TUP-23637):7.x missing local_project for big data spark batch job (#3671) 2019-08-13 11:42:04 +08:00
dmytroshekera
2733478a43 feat(TDI-38420):Add support for AND/OR filter in tServiceNowInput (#3813)
Ivan mention that I should use another type of component,
2019-08-12 17:03:37 +03:00
dmytroshekera
9b2180562c Dshekera/tdi 38420 service now input support and or condition (#3795)
* feat(TDI-38420):Add support for AND/OR filter in tServiceNowInput
2019-08-12 12:27:25 +03:00
Stéphane Bouchet
bec10957e5 TESB-26300: disable message dialog when checking build errors (#3654)
* TESB-23600 : do not open dialog anymore when checking errors

* TESB-23600 : refactor/cleanup
2019-08-09 14:23:28 +02:00
jzhao
d0a5929ad6 fix(TDI-42812):tRedshifBulkExec should hide access key and secret key when datasource is not S3(#3809) 2019-08-09 14:09:45 +08:00
vyu-talend
cc999c45b5 Vyu/tdi 40237 add eu (paris) region to t s3 connection (#3806)
* Fix(TDI-40237):Add new regions to tS3Connection

* Fix(TDI-40237):Add new regions

* Fix(TDI-40237):Further modifications

* Fix(TDI-40237)

* Fix(TDI-40237)

* Fix(TDI-40237)

* Fix(TDI-40237):tS3XXX
2019-08-09 11:16:05 +08:00
zshen-talend
5defbea32b feat(TDQ-17177): support "RAW" output flow for tPatternMasking component (#3790)
* feat(TDQ-17177): support "RAW" output flow for tPatternMasking component

* feat(TDQ-17177): added new output flow "FLOW_OUTPUT" to support

propagate scheam change
2019-08-09 11:06:49 +08:00
qiongli
2583507d00 feat(TDQ-17366)Remove migration for tRuleSurvivorship (#3794) 2019-08-09 10:59:22 +08:00
jiezhang-tlnd
48b0e2ecbd fix(TUP-24159)XML metadata wizard showing inconsistent view on V711 (#3807)
comparing with V6 Studio
https://jira.talendforge.org/browse/TUP-24159
2019-08-09 10:54:31 +08:00
hzhao-talendbj
f1b204b226 fix(TUP-24034)tETLoutput schema getting changed to Built-in from (#3802)
repository when changes are made in the map
2019-08-08 19:59:45 +08:00
hcyi
7f5e8bfa11 fix(TUP-20933):tStatCatcher Statistics property is not shown in Advanced (#3798)
tab of Tacokit components
2019-08-08 19:26:26 +08:00
wang wei
063f1daeec fix(TDI-42737): [java 11] FTPS protocol for FTP components don`t work with linux and java 11 (#3764) 2019-08-08 10:08:09 +08:00
wang wei
d2621d049e fix(TDI-42747): fix the issue for tjdbcinput with dynamic column and inside talend object type like for database type blob(#3783) 2019-08-08 09:12:43 +08:00
vizotenko-talend
364d240804 fix(TDI-42495): unexpected behaviour of tArchive (#3801) 2019-08-07 14:04:36 +03:00
SunChaoqun
7398c13e69 TESB-26174:Error on Job with multiple tRunJob calling Call another Child (#3701)
job with tESBConsumer component
2019-08-07 12:10:31 +08:00
hcyi
ea4ae4c735 fix(TUP-21027):Suggestable widgets not save edited values (if not use (#3793)
custom value).
2019-08-06 10:54:05 +08:00
jiezhang-tlnd
5eb3f438ea fix(TUP-20372)Use Project Settings is not checked as default for job (#3787)
https://jira.talendforge.org/browse/TUP-20372
2019-08-06 10:49:28 +08:00
wang wei
dc1c743d48 fix(TDI-42687): Enable AWS ECS role for S3 component (#3770) 2019-08-06 10:07:18 +08:00
vyu-talend
b885667b38 Fix(TDI-40237):Add new regions to tS3Connection (#3750)
* Fix(TDI-40237):Add new regions to tS3Connection

* Fix(TDI-40237):Add new regions

* Fix(TDI-40237):Further modifications

* Fix(TDI-40237)

* Fix(TDI-40237)

* Fix(TDI-40237)

* Fix(TDI-40237):tS3XXX

* Fix(TDI-40237):Remove new region China (Ningxia)
2019-08-05 15:40:25 +08:00
Chao MENG
4019138048 fix(TUP-20951): Validation parameters need to be removed from (#3786)
serialization
https://jira.talendforge.org/browse/TUP-20951
2019-08-05 11:24:38 +08:00
hcyi
eb00028926 fix(TUP-23513):NPE shows in errorlog view when clicking [...] button (#3785)
next to SQL Query.
2019-08-05 11:02:05 +08:00
hcyi
f439623eab fix(TUP-23515):tacokit dataset won't pop up after clicking OK on "Edit (#3791)
parameters using repository".
2019-08-05 10:50:43 +08:00
Romain Manni-Bucau
8e65ae765f chore(monitoring/TUP-23824): Ensure monitoring feature can rely on system properties to configure the audit logger and does not hardcode a local file name+location+maxsize (#3699)
* Ensure monitoring feature can rely on system properties to configure the audit logger and does not hardcode a local file name+location+maxsize

* support system property monitoring even when no arg is passed to the main
2019-08-02 10:09:35 +02:00
Emmanuel GALLOIS
84edd12af5 fix TalendExcel imports (#3788) 2019-08-01 16:06:49 +02:00
Emmanuel GALLOIS
cb8308df05 feat(TDI-41662): Password protected Excel files management (#3743)
* feat(TDI-41662): add PASSWORD property
* feat(TDI-41662): set PASSWORD property visible only for Excel 2007++
* feat(TDI-41662): add password decryption in Talend ExcelReader wrapper
- warning this is an external lib
* feat(TDI-41662): bump simpleexcel lib release to latest updates
- not sure that MANIFEST.MF should be updated but once build there was
no changes to it, so in case of...
* feat(TDI-41662): implement decryption in javajet
* feat(TDI-41662): add properties to tFileOutputExcel
* fix(TDI-41662): correct gav
* fix(TDI-41662): bump simpleexcel version
* fix(TDI-41662): add password protection on writing excel file
* fix(TDI-41662): bump talendExcel version
* fix(TDI-41662): add password protection on writing excel file (javajet)
* feat(TDI-41662): close workbook when not protected
* feat(TDI-41662): fix some stuff about tFileOutputExcel
* feat(TDI-41662): remove old parse methods to ExcelReader
* feat(TDI-41662): close workbook in any cases
* feat(TDI-41662): use agile encryption mode only in TalendExcel
* feat(TDI-41662): use agile encryption mode only in javajet comp
* feat(TDI-41662): merge master and bump TalendExcel version
* feat(TDI-41662): fix classpath version for TalendExcel
- not sure that's necessary but...
2019-08-01 11:04:13 +02:00
hzhao-talendbj
62fe9a0f38 fix(TUP-22893)Icon becomes old version after openning SQL builder (#3775)
* fix(TUP-22893)Icon becomes old version after openning SQL builder

* fix(TUP-22893)Icon becomes old version after openning SQL builder
2019-08-01 16:29:24 +08:00
Chao MENG
ac80911050 fix(TUP-23514): NPE shows in errorlog view when Selecting Repository in (#3772)
Basic settings
https://jira.talendforge.org/browse/TUP-23514
2019-08-01 16:17:58 +08:00
hcyi
7c682778ec fix(TUP-23626):tacokit problem with drag and drop variables from (#3773)
Outline.
2019-08-01 16:15:10 +08:00
SunChaoqun
4159e31ef1 TESB-24597:Docker Support for Microservice - CI Part (#3782) 2019-07-31 18:38:00 +08:00
Chao MENG
b7ed047c91 fix(TUP-20943): Code generates with an error when Pattern check fails if (#3774)
contain back slash
https://jira.talendforge.org/browse/TUP-20943
2019-07-31 17:39:33 +08:00
hwang-talend
be5a9a7dba bugfix(TUP-23894):Fix all failed junit for both java 8 and 11 (#3779)
bugfix(TUP-23894):Fix all failed junit for both java 8 and 11
2019-07-31 16:59:27 +08:00
apoltavtsev
0f0ead31aa TESB-24900 Undeploying a Data Service causes "Exception in opening zip file" error 2019-07-31 09:33:09 +03:00
SunChaoqun
70f45909b9 TESB-24597:Docker Support for Microservice - CI Part (#3776) 2019-07-31 11:09:37 +08:00
hwang-talend
3e0d0538bb bugfix(TUP-23967):Studio can't run jobs when path has spaces, Windows 10 (#3768)
bugfix(TUP-23967):Studio can't run jobs when path has spaces, Windows 10
2019-07-31 10:08:14 +08:00
Chao MENG
40d9315e48 Cmeng/bugfix/tup 23914 json wizard encoding (#3771)
* fix(TUP-23914): JSON with accented characters is not being recognized
when creating a new JSON file in Metadata.
https://jira.talendforge.org/browse/TUP-23914
2019-07-31 10:05:19 +08:00
wang wei
0946243005 fix(TDI-42705): tSAPBwInput component gives compilation error (#3739) 2019-07-31 09:52:27 +08:00
Roman
d544cfbbd8 feat(TDI-36668): Add possibility to write xlsm files (#3766) 2019-07-29 16:58:22 +03:00
hwang-talend
38d73d1c3b bugfix(TUP-23894):Fix all failed junit for both java 8 and 11 (#3761)
* bugfix(TUP-23894):Fix all failed junit for both java 8 and 11
2019-07-29 18:05:07 +08:00
Dmytro Sylaiev
a257812d17 Dsylaiev/tdi 37257 timeout sftp (#3756)
* feat(TDI-37257): Add connection timeout for SFTP

* feat(TDI-37257): Implement timeout feature for FTP and FTPS
2019-07-29 12:56:03 +03:00
Jane Ding
4224c49747 fix(TUP-22943)checkbox with empty item name display on window Save (#3755)
Resources
https://jira.talendforge.org/browse/TUP-22943
2019-07-29 15:12:01 +08:00
Jane Ding
2a48cfd3c5 fix(TUP-22880)Resources: Job cannot use resources from reference (#3759)
project;
https://jira.talendforge.org/browse/TUP-22880
2019-07-29 15:07:34 +08:00
hcyi
c4adf887db fix(TUP-21687):Table value is not visible after filling component properties from metadata. (#3747)
* fix(TUP-21687):Table value is not visible after filling component
properties from metadata.

* fix(TUP-21687):add/update junits for Table value is not visible after
filling component properties from metadata.

* fix(TUP-21687):add/update junits for Table value is not visible after
filling component properties from metadata.
2019-07-29 15:02:34 +08:00
hcyi
18422df2a4 fix(TUP-20928):Suggestable doesn't work for TextArea in Studio (#3765) 2019-07-29 14:56:00 +08:00
SunChaoqun
0985646d21 TESB-24597:Docker Support for Microservice - CI Part (#3730)
* TESB-24597:Docker Support for Microservice - CI Part

* TESB-24597:Docker Support for Microservice - CI Part

* TESB-24597:Docker Support for Microservice - CI Part
2019-07-26 10:39:40 +08:00
slushatel
481edbbc18 fix(TDI-42634): create output directory if not exists (#3760) 2019-07-25 16:15:55 +03:00
Hanna Liashchuk
02df23ffeb (feature(TBD-8945): added drop table checkbox to OAuth authentication mode (#3713) 2019-07-25 13:45:47 +03:00
Jane Ding
957704c0a4 fix(TUP-23789)Pom files are not generated sometimes for the jobs in (#3738)
* fix(TUP-23789)Pom files are not generated sometimes for the jobs in
7.1.1
https://jira.talendforge.org/browse/TUP-23789

* fix(TUP-23789)Pom files are not generated sometimes for the jobs in
7.1.1
https://jira.talendforge.org/browse/TUP-23789
Add a junit.
2019-07-24 16:18:59 +08:00
Roman
cb85a154f8 fix(TDI-42457): correct Postgre supported DB version (#3733) 2019-07-24 10:11:57 +03:00
hcyi
c7c9d4bf2d fix(TUP-23898):Create new plugin for junit tests of tacokit studio-integration. (#3740)
* fix(TUP-23898):Create new plugin for junit tests of tacokit
studio-integration.

* fix(TUP-23898):Create new plugin for junit tests of tacokit
studio-integration.

* fix(TUP-23898):Create new plugin for junit tests of tacokit
studio-integration.

* fix(TUP-23898):Create new plugin for junit tests of tacokit
studio-integration.
2019-07-23 18:29:29 +08:00
hcyi
4271923ca3 fix(TUP-23264):Studio can't find dependencies for tacokit components. (#3720)
* fix(TUP-23264):Studio can't find dependencies for tacokit components.

* fix(TUP-23264):add junits for Studio can't find dependencies for tacokit
components.
2019-07-23 12:03:00 +08:00
jiezhang-tlnd
70f0e7a076 fix(TUP-23588)For tCreateTable, the Database Type is always Mysql after dragging and dropping from database Metadata (#3714)
* fix(TUP-23588)For tCreateTable, the Database Type is always Mysql after
dragging and dropping from database Metadata
https://jira.talendforge.org/browse/TUP-23588

* junit(TUP-23588)For tCreateTable, the Database Type is always Mysql
after dragging and dropping from database Metadata

* fix(TUP-23588)For tCreateTable, the Database Type is always Mysql after
dragging and dropping from database Metadata

* fix(TUP-23588)For tCreateTable, the Database Type is always Mysql after
dragging and dropping from database Metadata

* fix(TUP-23588)For tCreateTable, the Database Type is always Mysql after
dragging and dropping from database Metadata

* fix(TUP-23588)For tCreateTable, the Database Type is always Mysql after
dragging and dropping from database Metadata
2019-07-23 11:20:25 +08:00
pyzhou
6e0af4dd99 fix(TDI-42674):Redshift missing jar for SSO (#3731)
* fix(TDI-42674):Redshift missing jar for SSO

* fix(TDI-42674):change mvn path

* fix(TDI-42674):change require to require_if
2019-07-22 17:23:21 +08:00
clesaec
26fbb4e014 TDI-42679 : use new File(rep, file) constructor (#3724) 2019-07-19 13:25:34 +02:00
dmytroshekera
28b472384b bugfix(TDI-42364): When building a Standalone job. (#3734)
Generated Powershell scripts, is wrong.
2019-07-19 12:51:04 +03:00
Zhiwei Xue
c0f93ad66b Revert fix of TUP-23729 (#3732)
* Revert "fix(TUP-23729):Memory issue found when run TUJ (#3705)"

This reverts commit 1d706bc243.

* Revert "fix(TUP-23729):Memory issue found when run TUJ (#3689)"

This reverts commit 5069b086d7.
2019-07-18 18:25:34 +08:00
slushatel
7a53a97ceb fix(TDI-42634): get attachments recursively (#3700)
* fix(TDI-42634): get attachments recursively

* fix(TDI-42634): change indents to 'space'
2019-07-18 10:55:05 +03:00
pyzhou
1c89042c03 fix(TDI-42572):upgrade EXAUtil version (#3719)
* fix(TDI-42572):upgrade EXAUtil version

* Change the mvn path
2019-07-18 10:01:06 +08:00
dmytroshekera
9133ff39d1 bugfix(TDI-42581): tSOAP doesn't support SOAPActions for Soap 1.2 (#3727)
param 'action' should not be empty
2019-07-17 14:40:51 +03:00
Zhiwei Xue
1d706bc243 fix(TUP-23729):Memory issue found when run TUJ (#3705) 2019-07-17 17:01:39 +08:00
dmytroshekera
24031c6bb0 Dshekera/tdi 42581 t soap doesnt support soap actions for soap 1 2 (#3725)
* bugfix(TDI-42581): tSOAP doesn't support SOAPActions for Soap 1.2
2019-07-16 16:44:14 +03:00
Chao MENG
5d7deb4f08 fix(TUP-23508): TOS can't use memory run (#3698)
https://jira.talendforge.org/browse/TUP-23508
2019-07-16 17:44:26 +08:00
kjwang-talend
2f323ea5ec TUP-23841 Build job with 'OSGI Bundle For ESB' get error pop-up (#3711)
https://jira.talendforge.org/browse/TUP-23841
2019-07-16 10:30:48 +08:00
kjwang-talend
0099cf4b7f Fix TUP-23809 [Studio] changes in column order are not saved (no (#3708)
propagate changes dialog)
https://jira.talendforge.org/browse/TUP-23809
2019-07-16 10:04:02 +08:00
vyu-talend
ad8b244f13 Fix(TDI-42572):Improvement for generating correct format to Exasol (#3677)
* Fix(TDI-42572):Improvement for generating correct format to Exasol

* Fix(TDI-42572):Using checkbox
2019-07-16 09:37:04 +08:00
slushatel
b4196f3ade feat(TDI-42570): stop process if parent thread has terminated, add log wrapper for tos studio (#3715) 2019-07-15 14:52:28 +03:00
Hanna Liashchuk
a6cc1fd725 fix(TBD-8951): skeap header for Service Account (#3716) 2019-07-15 14:18:45 +03:00
Hanna Liashchuk
6d2c08fb02 Revert "(feature(TBD-8945): added drop table checkbox to OAuth authentication mode (#3706)" (#3712)
This reverts commit b37f4d2169.
2019-07-15 13:04:28 +03:00
Hanna Liashchuk
b37f4d2169 (feature(TBD-8945): added drop table checkbox to OAuth authentication mode (#3706) 2019-07-15 12:55:50 +03:00
zshen-talend
18a6bd6b9e chor(TDQ-17069): remove unused code (#3617) 2019-07-15 15:53:33 +08:00
hcyi
12025462fd fix(TUP-23743):ERROR when Job in MDM project calling Job in Reference DI project:import routines.DataQuality cannot be resolved. (#3688)
* fix(TUP-23743):ERROR when Job in MDM project calling Job in Reference DI
project:import routines.DataQuality cannot be resolved.

* fix(TUP-23743):ERROR when Job in MDM project calling Job in Reference DI
project:import routines.DataQuality cannot be resolved.
2019-07-15 11:39:36 +08:00
hcyi
8a17273fff fix(TUP-23550):Select query for DBInput component page will list Couchbase. (#3704)
* fix(TUP-23550):Select query for DBInput component page will list
Couchbase.

* fix(TUP-23550):add junits
2019-07-15 10:33:04 +08:00
zshen-talend
4666ad0ba2 feat(TDQ-16777 TDQ-17239): added one migration for added encoding type (#3690)
of combo list.
2019-07-12 16:07:34 +08:00
Hanna Liashchuk
e99e3a88ac task(TBD-8950): Change redundant code appearance in tBigQueryBulkExec (#3703)
* (TBD-8950): code refactoring

* fix(TBD-8950): added postfix
2019-07-11 16:47:06 +03:00
Roman
eef7ee7939 fix(TDI-42436): copy Redshift components to cloud folder (#3702)
* fix(TDI-42436): copy Redshift components to cloud folder
2019-07-11 11:46:36 +03:00
wang wei
323b7fdf86 fix(TDI-42423): fix the tcreatetable for teradata with jdbc parameters(#3680) 2019-07-10 09:50:06 +08:00
slushatel
9e8d8785c2 Tdi 42570 stop process when thread stops (#3676)
* feat(TDI-42570): stop process if parent thread has terminated

* feat(TDI-42570): stop process if parent thread has terminated

* feat(TDI-42570): stop process if parent thread has terminated. improve indentation

* feat(TDI-42570): stop process if parent thread has terminated. improve indentation
2019-07-09 14:54:26 +03:00
hcyi
463d35acb2 fix(TUP-23790):Can't guess schema for metadata (#3691)
item(delimited/positional/regex/excel/ldif/ldap)
2019-07-09 18:21:16 +08:00
Zhiwei Xue
5069b086d7 fix(TUP-23729):Memory issue found when run TUJ (#3689) 2019-07-09 18:11:06 +08:00
Jane Ding
18694afab5 fix(TUP-23663)The connection row between components is getting activated (#3687)
* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663
2019-07-09 17:49:22 +08:00
257 changed files with 6572 additions and 3392 deletions

View File

@@ -55,5 +55,6 @@
<plugin id="org.talend.testutils" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
<plugin id="org.talend.updates.runtime.test" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="org.talend.utils.test" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="org.talend.sdk.component.studio-integration.test" download-size="0" install-size="0" version="0.0.0" fragment="true" unpack="false"/>
<plugin id="test.all.test.suite" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
</feature>

View File

@@ -59,88 +59,76 @@ if((codePart.equals(ECodePart.END))&&(stat || logstashCurrent)){
List<String> needToEndConnNames = new ArrayList<String>();
INode nextNode = node.getOutgoingConnections(EConnectionType.ITERATE).get(0).getTarget();
NodeUtil.fillConnectionsForStat(needToEndConnNames, nextNode);
%>
<%if(stat) {%>
if(execStat){
<%
for(String connName : needToEndConnNames){
if(!needToEndConnNames.isEmpty()) {
if(stat && logstashCurrent) {
%>
runStat.updateStatOnConnection("<%=connName%>"+iterateId,2, 0);
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,2,0<%for(String connName : needToEndConnNames){%>,"<%=connName%>"<%}%>);
<%
} else {
if(stat) {%>
if(execStat){
runStat.updateStatOnConnection(iterateId,2,0<%for(String connName : needToEndConnNames){%>,"<%=connName%>"<%}%>);
}
<%}%>
<%if(logstashCurrent) {//now only finish the log, not send, TODO%>
if(enableLogStash){
runStat.log(iterateId,2,0<%for(String connName : needToEndConnNames){%>,"<%=connName%>"<%}%>);
}
<%
}
}
%>
}
<%}%>
<%if(logstashCurrent) {//now only finish the log, not send, TODO%>
if(enableLogStash){
<%
for(String connName : needToEndConnNames){
%>
runStat.logStatOnConnection("<%=connName%>"+iterateId,2, 0);
<%
}
%>
}
<%
}
%>
<%
}
if(connSet.size()>0){
for(IConnection con:connSet){
INode source = con.getSource();
String sourceNodeId = source.getUniqueName();
String sourceNodeComponent = source.getComponent().getName();
%>
<%if(stat) {%>
if(execStat){
if(resourceMap.get("inIterateVComp") == null || !((Boolean)resourceMap.get("inIterateVComp"))){
runStat.updateStatOnConnection("<%=con.getUniqueName()%>"+iterateId,2, 0);
}
if(connSet.size()>0) {
if(stat && logstashCurrent && (connSet.size()==1)) {//the most common case, write this ugly logic for 65535 issue
for(IConnection con:connSet){
INode source = con.getSource();
String sourceNodeId = source.getUniqueName();
String sourceNodeComponent = source.getComponent().getName();
for (INode jobStructureCatcher : jobCatcherNodes) {
%>
if(runStat.updateStatAndLog(execStat,enableLogStash,resourceMap,iterateId,"<%=con.getUniqueName()%>",2,0,
<%=jobStructureCatcher.getUniqueName()%>,"<%=sourceNodeId%>","<%=sourceNodeComponent%>","<%=node.getUniqueName()%>","<%=node.getComponent().getName()%>","<%="REJECT".equals(con.getConnectorName()) ? "reject" : "output"%>")) {
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
}
<%
break;
}
}
} else {
if(stat){
%>
if(execStat){
runStat.updateStat(resourceMap,iterateId,2,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash){
if(resourceMap.get("inIterateVComp") == null || !((Boolean)resourceMap.get("inIterateVComp"))){
<%
if(logstashCurrent){
%>
if(enableLogStash) {
<%
for(IConnection con:connSet){
INode source = con.getSource();
String sourceNodeId = source.getUniqueName();
String sourceNodeComponent = source.getComponent().getName();
for (INode jobStructureCatcher : jobCatcherNodes) {
%>
RunStat.StatBean talend_statebean = runStat.logStatOnConnection("<%=con.getUniqueName()%>"+iterateId,2, 0);
<%=jobStructureCatcher.getUniqueName() %>.addConnectionMessage(
"<%=sourceNodeId%>",
"<%=sourceNodeComponent%>",
false,
"<%="REJECT".equals(con.getConnectorName()) ? "reject" : "output"%>",
"<%=con.getUniqueName()%>",
talend_statebean.getNbLine(),
talend_statebean.getStartTime(),
talend_statebean.getEndTime()
);
<%=jobStructureCatcher.getUniqueName() %>.addConnectionMessage(
"<%=node.getUniqueName()%>",
"<%=node.getComponent().getName()%>",
true,
"input",
"<%=con.getUniqueName()%>",
talend_statebean.getNbLine(),
talend_statebean.getStartTime(),
talend_statebean.getEndTime()
);
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
if(runStat.log(resourceMap,iterateId,"<%=con.getUniqueName()%>",2,0,
<%=jobStructureCatcher.getUniqueName()%>,"<%=sourceNodeId%>","<%=sourceNodeComponent%>","<%=node.getUniqueName()%>","<%=node.getComponent().getName()%>","<%="REJECT".equals(con.getConnectorName()) ? "reject" : "output"%>")) {
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
}
<%
break;
}
%>
}
}
%>
}
<%
}
<%}%>
<%
}
}
}
}
%>

View File

@@ -146,83 +146,82 @@
boolean logstashCurrent = !cid.startsWith("tJobStructureCatcher") && !cid.startsWith("talend") && enableLogStash;
if ((codePart.equals(ECodePart.BEGIN))&&(stat || logstashCurrent)&&connSet.size()>0) {
for(IConnection con:connSet){
%>
if (<%if(stat){%>execStat<%}%><%if(stat && logstashCurrent){%> || <%}%><%if(logstashCurrent){%>enableLogStash<%}%>) {
if(resourceMap.get("inIterateVComp") == null){
<%if(containsTPartitioner){%>
java.util.concurrent.ConcurrentHashMap<Object, Object> concurrentHashMap_<%=con.getUniqueName() %> = (java.util.concurrent.ConcurrentHashMap) globalMap.get("concurrentHashMap");
concurrentHashMap_<%=con.getUniqueName() %>.putIfAbsent("<%=con.getUniqueName() %>" + iterateLoop,new java.util.concurrent.atomic.AtomicInteger(0));
java.util.concurrent.atomic.AtomicInteger stats_<%=con.getUniqueName() %> = (java.util.concurrent.atomic.AtomicInteger) concurrentHashMap_<%=con.getUniqueName() %>.get("<%=con.getUniqueName() %>" + iterateLoop);
int step_<%=con.getUniqueName() %> = stats_<%=con.getUniqueName() %>.incrementAndGet()<=1?0:1;
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId, step_<%=con.getUniqueName()%>, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName()%>"+iterateId, step_<%=con.getUniqueName()%>, 0);
}
<%}%>
<%}else{%>
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId, 0, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName()%>"+iterateId, 0, 0);
}
<%}%>
<%}%>
if(containsTPartitioner) {
%>
if(<%if(stat){%>execStat<%}%><%if(stat && logstashCurrent){%> || <%}%><%if(logstashCurrent){%>enableLogStash<%}%>) {
runStat.updateStatOnConnectionAndLog(resourceMap,globalMap,iterateLoop,iterateId,<%if(stat){%>execStat<%} else {%>false<%}%>,enableLogStash,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
} else {
if(stat && logstashCurrent) {
%>
runStat.updateStatAndLog(execStat,enableLogStash,resourceMap,iterateId,0,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
<%
} else {
if(stat) {
%>
if(execStat) {
runStat.updateStatOnConnection(resourceMap,iterateId,0,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
}
<%
if(logstashCurrent) {
%>
if(enableLogStash) {
runStat.log(resourceMap,iterateId,0,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
}
}
}
if((codePart.equals(ECodePart.MAIN))&&(stat || logstashCurrent)&&connSet.size()>0){
for(IConnection con:connSet){
%>
//<%=con.getUniqueName()%>
//<%=(String)codeGenArgument.getIncomingName()%>
<%if (!node.getComponent().useMerge()) {%>
<%if(stat) {%>
if(execStat){
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
}
<%}%>
<%
} else if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){
if(!node.getComponent().useMerge()) {
if(stat && logstashCurrent) {
%>
<%if(stat) {%>
if(execStat){
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,1,1<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
<%
} else {
if(stat) {
%>
if(execStat){
runStat.updateStatOnConnection(iterateId,1,1<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
if(logstashCurrent) {
%>
if(enableLogStash) {
runStat.log(iterateId,1,1<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
<%}%>
<%}%>
<%
}
} else {
for(IConnection connection:connSet){
if(connection.getUniqueName().equals((String)codeGenArgument.getIncomingName())){
if(stat && logstashCurrent) {
%>
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,1,1<%for(IConnection con : connSet){if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){%>,"<%=con.getUniqueName()%>"<%}}%>);
<%
} else {
if(stat) {%>
if(execStat){
runStat.updateStatOnConnection(iterateId,1,1<%for(IConnection con : connSet){if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){%>,"<%=con.getUniqueName()%>"<%}}%>);
}
<%}
if(logstashCurrent) {%>
if(enableLogStash) {
runStat.log(iterateId,1,1<%for(IConnection con : connSet){if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){%>,"<%=con.getUniqueName()%>"<%}}%>);
}
<%}
}
}
}
}
}
@@ -251,7 +250,7 @@
for (INode jobStructureCatcher : jobCatcherNodes) {
%>
if(enableLogStash) {
<%=jobStructureCatcher.getUniqueName() %>.addComponentMessage("<%=node.getUniqueName()%>", "<%=node.getComponent().getName()%>");
<%=jobStructureCatcher.getUniqueName() %>.addCM("<%=node.getUniqueName()%>", "<%=node.getComponent().getName()%>");
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
}
<%
@@ -266,46 +265,37 @@
List<String> needToStartConnNames = new ArrayList<String>();
INode nextNode = node.getOutgoingConnections(EConnectionType.ITERATE).get(0).getTarget();
NodeUtil.fillConnectionsForStat(needToStartConnNames, nextNode);
if(needToStartConnNames.isEmpty()) {
//do nothing
} else if(containsTPartitioner){
%>
if(<%if(stat){%>execStat<%}%><%if(stat && logstashCurrent){%> || <%}%><%if(logstashCurrent){%>enableLogStash<%}%>){
<%
for(String connName : needToStartConnNames){
%>
<%if(containsTPartitioner){%>
java.util.concurrent.ConcurrentHashMap<Object, Object> concurrentHashMap_<%=connName%> = (java.util.concurrent.ConcurrentHashMap) globalMap.get("concurrentHashMap");
concurrentHashMap_<%=connName%>.putIfAbsent("<%=connName%>" + iterateLoop,new java.util.concurrent.atomic.AtomicInteger(0));
java.util.concurrent.atomic.AtomicInteger stats_<%=connName%> = (java.util.concurrent.atomic.AtomicInteger) concurrentHashMap_<%=connName%>.get("<%=connName%>" + iterateLoop);
int step_<%=connName %> = stats_<%=connName%>.incrementAndGet()<=1?0:1;
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=connName%>"+iterateId, step_<%=connName%>, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=connName%>"+iterateId, step_<%=connName%>, 0);
}
<%}%>
<%}else{%>
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=connName%>"+iterateId, 0, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=connName%>"+iterateId, 0, 0);
}
<%}%>
<%}%>
<%
}
%>
runStat.updateStatOnConnectionAndLog(globalMap,iterateLoop,iterateId,<%if(stat){%>execStat<%} else {%>false<%}%>,enableLogStash,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
}
<%
} else {
if(stat && logstashCurrent) {
%>
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,0,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
<%
} else {
if(stat){
%>
if(execStat){
runStat.updateStatOnConnection(iterateId,0,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
}
<%
}
%>
<%if(logstashCurrent) {%>
if(enableLogStash){
runStat.log(iterateId,0,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
}
<%
}
}
}
<%
}else if(codePart.equals(ECodePart.MAIN)){
%>
resourceMap.put("inIterateVComp", true);

View File

@@ -445,6 +445,7 @@
lastStr = "";
}
}
enableLogStash = "true".equalsIgnoreCase(System.getProperty("monitoring"));
<%if(isLog4jEnabled){%>
if(!"".equals(log4jLevel)){
@@ -493,7 +494,11 @@
properties_<%=jobCatcherNode.getUniqueName()%>.setProperty("appender.file.maxsize", "52428800");
properties_<%=jobCatcherNode.getUniqueName()%>.setProperty("appender.file.maxbackup", "20");
properties_<%=jobCatcherNode.getUniqueName()%>.setProperty("host", "false");
System.getProperties().stringPropertyNames().stream()
.filter(it -> it.startsWith("monitoring.audit.logger.properties."))
.forEach(key -> properties_<%=jobCatcherNode.getUniqueName()%>.setProperty(key.substring("monitoring.audit.logger.properties.".length()), System.getProperty(key)));
auditLogger_<%=jobCatcherNode.getUniqueName()%> = org.talend.job.audit.JobEventAuditLoggerFactory.createJobAuditLogger(properties_<%=jobCatcherNode.getUniqueName()%>);
}
<%
@@ -1509,12 +1514,10 @@ if (execStat) {
}
} else if (arg.startsWith("--log4jLevel=")) {
log4jLevel = arg.substring(13);
} else if (arg.startsWith("--monitoring=")) {//for trunjob call
enableLogStash = "true".equalsIgnoreCase(arg.substring(13));
}
if(!enableLogStash) {
enableLogStash = "true".equalsIgnoreCase(System.getProperty("monitoring"));
} else if (arg.startsWith("--monitoring") && arg.contains("=")) {//for trunjob call
final int equal = arg.indexOf('=');
final String key = arg.substring("--".length(), equal);
System.setProperty(key, arg.substring(equal + 1));
}
}

View File

@@ -139,7 +139,7 @@ for (INode node : process.getNodesOfType("tRESTClient")) {
}
boolean talendEsbJobFactory = actAsProvider || !process.getNodesOfType("tRouteInput").isEmpty();
boolean talendEsbJob = talendEsbJobFactory || actAsConsumer || ProcessorUtilities.isEsbJob(process.getId(), process.getVersion());
boolean talendEsbJob = talendEsbJobFactory || actAsConsumer || ProcessorUtilities.isEsbJob(process);
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(process, "__LOG4J_ACTIVATE__"));

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>commons-net-ftps-proxy</artifactId>
<version>3.6.1-talend-20190128</version>
<version>3.6.1-talend-20190726</version>
<name>commons-net-talend</name>

View File

@@ -29,11 +29,8 @@ public class SSLSessionReuseFTPSClient extends FTPSClient {
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
final Method getHostMethod = socket.getClass().getDeclaredMethod("getHost");
getHostMethod.setAccessible(true);
Object host = getHostMethod.invoke(socket);
final String key =
String.format("%s:%s", host, String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT);
String.format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT);
putMethod.invoke(cache, key, session);
} catch (Exception e) {
e.printStackTrace();

View File

@@ -7,7 +7,7 @@
<groupId>net.sf.json-lib</groupId>
<artifactId>json-lib</artifactId>
<packaging>jar</packaging>
<version>2.4.1-talend</version>
<version>2.4.2-talend</version>
<name>json-lib</name>
<properties>

View File

@@ -15,6 +15,7 @@
*/
package net.sf.json.util;
import java.math.BigDecimal;
import net.sf.json.JSONArray;
import net.sf.json.JSONException;
import net.sf.json.JSONNull;
@@ -414,7 +415,7 @@ public class JSONTokener {
}
try{
return NumberUtils.createNumber(s);
return createNumber(s);
}catch( Exception e ){
return s;
}
@@ -435,6 +436,26 @@ public class JSONTokener {
return s;
}
/**
* This method has been added to fix https://jira.talendforge.org/browse/TDI-42689
*
* @param s The String representation of the number
* @return The Number instance
*/
private Number createNumber(String s){
boolean isDecimal = s.indexOf('.') != -1;
if(isDecimal){
Double d = Double.valueOf(s);
if(Double.POSITIVE_INFINITY == Math.abs(d)){
return new BigDecimal(s);
}
return d;
}
return NumberUtils.createNumber(s);
}
/**
* Look at the next character in the source string.
*

View File

@@ -184,14 +184,12 @@ public final class JSONUtils {
return Integer.class;
}else if( isLong( n ) ){
return Long.class;
}else if( isFloat( n ) ){
return Float.class;
}else if( isBigInteger( n ) ){
return BigInteger.class;
}else if( isBigDecimal( n ) ){
return BigDecimal.class;
}else if( isDouble( n ) ){
return Double.class;
}else if( isBigDecimal( n ) ){
return BigDecimal.class;
}else{
throw new JSONException( "Unsupported type" );
}

View File

@@ -27,6 +27,7 @@ import net.sf.json.processors.PropertyNameProcessor;
import net.sf.json.sample.BeanA;
import net.sf.json.sample.BeanB;
import net.sf.json.sample.BeanC;
import net.sf.json.sample.BeanD;
import net.sf.json.sample.BeanFoo;
import net.sf.json.sample.BeanWithFunc;
import net.sf.json.sample.ChildBean;
@@ -1042,6 +1043,17 @@ public class TestJSONObject extends TestCase {
JSONArray.toArray( jsonObject.getJSONArray( "intarray" ) ) );
}
public void testToBean_BeanD() {
String json = "{bool:true,integer:1,string:\"json\",doublearray:[4.2424245783E7, 123456789.2424245783E7, 6.0]}";
JSONObject jsonObject = JSONObject.fromObject( json );
BeanD bean = (BeanD) JSONObject.toBean( jsonObject, BeanD.class );
assertEquals( jsonObject.get( "bool" ), Boolean.valueOf( bean.isBool() ) );
assertEquals( jsonObject.get( "integer" ), new Integer( bean.getInteger() ) );
assertEquals( jsonObject.get( "string" ), bean.getString() );
Assertions.assertEquals( bean.getDoublearray(),
JSONArray.toArray( jsonObject.getJSONArray( "doublearray" ) ) );
}
public void testToBean_ClassBean() {
JSONObject json = new JSONObject();
json.element( "klass", "java.lang.Object" );
@@ -1050,9 +1062,29 @@ public class TestJSONObject extends TestCase {
assertEquals( Object.class, bean.getKlass() );
}
public void testToBean_DynaBean__BigInteger_BigDecimal() {
public void testToBean_DynaBean__BigInteger_Double() {
BigInteger l = new BigDecimal( "1.7976931348623157E308" ).toBigInteger();
BigDecimal m = new BigDecimal( "1.7976931348623157E307" ).add( new BigDecimal( "0.0001" ) );
JSONObject json = new JSONObject().element( "i", BigInteger.ZERO )
.element( "d", MorphUtils.BIGDECIMAL_ONE )
.element( "bi", l )
.element( "bd", m );
Object bean = JSONObject.toBean( json );
Object i = ((MorphDynaBean) bean).get( "i" );
Object d = ((MorphDynaBean) bean).get( "d" );
assertTrue( i instanceof Integer );
assertTrue( d instanceof Integer );
Object bi = ((MorphDynaBean) bean).get( "bi" );
Object bd = ((MorphDynaBean) bean).get( "bd" );
assertTrue( bi instanceof BigInteger );
assertTrue( bd instanceof Double );
}
public void testToBean_DynaBean__BigInteger_BigDecimal() {
BigInteger l = new BigDecimal( "1.7976931348623157E308" ).toBigInteger();
BigDecimal m = new BigDecimal( "-1.7976931348623157E309" ).add( new BigDecimal( "0.0001" ) );
JSONObject json = new JSONObject().element( "i", BigInteger.ZERO )
.element( "d", MorphUtils.BIGDECIMAL_ONE )
.element( "bi", l )

View File

@@ -0,0 +1,41 @@
/*
* Copyright 2002-2009 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package net.sf.json.sample;
import org.apache.commons.lang.builder.ToStringBuilder;
import org.apache.commons.lang.builder.ToStringStyle;
import java.math.BigDecimal;
/**
* @author Andres Almiray <aalmiray@users.sourceforge.net>
*/
public class BeanD extends BeanA {
private Double[] doublearray = new Double[3];
public Double[] getDoublearray() {
return doublearray;
}
public void setDoublearray(Double[] doublearray) {
this.doublearray = doublearray;
}
public String toString() {
return ToStringBuilder.reflectionToString( this, ToStringStyle.MULTI_LINE_STYLE );
}
}

View File

@@ -17,6 +17,7 @@
package net.sf.json.util;
import java.io.StringWriter;
import java.math.BigDecimal;
import junit.framework.TestCase;
import net.sf.json.JSONFunction;
@@ -93,7 +94,7 @@ public class TestJSONBuilder extends TestCase {
.endObject();
JSONObject jsonObj = JSONObject.fromObject( w.toString() );
assertEquals( Boolean.TRUE, jsonObj.get( "bool" ) );
assertEquals( new Double( 1.1d ), jsonObj.get( "numDouble" ) );
assertEquals( Double.valueOf( "1.1" ), jsonObj.get( "numDouble" ) );
assertEquals( new Long( 2 ).longValue(), ((Number) jsonObj.get( "numInt" )).longValue() );
assertEquals( "text", jsonObj.get( "text" ) );
assertTrue( JSONUtils.isFunction( jsonObj.get( "func" ) ) );

View File

@@ -20,6 +20,8 @@ import junit.framework.TestCase;
import net.sf.json.JSONFunction;
import net.sf.json.JSONObject;
import java.math.BigDecimal;
/**
* @author Andres Almiray <aalmiray@users.sourceforge.net>
*/
@@ -85,7 +87,7 @@ public class TestJSONStringer extends TestCase {
.endObject();
JSONObject jsonObj = JSONObject.fromObject( b.toString() );
assertEquals( Boolean.TRUE, jsonObj.get( "bool" ) );
assertEquals( new Double( 1.1d ), jsonObj.get( "numDouble" ) );
assertEquals( Double.valueOf( "1.1" ), jsonObj.get( "numDouble" ) );
assertEquals( new Long( 2 ).longValue(), ((Number) jsonObj.get( "numInt" )).longValue() );
assertEquals( "text", jsonObj.get( "text" ) );
assertTrue( JSONUtils.isFunction( jsonObj.get( "func" ) ) );

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>simpleexcel-2.1-20190507</artifactId>
<artifactId>simpleexcel-2.2-20190722</artifactId>
<version>6.0.0</version>
<packaging>jar</packaging>

View File

@@ -12,17 +12,21 @@
// ============================================================================
package com.talend.excel.xssf.event;
import org.apache.poi.ooxml.util.PackageHelper;
import org.apache.poi.openxml4j.opc.OPCPackage;
import org.apache.poi.poifs.crypt.Decryptor;
import org.apache.poi.poifs.crypt.EncryptionInfo;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.xssf.eventusermodel.ReadOnlySharedStringsTable;
import org.apache.poi.xssf.eventusermodel.XSSFReader;
import org.apache.poi.xssf.model.StylesTable;
import org.apache.poi.ooxml.util.PackageHelper;
import org.xml.sax.ContentHandler;
import org.xml.sax.InputSource;
import org.xml.sax.XMLReader;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.File;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.LinkedHashMap;
@@ -32,7 +36,6 @@ import java.util.concurrent.FutureTask;
/**
* created by wwang on 2012-9-27 Detailled comment
*
*/
public class ExcelReader implements Callable {
@@ -46,6 +49,8 @@ public class ExcelReader implements Callable {
private String charset = "UTF-8";
private String password = null;
private java.io.InputStream is;
private List<String> sheetNames = new ArrayList<String>();
@@ -64,15 +69,17 @@ public class ExcelReader implements Callable {
task = new Thread(futureTask);
}
public void parse(String fileURL, String charset) {
public void parse(String fileURL, String charset, String password) {
this.fileURL = fileURL;
this.charset = charset;
this.password = password;
task.start();
}
public void parse(java.io.InputStream is, String charset) {
public void parse(java.io.InputStream is, String charset, String password) {
this.is = is;
this.charset = charset;
this.password = password;
task.start();
}
@@ -120,11 +127,25 @@ public class ExcelReader implements Callable {
public Object call() throws Exception {
OPCPackage pkg = null;
POIFSFileSystem fs = null;
try {
if (fileURL != null) {
pkg = OPCPackage.open(fileURL);
if (password != null) {
if (fileURL != null) {
fs = new POIFSFileSystem(new File(fileURL));
} else {
fs = new POIFSFileSystem(is);
}
Decryptor d = Decryptor.getInstance(new EncryptionInfo(fs));
if (!d.verifyPassword(password)) {
throw new RuntimeException("Error: Cannot decrypt Excel file. Invalid password.");
}
pkg = OPCPackage.open(d.getDataStream(fs));
} else {
pkg = PackageHelper.open(is);
if (fileURL != null) {
pkg = OPCPackage.open(fileURL);
} else {
pkg = PackageHelper.open(is);
}
}
XSSFReader r = new XSSFReader(pkg);
@@ -204,6 +225,9 @@ public class ExcelReader implements Callable {
if (pkg != null) {
pkg.revert();
}
if (fs != null) {
fs.close();
}
cache.notifyErrorOccurred();
}
return null;

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend-soap</artifactId>
<version>2.1-20190513</version>
<version>2.1-20190716</version>
<packaging>jar</packaging>
<name>talend-soap</name>

View File

@@ -124,7 +124,9 @@ public class SOAPUtil {
}
SOAPMessage message = messageFactory.createMessage();
MimeHeaders mimeHeaders = message.getMimeHeaders();
mimeHeaders.setHeader("SOAPAction", soapAction);
setSoapAction(version, soapAction, mimeHeaders);
if (basicAuth) {
addBasicAuthHeader(mimeHeaders, username, password);
}
@@ -263,8 +265,8 @@ public class SOAPUtil {
}
SOAPMessage message = messageFactory.createMessage();
MimeHeaders mimeHeaders = message.getMimeHeaders();
mimeHeaders.setHeader("SOAPAction", soapAction);
SOAPPart soapPart = message.getSOAPPart();
setSoapAction(version, soapAction, mimeHeaders);
SOAPPart soapPart = message.getSOAPPart();
String encoding = getEncoding(soapMessage);
@@ -291,8 +293,20 @@ public class SOAPUtil {
return null;
}
}
private String getEncoding(String text) {
/* https://jira.talendforge.org/browse/TDI-42581 skip add SOAPAction directly to header v1.2 */
private void setSoapAction(String version, String soapAction, MimeHeaders mimeHeaders) {
if (SOAP12.equals(version)) {
// in soap version 1.2 param 'action' optional and should not be empty
if( soapAction != null && !soapAction.trim().isEmpty()) {
mimeHeaders.setHeader("Content-Type", "application/soap+xml; charset=utf-8; action=\"" + soapAction + "\"");
}
} else {
mimeHeaders.setHeader("SOAPAction", soapAction);
}
}
private String getEncoding(String text) {
String result = Charset.defaultCharset().name();
if(text == null) {

View File

@@ -7,7 +7,7 @@
<groupId>org.talend.libraries</groupId>
<artifactId>talend-codegen-utils</artifactId>
<!-- release for revert version of library -->
<version>0.25.4</version>
<version>0.28.0</version>
<packaging>jar</packaging>
<properties>

View File

@@ -51,6 +51,8 @@ public final class TypeConverter {
public static final String LIST = "id_List";
public static final String OBJECT = "id_Object";
private TypeConverter() {
// Class provides static utility methods and shouldn't be instantiated
}
@@ -167,6 +169,8 @@ public final class TypeConverter {
return SHORT;
case "java.util.Date":
return DATE;
case "java.lang.Object":
return OBJECT;
default:
throw new UnsupportedOperationException("Unrecognized java class " + javaClass);
}

View File

@@ -5,9 +5,9 @@
>
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<groupId>org.talend.components.lib</groupId>
<artifactId>talend-db-exasol</artifactId>
<version>2.1.2</version>
<version>2.1.4</version>
<packaging>jar</packaging>
<name>talend-db-exasol</name>

View File

@@ -94,12 +94,11 @@ public class EXABulkUtil {
private String createNumberFormat(Integer length, Integer precision, boolean hasGroups) {
if (length != null && length.intValue() > 0) {
StringBuilder sb = new StringBuilder();
int numGroups = (length.intValue() / 3) + 1;
for (int i = 0; i < numGroups; i++) {
if (i > 0 && hasGroups) {
sb.append("G");
}
sb.append("999");
for (int i = length - 1; i >= 0; i--) {
if(hasGroups && i < length - 1 && i > 0 && (i % 3 == 2)) {
sb.append("G");
}
sb.append("9");
}
if (precision != null && precision.intValue() > 0) {
sb.append("D");

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talendExcel-1.4-20190531</artifactId>
<artifactId>talendExcel-1.5-20190731</artifactId>
<version>6.0.0</version>
<packaging>jar</packaging>

View File

@@ -3,6 +3,7 @@ package org.talend;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Date;
@@ -10,17 +11,23 @@ import java.util.HashMap;
import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.apache.poi.poifs.crypt.EncryptionInfo;
import org.apache.poi.poifs.crypt.EncryptionMode;
import org.apache.poi.poifs.crypt.Encryptor;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.CellType;
import org.apache.poi.ss.usermodel.Font;
import org.apache.poi.ss.usermodel.FormulaEvaluator;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.usermodel.WorkbookFactory;
import org.apache.poi.xssf.streaming.SXSSFSheet;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.ss.usermodel.FormulaEvaluator;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbookType;
public class ExcelTool {
@@ -70,6 +77,8 @@ public class ExcelTool {
private boolean isTrackAllColumns = false;
private String password = null;
public ExcelTool() {
cellStylesMapping = new HashMap<>();
}
@@ -110,21 +119,7 @@ public class ExcelTool {
initPreXlsx(fileName);
}
if (appendWorkbook) {
InputStream inp = new FileInputStream(fileName);
wb = WorkbookFactory.create(inp);
sheet = wb.getSheet(sheetName);
if (sheet != null) {
if (appendSheet) {
if (sheet.getLastRowNum() != 0 || sheet.getRow(0) != null) {
curY = sheet.getLastRowNum() + 1;
}
} else {
wb.removeSheetAt(wb.getSheetIndex(sheetName));
sheet = wb.createSheet(sheetName);
}
} else {
sheet = wb.createSheet(sheetName);
}
appendActionForFile(fileName);
} else {
xlsxFile.delete();
wb = new SXSSFWorkbook(rowAccessWindowSize);
@@ -140,6 +135,47 @@ public class ExcelTool {
}
}
public void prepareXlsmFile(String fileName) throws Exception {
File xlsmFile = new File(fileName);
if (xlsmFile.exists()) {
if (isAbsY && keepCellFormat) {
initPreXlsx(fileName);
}
if (appendWorkbook) {
appendActionForFile(fileName);
} else {
xlsmFile.delete();
wb = new SXSSFWorkbook(new XSSFWorkbook(XSSFWorkbookType.XLSM), rowAccessWindowSize);
sheet = wb.createSheet(sheetName);
}
} else {
wb = new SXSSFWorkbook(new XSSFWorkbook(XSSFWorkbookType.XLSM), rowAccessWindowSize);
sheet = wb.createSheet(sheetName);
}
if (isAbsY) {
startX = absX;
curY = absY;
}
}
private void appendActionForFile(String fileName) throws IOException {
InputStream inp = new FileInputStream(fileName);
wb = WorkbookFactory.create(inp);
sheet = wb.getSheet(sheetName);
if (sheet != null) {
if (appendSheet) {
if (sheet.getLastRowNum() != 0 || sheet.getRow(0) != null) {
curY = sheet.getLastRowNum() + 1;
}
} else {
wb.removeSheetAt(wb.getSheetIndex(sheetName));
sheet = wb.createSheet(sheetName);
}
} else {
sheet = wb.createSheet(sheetName);
}
}
/**
*
* @return start insert row index.
@@ -302,13 +338,14 @@ public class ExcelTool {
}
public void writeExcel(OutputStream outputStream) throws Exception {
try {
wb.write(outputStream);
} finally {
if(outputStream != null) {
outputStream.close();
}
}
try {
wb.write(outputStream);
wb.close();
} finally {
if (outputStream != null) {
outputStream.close();
}
}
}
public void writeExcel(String fileName, boolean createDir) throws Exception {
@@ -319,14 +356,29 @@ public class ExcelTool {
pFile.mkdirs();
}
}
FileOutputStream fileOutput = new FileOutputStream(fileName);
if (appendWorkbook && appendSheet && recalculateFormula) {
evaluateFormulaCell();
}
FileOutputStream fileOutput = new FileOutputStream(fileName);
POIFSFileSystem fs = null;
try {
wb.write(fileOutput);
if (password == null) {
wb.write(fileOutput);
} else {
fs = new POIFSFileSystem();
Encryptor encryptor = new EncryptionInfo(EncryptionMode.agile).getEncryptor();
encryptor.confirmPassword(password);
OutputStream encryptedDataStream = encryptor.getDataStream(fs);
wb.write(encryptedDataStream);
encryptedDataStream.close(); // this is mandatory to do that at that point
fs.writeFilesystem(fileOutput);
}
} finally {
fileOutput.close();
wb.close();
fileOutput.close();
if (fs != null) {
fs.close();
}
}
}
@@ -349,4 +401,9 @@ public class ExcelTool {
((SXSSFSheet) sheet).flushRows();
}
}
public void setPasswordProtection(String password) {
this.password = password;
}
}

View File

@@ -7,6 +7,7 @@
<FAMILIES>
<FAMILY>Databases/DB Specifics/Amazon/Redshift</FAMILY>
<FAMILY>Cloud/Amazon/Redshift</FAMILY>
</FAMILIES>
<DOCUMENTATION>

View File

@@ -24,13 +24,15 @@
String dataset = ElementParameterParser.getValue(node, "__DATASET__");
String table = ElementParameterParser.getValue(node, "__TABLE__");
String gsFile = ElementParameterParser.getValue(node, "__GS_FILE__");
boolean setFieldDelimiter = ElementParameterParser.getBooleanValue(node, "__SET_FIELD_DELIMITER__");
String fieldDelimiter = ElementParameterParser.getValue(node, "__FIELD_DELIMITER__");
String actionOnData = ElementParameterParser.getValue(node, "__ACTION_ON_DATA__");
boolean dieOnError = "true".equals(ElementParameterParser.getValue(node, "__DIE_ON_ERROR__"));
boolean dieOnError = ElementParameterParser.getBooleanValue(node, "__DIE_ON_ERROR__");
boolean createTableIfNotExist = ElementParameterParser.getBooleanValue(node, "__CREATE_TABLE_IF_NOT_EXIST__");
String gsFileHeader = ElementParameterParser.getValue(node, "__GS_FILE_HEADER__");
String tokenFile = ElementParameterParser.getValue(node,"__TOKEN_NAME__");
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
boolean isLog4jEnabled = ElementParameterParser.getBooleanValue(node.getProcess(), "__LOG4J_ACTIVATE__");
boolean dropTable = ElementParameterParser.getBooleanValue(node, "__DROP__");
String passwordFieldName = "";
@@ -201,7 +203,7 @@
} else {
throw new IllegalArgumentException("authentication mode should be either \"SERVICEACCOUNT\" or \"OAUTH\", but it is " + authMode);
}
boolean bulkFileAlreadyExists = "true".equals(ElementParameterParser.getValue(node, "__BULK_FILE_ALREADY_EXIST__"));
boolean bulkFileAlreadyExists = ElementParameterParser.getBooleanValue(node, "__BULK_FILE_ALREADY_EXIST__");
String accessKey = ElementParameterParser.getValue(node, "__GS_ACCESS_KEY__");
String secretKey = ElementParameterParser.getValue(node, "__GS_SECRET_KEY__");
String localFilename = ElementParameterParser.getValue(node, "__GS_LOCAL_FILE__");
@@ -234,6 +236,7 @@
}
%>
gsService_<%=cid%>.putObject(<%=bucketName%>, fileObject_<%=cid%>);
/* ----END-UPLOADING-FILE---- */
<%
if(isLog4jEnabled){
%>
@@ -242,7 +245,6 @@
}
}
%>
/* ----END-UPLOADING-FILE---- */
<%
if (authMode.equals("OAUTH")) {
@@ -259,98 +261,130 @@
com.google.api.services.bigquery.model.JobConfiguration config_<%=cid%> = new com.google.api.services.bigquery.model.JobConfiguration();
com.google.api.services.bigquery.model.JobConfigurationLoad queryLoad_<%=cid%> = new com.google.api.services.bigquery.model.JobConfigurationLoad();
if (<%=ElementParameterParser.getBooleanValue(node, "__CREATE_TABLE_IF_NOT_EXIST__")%>) {
com.google.api.services.bigquery.model.TableSchema schema_<%=cid%> = new com.google.api.services.bigquery.model.TableSchema();
<%
if(isLog4jEnabled){
%>
log.info("<%=cid%> - Table field schema:");
<%
}
%>
java.util.List<com.google.api.services.bigquery.model.TableFieldSchema> fields_<%=cid%> = new java.util.ArrayList<com.google.api.services.bigquery.model.TableFieldSchema>();
<%
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null) && (metadatas.size() > 0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata != null) {
List<IMetadataColumn> columns = metadata.getListColumns();
int nbColumns = columns.size();
for (int i = 0; i < nbColumns; i++ ) {
IMetadataColumn column = columns.get(i);
String columnName = column.getLabel();
String typeToGenerate = "string";
if("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "float";
}else if("id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType()) || "id_Short".equals(column.getTalendType())) {
typeToGenerate = "integer";
} else if("id_Character".equals(column.getTalendType())) {
typeToGenerate = "string";
} else if("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "numeric";
} else if("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "boolean";
} else if("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "date";
}else if(pattern.length() > 12){
typeToGenerate = "timestamp";
}else{
typeToGenerate = "string";
}
}
%>
<%
String modeType = null;
if (!column.isNullable()) {
modeType = "REQUIRED";
} else {
modeType = "NULLABLE";
}
%>
com.google.api.services.bigquery.model.TableFieldSchema <%=columnName%>_<%=cid%> = new com.google.api.services.bigquery.model.TableFieldSchema();
<%=columnName%>_<%=cid%>.setName("<%=columnName%>");
<%=columnName%>_<%=cid%>.setType("<%=typeToGenerate%>");
<%=columnName%>_<%=cid%>.setMode("<%=modeType%>");
fields_<%=cid%>.add(<%=columnName%>_<%=cid%>);
<%
<%if (dropTable) {%>
try {
bigqueryclient_<%=cid%>.tables().delete(PROJECT_ID_<%=cid%>, <%=dataset%>, <%=table%>).execute();
} catch (com.google.api.client.googleapis.json.GoogleJsonResponseException e_<%=cid%>) {
if (e_<%=cid%>.getDetails().getCode() != 404) {
<%
if(dieOnError) {
%>
throw e_<%=cid%>;
<%
} else {
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[<%=i%>] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\",\"mode\":\"<%=modeType%>\"}");
log.error("<%=cid%> - "+ e_<%=cid%>.getDetails().getMessage());
<%
}
%>
System.err.println(e_<%=cid%>.getDetails().getMessage());
<%
}
%>
}
}
%>
<%}%>
schema_<%=cid%>.setFields(fields_<%=cid%>);
queryLoad_<%=cid%>.setSchema(schema_<%=cid%>);
<%if (createTableIfNotExist) { %>
}
<%
if("true".equals(ElementParameterParser.getValue(node, "__CREATE_TABLE_IF_NOT_EXIST__"))) {
%>
queryLoad_<%=cid%>.setCreateDisposition("CREATE_IF_NEEDED");
try {
com.google.api.services.bigquery.model.Table getTable = bigqueryclient_<%=cid%>.tables().get(PROJECT_ID_<%=cid%>, <%=dataset%>, <%=table%>).execute();
queryLoad_<%=cid%>.setCreateDisposition("CREATE_NEVER");
} catch (com.google.api.client.googleapis.json.GoogleJsonResponseException e_<%=cid%>){
if (e_<%=cid%>.getDetails().getCode() == 404) {
com.google.api.services.bigquery.model.TableSchema schema_<%=cid%> = new com.google.api.services.bigquery.model.TableSchema();
<%
if(isLog4jEnabled){
%>
log.info("<%=cid%> - Table field schema:");
<%
}
%>
java.util.List<com.google.api.services.bigquery.model.TableFieldSchema> fields_<%=cid%> = new java.util.ArrayList<com.google.api.services.bigquery.model.TableFieldSchema>();
<%
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null) && (metadatas.size() > 0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata != null) {
List<IMetadataColumn> columns = metadata.getListColumns();
int nbColumns = columns.size();
for (int i = 0; i < nbColumns; i++ ) {
IMetadataColumn column = columns.get(i);
String columnName = column.getLabel();
String typeToGenerate = "string";
if("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "float";
}else if("id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType()) || "id_Short".equals(column.getTalendType())) {
typeToGenerate = "integer";
} else if("id_Character".equals(column.getTalendType())) {
typeToGenerate = "string";
} else if("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "numeric";
} else if("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "boolean";
} else if("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "date";
}else if(pattern.length() > 12){
typeToGenerate = "timestamp";
}else{
typeToGenerate = "string";
}
}
String modeType = (!column.isNullable()) ? "REQUIRED" : "NULLABLE";
%>
com.google.api.services.bigquery.model.TableFieldSchema <%=columnName%>_<%=cid%> = new com.google.api.services.bigquery.model.TableFieldSchema();
<%=columnName%>_<%=cid%>.setName("<%=columnName%>");
<%=columnName%>_<%=cid%>.setType("<%=typeToGenerate%>");
<%=columnName%>_<%=cid%>.setMode("<%=modeType%>");
fields_<%=cid%>.add(<%=columnName%>_<%=cid%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[<%=i%>] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\",\"mode\":\"<%=modeType%>\"}");
<%
}
}
}
}
%>
schema_<%=cid%>.setFields(fields_<%=cid%>);
queryLoad_<%=cid%>.setSchema(schema_<%=cid%>);
queryLoad_<%=cid%>.setCreateDisposition("CREATE_IF_NEEDED");
} else {
<%
if(dieOnError) {
%>
throw e_<%=cid%>;
<%
} else {
if(isLog4jEnabled){
%>
log.error("<%=cid%> - "+ e_<%=cid%>.getDetails().getMessage());
<%
}
%>
System.err.println(e_<%=cid%>.getDetails().getMessage());
<%
}
%>
}
}
<%
} else {
%>
queryLoad_<%=cid%>.setCreateDisposition("CREATE_NEVER");
<%
}
if("true".equals(ElementParameterParser.getValue(node, "__SET_FIELD_DELIMITER__"))) {
if(setFieldDelimiter) {
%>
queryLoad_<%=cid%>.setFieldDelimiter(<%=fieldDelimiter%>);
<%
}
%>
queryLoad_<%=cid%>.setAllowQuotedNewlines(true);
queryLoad_<%=cid%>.setWriteDisposition("WRITE_<%=actionOnData%>");
com.google.api.services.bigquery.model.TableReference destinationTable_<%=cid%> = new com.google.api.services.bigquery.model.TableReference();
destinationTable_<%=cid%>.setProjectId(PROJECT_ID_<%=cid%>);
@@ -358,8 +392,8 @@
destinationTable_<%=cid%>.setTableId(<%=table%>);
queryLoad_<%=cid%>.setDestinationTable(destinationTable_<%=cid%>);
queryLoad_<%=cid%>.setSourceUris(java.util.Arrays.asList(<%=ElementParameterParser.getValue(node, "__GS_FILE__")%>));
queryLoad_<%=cid%>.setSkipLeadingRows(<%=ElementParameterParser.getValue(node, "__GS_FILE_HEADER__")%>);
queryLoad_<%=cid%>.setSourceUris(java.util.Arrays.asList(<%=gsFile%>));
queryLoad_<%=cid%>.setSkipLeadingRows(<%=gsFileHeader%>);
queryLoad_<%=cid%>.setNullMarker("\\N");
config_<%=cid%>.setLoad(queryLoad_<%=cid%>);
@@ -463,9 +497,11 @@
%>
/* ----START-CREATING-JOB (Cloud API)---- */
com.google.cloud.bigquery.TableId tableId_<%=cid%> = com.google.cloud.bigquery.TableId.of(<%=projectId%>, <%=dataset%>, <%=table%>);
com.google.cloud.bigquery.Table table_<%=cid%> = bigquery_<%=cid%>.getTable(tableId_<%=cid%>);
com.google.cloud.bigquery.LoadJobConfiguration.Builder loadJobBuilder_<%=cid%> = com.google.cloud.bigquery.LoadJobConfiguration.newBuilder(tableId_<%=cid%>, <%=gsFile%>);
if (<%=ElementParameterParser.getBooleanValue(node, "__DROP__")%> && table_<%=cid%> != null) {
boolean dropTable_<%=cid%> = <%=dropTable%>;
if (dropTable_<%=cid%> && bigquery_<%=cid%>.getTable(tableId_<%=cid%>) != null) {
boolean deleted = bigquery_<%=cid%>.delete(tableId_<%=cid%>);
if (deleted) {
<%
@@ -479,123 +515,100 @@
throw new RuntimeException("Unable to delete table " + tableId_<%=cid%>);
}
}
if (<%=ElementParameterParser.getBooleanValue(node, "__DROP__")%> || <%=ElementParameterParser.getBooleanValue(node, "__CREATE_TABLE_IF_NOT_EXIST__")%>) {
java.util.List<com.google.cloud.bigquery.Field> fields_<%=cid%> = new java.util.ArrayList<>();
<%
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null) && (metadatas.size() > 0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata != null) {
List<IMetadataColumn> columns = metadata.getListColumns();
int nbColumns = columns.size();
for (int i = 0; i < nbColumns; i++ ) {
IMetadataColumn column = columns.get(i);
String columnName = column.getLabel();
String typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
if("id_String".equals(column.getTalendType()) || "id_Character".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
} else if ("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.FLOAT";
} else if ("id_Short".equals(column.getTalendType()) || "id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.INTEGER";
} else if ("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.NUMERIC";
} else if ("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.BOOLEAN";
} else if ("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.DATE";
}else if(pattern.length() > 12){
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.TIMESTAMP";
}else{
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
}
}
String modeType = "NULLABLE";
if (!column.isNullable()) {
modeType = "REQUIRED";
}
<%if(createTableIfNotExist){%>
if(bigquery_<%=cid%>.getTable(tableId_<%=cid%>) == null){
java.util.List<com.google.cloud.bigquery.Field> fields_<%=cid%> = new java.util.ArrayList<>();
<%
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null) && (metadatas.size() > 0)) {
IMetadataTable metadata = metadatas.get(0);
if (metadata != null) {
List<IMetadataColumn> columns = metadata.getListColumns();
int nbColumns = columns.size();
for (int i = 0; i < nbColumns; i++ ) {
IMetadataColumn column = columns.get(i);
String columnName = column.getLabel();
String typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
if("id_String".equals(column.getTalendType()) || "id_Character".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
} else if ("id_Float".equals(column.getTalendType()) || "id_Double".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.FLOAT";
} else if ("id_Short".equals(column.getTalendType()) || "id_Integer".equals(column.getTalendType()) || "id_Long".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.INTEGER";
} else if ("id_BigDecimal".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.NUMERIC";
} else if ("id_Boolean".equals(column.getTalendType())) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.BOOLEAN";
} else if ("id_Date".equals(column.getTalendType())) {
String pattern = column.getPattern();
if(pattern.length() == 12 || pattern.isEmpty() || "\"\"".equals(pattern)) {
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.DATE";
}else if(pattern.length() > 12){
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.TIMESTAMP";
}else{
typeToGenerate = "com.google.cloud.bigquery.LegacySQLTypeName.STRING";
}
}
String modeType = (!column.isNullable()) ? "REQUIRED" : "NULLABLE";
%>
com.google.cloud.bigquery.Field field_<%=i%> = com.google.cloud.bigquery.Field.newBuilder("<%=columnName%>", <%=typeToGenerate%>)
.setMode(com.google.cloud.bigquery.Field.Mode.valueOf("<%=modeType%>"))
.build();
fields_<%=cid%>.add(field_<%=i%>);
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[<%=i%>] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\"}");
<%
<%
if(isLog4jEnabled){
%>
log.debug("<%=cid%> - Field index[<%=i%>] {\"name\":\"<%=columnName%>\",\"type\":\"<%=typeToGenerate%>\"}");
<%
}
}
}
}
}
%>
%>
com.google.cloud.bigquery.Schema schema_<%=cid%> = com.google.cloud.bigquery.Schema.of(fields_<%=cid%>);
com.google.cloud.bigquery.TableInfo tableInfo_<%=cid%> = com.google.cloud.bigquery.TableInfo.newBuilder(tableId_<%=cid%>, com.google.cloud.bigquery.StandardTableDefinition.of(schema_<%=cid%>)).build();
table_<%=cid%> = bigquery_<%=cid%>.create(tableInfo_<%=cid%>);
loadJobBuilder_<%=cid%>.setSchema(schema_<%=cid%>);
}
<%
if("true".equals(ElementParameterParser.getValue(node, "__CREATE_TABLE_IF_NOT_EXIST__"))) {
%>
com.google.cloud.bigquery.Schema schema_<%=cid%> = com.google.cloud.bigquery.Schema.of(fields_<%=cid%>);
com.google.cloud.bigquery.TableInfo tableInfo_<%=cid%> = com.google.cloud.bigquery.TableInfo.newBuilder(tableId_<%=cid%>, com.google.cloud.bigquery.StandardTableDefinition.of(schema_<%=cid%>)).build();
com.google.cloud.bigquery.Table table_<%=cid%> = bigquery_<%=cid%>.create(tableInfo_<%=cid%>);
loadJobBuilder_<%=cid%>.setSchema(schema_<%=cid%>);
loadJobBuilder_<%=cid%>.setCreateDisposition(com.google.cloud.bigquery.JobInfo.CreateDisposition.CREATE_IF_NEEDED);
<%
} else {
%>
loadJobBuilder_<%=cid%>.setCreateDisposition(com.google.cloud.bigquery.JobInfo.CreateDisposition.CREATE_NEVER);
<%
}
%>
<%} else {%>
loadJobBuilder_<%=cid%>.setCreateDisposition(com.google.cloud.bigquery.JobInfo.CreateDisposition.CREATE_NEVER);
<%}%>
<%
if("APPEND".equals(actionOnData)) {
%>
loadJobBuilder_<%=cid%>.setWriteDisposition(com.google.cloud.bigquery.JobInfo.WriteDisposition.WRITE_APPEND);
<%
} else if("TRUNCATE".equals(actionOnData)) {
%>
loadJobBuilder_<%=cid%>.setWriteDisposition(com.google.cloud.bigquery.JobInfo.WriteDisposition.WRITE_TRUNCATE);
<%
}
else {
%>
loadJobBuilder_<%=cid%>.setWriteDisposition(com.google.cloud.bigquery.JobInfo.WriteDisposition.WRITE_EMPTY);
<%
}
%>
loadJobBuilder_<%=cid%>.setDestinationTable(tableId_<%=cid%>);
com.google.cloud.bigquery.CsvOptions.Builder csvOptions_<%=cid%> = com.google.cloud.bigquery.CsvOptions.newBuilder();
csvOptions_<%=cid%>.setAllowQuotedNewLines(true);
loadJobBuilder_<%=cid%>.setWriteDisposition(com.google.cloud.bigquery.JobInfo.WriteDisposition.WRITE_<%=actionOnData%>);
loadJobBuilder_<%=cid%>.setDestinationTable(tableId_<%=cid%>);
com.google.cloud.bigquery.CsvOptions.Builder csvOptions_<%=cid%> = com.google.cloud.bigquery.CsvOptions.newBuilder();
csvOptions_<%=cid%>.setAllowQuotedNewLines(true);
csvOptions_<%=cid%>.setSkipLeadingRows(<%=gsFileHeader%>);
<%if("true".equals(ElementParameterParser.getValue(node, "__SET_FIELD_DELIMITER__"))) {
%>
loadJobBuilder_<%=cid%>.setFormatOptions(csvOptions_<%=cid%>.setFieldDelimiter(<%=fieldDelimiter%>).build());
<%
}
%>
<%if(setFieldDelimiter) {
%>
loadJobBuilder_<%=cid%>.setFormatOptions(csvOptions_<%=cid%>.setFieldDelimiter(<%=fieldDelimiter%>).build());
<%
}
%>
loadJobBuilder_<%=cid%>.setNullMarker("\\N");
com.google.cloud.bigquery.Job job_<%=cid%> = bigquery_<%=cid%>.create(com.google.cloud.bigquery.JobInfo.of(loadJobBuilder_<%=cid%>.build()));
job_<%=cid%> = job_<%=cid%>.waitFor(com.google.cloud.RetryOption.initialRetryDelay(org.threeten.bp.Duration.ofSeconds(1)));
if (job_<%=cid%> != null && job_<%=cid%>.getStatus().getError() == null) {
<%
if(isLog4jEnabled){
%>
log.info("<%=cid%> - Job execution status: " + job_<%=cid%>.getStatus());
<%
loadJobBuilder_<%=cid%>.setNullMarker("\\N");
com.google.cloud.bigquery.Job job_<%=cid%> = bigquery_<%=cid%>.create(com.google.cloud.bigquery.JobInfo.of(loadJobBuilder_<%=cid%>.build()));
job_<%=cid%> = job_<%=cid%>.waitFor(com.google.cloud.RetryOption.initialRetryDelay(org.threeten.bp.Duration.ofSeconds(1)));
if (job_<%=cid%> != null && job_<%=cid%>.getStatus().getError() == null) {
<%
if(isLog4jEnabled){
%>
log.info("<%=cid%> - Job execution status: " + job_<%=cid%>.getStatus());
<%
}
%>
} else {
List<com.google.cloud.bigquery.BigQueryError> errorList = job_<%=cid%>.getStatus().getExecutionErrors();
throw new RuntimeException("Job failed: " + errorList.get(errorList.size() - 1));
}
%>
} else {
List<com.google.cloud.bigquery.BigQueryError> errorList = job_<%=cid%>.getStatus().getExecutionErrors();
throw new RuntimeException("Job failed: " + errorList.get(errorList.size() - 1));
}
/* ----END-CREATING-JOB (Cloud API)---- */
<%

View File

@@ -173,7 +173,7 @@
SHOW_IF="(SET_FIELD_DELIMITER == 'true')">
<DEFAULT>","</DEFAULT>
</PARAMETER>
<PARAMETER NAME="DROP" FIELD="CHECK" REQUIRED="true" NUM_ROW="25" SHOW_IF="AUTH_MODE == 'SERVICEACCOUNT'">
<PARAMETER NAME="DROP" FIELD="CHECK" REQUIRED="true" NUM_ROW="25">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="ENCODING" FIELD="ENCODING_TYPE" NUM_ROW="30"

View File

@@ -53,6 +53,7 @@
<PARAMETER
NAME="DBTYPE"
REPOSITORY_VALUE="DBTYPE"
FIELD="CLOSED_LIST"
NUM_ROW="10"
>

View File

@@ -1564,7 +1564,7 @@ class TeradataManager extends Manager {
private boolean useExistingConnection;
private String connection;
private static final String SEPARATOR = ",";
private static final String SEPARATOR = "/";
protected TeradataManager(String host, String port, String dbName, String tableName,
String userName, String cid, boolean useExistingConnection, String connection, String additionalParams ) {

View File

@@ -118,7 +118,7 @@ DB_VERTICA_VERSION.ITEM.VERTICA_7_1_X=Vertica 7.1.X
DB_VERTICA_VERSION.ITEM.VERTICA_9_0=Vertica 9
DB_POSTGRE_VERSION.NAME=DB Version
DB_POSTGRE_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_POSTGRE_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_POSTGRE_VERSION.ITEM.V9_X=v9 and later
QUERY.NAME=Query

View File

@@ -36,6 +36,8 @@ skeleton="@{org.talend.designer.components.localprovider}/components/templates/d
String defaultDateFormat = ElementParameterParser.getValue(node, "__DEFAULT_DATE_FORMAT__");
String defaultTimestampFormat = ElementParameterParser.getValue(node, "__DEFAULT_TIMESTAMP_FORMAT__");
String numGroupSep = ElementParameterParser.getValue(node, "__THOUSANDS_SEPARATOR__");
boolean useLenPrecision = "true".equals(ElementParameterParser.getValue(node, "__USE_PRECISION_LENGTH_FROM_SCHEMA__"));
if (numGroupSep == null|| numGroupSep.trim().isEmpty()) {
numGroupSep = "null";
}
@@ -207,12 +209,24 @@ org.talend.database.exasol.imp.EXABulkUtil <%=cid%> = new org.talend.database.ex
String lengthStr = "null";
String precisionStr = "null";
Integer length = column.getLength();
if (length != null) {
lengthStr = String.valueOf(length);
}
if(useLenPrecision) {
if (length != null) {
lengthStr = String.valueOf(length);
}
} else {
if (length != null && hasGroupSep) {
lengthStr = String.valueOf(length);
}
}
Integer precision = column.getPrecision();
if (precision != null) {
precisionStr = String.valueOf(precision);
if(useLenPrecision) {
if (precision != null) {
precisionStr = String.valueOf(precision);
}
} else {
if (precision != null && hasGroupSep) {
precisionStr = String.valueOf(precision);
}
}
String talendType = column.getTalendType();
if ("id_Date".equals(talendType)

View File

@@ -635,14 +635,23 @@
>
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="USE_PRECISION_LENGTH_FROM_SCHEMA"
FIELD="CHECK"
NUM_ROW="115"
REQUIRED="true"
>
<DEFAULT>false</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-EXASolution" MODULE="exajdbc-6.0.9302.jar" MVN="mvn:org.talend.libraries/exajdbc-6.0.9302/6.3.0" REQUIRED="true" />
<IMPORT NAME="Talend-DB-Exasol-Util" MODULE="talend-db-exasol-2.1.2.jar" MVN="mvn:org.talend.libraries/talend-db-exasol-2.1.2/6.2.0"
UrlPath="platform:/plugin/org.talend.libraries.jdbc.exasol/lib/talend-db-exasol-2.1.2.jar" REQUIRED="true" />
<IMPORT NAME="Talend-DB-Exasol-Util" MODULE="talend-db-exasol-2.1.4.jar" MVN="mvn:org.talend.components.lib/talend-db-exasol/2.1.4"
UrlPath="platform:/plugin/org.talend.libraries.jdbc.exasol/lib/talend-db-exasol-2.1.4.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -108,4 +108,6 @@ NB_LINE_INSERTED.NAME=Rows inserted
NB_LINE_DELETED.NAME=Rows deleted
SOURCE_QUERY.NAME=Source query
TEST_MODE.NAME=Test mode (no statements are executed)
TEST_MODE.NAME=Test mode (no statements are executed)
USE_PRECISION_LENGTH_FROM_SCHEMA.NAME=Use precision and length from schema

View File

@@ -178,7 +178,7 @@
<IMPORTS>
<IMPORT NAME="Java_DOM4J1.6" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar" REQUIRED_IF="READ_BY == 'XPATH'" BundleID="" />
<IMPORT NAME="Java_JAXEN1.1" MODULE="jaxen-1.1.1.jar" MVN="mvn:org.talend.libraries/jaxen-1.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/jaxen-1.1.1.jar" REQUIRED_IF="READ_BY == 'XPATH'" BundleID="" />
<IMPORT NAME="json-lib" MODULE="json-lib-2.4.1-talend.jar" MVN="mvn:net.sf.json-lib/json-lib/2.4.1-talend" REQUIRED_IF="READ_BY == 'XPATH'" />
<IMPORT NAME="json-lib" MODULE="json-lib-2.4.2-talend.jar" MVN="mvn:net.sf.json-lib/json-lib/2.4.2-talend" REQUIRED_IF="READ_BY == 'XPATH'" />
<IMPORT NAME="commons_lang" MODULE="commons-lang-2.6.jar" MVN="mvn:commons-lang/commons-lang/2.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-lang-2.6.jar" REQUIRED_IF="READ_BY == 'XPATH'" />
<IMPORT NAME="commons_logging" MODULE="commons-logging-1.1.1.jar" MVN="mvn:org.talend.libraries/commons-logging-1.1.1/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.1.1.v201101211721.jar" REQUIRED_IF="READ_BY == 'XPATH'" />
<IMPORT NAME="ezmorph" MODULE="ezmorph-1.0.6.jar" MVN="mvn:org.talend.libraries/ezmorph-1.0.6/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.jackson/lib/ezmorph-1.0.6.jar" REQUIRED_IF="READ_BY == 'XPATH'" />

View File

@@ -1,88 +0,0 @@
<%@ jet
%>
boolean retry_<%=cid%> = false;
int retry_count_<%=cid%> = 0;
int retry_max_<%=cid%> = 5;
com.jcraft.jsch.Session session_<%=cid%> = null;
com.jcraft.jsch.Channel channel_<%=cid%> = null;
do {
retry_<%=cid%> = false;
com.jcraft.jsch.JSch jsch_<%=cid%> = new com.jcraft.jsch.JSch();
<%if (("PUBLICKEY").equals(authMethod)){%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - SFTP authentication using a public key.");
log.debug("<%=cid%> - Private key: '" + <%=privateKey%> + "'.");
<%}%>
jsch_<%=cid%>.addIdentity(<%=privateKey %>, defaultUserInfo_<%=cid%>.getPassphrase());
<%}%>
session_<%=cid%> = jsch_<%=cid%>.getSession(<%=user%>, <%=host%>, <%=port%>);
session_<%=cid%>.setConfig("PreferredAuthentications", "publickey,password,keyboard-interactive,gssapi-with-mic");
<%if (("PASSWORD").equals(authMethod)) {%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - SFTP authentication using a password.");
<%}%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
session_<%=cid%>.setPassword(decryptedPassword_<%=cid%>);
<%}%>
session_<%=cid%>.setUserInfo(defaultUserInfo_<%=cid%>);
<%if (!useProxy) {%>
if(("true").equals(System.getProperty("http.proxySet")) ){
com.jcraft.jsch.ProxyHTTP proxy_<%=cid%> = new com.jcraft.jsch.ProxyHTTP(System.getProperty("http.proxyHost"),Integer.parseInt(System.getProperty("http.proxyPort")));
if(!"".equals(System.getProperty("http.proxyUser"))){
proxy_<%=cid%>.setUserPasswd(System.getProperty("http.proxyUser"),System.getProperty("http.proxyPassword"));
}
session_<%=cid%>.setProxy(proxy_<%=cid%>);
}
<%}%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user%> + "'.");
<%}%>
channel_<%=cid%> = null;
try {
session_<%=cid%>.connect();
channel_<%=cid%> = session_<%=cid%>.openChannel("sftp");
channel_<%=cid%>.connect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
} catch (com.jcraft.jsch.JSchException e_<%=cid%>) {
try {
if(channel_<%=cid%>!=null) {
channel_<%=cid%>.disconnect();
}
if(session_<%=cid%>!=null) {
session_<%=cid%>.disconnect();
}
} catch(java.lang.Exception ce_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - close sftp connection failed : " + ce_<%=cid%>.getClass() + " : " + ce_<%=cid%>.getMessage());
<%}%>
}
String message_<%=cid%> = new TalendException(null, null, null).getExceptionCauseMessage(e_<%=cid%>);
if(message_<%=cid%>.contains("Signature length not correct") || message_<%=cid%>.contains("connection is closed by foreign host")) {
retry_<%=cid%> = true;
retry_count_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - connect: Signature length not correct or connection is closed by foreign host, so retry, retry time : " + retry_count_<%=cid%>);
<%}%>
} else {
throw e_<%=cid%>;
}
}
} while(retry_<%=cid%> && (retry_count_<%=cid%> < retry_max_<%=cid%>));
com.jcraft.jsch.ChannelSftp c_<%=cid%> = (com.jcraft.jsch.ChannelSftp)channel_<%=cid%>;

View File

@@ -28,10 +28,11 @@ String authMethod = ElementParameterParser.getValue(node, "__AUTH_METHOD__");
String privateKey = ElementParameterParser.getValue(node, "__PRIVATEKEY__");
boolean ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String sEncoding = ElementParameterParser.getValue(node, "__ENCODING__");
String encoding = ElementParameterParser.getValue(node, "__ENCODING__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
String passwordFieldName = "";
@@ -60,37 +61,18 @@ if (ElementParameterParser.canEncrypt(node, passwordFieldName)) {
});
<%
}
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
if (!sftp && !ftps) { // *** ftp *** //
if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
passwordFieldName = "__PASS__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=sEncoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASS__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
globalMap.put("conn_<%=cid%>",ftp_<%=cid %>);
globalMap.put("conn_<%=cid%>",ftp_<%=cid %>);
<%
} else if (!ftps) { // *** sftp *** //
%>
@@ -138,99 +120,20 @@ if (!sftp && !ftps) { // *** ftp *** //
passwordFieldName = "__PASS__";
%>
<%@ include file="sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=sEncoding%>);
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
<%}%>
globalMap.put("conn_<%=cid%>", c_<%=cid%>);
<%
} else { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
MyTrust_<%=cid%> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=sEncoding %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%
passwordFieldName = "__PASS__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
%>
globalMap.put("conn_<%=cid%>", ftp_<%=cid %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
globalMap.put("conn_<%=cid%>", ftp_<%=cid %>);
<%
}
%>

View File

@@ -232,6 +232,11 @@
NUM_ROW="95" SHOW_IF="(FTPS=='true')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -239,7 +244,7 @@
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar" MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" REQUIRED_IF="(FTPS == 'true')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar" MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" REQUIRED_IF="(FTPS == 'true')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -40,4 +40,5 @@ SECURITY_MODE.NAME=Security Mode
SECURITY_MODE.ITEM.IMPLICIT=Implicit
SECURITY_MODE.ITEM.EXPLICIT=Explicit
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -43,6 +43,7 @@ boolean sftp = false;
boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
if (("true").equals(useExistingConn)) {
List<? extends INode> nodeList = node.getProcess().getGeneratingNodes();
@@ -55,6 +56,9 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -129,7 +133,7 @@ int nb_file_<%=cid%> = 0;
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -167,101 +171,18 @@ int nb_file_<%=cid%> = 0;
<%
} else if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
if (moveToCurrentDir) {
%>
@@ -303,36 +224,12 @@ int nb_file_<%=cid%> = 0;
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username: '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);

View File

@@ -288,6 +288,11 @@
NUM_ROW="95" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -296,8 +301,8 @@
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -56,4 +56,5 @@ IGNORE_FAILURE_AT_QUIT.NAME=Ignore Failure At Quit
PERL5_REGEX.NAME=Use Perl5 Regex Expressions as Filemask (Unchecked means Glob Expressions)
MOVE_TO_THE_CURRENT_DIRECTORY.NAME=Move to the current directory
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -42,6 +42,7 @@ boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -57,6 +58,9 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
String passwordFieldName = "";
@@ -134,7 +138,7 @@ if (sftp) { // *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -154,90 +158,18 @@ if (sftp) { // *** sftp *** //
<%
}
} else if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
} else { // *** ftp *** //
@@ -251,36 +183,10 @@ if (sftp) { // *** sftp *** //
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username: '" +<%=username %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=username %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host%> + "' has succeeded.");
<%}%>
<%
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
}
}%>

View File

@@ -276,6 +276,11 @@ COMPATIBILITY="ALL"
NUM_ROW="95" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -283,8 +288,8 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -56,4 +56,5 @@ CONNECT_MODE.ITEM.PASSIVE=Passive
IGNORE_FAILURE_AT_QUIT.NAME=Ignore Failure At Quit
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -42,6 +42,7 @@ boolean sftp = false;
boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
if (("true").equals(useExistingConn)) {
@@ -56,6 +57,9 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
String passwordFieldName = "";
@@ -151,7 +155,7 @@ if (sftp) {// *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -233,93 +237,18 @@ if (sftp) {// *** sftp *** //
nb_file_<%=cid%>++;
<%
} else if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
if(isLog4jEnabled){
%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%
}
passwordFieldName = "__PASSWORD__";
%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
String rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
<%
}
@@ -401,35 +330,10 @@ if (sftp) {// *** sftp *** //
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
String remotedir_<%=cid %> = (<%=remotedir%>).replaceAll("\\\\","/");
ftp_<%=cid %>.chdir(remotedir_<%=cid %>);

View File

@@ -254,6 +254,11 @@
NUM_ROW="95" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -261,8 +266,8 @@
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -51,4 +51,5 @@ FILES.NAME=Files
FILES.ITEM.FILEMASK=Filemask
MOVE_TO_THE_CURRENT_DIRECTORY.NAME=Move to the current directory
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -46,6 +46,7 @@ boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
log4jFileUtil.componentStartInfo(node);
@@ -62,6 +63,9 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -153,9 +157,9 @@ if (sftp) { // *** sftp *** //
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
<%}%>
@@ -203,90 +207,18 @@ if (sftp) { // *** sftp *** //
}
<%
} else if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE); //to calculate md5 for binary files correctly
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
%>
@@ -341,36 +273,11 @@ if (sftp) { // *** sftp *** //
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=username %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
<%
//add feature 19709,add the "Transfer mode" option,
//the default transfer mode is ASCII,we don't change the mode when ascii is choosed.

View File

@@ -251,6 +251,11 @@
NUM_ROW="95" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -258,8 +263,8 @@
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -50,4 +50,5 @@ MODE.ITEM.ASCII=ascii
MODE.ITEM.BINARY=binary
MODE.NAME=Transfer mode
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -49,6 +49,7 @@ boolean sftp = false;
boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -66,6 +67,9 @@ if ("true".equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
%>
int nb_file_<%=cid%> = 0;
@@ -286,7 +290,7 @@ if (sftp) { // *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -528,41 +532,11 @@ if (sftp) { // *** sftp *** //
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}
if (("ACTIVE").equals(connectMode)){
%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%
}else{
%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%
}
%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%} %>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%} %>
msg_<%=cid%>.clearAll();
FTPGetter_<%=cid%> getter_<%=cid%> = new FTPGetter_<%=cid%>();
getter_<%=cid%>.ftpClient = ftp_<%=cid %>;
@@ -650,96 +624,8 @@ if (sftp) { // *** sftp *** //
<%
if ("false".equals(useExistingConn)){
%>
<% /* create connection */ %>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid%> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid%> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid%> = null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid%>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
String conn= "conn_" + connection;
%>

View File

@@ -294,6 +294,11 @@ COMPATIBILITY="ALL"
NUM_ROW="100" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -302,8 +307,8 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -68,4 +68,5 @@ PRINT_MESSAGE.NAME=Print message
PERL5_REGEX.NAME=Use Perl5 Regex Expressions as Filemask (Unchecked means Glob Expressions)
MOVE_TO_THE_CURRENT_DIRECTORY.NAME=Move to the current directory
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -46,6 +46,7 @@ boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -62,8 +63,11 @@ if (("true").equals(useExistingConn)) {
}
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
}
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
//The following part support the socks proxy for FTP and SFTP (Socks V4 or V5, they are all OK).
//And it can not work with the FTP proxy directly, only support the socks proxy.
@@ -150,8 +154,8 @@ int nb_file_<%=cid%> = 0;
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -242,37 +246,11 @@ int nb_file_<%=cid%> = 0;
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
// msg_<%=cid%> likes a String[] to save the message from transfer.
@@ -335,96 +313,8 @@ int nb_file_<%=cid%> = 0;
<%
if(("false").equals(useExistingConn)){
%>
<% /* create connection */ %>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid%> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid%> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid%> = null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid%>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
String conn= "conn_" + connection;
%>

View File

@@ -339,6 +339,11 @@ COMPATIBILITY="ALL"
NUM_ROW="100" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -346,8 +351,8 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -73,4 +73,5 @@ FTPSOVERWRITE.ITEM.NEVER=never
FTPSOVERWRITE.ITEM.ALWAYS=always
FTPSOVERWRITE.NAME=Overwrite file
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -42,6 +42,7 @@ boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
boolean dieOnError = ("true").equals(ElementParameterParser.getValue(node,"__DIE_ON_ERROR__"));
@@ -57,6 +58,9 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -131,9 +135,9 @@ if (sftp) { // *** sftp *** //
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
<%}%>
@@ -173,99 +177,18 @@ if (sftp) { // *** sftp *** //
<%
} else if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
%>
@@ -308,36 +231,11 @@ if (sftp) { // *** sftp *** //
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%

View File

@@ -257,6 +257,11 @@ COMPATIBILITY="ALL"
NUM_ROW="95" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -264,8 +269,8 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -55,4 +55,5 @@ CONNECT_MODE.ITEM.PASSIVE=Passive
IGNORE_FAILURE_AT_QUIT.NAME=Ignore Failure At Quit
MOVE_TO_THE_CURRENT_DIRECTORY.NAME=Move to the current directory
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -41,6 +41,7 @@ boolean sftp = false;
boolean ftps = false;
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
String timeoutValue = ElementParameterParser.getValue(node, "__CONNECTION_TIMEOUT__");
boolean useRemoteVerification = ("true").equals(ElementParameterParser.getValue(node, "__REMOTE_VERIFICATION__"));
if (("true").equals(useExistingConn)) {
@@ -55,6 +56,9 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
int connectionTimeout_<%=cid %> = Integer.valueOf(<%=timeoutValue %>);
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -131,7 +135,7 @@ if (sftp) {// *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -168,92 +172,19 @@ if (sftp) {// *** sftp *** //
<%
} else if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
/* creating connection */
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
if(isLog4jEnabled){
%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%
}
passwordFieldName = "__PASSWORD__";
%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
if (moveToCurrentDir) {
%>
@@ -295,36 +226,11 @@ if (sftp) {// *** sftp *** //
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<% if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%for (int i = 0; i < files.size(); i++) {

View File

@@ -245,6 +245,11 @@ COMPATIBILITY="ALL"
NUM_ROW="95" SHOW_IF="(FTPS=='true') AND (USE_EXISTING_CONNECTION == 'false')" REPOSITORY_VALUE="REMOTE_VERIFICATION">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CONNECTION_TIMEOUT" FIELD="TEXT"
NUM_ROW="96" SHOW_IF="USE_EXISTING_CONNECTION == 'false'" REPOSITORY_VALUE="CONNECTION_TIMEOUT">
<DEFAULT>0</DEFAULT>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
@@ -253,8 +258,8 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190726" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190726.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -52,4 +52,5 @@ IGNORE_FAILURE_AT_QUIT.NAME=Ignore Failure At Quit
PERL5_REGEX.NAME=Use Perl5 Regex Expressions as Filemask (Unchecked means Glob Expressions)
MOVE_TO_THE_CURRENT_DIRECTORY.NAME=Move to the current directory
USE_ENCODING.NAME=Filename encoding
USE_ENCODING.NAME=Filename encoding
CONNECTION_TIMEOUT.NAME=Connection timeout (milliseconds)

View File

@@ -9,11 +9,11 @@
%>
<%@ include file="../templates/Log4j/Log4jFileUtil.javajet"%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
String cid = node.getUniqueName();
String directory = ElementParameterParser.getValue(node, "__SOURCE__");
String zipFile = ElementParameterParser.getValue(node, "__TARGET__");
@@ -24,16 +24,16 @@
boolean overwrite = ("true").equals(ElementParameterParser.getValue(node, "__OVERWRITE__"));
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
boolean isEncrypted = ("true").equals(ElementParameterParser.getValue(node, "__ENCRYPT_FILES__"));
List<Map<String, String>> masks = (List<Map<String,String>>)ElementParameterParser.getObjectValue(node, "__MASK__");
List<Map<String, String>> masks = (List<Map<String,String>>)ElementParameterParser.getObjectValue(node, "__MASK__");
String zip64Mode = ElementParameterParser.getValue(node, "__ZIP64_MODE__");
String encryptMethod = ElementParameterParser.getValue(node, "__ENCRYPT_METHOD__");
String aesKeyStrength = ElementParameterParser.getValue(node, "__AES_KEY_STRENGTH__");
String archiveFormat = ElementParameterParser.getValue(node, "__ARCHIVE_FORMAT__");
String sourceFile = ElementParameterParser.getValue(node, "__SOURCE_FILE__");
boolean isSyncFlush = "true".equals(ElementParameterParser.getValue(node, "__USE_SYNC_FLUSH__"));
log4jFileUtil.componentStartInfo(node);
@@ -47,6 +47,11 @@
<%
}
%>
if (java.nio.file.Files.notExists(java.nio.file.Paths.get(sourceFile_<%=cid %>), java.nio.file.LinkOption.NOFOLLOW_LINKS)){
throw new java.io.FileNotFoundException(sourceFile_<%=cid %> + " (The system cannot find the path specified)");
}
String zipFile_<%=cid %> = <%=zipFile %>;
com.talend.compress.zip.Zip zip_<%=cid%> = new com.talend.compress.zip.Zip(sourceFile_<%=cid %>, zipFile_<%=cid %>);

View File

@@ -55,6 +55,17 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
String mode = ElementParameterParser.getValue(node,"__GENERATION_MODE__");
final boolean eventMode = "EVENT_MODE".equals(mode);
if (version07) {
String passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
String password_<%=cid%> = decryptedPassword_<%=cid%>;
if (password_<%=cid%>.isEmpty()){
password_<%=cid%> = null;
}
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
int dynamic_index = -1;
@@ -692,9 +703,9 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
org.apache.poi.xssf.usermodel.XSSFWorkbook workbook_<%=cid%> = null;
if(source_<%=cid %> instanceof String){
workbook_<%=cid%> = new org.apache.poi.xssf.usermodel.XSSFWorkbook((String)source_<%=cid %>);
workbook_<%=cid%> = (org.apache.poi.xssf.usermodel.XSSFWorkbook) org.apache.poi.ss.usermodel.WorkbookFactory.create(new java.io.File((String)source_<%=cid %>), password_<%=cid%>);
} else if(source_<%=cid %> instanceof java.io.InputStream) {
workbook_<%=cid%> = new org.apache.poi.xssf.usermodel.XSSFWorkbook((java.io.InputStream)source_<%=cid %>);
workbook_<%=cid%> = (org.apache.poi.xssf.usermodel.XSSFWorkbook) org.apache.poi.ss.usermodel.WorkbookFactory.create((java.io.InputStream)source_<%=cid %>, password_<%=cid%>);
} else{
workbook_<%=cid%> = null;
throw new java.lang.Exception("The data source should be specified as Inputstream or File Path!");
@@ -1266,9 +1277,9 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
char decimalChar_<%=cid %> = df_<%=cid %>.getDecimalFormatSymbols().getDecimalSeparator();
if(source_<%=cid %> instanceof String){
excelReader_<%=cid%>.parse((String)source_<%=cid %>,<%=encoding %>);
excelReader_<%=cid%>.parse((String)source_<%=cid %>,<%=encoding %>, password_<%=cid%>);
} else if(source_<%=cid %> instanceof java.io.InputStream) {
excelReader_<%=cid%>.parse((java.io.InputStream)source_<%=cid %>,<%=encoding %>);
excelReader_<%=cid%>.parse((java.io.InputStream)source_<%=cid %>,<%=encoding %>, password_<%=cid%>);
}
<%
if(hasDynamic){

View File

@@ -39,6 +39,10 @@
<DEFAULT>"__COMP_DEFAULT_FILE_DIR__/in.xls"</DEFAULT>
</PARAMETER>
<PARAMETER NAME="PASSWORD" FIELD="PASSWORD" REQUIRED="false" SHOW_IF="(VERSION_2007 == 'true')" NUM_ROW="25">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="ALL_SHEETS" FIELD="CHECK" REQUIRED="true" NUM_ROW="30" REPOSITORY_VALUE="SELECT_ALL_SHEETS">
<DEFAULT>false</DEFAULT>
</PARAMETER>
@@ -182,7 +186,7 @@
<IMPORT NAME="poi-ooxml-schemas" MODULE="poi-ooxml-schemas-4.1.0-20190523141255_modified_talend.jar" MVN="mvn:org.apache.poi/poi-ooxml-schemas/4.1.0-20190523141255_modified_talend" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/poi-ooxml-schemas-4.1.0-20190523141255_modified_talend.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="poi-scratchpad" MODULE="poi-scratchpad-4.1.0-20190523141255_modified_talend.jar" MVN="mvn:org.apache.poi/poi-scratchpad/4.1.0-20190523141255_modified_talend" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/poi-scratchpad-4.1.0-20190523141255_modified_talend.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="xmlbeans" MODULE="xmlbeans-3.1.0.jar" MVN="mvn:org.apache.xmlbeans/xmlbeans/3.1.0" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="simpleexcel" MODULE="simpleexcel-2.1-20190507.jar" MVN="mvn:org.talend.libraries/simpleexcel-2.1-20190507/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/simpleexcel-2.1-20190507.jar" REQUIRED_IF="(VERSION_2007 == 'true') AND GENERATION_MODE == 'EVENT_MODE')" />
<IMPORT NAME="simpleexcel" MODULE="simpleexcel-2.2-20190722.jar" MVN="mvn:org.talend.libraries/simpleexcel-2.2-20190722/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/simpleexcel-2.2-20190722.jar" REQUIRED_IF="(VERSION_2007 == 'true') AND GENERATION_MODE == 'EVENT_MODE')" />
<IMPORT NAME="commons-collections4" MODULE="commons-collections4-4.1.jar" MVN="mvn:org.talend.libraries/commons-collections4-4.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/commons-collections4-4.1.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="commons-compress" MODULE="commons-compress-1.18.jar" MVN="mvn:org.apache.commons/commons-compress/1.18" REQUIRED_IF="(VERSION_2007 == 'true')" />

View File

@@ -8,6 +8,7 @@ DIE_ON_ERROR.NAME=Die on error
ENCODING.NAME=Encoding
END_OF_FILE.NAME=End Of File
FILENAME.NAME=File Name/Input Stream
PASSWORD.NAME=Password
FIRST_COLUMN.NAME=First column
FOOTER.NAME=Footer
HEADER.NAME=Header
@@ -29,7 +30,7 @@ STOPREAD_ON_EMPTYROW.NAME=Stop to read on empty row
SUPPRESS_WARN.NAME=Ignore the warning
THOUSANDS_SEPARATOR.NAME=Thousands separator
WITH_FORMAT.NAME=With format
VERSION_2007.NAME=Read excel2007 file format(xlsx)
VERSION_2007.NAME=Read excel2007 file format (xlsx / xlsm)
TRIMALL.NAME=Trim all column
TRIMSELECT.ITEM.TRIM=Trim
TRIMSELECT.NAME=Check column to trim

View File

@@ -165,7 +165,7 @@
REQUIRED_IF="(READ_BY == 'XPATH')" BundleID="" />
<IMPORT NAME="Java_JAXEN1.1" MODULE="jaxen-1.1.1.jar" MVN="mvn:org.talend.libraries/jaxen-1.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/jaxen-1.1.1.jar"
REQUIRED_IF="(READ_BY == 'XPATH')" BundleID="" />
<IMPORT NAME="json-lib" MODULE="json-lib-2.4.1-talend.jar" MVN="mvn:net.sf.json-lib/json-lib/2.4.1-talend" REQUIRED_IF="(READ_BY == 'XPATH')" />
<IMPORT NAME="json-lib" MODULE="json-lib-2.4.2-talend.jar" MVN="mvn:net.sf.json-lib/json-lib/2.4.2-talend" REQUIRED_IF="(READ_BY == 'XPATH')" />
<IMPORT NAME="commons_lang" MODULE="commons-lang-2.6.jar" MVN="mvn:commons-lang/commons-lang/2.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-lang-2.6.jar"
REQUIRED_IF="(READ_BY == 'XPATH')" />
<IMPORT NAME="commons_logging" MODULE="commons-logging-1.1.1.jar" MVN="mvn:org.talend.libraries/commons-logging-1.1.1/6.0.0" UrlPath="platform:/base/plugins/org.apache.commons.logging_1.1.1.v201101211721.jar"

View File

@@ -66,25 +66,26 @@
}
}
DecodeString_<%=cid%> decode_<%=cid%> = new DecodeString_<%=cid%>();
try{
fileInput<%=cid%> = new java.io.FileInputStream(<%=filename %>);
javax.mail.Session session_<%=cid %> = javax.mail.Session.getInstance(System.getProperties(), null);
javax.mail.internet.MimeMessage msg_<%=cid %> = new javax.mail.internet.MimeMessage(session_<%=cid %>, fileInput<%=cid%>);
javax.mail.internet.MimeMessage msg_<%=cid %> = new javax.mail.internet.MimeMessage(session_<%=cid %>, fileInput<%=cid%>);
java.util.List<String> list_<%=cid %> = new java.util.ArrayList<String>();
for (int i_<%=cid %> =0;i_<%=cid %> < mailParts_<%=cid %>.length;i_<%=cid %>++) {
String part_<%=cid %> = mailParts_<%=cid %>[i_<%=cid %>];
String sep_<%=cid%>= mailSeparator_<%=cid %>[i_<%=cid %>];
if(part_<%=cid %>.equalsIgnoreCase("body")) {
for (int i_<%=cid %> = 0; i_<%=cid %> < mailParts_<%=cid %>.length; i_<%=cid %>++) {
String part_<%=cid %> = mailParts_<%=cid %>[i_<%=cid %>];
String sep_<%=cid%>= mailSeparator_<%=cid %>[i_<%=cid %>];
if(part_<%=cid %>.equalsIgnoreCase("body")) {
if(msg_<%=cid %>.isMimeType("multipart/*")) {
javax.mail.Multipart mp<%=cid%> = (javax.mail.Multipart) msg_<%=cid %>.getContent();
for (int i = 0; i < mp<%=cid%>.getCount(); i++) {
javax.mail.BodyPart mpart<%=cid%> = mp<%=cid%>.getBodyPart(i);
String disposition<%=cid%> = mpart<%=cid%>.getDisposition();
if (!((disposition<%=cid%> != null) && ((disposition<%=cid%>
.equals(javax.mail.Part.ATTACHMENT)) || (disposition<%=cid%>.equals(javax.mail.Part.INLINE))))) {
// the following extract the body part(text/plain + text/html)
try{
for (int i = 0; i < mp<%=cid%>.getCount(); i++) {
javax.mail.BodyPart mpart<%=cid%> = mp<%=cid%>.getBodyPart(i);
String disposition<%=cid%> = mpart<%=cid%>.getDisposition();
if (!((disposition<%=cid%> != null) && ((disposition<%=cid%>
.equals(javax.mail.Part.ATTACHMENT)) || (disposition<%=cid%>.equals(javax.mail.Part.INLINE))))) {
// the following extract the body part(text/plain + text/html)
try{
Object content_<%=cid %> = mpart<%=cid %>.getContent();
if (content_<%=cid %> instanceof javax.mail.internet.MimeMultipart) {
javax.mail.internet.MimeMultipart mimeMultipart_<%=cid %> = (javax.mail.internet.MimeMultipart) content_<%=cid %>;
@@ -124,105 +125,104 @@
<%}%>
}
<%
//both attachment and message context in the email,bug TDI-19065
//both attachment and message context in the email,bug TDI-19065
%>
}else if(disposition<%=cid%> != null && disposition<%=cid%>.equals(javax.mail.Part.INLINE)){
list_<%=cid %>.add(mpart<%=cid%>.getContent().toString());
}
}
}
} else {
java.io.InputStream in_<%=cid %> = msg_<%=cid %>.getInputStream();
byte[] buffer_<%=cid %> = new byte[1024];
int length_<%=cid %> = 0;
java.io.ByteArrayOutputStream baos_<%=cid %> = new java.io.ByteArrayOutputStream();
while ((length_<%=cid %> = in_<%=cid %>.read(buffer_<%=cid %>, 0, 1024)) != -1) {
baos_<%=cid %>.write(buffer_<%=cid %>, 0, length_<%=cid %>);
}
String contentType_<%=cid%> = msg_<%=cid%>.getContentType();
String charsetName_<%=cid%> = "";
if(contentType_<%=cid%>!=null && contentType_<%=cid%>.trim().length()>0){
javax.mail.internet.ContentType cy_<%=cid%> = new javax.mail.internet.ContentType(contentType_<%=cid%>);
charsetName_<%=cid%> = cy_<%=cid%>.getParameter("charset");
}
if(charsetName_<%=cid%>!=null && charsetName_<%=cid%>.length()>0){
list_<%=cid %>.add(baos_<%=cid %>.toString(charsetName_<%=cid%>));
}else{
list_<%=cid %>.add(baos_<%=cid %>.toString());
}
in_<%=cid %>.close();
baos_<%=cid %>.close();
byte[] buffer_<%=cid %> = new byte[1024];
int length_<%=cid %> = 0;
java.io.ByteArrayOutputStream baos_<%=cid %> = new java.io.ByteArrayOutputStream();
while ((length_<%=cid %> = in_<%=cid %>.read(buffer_<%=cid %>, 0, 1024)) != -1) {
baos_<%=cid %>.write(buffer_<%=cid %>, 0, length_<%=cid %>);
}
String contentType_<%=cid%> = msg_<%=cid%>.getContentType();
String charsetName_<%=cid%> = "";
if(contentType_<%=cid%>!=null && contentType_<%=cid%>.trim().length()>0){
javax.mail.internet.ContentType cy_<%=cid%> = new javax.mail.internet.ContentType(contentType_<%=cid%>);
charsetName_<%=cid%> = cy_<%=cid%>.getParameter("charset");
}
if(charsetName_<%=cid%>!=null && charsetName_<%=cid%>.length()>0){
list_<%=cid %>.add(baos_<%=cid %>.toString(charsetName_<%=cid%>));
}else{
list_<%=cid %>.add(baos_<%=cid %>.toString());
}
in_<%=cid %>.close();
baos_<%=cid %>.close();
}
}else if(part_<%=cid %>.equalsIgnoreCase("header")){
java.util.Enumeration em = msg_<%=cid %>.getAllHeaderLines();
int em_count=0;
String tempStr_<%=cid %>="";
}else if(part_<%=cid %>.equalsIgnoreCase("header")){
java.util.Enumeration em = msg_<%=cid %>.getAllHeaderLines();
int em_count=0;
String tempStr_<%=cid %>="";
while (em.hasMoreElements()) {
tempStr_<%=cid %> = tempStr_<%=cid %> + (String) em.nextElement() + sep_<%=cid%> ;
}
list_<%=cid%>.add(decode_<%=cid%>.decode(tempStr_<%=cid%>));
}else{
if(("true").equals(mailChecked_<%=cid %>[i_<%=cid%>])){
} else {
if(("true").equals(mailChecked_<%=cid %>[i_<%=cid%>])){
String[] sa_<%=cid%> = msg_<%=cid %>.getHeader(part_<%=cid%>);
String tempStr_<%=cid%>="";
for(int i=0;i<sa_<%=cid%>.length;i++){
tempStr_<%=cid%>=tempStr_<%=cid%>+sa_<%=cid%>[i] + sep_<%=cid%>;
}
list_<%=cid%>.add(decode_<%=cid%>.decode(tempStr_<%=cid%>));
}else{
String content_<%=cid %> = msg_<%=cid %>.getHeader(part_<%=cid %>, null);
list_<%=cid %>.add(decode_<%=cid%>.decode(content_<%=cid %>));
}
}
}
//attachment Deal
if(msg_<%=cid %>.isMimeType("multipart/*")){
javax.mail.Multipart mp<%=cid%> = (javax.mail.Multipart) msg_<%=cid %>.getContent();
String attachfileName<%=cid%> = "";
String path<%=cid%> = "";
java.io.BufferedOutputStream out<%=cid%> = null;
java.io.BufferedInputStream in<%=cid%> = null;
for (int i = 0; i < mp<%=cid%>.getCount(); i++) {
javax.mail.BodyPart mpart<%=cid%> = mp<%=cid%>.getBodyPart(i);
String disposition<%=cid%> = mpart<%=cid%>.getDisposition();
<%
// fixed bug TDI-8586,to deal with attachments download
%>
if (mpart<%=cid%>.getFileName()!=null
&& ((disposition<%=cid%> != null && (disposition<%=cid%>.equals(javax.mail.Part.ATTACHMENT) || disposition<%=cid%>.equals(javax.mail.Part.INLINE)))
|| disposition<%=cid%>==null)) { <%// TDI-29179 %>
attachfileName<%=cid%> = mpart<%=cid%>.getFileName();
attachfileName<%=cid%> = javax.mail.internet.MimeUtility.decodeText(attachfileName<%=cid%>);
if(!(<%=directory%>).endsWith("/")){
path<%=cid%> = <%=directory%> + "/";
}else{
path<%=cid%> =<%=directory%>;
}
path<%=cid%> = path<%=cid%> + attachfileName<%=cid%>;
<% if(isLog4jEnabled){ %>
log.info("<%= cid %> - Extracted attachment: '"+attachfileName<%=cid%>+"'.");
<% } %>
java.io.File attachFile = new java.io.File(path<%=cid%>);
out<%=cid%> = new java.io.BufferedOutputStream(new java.io.FileOutputStream(attachFile));
in<%=cid%> = new java.io.BufferedInputStream(mpart<%=cid%>.getInputStream());
int buffer<%=cid%> = 0;
while ((buffer<%=cid%> = in<%=cid%>.read()) != -1) {
out<%=cid%>.write(buffer<%=cid%>);
out<%=cid%>.flush();
}
out<%=cid%>.close();
in<%=cid%>.close();
}
}
}else{
String content_<%=cid %> = msg_<%=cid %>.getHeader(part_<%=cid %>, null);
list_<%=cid %>.add(decode_<%=cid%>.decode(content_<%=cid %>));
}
}
}
//attachment Deal
class MessagePartProcessor {
void saveAttachment(javax.mail.Part mpart) throws IOException, javax.mail.MessagingException {
if (mpart.getFileName() != null && (
mpart.getDisposition() == null ||
(mpart.getDisposition().equals(javax.mail.Part.ATTACHMENT) || mpart.getDisposition().equals(javax.mail.Part.INLINE))
)) {
String attachFileName = javax.mail.internet.MimeUtility.decodeText(mpart.getFileName());
String path = <%=directory%>;
if(!path.endsWith("/")){
path = path + "/";
}
path = path + attachFileName;
<% if(isLog4jEnabled){ %>
log.info("<%= cid %> - Extracted attachment: '" + attachFileName + "'.");
<% } %>
java.io.File attachFile = new java.io.File(path);
java.io.BufferedOutputStream out = new java.io.BufferedOutputStream(new java.io.FileOutputStream(attachFile));
java.io.BufferedInputStream in = new java.io.BufferedInputStream(mpart.getInputStream());
int buffer = 0;
while ((buffer = in.read()) != -1) {
out.write(buffer);
out.flush();
}
out.close();
in.close();
}
}
// recursively process body parts
void processPart(javax.mail.Part part) throws javax.mail.MessagingException, IOException {
if (part.isMimeType("multipart/*")) {
javax.mail.Multipart multipartContent = (javax.mail.Multipart) part.getContent();
for (int i = 0; i < multipartContent.getCount(); i++) {
javax.mail.Part mpart = multipartContent.getBodyPart(i);
saveAttachment(mpart);
processPart(mpart);
}
}
}
}
new MessagePartProcessor().processPart(msg_<%=cid %>);
// for output
<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
@@ -282,10 +282,10 @@
for (int i=1;i<conns.size();i++) {
IConnection conn2 = conns.get(i);
if ((conn2.getName().compareTo(firstConnName)!=0)&&(conn2.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA))) {
for (IMetadataColumn column: metadata.getListColumns()) {%>
<%=conn2.getName() %>.<%=column.getLabel() %> = <%=firstConnName %>.<%=column.getLabel() %>;
<%
}
for (IMetadataColumn column: metadata.getListColumns()) {%>
<%=conn2.getName() %>.<%=column.getLabel() %> = <%=firstConnName %>.<%=column.getLabel() %>;
<%
}
}
}
}

View File

@@ -37,6 +37,17 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
}else{
globalMap.put("<%=cid %>_EXPORTED_FILE_PATH",<%=directory%>);
}
// create output directory if not exists
String path = <%=directory%>;
if(!path.endsWith("/")){
path = path + "/";
}
java.io.File outputDir = new java.io.File(path);
if (!outputDir.exists()) {
outputDir.mkdirs();
}
boolean hasData_<%=cid%> = false;
<%
if("MIME".equals(mailType)) {

View File

@@ -69,6 +69,16 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
if (metadata!=null) {
List<IMetadataColumn> columns = metadata.getListColumns();
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
boolean protectFile = ("true").equals(ElementParameterParser.getValue(node, "__PROTECT_FILE__")) ? true : false;
if (protectFile) {
String passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
String password_<%=cid%> = decryptedPassword_<%=cid%>;
<%
}
boolean hasDynamic = false;
int dynamic_index = 0;
String datePattern = "\"dd-MM-yyyy\"";
@@ -416,13 +426,13 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
<%
}
}
%>
%>
writableSheet_<%=cid %> = writeableWorkbook_<%=cid %>.getSheet(<%=sheetname %>);
if(writableSheet_<%=cid %> == null){
writableSheet_<%=cid %> = writeableWorkbook_<%=cid %>.createSheet(<%=sheetname %>, writeableWorkbook_<%=cid %>.getNumberOfSheets());
}
<%if(!useStream && isAppendFile && !isAppendSheet){%>
else {
@@ -479,7 +489,7 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
}else{
%>
if(true){
throw new RuntimeException("Date pattern must be set for column <%=column.getLabel() %> in the schema of component <%=cid %>!");
throw new RuntimeException("Date pattern must be set for column <%=column.getLabel() %> in the schema of component <%=cid %>! ");
}
<% }
}
@@ -760,6 +770,9 @@ if(!hasDynamic && isIncludeHeader){
%>
int nb_line_<%=cid%> = 0;
org.talend.ExcelTool xlsxTool_<%=cid%> = new org.talend.ExcelTool();
<% if (protectFile) { %>
xlsxTool_<%=cid%>.setPasswordProtection(password_<%=cid%>);
<%}%>
<%
if(flushOnRow && (useStream || !isAppendFile)){
%>
@@ -780,7 +793,15 @@ if(!hasDynamic && isIncludeHeader){
java.util.concurrent.ConcurrentHashMap<java.lang.Object, java.lang.Object> chm_<%=cid%> = (java.util.concurrent.ConcurrentHashMap<java.lang.Object, java.lang.Object>) globalMap.get("concurrentHashMap");
java.lang.Object lockObj_<%=cid%> = chm_<%=cid%>.computeIfAbsent("EXCEL_OUTPUT_LOCK_OBJ_<%=cid%>", k -> new Object());
synchronized (lockObj_<%=cid%>) {
xlsxTool_<%=cid%>.prepareXlsxFile(fileName_<%=cid%>);
<%
String rowFileName = filename.replaceAll("\"", "");
String fileExtension = rowFileName.substring(rowFileName.lastIndexOf(".") + 1);
if (fileExtension.equalsIgnoreCase("xlsm")){
%>
xlsxTool_<%=cid%>.prepareXlsmFile(fileName_<%=cid%>);
<%} else {%>
xlsxTool_<%=cid%>.prepareXlsxFile(fileName_<%=cid%>);
<%}%>
}
<%}else{%>
xlsxTool_<%=cid%>.prepareStream();

View File

@@ -149,7 +149,14 @@
</ITEMS>
</PARAMETER>
<PARAMETER
<PARAMETER NAME="PROTECT_FILE" FIELD="CHECK" REQUIRED="false" SHOW_IF="(VERSION_2007 == 'true') AND (USESTREAM=='false')" NUM_ROW="72">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="PASSWORD" FIELD="PASSWORD" REQUIRED="true" SHOW_IF="isShow[PROTECT_FILE] AND (PROTECT_FILE == 'true')" NUM_ROW="72">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="SCHEMA"
FIELD="SCHEMA_TYPE"
REQUIRED="true"
@@ -220,7 +227,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java_Excel" MODULE="jxl.jar" MVN="mvn:org.talend.libraries/jxl/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.jexcel/lib/jxl.jar" REQUIRED_IF="(VERSION_2007 == 'false')" />
<IMPORT NAME="TALEND_EXCEL" MODULE="talendExcel-1.4-20190531.jar" MVN="mvn:org.talend.libraries/talendExcel-1.4-20190531/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/talendExcel-1.4-20190531.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="TALEND_EXCEL" MODULE="talendExcel-1.5-20190731.jar" MVN="mvn:org.talend.libraries/talendExcel-1.5-20190731/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/talendExcel-1.5-20190731.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="commons-lang3" MODULE="commons-lang3-3.8.1.jar" MVN="mvn:org.apache.commons/commons-lang3/3.8.1" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="dom4j" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar" REQUIRED_IF="(VERSION_2007 == 'true')" BundleID="" />
<IMPORT NAME="geronimo-stax-api" MODULE="geronimo-stax-api_1.0_spec-1.0.1.jar" MVN="mvn:org.talend.libraries/geronimo-stax-api_1.0_spec-1.0.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar" REQUIRED_IF="(VERSION_2007 == 'true')" BundleID="" />

View File

@@ -74,6 +74,8 @@ IS_ALL_AUTO_SZIE.NAME=Define all columns auto size
KEEP_LEADING_ZEROS.NAME=Keep leading zeros
LONG_NAME=Writes the cells row by row in a Microsoft Excel file
NB_LINE.NAME=Number of line
PROTECT_FILE.NAME=Protect file
PASSWORD.NAME=Password
SCHEMA.NAME=Schema
SHEETNAME.NAME=Sheet name
THOUSANDS_SEPARATOR.NAME=Thousands separator
@@ -84,8 +86,8 @@ FIRST_CELL_Y_ABSOLUTE.NAME=Is absolute Y pos.
KEEP_CELL_FORMATING.NAME=Keep existing cell format
USESTREAM.NAME=Use Output Stream
STREAMNAME.NAME=Output Stream
VERSION_2007.NAME=Write excel2007 file format(xlsx)
VERSION_2007.NAME=Write excel2007 file format (xlsx / xlsm)
FLUSHONROW.NAME=Custom the flush buffer size
FLUSHONROW_NUM.NAME=Row number
RECALCULATE_FORMULA.NAME=Recalculate formula
RECALCULATE_FORMULA.NAME=Recalculate formula

View File

@@ -185,14 +185,14 @@ imports="
%>
for (org.jets3t.service.model.GSObject objectSummary_<%=cid%> : objects_<%=cid%>) {
String objkey_<%=cid%> = objectSummary_<%=cid%>.getKey();
java.io.File file_<%=cid%> = new java.io.File(<%=outputsDir%>+objkey_<%=cid%>);
java.io.File file_<%=cid%> = new java.io.File(<%=outputsDir%>, objkey_<%=cid%>);
org.jets3t.service.model.GSObject obj_<%=cid%> = service_<%=cid%>.getObject(currentBucketName_<%=cid%>, objkey_<%=cid%>);
<%
}else{
%>
for (java.util.Map<String, String> map_<%=cid %>: list_<%=cid%>) {
String currentBucketName_<%=cid%>=map_<%=cid %>.get("BUCKET_NAME");
java.io.File file_<%=cid%> = new java.io.File(<%=outputsDir%>+map_<%=cid %>.get("OBJECT_NEWNAME"));
java.io.File file_<%=cid%> = new java.io.File(<%=outputsDir%>, map_<%=cid %>.get("OBJECT_NEWNAME"));
org.jets3t.service.model.GSObject obj_<%=cid%> =null;
try {
obj_<%=cid%> = service_<%=cid%>.getObject(currentBucketName_<%=cid%>, map_<%=cid %>.get("OBJECT_KEY"));
@@ -275,4 +275,4 @@ imports="
obj_<%=cid%>.closeDataInputStream();
<%
}
%>
%>

View File

@@ -52,6 +52,6 @@ SCHEMA.NAME=Schema
NOTE.NAME=Note
CONNECTION.NAME=Component List
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -17,5 +17,5 @@ AUTO_COMMIT.NAME = Auto Commit
TYPE.NAME=Database Driver
PROPERTY.NAME=Property Type
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later

View File

@@ -39,6 +39,6 @@ GUESS_SCHEMA.NAME=Guess Schema
CONNECTION.NAME=Component List
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -25,7 +25,7 @@ PORT.NAME=Port
TABLE.NAME=Table
FIELD_OPTIONS.NAME=Field options
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
SQL_SYNTAX.NAME=Sql Syntax
ADD_COLS.NAME=Additional columns

View File

@@ -50,5 +50,5 @@ STANDARD_CONFORMING_STRING.NAME=Use standard_conforming_string ON
PROPERTY_DB.NAME=Property Type
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later

View File

@@ -61,6 +61,6 @@ SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Time=Time
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Null=Null
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_VALUE=Parameter Value
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -71,6 +71,6 @@ DIE_ON_ERROR.NAME=Die on error
REJECT.LINK=Rejects
REJECT.MENU=Rejects
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -84,6 +84,6 @@ PROPERTY_DB.NAME=Property Type
CONNECTION.NAME=Component List
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -57,7 +57,7 @@ SCHEMA.NAME=Schema
NOTE.NAME=Note
CONNECTION.NAME=Component List
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
LOCAL_FILE.NAME=Use local file for copy (for DB server 8.2 or newer)
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -17,7 +17,7 @@ AUTO_COMMIT.NAME = Auto Commit
TYPE.NAME=Database Driver
PROPERTY.NAME=Property Type
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
SPECIFY_DATASOURCE_ALIAS.NAME=Specify a data source alias

View File

@@ -39,7 +39,7 @@ GUESS_SCHEMA.NAME=Guess Schema
CONNECTION.NAME=Component List
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
SPECIFY_DATASOURCE_ALIAS.NAME=Specify a data source alias

View File

@@ -69,6 +69,8 @@ skeleton="../templates/db_output_bulk.skeleton"
boolean savePoint = ("true").equals(ElementParameterParser.getValue(node,"__SAVE_POINT__"));
String dieOnError = ElementParameterParser.getValue(node, "__DIE_ON_ERROR__");
boolean convertToLowercase = ("true").equals(ElementParameterParser.getValue(node, "__CONVERT_COLUMN_TABLE_TO_LOWERCASE__"));
String rejectConnName = null;
List<? extends IConnection> rejectConns = node.getOutgoingConnections("REJECT");
@@ -98,6 +100,18 @@ List<Column> stmtStructure = null;
if(columnList != null && columnList.size() > 0) {
stmtStructure = getManager(dbmsId, cid).createColumnList(columnList, useFieldOptions, fieldOptions, addCols);
isDynamic = isDynamic && !getManager(dbmsId, cid).isDynamicColumnReplaced();
if(convertToLowercase){
for(Column column : stmtStructure) {
if(column.isReplaced()) {
for (Column replacedColumn : column.getReplacement()) {
replacedColumn.setColumnName(replacedColumn.getColumnName().toLowerCase());
}
} else if(!column.isDynamic()){
column.setColumnName(column.getColumnName().toLowerCase());
}
}
}
}
%>
@@ -116,9 +130,9 @@ if(("true").equals(useExistingConn)) {
String tableName_<%=cid%> = null;
if(dbschema_<%=cid%> == null || dbschema_<%=cid%>.trim().length() == 0) {
tableName_<%=cid%> = <%=table%>;
tableName_<%=cid%> = (<%=table%>)<%=convertToLowercase ? ".toLowerCase()" : ""%>;
} else {
tableName_<%=cid%> = dbschema_<%=cid%> + "\".\"" + <%=table%>;
tableName_<%=cid%> = dbschema_<%=cid%> + "\".\"" + (<%=table%>)<%=convertToLowercase ? ".toLowerCase()" : ""%>;
}
<%

View File

@@ -393,6 +393,14 @@
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="CONVERT_COLUMN_TABLE_TO_LOWERCASE"
FIELD="CHECK"
NUM_ROW="25"
>
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="USE_BATCH_SIZE"
FIELD="CHECK"

View File

@@ -49,6 +49,8 @@ skeleton="../templates/db_output_bulk.skeleton"
boolean useBatchSize = ("true").equals(ElementParameterParser.getValue(node,"__USE_BATCH_SIZE__"));
String batchSize=ElementParameterParser.getValue(node,"__BATCH_SIZE__");
boolean convertToLowercase = ("true").equals(ElementParameterParser.getValue(node, "__CONVERT_COLUMN_TABLE_TO_LOWERCASE__"));
//feature:22704
boolean savePoint = ("true").equals(ElementParameterParser.getValue(node,"__SAVE_POINT__"));
@@ -116,10 +118,14 @@ skeleton="../templates/db_output_bulk.skeleton"
for(Column colStmt:stmtStructure){
if (!colStmt.isReplaced()&&!colStmt.isAddCol()&&!colStmt.isDynamic()) {
colStruct.add(colStmt);
if (convertToLowercase)
colStmt.setColumnName(colStmt.getColumnName().toLowerCase());
} else if ( colStmt.isReplaced() ) {
List <Column> replacements = colStmt.getReplacement() ;
for (int i=0; i < replacements.size() ; i++) {
Column columnTest =columnTest = replacements.get(i);
if (convertToLowercase)
columnTest.setColumnName(columnTest.getColumnName().toLowerCase());
String sqlExpression = columnTest.getSqlStmt();
if (sqlExpression.contains("?")) {
colStruct.add(colStmt);
@@ -150,17 +156,31 @@ skeleton="../templates/db_output_bulk.skeleton"
StringBuilder updateWhereStmt = actionSQLMap.get(UPDATE_WHERE_STMT);
StringBuilder deleteWhereStmt = actionSQLMap.get(DELETE_WHERE_STMT);
talendDynCol = getDynamicColumn();
String insertColNameStr = insertColName.toString();
String updateSetStmtStr = updateSetStmt.toString();
String updateWhereStmtStr = updateWhereStmt.toString();
String deleteWhereStmtStr = deleteWhereStmt.toString();
if(("INSERT").equals(dataAction)) {
List< ? extends IConnection> conns_dynamic = node.getIncomingConnections();
if(conns_dynamic!=null && conns_dynamic.size()>0){
IConnection conn = conns_dynamic.get(0);
if(!("".equals(insertColName.toString()))) {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" (<%=insertColName.toString()%>, "+DynamicUtils.getInsertIntoStmtColumnsList(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+") VALUES (<%=insertValueStmt.toString()%>, "+DynamicUtils.getInsertIntoStmtValuesList(<%=conn.getName()%>.<%=getDynamicColumn()%>)+")";
routines.system.Dynamic dynamic_<%=cid%> = <%=conn.getName()%>.<%=getDynamicColumn()%>;
if (<%=convertToLowercase%>) {
for (int i = 0; i < dynamic_<%=cid%>.getColumnCount(); i++) {
routines.system.DynamicMetadata column = dynamic_<%=cid%>.getColumnMetadata(i);
column.setName(column.getName().toLowerCase());
}
}
<%
if(!("".equals(insertColNameStr))) {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" (<%=insertColNameStr%>, "+DynamicUtils.getInsertIntoStmtColumnsList(dynamic_<%=cid%>, "<%=dbmsId %>")+") VALUES (<%=insertValueStmt.toString()%>, "+DynamicUtils.getInsertIntoStmtValuesList(dynamic_<%=cid%>)+")";
<% } else {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" ("+DynamicUtils.getInsertIntoStmtColumnsList(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+") VALUES ("+DynamicUtils.getInsertIntoStmtValuesList(<%=conn.getName()%>.<%=getDynamicColumn()%>)+")";
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" ("+DynamicUtils.getInsertIntoStmtColumnsList(dynamic_<%=cid%>, "<%=dbmsId %>")+") VALUES ("+DynamicUtils.getInsertIntoStmtValuesList(dynamic_<%=cid%>)+")";
<%
}
}
@@ -188,12 +208,21 @@ skeleton="../templates/db_output_bulk.skeleton"
List< ? extends IConnection> conns_dynamic = node.getIncomingConnections();
if(conns_dynamic!=null && conns_dynamic.size()>0){
IConnection conn = conns_dynamic.get(0);
if(!("".equals(updateSetStmt.toString()))) {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET <%=updateSetStmt.toString()%>, "+DynamicUtils.getUpdateSet(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmt.toString()%>";
routines.system.Dynamic dynamic_<%=cid%> = <%=conn.getName()%>.<%=getDynamicColumn()%>;
if (<%=convertToLowercase%>) {
for (int i = 0; i < dynamic_<%=cid%>.getColumnCount(); i++) {
routines.system.DynamicMetadata column = dynamic_<%=cid%>.getColumnMetadata(i);
column.setName(column.getName().toLowerCase());
}
}
<%
if(!("".equals(updateSetStmtStr))) {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET <%=updateSetStmtStr%>, "+DynamicUtils.getUpdateSet(dynamic_<%=cid%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmtStr%>";
<% } else {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET "+DynamicUtils.getUpdateSet(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmt.toString()%>";
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET "+DynamicUtils.getUpdateSet(dynamic_<%=cid%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmtStr%>";
<%
}
}
@@ -205,25 +234,34 @@ skeleton="../templates/db_output_bulk.skeleton"
List< ? extends IConnection> conns_dynamic = node.getIncomingConnections();
if(conns_dynamic!=null && conns_dynamic.size()>0){
IConnection conn = conns_dynamic.get(0);
if(!("".equals(insertColName.toString()))) {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" (<%=insertColName.toString()%>, "+DynamicUtils.getInsertIntoStmtColumnsList(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+") VALUES (<%=insertValueStmt.toString()%>, "+DynamicUtils.getInsertIntoStmtValuesList(<%=conn.getName()%>.<%=getDynamicColumn()%>)+")";
routines.system.Dynamic dynamic_<%=cid%> = <%=conn.getName()%>.<%=getDynamicColumn()%>;
if (<%=convertToLowercase%>) {
for (int i = 0; i < dynamic_<%=cid%>.getColumnCount(); i++) {
routines.system.DynamicMetadata column = dynamic_<%=cid%>.getColumnMetadata(i);
column.setName(column.getName().toLowerCase());
}
}
<%
if(!("".equals(insertColNameStr))) {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" (<%=insertColNameStr%>, "+DynamicUtils.getInsertIntoStmtColumnsList(dynamic_<%=cid%>, "<%=dbmsId %>")+") VALUES (<%=insertValueStmt.toString()%>, "+DynamicUtils.getInsertIntoStmtValuesList(dynamic_<%=cid%>)+")";
<% } else {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" ("+DynamicUtils.getInsertIntoStmtColumnsList(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+") VALUES ("+DynamicUtils.getInsertIntoStmtValuesList(<%=conn.getName()%>.<%=getDynamicColumn()%>)+")";
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" ("+DynamicUtils.getInsertIntoStmtColumnsList(dynamic_<%=cid%>, "<%=dbmsId %>")+") VALUES ("+DynamicUtils.getInsertIntoStmtValuesList(dynamic_<%=cid%>)+")";
<%
}
if(!("".equals(updateSetStmt.toString()))) {
if(!("".equals(updateSetStmtStr))) {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET <%=updateSetStmt.toString()%>, "+DynamicUtils.getUpdateSet(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmt.toString()%>";
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET <%=updateSetStmtStr%>, "+DynamicUtils.getUpdateSet(dynamic_<%=cid%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmtStr%>";
<% } else {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET "+DynamicUtils.getUpdateSet(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmt.toString()%>";
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET "+DynamicUtils.getUpdateSet(dynamic_<%=cid%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmtStr%>";
<%
}
}
%>
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement("SELECT COUNT(1) FROM \"" + tableName_<%=cid%> + "\" WHERE <%=updateWhereStmt.toString()%>");
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement("SELECT COUNT(1) FROM \"" + tableName_<%=cid%> + "\" WHERE <%=updateWhereStmtStr%>");
resourceMap.put("pstmt_<%=cid %>", pstmt_<%=cid %>);
pstmtInsert_<%=cid %> = conn_<%=cid%>.prepareStatement(insert_<%=cid%>);
resourceMap.put("pstmtInsert_<%=cid %>", pstmtInsert_<%=cid %>);
@@ -234,20 +272,29 @@ skeleton="../templates/db_output_bulk.skeleton"
List< ? extends IConnection> conns_dynamic = node.getIncomingConnections();
if(conns_dynamic!=null && conns_dynamic.size()>0){
IConnection conn = conns_dynamic.get(0);
if(!("".equals(insertColName.toString()))) {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" (<%=insertColName.toString()%>, "+DynamicUtils.getInsertIntoStmtColumnsList(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+") VALUES (<%=insertValueStmt.toString()%>, "+DynamicUtils.getInsertIntoStmtValuesList(<%=conn.getName()%>.<%=getDynamicColumn()%>)+")";
routines.system.Dynamic dynamic_<%=cid%> = <%=conn.getName()%>.<%=getDynamicColumn()%>;
if (<%=convertToLowercase%>) {
for (int i = 0; i < dynamic_<%=cid%>.getColumnCount(); i++) {
routines.system.DynamicMetadata column = dynamic_<%=cid%>.getColumnMetadata(i);
column.setName(column.getName().toLowerCase());
}
}
<%
if(!("".equals(insertColNameStr))) {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" (<%=insertColNameStr%>, "+DynamicUtils.getInsertIntoStmtColumnsList(dynamic_<%=cid%>, "<%=dbmsId %>")+") VALUES (<%=insertValueStmt.toString()%>, "+DynamicUtils.getInsertIntoStmtValuesList(dynamic_<%=cid%>)+")";
<% } else {
%>
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" ("+DynamicUtils.getInsertIntoStmtColumnsList(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+") VALUES ("+DynamicUtils.getInsertIntoStmtValuesList(<%=conn.getName()%>.<%=getDynamicColumn()%>)+")";
String insert_<%=cid%> = "INSERT INTO \"" + tableName_<%=cid%> + "\" ("+DynamicUtils.getInsertIntoStmtColumnsList(dynamic_<%=cid%>, "<%=dbmsId %>")+") VALUES ("+DynamicUtils.getInsertIntoStmtValuesList(dynamic_<%=cid%>)+")";
<%
}
if(!("".equals(updateSetStmt.toString()))) {
if(!("".equals(updateSetStmtStr))) {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET <%=updateSetStmt.toString()%>, "+DynamicUtils.getUpdateSet(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmt.toString()%>";
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET <%=updateSetStmtStr%>, "+DynamicUtils.getUpdateSet(dynamic_<%=cid%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmtStr%>";
<% } else {
%>
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET "+DynamicUtils.getUpdateSet(<%=conn.getName()%>.<%=getDynamicColumn()%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmt.toString()%>";
String update_<%=cid%> = "UPDATE \"" + tableName_<%=cid%> + "\" SET "+DynamicUtils.getUpdateSet(dynamic_<%=cid%>, "<%=dbmsId %>")+" WHERE <%=updateWhereStmtStr%>";
<%
}
}
@@ -259,7 +306,7 @@ skeleton="../templates/db_output_bulk.skeleton"
<%
} else if (("DELETE").equals(dataAction)) {
%>
String delete_<%=cid%> = "DELETE FROM \"" + tableName_<%=cid%> + "\" WHERE <%=deleteWhereStmt.toString()%>";
String delete_<%=cid%> = "DELETE FROM \"" + tableName_<%=cid%> + "\" WHERE <%=deleteWhereStmtStr%>";
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(delete_<%=cid%>);
resourceMap.put("pstmt_<%=cid %>", pstmt_<%=cid %>);
<%

View File

@@ -91,11 +91,13 @@ BATCH_SIZE.NAME= Batch Size
SAVE_POINT.NAME= Use save point
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
NB_LINE_REJECTED.NAME=Number Of Rejected Lines
SPECIFY_DATASOURCE_ALIAS.NAME=Specify a data source alias
DATASOURCE.NAME=Data source
DATASOURCE_ALIAS.NAME=Data source alias
PROPERTIES.NAME=Additional JDBC Parameters
PROPERTIES.NAME=Additional JDBC Parameters
CONVERT_COLUMN_TABLE_TO_LOWERCASE.NAME=Convert columns and table to lowercase

View File

@@ -51,7 +51,7 @@ STANDARD_CONFORMING_STRING.NAME=Use standard_conforming_string ON
PROPERTY_DB.NAME=Property Type
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
LOCAL_FILE.NAME=Use local file for copy (for DB server 8.2 or newer)
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -61,7 +61,7 @@ SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Time=Time
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Null=Null
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_VALUE=Parameter Value
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
SPECIFY_DATASOURCE_ALIAS.NAME=Specify a data source alias

View File

@@ -71,6 +71,6 @@ DIE_ON_ERROR.NAME=Die on error
REJECT.LINK=Rejects
REJECT.MENU=Rejects
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -84,6 +84,6 @@ PROPERTY_DB.NAME=Property Type
CONNECTION.NAME=Component List
MAPPING.NAME=Mapping
DB_VERSION.NAME=DB Version
DB_VERSION.ITEM.PRIOR_TO_V9=Prior to v9
DB_VERSION.ITEM.PRIOR_TO_V9=v7.2 to v8.X
DB_VERSION.ITEM.V9_X=v9 and later
PROPERTIES.NAME=Additional JDBC Parameters

View File

@@ -134,12 +134,12 @@
<!-- S3 -->
<PARAMETER NAME="ACCESS_KEY" FIELD="TEXT" NUM_ROW="50"
REQUIRED="true" GROUP="S3_CONFIG" NOT_SHOW_IF="isShow[ASSUME_ROLE] AND (ASSUME_ROLE == 'true')">
REQUIRED="true" GROUP="S3_CONFIG" SHOW_IF="(DATA_SOURCE_TYPE == 'S3') AND (ASSUME_ROLE == 'false')">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="SECRET_KEY" FIELD="PASSWORD" NUM_ROW="50"
REQUIRED="true" GROUP="S3_CONFIG" NOT_SHOW_IF="isShow[ASSUME_ROLE] AND (ASSUME_ROLE == 'true')">
REQUIRED="true" GROUP="S3_CONFIG" SHOW_IF="(DATA_SOURCE_TYPE == 'S3') AND (ASSUME_ROLE == 'false')">
<DEFAULT>""</DEFAULT>
</PARAMETER>
@@ -419,7 +419,7 @@
UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-logging-1.1.3.jar"
REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="commons-codec-1.6.jar" MODULE="commons-codec-1.6.jar"
MVN="mvn:org.talend.libraries/commons-codec-1.6/6.0.0"
MVN="mvn:commons-codec/commons-codec/1.6"
UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-codec-1.6.jar"
REQUIRED="true" />
</IMPORTS>

View File

@@ -207,6 +207,7 @@
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar" MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9" REQUIRED_IF="(JDBC_URL == 'SSO')" />
<IMPORT NAME="httpclient-4.5.5.jar" MODULE="httpclient-4.5.5.jar" MVN="mvn:org.apache.httpcomponents/httpclient/4.5.5" REQUIRED_IF="(JDBC_URL == 'SSO')" />
<IMPORT NAME="joda-time-2.8.1.jar" MODULE="joda-time-2.8.1.jar" MVN="mvn:joda-time/joda-time/2.8.1" REQUIRED_IF="(JDBC_URL == 'SSO')" />
<IMPORT NAME="commons-codec-1.6.jar" MODULE="commons-codec-1.6.jar" MVN="mvn:commons-codec/commons-codec/1.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-codec-1.6.jar" REQUIRED_IF="(JDBC_URL == 'SSO')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -288,6 +288,7 @@
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar" MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="httpclient-4.5.5.jar" MODULE="httpclient-4.5.5.jar" MVN="mvn:org.apache.httpcomponents/httpclient/4.5.5" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="joda-time-2.8.1.jar" MODULE="joda-time-2.8.1.jar" MVN="mvn:joda-time/joda-time/2.8.1" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="commons-codec-1.6.jar" MODULE="commons-codec-1.6.jar" MVN="mvn:commons-codec/commons-codec/1.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-codec-1.6.jar" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -378,6 +378,7 @@
<IMPORT NAME="httpcore-4.4.9.jar" MODULE="httpcore-4.4.9.jar" MVN="mvn:org.apache.httpcomponents/httpcore/4.4.9" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="httpclient-4.5.5.jar" MODULE="httpclient-4.5.5.jar" MVN="mvn:org.apache.httpcomponents/httpclient/4.5.5" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="joda-time-2.8.1.jar" MODULE="joda-time-2.8.1.jar" MVN="mvn:joda-time/joda-time/2.8.1" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="commons-codec-1.6.jar" MODULE="commons-codec-1.6.jar" MVN="mvn:commons-codec/commons-codec/1.6" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-codec-1.6.jar" REQUIRED_IF="(JDBC_URL == 'SSO') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -146,20 +146,28 @@
GROUP="S3_CONFIG">
<ITEMS DEFAULT="DEFAULT">
<ITEM NAME="DEFAULT" VALUE="DEFAULT" />
<ITEM NAME="AP_SOUTH_1" VALUE="&quot;ap-south-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_1" VALUE="&quot;ap-southeast-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_2" VALUE="&quot;ap-southeast-2&quot;" />
<ITEM NAME="AP_NORTHEAST_1" VALUE="&quot;ap-northeast-1&quot;" />
<ITEM NAME="CN_NORTH_1" VALUE="&quot;cn-north-1&quot;" />
<ITEM NAME="EU_WEST_1" VALUE="&quot;eu-west-1&quot;" />
<ITEM NAME="EU_CENTRAL_1" VALUE="&quot;eu-central-1&quot;" />
<ITEM NAME="GovCloud" VALUE="&quot;us-gov-west-1&quot;" />
<ITEM NAME="SA_EAST_1" VALUE="&quot;sa-east-1&quot;" />
<ITEM NAME="US_EAST_1" VALUE="&quot;us-east-1&quot;" />
<ITEM NAME="US_WEST_1" VALUE="&quot;us-west-1&quot;" />
<ITEM NAME="US_WEST_2" VALUE="&quot;us-west-2&quot;" />
</ITEMS>
<ITEM NAME="DEFAULT" VALUE="DEFAULT" />
<ITEM NAME="GovCloud" VALUE="&quot;us-gov-west-1&quot;" />
<ITEM NAME="US_EAST_1" VALUE="&quot;us-east-1&quot;" />
<ITEM NAME="US_WEST_1" VALUE="&quot;us-west-1&quot;" />
<ITEM NAME="US_WEST_2" VALUE="&quot;us-west-2&quot;" />
<ITEM NAME="US_EAST_2" VALUE="&quot;us-east-2&quot;" />
<ITEM NAME="SA_EAST_1" VALUE="&quot;sa-east-1&quot;" />
<ITEM NAME="EU_WEST_1" VALUE="&quot;eu-west-1&quot;" />
<ITEM NAME="EU_CENTRAL_1" VALUE="&quot;eu-central-1&quot;" />
<ITEM NAME="EU_WEST_2" VALUE="&quot;eu-west-2&quot;" />
<ITEM NAME="EU_WEST_3" VALUE="&quot;eu-west-3&quot;" />
<ITEM NAME="EU_NORTH_1" VALUE="&quot;eu-north-1&quot;" />
<ITEM NAME="CN_NORTH_1" VALUE="&quot;cn-north-1&quot;" />
<ITEM NAME="CN_NORTHWEST_1" VALUE="&quot;cn-northwest-1&quot;" />
<ITEM NAME="AP_EAST_1" VALUE="&quot;ap-east-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_1" VALUE="&quot;ap-southeast-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_2" VALUE="&quot;ap-southeast-2&quot;" />
<ITEM NAME="AP_NORTHEAST_1" VALUE="&quot;ap-northeast-1&quot;" />
<ITEM NAME="AP_NORTHEAST_2" VALUE="&quot;ap-northeast-2&quot;" />
<ITEM NAME="AP_SOUTH_1" VALUE="&quot;ap-south-1&quot;" />
<ITEM NAME="CA_CENTRAL_1" VALUE="&quot;ca-central-1&quot;" />
</ITEMS>
</PARAMETER>
<PARAMETER

View File

@@ -66,14 +66,22 @@ REGION.ITEM.GovCloud=AWS GovCloud (US)
REGION.ITEM.US_EAST_1=US Standard
REGION.ITEM.US_WEST_1=US West (N. California)
REGION.ITEM.US_WEST_2=US West (Oregon)
REGION.ITEM.US_EAST_2=US East (Ohio)
REGION.ITEM.SA_EAST_1=South America (Sao Paulo)
REGION.ITEM.EU_WEST_1=EU (Ireland)
REGION.ITEM.EU_CENTRAL_1=EU (Frankfurt)
REGION.ITEM.CN_NORTH_1=China (Beijing)
REGION.ITEM.EU_WEST_2=EU (London)
REGION.ITEM.EU_WEST_3=EU (Paris)
REGION.ITEM.EU_NORTH_1=EU (Stockholm)
REGION.ITEM.CN_NORTH_1=China (Beijing)
REGION.ITEM.CN_NORTHWEST_1=China (Ningxia)
REGION.ITEM.AP_EAST_1=Asia Pacific (Hong Kong)
REGION.ITEM.AP_SOUTHEAST_1=Asia Pacific (Singapore)
REGION.ITEM.AP_SOUTHEAST_2=Asia Pacific (Sydney)
REGION.ITEM.AP_NORTHEAST_1=Asia Pacific (Tokyo)
REGION.ITEM.AP_NORTHEAST_2=Asia Pacific (Seoul)
REGION.ITEM.AP_SOUTH_1=Asia Pacific (Mumbai)
REGION.ITEM.SA_EAST_1=South America (Sao Paulo)
REGION.ITEM.CA_CENTRAL_1=Canada (Central)
INHERIT_CREDENTIALS.NAME=Inherit credentials from AWS role

View File

@@ -228,20 +228,28 @@
GROUP="S3_CONFIG">
<ITEMS DEFAULT="DEFAULT">
<ITEM NAME="DEFAULT" VALUE="DEFAULT" />
<ITEM NAME="AP_SOUTH_1" VALUE="&quot;ap-south-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_1" VALUE="&quot;ap-southeast-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_2" VALUE="&quot;ap-southeast-2&quot;" />
<ITEM NAME="AP_NORTHEAST_1" VALUE="&quot;ap-northeast-1&quot;" />
<ITEM NAME="CN_NORTH_1" VALUE="&quot;cn-north-1&quot;" />
<ITEM NAME="EU_WEST_1" VALUE="&quot;eu-west-1&quot;" />
<ITEM NAME="EU_CENTRAL_1" VALUE="&quot;eu-central-1&quot;" />
<ITEM NAME="GovCloud" VALUE="&quot;us-gov-west-1&quot;" />
<ITEM NAME="SA_EAST_1" VALUE="&quot;sa-east-1&quot;" />
<ITEM NAME="US_EAST_1" VALUE="&quot;us-east-1&quot;" />
<ITEM NAME="US_WEST_1" VALUE="&quot;us-west-1&quot;" />
<ITEM NAME="US_WEST_2" VALUE="&quot;us-west-2&quot;" />
</ITEMS>
<ITEM NAME="DEFAULT" VALUE="DEFAULT" />
<ITEM NAME="GovCloud" VALUE="&quot;us-gov-west-1&quot;" />
<ITEM NAME="US_EAST_1" VALUE="&quot;us-east-1&quot;" />
<ITEM NAME="US_WEST_1" VALUE="&quot;us-west-1&quot;" />
<ITEM NAME="US_WEST_2" VALUE="&quot;us-west-2&quot;" />
<ITEM NAME="US_EAST_2" VALUE="&quot;us-east-2&quot;" />
<ITEM NAME="SA_EAST_1" VALUE="&quot;sa-east-1&quot;" />
<ITEM NAME="EU_WEST_1" VALUE="&quot;eu-west-1&quot;" />
<ITEM NAME="EU_CENTRAL_1" VALUE="&quot;eu-central-1&quot;" />
<ITEM NAME="EU_WEST_2" VALUE="&quot;eu-west-2&quot;" />
<ITEM NAME="EU_WEST_3" VALUE="&quot;eu-west-3&quot;" />
<ITEM NAME="EU_NORTH_1" VALUE="&quot;eu-north-1&quot;" />
<ITEM NAME="CN_NORTH_1" VALUE="&quot;cn-north-1&quot;" />
<ITEM NAME="CN_NORTHWEST_1" VALUE="&quot;cn-northwest-1&quot;" />
<ITEM NAME="AP_EAST_1" VALUE="&quot;ap-east-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_1" VALUE="&quot;ap-southeast-1&quot;" />
<ITEM NAME="AP_SOUTHEAST_2" VALUE="&quot;ap-southeast-2&quot;" />
<ITEM NAME="AP_NORTHEAST_1" VALUE="&quot;ap-northeast-1&quot;" />
<ITEM NAME="AP_NORTHEAST_2" VALUE="&quot;ap-northeast-2&quot;" />
<ITEM NAME="AP_SOUTH_1" VALUE="&quot;ap-south-1&quot;" />
<ITEM NAME="CA_CENTRAL_1" VALUE="&quot;ca-central-1&quot;" />
</ITEMS>
</PARAMETER>
<PARAMETER NAME="BUCKET" FIELD="TEXT" NUM_ROW="95" REQUIRED="true"
@@ -538,7 +546,7 @@
UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-logging-1.1.3.jar"
REQUIRED="true" />
<IMPORT NAME="commons-codec-1.6.jar" MODULE="commons-codec-1.6.jar"
MVN="mvn:org.talend.libraries/commons-codec-1.6/6.0.0"
MVN="mvn:commons-codec/commons-codec/1.6"
UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-codec-1.6.jar"
REQUIRED="true" />
</IMPORTS>

View File

@@ -135,14 +135,22 @@ REGION.ITEM.GovCloud=AWS GovCloud (US)
REGION.ITEM.US_EAST_1=US Standard
REGION.ITEM.US_WEST_1=US West (N. California)
REGION.ITEM.US_WEST_2=US West (Oregon)
REGION.ITEM.US_EAST_2=US East (Ohio)
REGION.ITEM.SA_EAST_1=South America (Sao Paulo)
REGION.ITEM.EU_WEST_1=EU (Ireland)
REGION.ITEM.EU_CENTRAL_1=EU (Frankfurt)
REGION.ITEM.CN_NORTH_1=China (Beijing)
REGION.ITEM.EU_WEST_2=EU (London)
REGION.ITEM.EU_WEST_3=EU (Paris)
REGION.ITEM.EU_NORTH_1=EU (Stockholm)
REGION.ITEM.CN_NORTH_1=China (Beijing)
REGION.ITEM.CN_NORTHWEST_1=China (Ningxia)
REGION.ITEM.AP_EAST_1=Asia Pacific (Hong Kong)
REGION.ITEM.AP_SOUTHEAST_1=Asia Pacific (Singapore)
REGION.ITEM.AP_SOUTHEAST_2=Asia Pacific (Sydney)
REGION.ITEM.AP_NORTHEAST_1=Asia Pacific (Tokyo)
REGION.ITEM.AP_NORTHEAST_2=Asia Pacific (Seoul)
REGION.ITEM.AP_SOUTH_1=Asia Pacific (Mumbai)
REGION.ITEM.SA_EAST_1=South America (Sao Paulo)
REGION.ITEM.CA_CENTRAL_1=Canada (Central)
INHERIT_CREDENTIALS.NAME=Inherit credentials from AWS role

Some files were not shown because too many files have changed in this diff Show More