Compare commits

...

86 Commits

Author SHA1 Message Date
dmytroshekera
a85bfe1fab fix(TPS-3404):An error occurs on tFileOutputExcel after Iterate when /tmp/profiles folder does not exist
* fix(TDI-42092)
2019-09-20 15:21:12 +03:00
dmytroshekera
f3b36e509c fix(TPS-3404):An error occurs on tFileOutputExcel after Iterate when /tmp/profiles folder does not exist
* fix(TDI-42092)
2019-09-19 18:09:24 +03:00
dmytroshekera
ce91c8f373 fix(TDI-42092): An error occurs on tFileOutputExcel after Iterate when /tmp/profiles folder does not exist (#3419) (#3937)
* fix(TDI-42092): An error occurs on tFileOutputExcel after
* Iterate when /tmp/profiles folder does not exist.
* reproduce in multithread env - each thread try create
* folder. First thread create. Second crash because folder exist
2019-09-18 14:37:41 +03:00
Denis Sergent
6b780bc24a TESB-26772 Adds a way to externalise osgi-exclude.properties file (#3859)
This dev adds the opportunity to externalise osgi-exclude.properties
resource file from org.talend.repository bundle better than using the
one included in org.talend.repository budle jar file.
2019-08-22 14:02:18 +02:00
apoltavtsev
22c1e8ad07 TESB-26293 Duplicated libraries in private & import packages of the build manifest file 2019-08-20 12:21:40 +03:00
apoltavtsev
8c2d941b22 TESB-25164 Intermittent "ClassNotFoundException: javax.mail.Address" errors 2019-08-20 12:10:22 +03:00
wang wei
26b1bd2cc5 fix(TDI-42705): tSAPBwInput component gives compilation error (#3739) 2019-07-31 09:59:59 +08:00
slushatel
03f1650e41 feat(TDI-42570): stop process if parent thread has terminated, add log wrapper for tos studio (#3715)
(cherry picked from commit b4196f3ade)
2019-07-15 15:58:05 +03:00
slushatel
cba4a8f3e0 Tdi 42570 stop process when thread stops (#3676)
* feat(TDI-42570): stop process if parent thread has terminated

* feat(TDI-42570): stop process if parent thread has terminated

* feat(TDI-42570): stop process if parent thread has terminated. improve indentation

* feat(TDI-42570): stop process if parent thread has terminated. improve indentation

(cherry picked from commit 9e8d8785c2)
2019-07-11 15:05:38 +03:00
Jane Ding
b35a54fc89 fix(TUP-23663)The connection row between components is getting activated (#3687) (#3695)
* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663

* fix(TUP-23663)The connection row between components is getting activated
unexpectedly when using tMSXMLInput component
https://jira.talendforge.org/browse/TUP-23663
2019-07-09 18:50:03 +08:00
Irene Wang
be20e84c8b Revert "DEVOPS-3416 Deploy all plugins for Black Duck scans"
This reverts commit 1ce0e429ca.
2019-06-28 12:17:19 +02:00
kjwang-talend
29147e644c TUP-22955 java.lang.ClassNotFoundException: org.gjt.mm.mysql.Driver (Fix can't guess query problem) (#3607)
* TUP-22955 java.lang.ClassNotFoundException: org.gjt.mm.mysql.Driver (Fix
can't guess query problem)
https://jira.talendforge.org/browse/TUP-22955

* TUP-22955 java.lang.ClassNotFoundException: org.gjt.mm.mysql.Driver (Fix
can't guess query problem)
https://jira.talendforge.org/browse/TUP-22955
2019-06-14 15:39:18 +08:00
vyu-talend
9764daa1cf Fix(TDI-42407):Fix the issue of lose data. 2019-05-31 18:31:57 +08:00
wchen-talend
abe8ea8ce4 fix(TUP-22955):java.lang.ClassNotFoundException: org.gjt.mm.mysql.Driver (#3526) (#3533) 2019-05-31 14:26:37 +08:00
jzhao
7603fbbb1b fix(TDI-42356):ClassCastException with dynamic schema and data source
alias from table with column of type nvarchar(max) (#3504)

* fix(TDI-42356):ClassCastException with dynamic schema and data source
alias from table with column of type nvarchar(max)

* TalendNTextImpl from talend lib to a patched jtds
* Add maven project to build patched jtds with TalendNTextImpl
* update talend lib talend_DB_mssqlUtil
* update depends on component side

* fix(TDI-42356):exclude jtds jar in osgi bundle

* fix(TDI-42356):fix missing changes for stats and logs with database
config

* fix(TDI-42356):change the GAV of build jar and maven path in
components

* fix(TDI-42356):change the version of jar
2019-05-31 10:47:15 +08:00
Jane Ding
82ec752f49 fix(TUP-22147)There is a NPE error when closing studio (#3443) (#3510)
* fix(TUP-22147)There is a NPE error when closing studio

* fix(TUP-22147)There is a NPE error when closing studio

Conflicts:
	main/plugins/org.talend.designer.runprocess/src/main/java/org/talend/designer/runprocess/java/TalendJavaProjectManager.java
2019-05-24 15:38:30 +08:00
vyu-talend
6691c25dfd Fix(TDI-42300):Make variable name unique 2019-05-10 11:04:52 +08:00
Carlos Carvajal
25a8ad0006 fix(TBD-8566): correct context var replacement in subqueries (#3420) (#3461) 2019-05-09 15:12:23 +02:00
wang wei
3cc382729f fix(TDI-42239): tLogCatcher and tAssertCatcher cause compilation error when used together(#3432) 2019-05-06 14:01:02 +08:00
hwang-talend
4203f4e155 Hwang/tup 22649 (#3413) (#3439)
bugfix(TUP-22649):tELTOracleMap : wrong Generated SQL Select query after
patch TPS-2944
2019-04-29 16:24:22 +08:00
wwang-talend
a5ff7e4a70 Revert "fix(TDI-41894): Randomly the job fails with "java.security.SignatureException: Invalid encoding for signature" error. (#3323)"
This reverts commit b5881ebf3e.
2019-04-22 11:45:03 +08:00
wang wei
4aefce88e7 fix(TDI-42141): tReplacelist not replacing in order (#3398) 2019-04-22 11:08:59 +08:00
Zhiwei Xue
df79455195 fix(TUP-22619):Miss item files in final zip when publishing job in (#3389)
commandline
2019-04-11 17:21:52 +08:00
wang wei
b5881ebf3e fix(TDI-41894): Randomly the job fails with "java.security.SignatureException: Invalid encoding for signature" error. (#3323) 2019-04-11 10:40:52 +08:00
hcyi
0a5b0f66ab fix(TUP-22521):tHiveMap- Unexpected SQL syntax for 'case when (#3374)
...then....else' condition.
2019-04-08 15:26:32 +08:00
Jane Ding
8b0bc633e9 fix(TUP-21920)Nullpointer on build job (#3336) (#3370)
https://jira.talendforge.org/browse/TUP-21920
Loop dependency classpath issue caused main class not found
2019-04-03 16:20:15 +08:00
wang wei
1b4dd545cc fix(TDI-42072): NullPointerException when using the tDie component in the child job and execute the job from the parent job.(#3352) 2019-04-02 16:21:14 +08:00
Jane Ding
16bad7bf5b fix(TUP-22342)Joblet referring to objects that do not exist (#3355) (#3360)
https://jira.talendforge.org/browse/TUP-22342
2019-04-02 15:33:13 +08:00
Dmytro Sylaiev
d0282cc782 fix(TDI-42024): Change execProt and execPBSZ order (#3333) 2019-04-01 13:07:25 +03:00
wang wei
d0758cdbb0 fix(TDI-41898): Comment in the query in tJDBCInput component does not work if the job is migrated from 6.2.1 to 7.1.1(#3334) 2019-03-28 14:20:48 +08:00
apoltavtsev
4721797b9f Typo removed 2019-03-21 09:49:20 +03:00
apoltavtsev
7c1871e54b TESB-24730 Services no longer accessible ( NPE) after route deployment 2019-03-21 09:44:00 +03:00
chmyga
4c64213134 fix(TDI-41951): special chars in column comment (#3318)
* Use text escape method usage in SPSOutput and SalesforceWaveBulkExec components
2019-03-19 14:49:34 +02:00
kjwang-talend
af41a8e0c8 kjwang/Fix-TUP-21903 tSalesForceConnection fix junit (#3314) (#3315)
https://jira.talendforge.org/browse/TUP-21903
2019-03-15 14:59:11 +08:00
kjwang-talend
51f2d1572f fix(TUP-21903):tSalesForceConnection JWT flow does not work with Context (#3254) (#3311)
* fix(TUP-21903):tSalesForceConnection JWT flow does not work with Context
group variable set for connection type Oauth.
2019-03-13 16:34:38 +08:00
pyzhou
e000cf43bd fix(TDI-41789): Redshift change migration logic (#3304)
Conflicts:
	main/plugins/org.talend.repository/plugin.xml
2019-03-12 17:08:10 +08:00
Dmytro Sylaiev
03d23179e4 fix(TDI-41897): Fix compile error on generated code (#3294) 2019-03-07 15:01:36 +02:00
Dmytro Sylaiev
c8ba05ff57 fix(TDI-41808): Fix showing pass as clear text NZLoad (#3269)
* fix(TDI-41808): Fix showing pass as clear text NZLoad

* fix(TDI-41808): Rewrite fix to be more efficient

* chore(TDI-41808): Fix indentation

* fix(TDI-41808): Avoid to show real length of password

* fix(TDI-41808): Use constant from util class
2019-03-07 14:30:15 +02:00
Jane Ding
205a198148 Jding/backport70/tup 21920 nullpointer on build job (#3305)
* fix(TUP-21920) Nullpointer on build job (#3266)

* fix(TUP-21920)Nullpointer on build job
https://jira.talendforge.org/browse/TUP-21920
2019-03-07 18:34:32 +08:00
hwang-talend
c4ce8a8996 bugfix(TUP-21944):Problem with globalMap.get in tELTOracleOutput #3261 (#3302)
bugfix(TUP-21944):Problem with globalMap.get in tELTOracleOutput
2019-03-07 11:38:59 +08:00
hcyi
49fa541f79 fix(TUP-21777):[7.0.1] Metadata Wizard with SSL cannot retrieve cluster (#3296) (#3299)
settings.
2019-03-07 10:29:28 +08:00
hwang-talend
12e1588c2b bugfix(TUP-21944):Problem with globalMap.get in tELTOracleOutput (#3264)
bugfix(TUP-21944):Problem with globalMap.get in tELTOracleOutput component
2019-03-04 11:15:54 +08:00
Emmanuel GALLOIS
887a7d7527 fix(TDI-41795): migrate correctly connection and linked connection (#3251)
* fix(TDI-41795): migrate correctly connection and linked connection
* fix(TDI-41795): fix invalid char in properties files
* chore(TDI-41795): do net set value at each property, just once
* fix(TDI-41795): correct NPE for tGoogleDriveCopy/Put
* fix(TDI-41795): do not corrupt tcompv0 components
  - fix migration version and breaks
  - do not execute migration for tcompv0 components

(cherry picked from commit c23955d51f)
2019-02-26 09:50:42 +01:00
pyzhou-talend
0114340dac fix(TDI-41789):Version of migration task tRedshift. 2019-02-26 15:26:53 +08:00
pyzhou
8befa6eb84 Pyzhou/TDI-41789 t redshift fixed log path (#3260)
* fix(TDI-41789): tRedshift require log file path

* redirect log to console

* change tab to space

Conflicts:
	main/plugins/org.talend.repository/plugin.xml

Conflicts:
	main/plugins/org.talend.designer.components.localprovider/components/tRedshiftConnection/tRedshiftConnection_begin.javajet
	main/plugins/org.talend.designer.components.localprovider/components/tRedshiftInput/tRedshiftInput_begin.javajet
2019-02-26 15:21:50 +08:00
apoltavtsev
32f840177e TESB-24784 Propagation of Connection extra properties is added (#3247)
* TESB-24784 Propagation of Connection extra properties is added

* Propagation of Connection extra properties is added
2019-02-18 08:56:02 +01:00
apoltavtsev
07973a694e TESB-22085:Dynamic schema column type is always String in Runtime 2019-02-11 11:54:22 +03:00
hwang-talend
c7227d94ed Hwang/tup 21868 (#3205)
* bugfix(TUP-20479):ELT Maps generate incorrect SQL

* bugfix(TUP-21164):Context is not recognized as context variable in ELT
#2984
2019-01-31 11:11:28 +08:00
Tao Tao
7a8b88603a Ttao/backport 7.0/tup 20927 automatic scrolling in execution console log (#3207)
* fix(TUP-20927)Automatic scrolling in execution console prevent upper log (#2845)

* fix(TUP-20927)Automatic scrolling in execution console prevent upper log
messages checking
https://jira.talendforge.org/browse/TUP-20927

*     fix(TUP-20927)Automatic scrolling in execution console prevent upper log messages checking
    https://jira.talendforge.org/browse/TUP-20927

* Fixed(TUP-20927)Automatic scrolling in execution console prevent upper log messages checking (#2890)

https://jira.talendforge.org/browse/TUP-20927

* fix(TUP-20927)Automatic scrolling in execution console prevent upper log (#2929)

messages checking

* Fix(TUP-21161)can't use the up-arrow button or scroll bar to locate at (#2959)

the top when the gap is too small

https://jira.talendforge.org/browse/TUP-21161

* bugfix(TUP-21336): Run view console result is not synchronized when I (#3067)

switch different job on job editor

https://jira.talendforge.org/browse/TUP-21336
2019-01-30 11:06:42 +08:00
Dmytro Sylaiev
3eb4eb9038 fix(TDI-41678): Upgrade custom lib version to avoid compile error (#3202)
* fix(TDI-41678): Upgrade custom lib version to avoid compile error

* fix(TDI-41678): Make timestamp-version
2019-01-28 14:22:43 +02:00
Zhiwei Xue
058c2fa5ee fix(TUP-21532):re-fix child assembly problem. (#3181) (#3184) 2019-01-22 14:28:34 +08:00
hwang-talend
ddca2a0972 Hwang/tup 21164 (#3004) (#3011) (#3173)
* bugfix(TUP-21164):Context is not recognized as context variable in ELT
Map components and is not separated with string concat from SQL code
2019-01-17 16:03:50 +08:00
Dmytro Sylaiev
1b0b6132b6 feat(TDI-40990): Add http proxy support for ftps (#3080)
* feat(TDI-40990): Add http proxy support for ftps

* Avoid set PROT and PBSZ commands

* feat(TDI-40990): Allign using proxy marker

* feat(TDI-40990): Remove SSLReuse class from template

* feat(TDI-40990): Use correct constructor

* feat(TDI-40990): Bound FTP components to proxy-support ftps library

* feat(TDI-40990): Make talend-commons-net library included to studio

* Bound all ftp components except tFTPConnection to UrlPath

* feat(TDI-40990): Use HTTP proxy when it's set for FTPS

* chore(TDI-40990): Change package name for custom clients
2019-01-16 10:46:47 +02:00
Dmytro Sylaiev
a803bb1cc6 Dsylaiev/tdi 40990 support ftps http proxy commons net prot p (#3146)
* feat(TDI-40990): Add talend-commons-net library

* feat(TDI-40990): Add distribution section to deploy lib in nexus

* chore(TDI-40990): Use talend formatter

* feat(TDI-40990): Extract SSLSessionReuseFTPSClient behavior

* feat(TDI-40990): Rename ProxyClient

* chore(TDI-40990): Minor refactoring

* Use while instead of for

* chore(TDI-40990): Remove eclipse service files

* chore(TDI-40990): Change package name

* feat(TDI-40990): Extends talend-commons-net library to support PROT C

* Major refactoring

* chore(TDI-40990): Make methods order more reasonable

* chore(TDI-40990): Add comments

* Fix using default PROT level
2019-01-16 10:46:42 +02:00
zwxue
3c34e131ee fix(TUP-21532): fix run job and assembly problem. 2019-01-16 10:32:32 +08:00
Zhiwei Xue
705a1527bf fix(TUP-21596):exclude tLibraryLoad jars for independent subjob. (#3148) 2019-01-09 10:26:20 +08:00
Tao Tao
1651e72520 bugfix(TUP-21564):tMap settings - the "Options" window pop-up not (#3136) (#3139)
showing complete view

https://jira.talendforge.org/browse/TUP-21564
2019-01-08 15:35:06 +08:00
Dmytro Sylaiev
ab3b3a737a fix(TDI-41404): Avoid 64kb method issue when too many context paramet… (#3122)
* fix(TDI-41404): Avoid 64kb method issue when too many context paramet… (#3025)

* fix(TDI-41404): Avoid 64kb method issue when too many context parameter exists

* fix(TDI-41404): Regain missing warnings

* Dsylaiev/tdi 41404 fix 64kb issue t run job context (#3087)

* fix(TDI-41404): Avoid 64kb method issue when too many context parameter exists

* fix(TDI-41404): Regain missing warnings

* fix(TDI-41404): Fix wrong context processing in footer
2019-01-03 16:59:21 +02:00
Zhiwei Xue
30e6b50edc # WARNING: head commit changed in the meantime (#3132)
fix(TUP-21532):Standard DI job which refers to big data batch job having
S3 component and standard DI job having S3 connection component fails
with an error.
2019-01-02 15:22:10 +08:00
SunChaoqun
557dab2fec TESB-24354:SFTP proxy bean not recognized in runtime only (#3135) 2019-01-02 14:29:20 +08:00
Jane Ding
ecc2da2f72 fix(TUP-21333)OnSubJob Links Not Working (#3079) (#3102)
https://jira.talendforge.org/browse/TUP-21333
2018-12-24 11:44:15 +08:00
sponomarova
365723b2a8 fix(TBD-8144): 404 error fix for tBigQueryInput (#3098) 2018-12-21 11:32:28 +01:00
zwxue
5cff3bc2ea fix(TUP-21175):fix JUnit (#3086)
* fix(TUP-21175):fix JUnit (#3039)

* fix(TUP-21175): fix Junits. (#3064)
2018-12-19 19:11:49 +08:00
chmyga
0d5a857a4b fix(TDI-41469): tFileUnarchive zip64 format error (#3026) (#3066)
* fix(TDI-41469): tFileUnarchive zip64 format error

* Use apache commons compress in checkArchive lib

* fix(TDI-41469): tFileUnarchive zip64 format error

* Remove unused method
2018-12-17 15:51:28 +02:00
hcyi
5690c4d98f fix(TUP-21356):talend-bigdata-launcher-1.2.0-20170410.jar not found when (#3078)
spark job launched from standard job by jobserver.
2018-12-14 19:16:34 +08:00
Chao MENG
467a4e08c8 fix(TUP-21461): Routine libraries can't be updated to pom.xml of routine (#3075)
fix(TUP-21461): Routine libraries can't be updated to pom.xml of routine in special case
https://jira.talendforge.org/browse/TUP-21461
2018-12-14 18:25:22 +08:00
Tao Tao
39be232bee bugfix(TUP-21356):"talend-bigdata-launcher-1.2.0-20170410.jar" not found (#3054) (#3056)
when spark job launched from standard job by jobserver

https://jira.talendforge.org/browse/TUP-21356?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel
2018-12-10 14:44:21 +08:00
nrousseau
d410c6d6d2 fix(TUP-21175) avoid exception during junit execution 2018-12-10 10:20:07 +08:00
Jane Ding
883af43477 fix(TUP-21245)faile to check JDBC connection with context Migrate from (#3034) (#3048)
6.4.1 to 7.1.1
https://jira.talendforge.org/browse/TUP-21245
2018-12-06 10:14:41 +08:00
hcyi
bbcaae980e fix(TUP-21160):refix JDBC connection will not commit even if autocommit (#3033) (#3042)
is set to True.
2018-12-04 16:40:26 +08:00
hwang-talend
d18c9dab6c bugfix(TUP-21105):tDBConnection(Snowflake) issue within joblet. (#3028)
* bugfix(TUP-21105):tDBConnection(Snowflake) issue within joblet.

* bugfix(TUP-21105):tDBConnection(Snowflake) issue within joblet.
2018-12-04 14:30:06 +08:00
ypiel
466c74d1ff fix(TDI-41394) : Qa fix - tCreateTable+Vertica9 backport 7.0 (#3018)
* fix(TDI-41394) : Qa fix - tCreateTable+Vertica9

* fix(TDI-41394) : Add urlPath for MSSQL as it was before
2018-11-29 10:31:19 +01:00
zwxue
d2e41137ac fix(TUP-21252):tMap losing all links in testcase when we do some (#3015)
modification in the main job in the same component
2018-11-29 11:12:49 +08:00
hwang-talend
ee1973b7f7 bugfix(TUP-21118):Code view/test case still link last component even all (#2969) (#3006)
bugfix(TUP-21118):Code view/test case still link last component even all job editor closed
2018-11-27 16:24:50 +08:00
Stéphane Bouchet
45bafe6a60 fix(TESB-23687) : fixed tests (#2991) 2018-11-23 09:27:26 +01:00
hwang-talend
d59249c1ab Hwang/tup 21105 t db connection (#2953) (#2994)
* bugfix(TUP-21105):tDBConnection(Snowflake) issue within joblet.
2018-11-23 14:47:31 +08:00
Jane Ding
f376627f58 fix(TUP-21064)NPE happens when 'save as' routine (#2934) (#2990)
fix(TUP-21064)NPE happens when 'save as' routine
https://jira.talendforge.org/browse/TUP-21064
2018-11-23 10:14:36 +08:00
kjwang-talend
c48f3114df fix(TUP-21160):JDBC connection will not commit even if autocommit is set (#2976) (#2982)
to True.
2018-11-22 14:34:15 +08:00
ypiel
e274e7cc94 fix(TDI-41394) : tCreateTable vertica 9 support (#2967) 2018-11-19 12:49:37 +01:00
Jane Ding
a61fe68325 fix(TUP-21075)Problem of display in the tMap with Mojave (#2943) (#2955)
https://jira.talendforge.org/browse/TUP-21075
2018-11-15 11:53:26 +08:00
Dmytro Sylaiev
8e59bed87d fix(TDI-41330): Use sql TIME in setNull for SIQ (#2927) (#2937) 2018-11-12 13:55:28 +02:00
zwxue
32e9943a31 feat(TUP-20633):Add a command to regenerate all the poms of jobs in a (#2915)
(#2759)

project.

Conflicts:
	main/plugins/org.talend.designer.runprocess/src/main/java/org/talend/designer/runprocess/java/TalendJavaProjectManager.java
2018-11-08 10:50:00 +08:00
jzhao
3068461de4 fix(TDI-41176):'File Json' > 'Input Json' metadata does not show parent values in Preview with JsonPath (#2820) (#2922)
* add a new checkbox "Use the loop node as root" (tFileInputJson)

* fix tExtractJsonField

* fix wizard by set "Use the loop node as root" to false
2018-11-08 10:05:06 +08:00
mbasiuk-talend
5bb06d7bac Mbasiuk/tdi 40846 date validation fix (#2620) (#2918)
* fix(TDI-40846): improve date validation check

* fix(TDI-40846): correct formatting of code

* fix(TDI-40846): add check box use strict date check

* fix(TDI-40846): move to routines method
2018-11-07 11:36:12 +02:00
Dmytro Sylaiev
6ef7abafa6 fix(TDI-41276): Use correct SQLType for null Time (#2852) (#2911)
* fix(TDI-41276): Use correct SQLType for null Time

* fix(TDI-41276): Fix codegen error
2018-11-06 13:14:28 +02:00
Jane Ding
12a939d0dc fix(TUP-21075)Problem of display in the tMap with Mojave (#2900) (#2907)
* fix(TUP-21075)Problem of display in the tMap with Mojave
https://jira.talendforge.org/browse/TUP-21075

* fix(TUP-21075)use duplicate name for jdbc connection, no error show
https://jira.talendforge.org/browse/TUP-21075

* fix(TUP-21075)Problem of display in the tMap with Mojave
https://jira.talendforge.org/browse/TUP-21075
2018-11-06 16:07:30 +08:00
149 changed files with 4046 additions and 1097 deletions

68
PATCH_RELEASE_NOTE.md Normal file
View File

@@ -0,0 +1,68 @@
---
version: 7.0.1
module: https://talend.poolparty.biz/coretaxonomy/42
product:
- https://talend.poolparty.biz/coretaxonomy/23
---
# TPS-3404
| Info | Value |
| ---------------- | ---------------- |
| Patch Name | Patch\_20190920\_TPS-3404\_v1-7.0.1 |
| Release Date | 2019-10-11 |
| Target Version | 7.0.1.20190226\_1146-patch |
| Product affected | Talend Studio |
## Introduction
This is a self-contained patch.
**NOTE**: For information on how to obtain this patch, reach out to your Support contact at Talend.
## Fixed issues
This patch contains the following fixes:
- TPS-3404 [7.0.1] An error occurs on tFileOutputExcel after Iterate when /tmp/profiles folder does not exist (TDI-42092)
## Prerequisites
Consider the following requirements for your system:
- Talend Studio 7.0.1 must be installed.
- Patch TPS-2956 must be installed.
## Installation
### Installing the patch using Software update
1) Logon TAC and switch to Configuration->Software Update, then enter the correct values and save referring to the documentation: https://help.talend.com/reader/f7Em9WV_cPm2RRywucSN0Q/j9x5iXV~vyxMlUafnDejaQ
2) Switch to Software update page, where the new patch will be listed. The patch can be downloaded from here into the nexus repository.
3) On Studio Side: Logon Studio with remote mode, on the logon page the Update button is displayed: click this button to install the patch.
### Installing the patch using Talend Studio
1) Create a folder named "patches" under your studio installer directory and copy the patch .zip file to this folder.
2) Restart your studio: a window pops up, then click OK to install the patch, or restart the commandline and the patch will be installed automatically.
### Installing the patch using Commandline
Execute the following commands:
1. Talend-Studio-win-x86_64.exe -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002 --talendDebug
2. initRemote {tac_url} -ul {TAC login username} -up {TAC login password}
3. checkAndUpdate -tu {TAC login username} -tup {TAC login password}
## Uninstallation
Backup the Affected files list below. Uninstall the patch by restore the backup files.
## Affected files for this patch
The following files are installed by this patch:
- {Talend\_Studio\_path}/plugins/org.talend.designer.components.localprovider\_7.0.1.20190226\_1146-patch/components/tFileOutputExcel/tFileOutputExcel/_begin.javajet

View File

@@ -526,85 +526,107 @@
}
}
<% for (IContextParameter ctxParam :params)
{
%>
context.setContextType("<%=ctxParam.getName()%>", "<%=ctxParam.getType()%>");
<%
if (ctxParam.getType().equals("id_Password")) {
%>
String pwd_<%=ctxParam.getName()%>_value = context.getProperty("<%=ctxParam.getName()%>");
context.<%=ctxParam.getName()%> = null;
if(pwd_<%=ctxParam.getName()%>_value!=null) {
if(context_param.containsKey("<%=ctxParam.getName()%>")) {//no need to decrypt if it come from program argument or parent job runtime
context.<%=ctxParam.getName()%> = pwd_<%=ctxParam.getName()%>_value;
} else if (!pwd_<%=ctxParam.getName()%>_value.isEmpty()) {
try {
context.<%=ctxParam.getName()%> = routines.system.PasswordEncryptUtil.decryptPassword(pwd_<%=ctxParam.getName()%>_value);
context.put("<%=ctxParam.getName()%>",context.<%=ctxParam.getName()%>);
} catch (java.lang.RuntimeException e) {
//do nothing
}
}
}
<%
continue;
}
String typeToGenerate ="String";
if( !(ctxParam.getType().equals("id_File") || ctxParam.getType().equals("id_Directory") ||ctxParam.getType().equals("id_List Of Value")))
{
typeToGenerate=JavaTypesManager.getTypeToGenerate(ctxParam.getType(),true);
}
if(typeToGenerate.equals("java.util.Date"))
{
final int maxContextInOneMethodToProcess = 100;
int lastMethodNumber = 0;
%>
class ContextProcessing {
private void processContext_0() {
<%
for (int i = 1; i <= params.size(); i++) {
IContextParameter ctxParam = params.get(i-1);
if (i % maxContextInOneMethodToProcess == 0) {
/* close previous method and declare new */
lastMethodNumber++;
%>
}
%>
try{
String context_<%=ctxParam.getName()%>_value = context.getProperty("<%=ctxParam.getName()%>");
if (context_<%=ctxParam.getName()%>_value == null){
context_<%=ctxParam.getName()%>_value = "";
}
int context_<%=ctxParam.getName()%>_pos = context_<%=ctxParam.getName()%>_value.indexOf(";");
String context_<%=ctxParam.getName()%>_pattern = "yyyy-MM-dd HH:mm:ss";
if(context_<%=ctxParam.getName()%>_pos > -1){
context_<%=ctxParam.getName()%>_pattern = context_<%=ctxParam.getName()%>_value.substring(0, context_<%=ctxParam.getName()%>_pos);
context_<%=ctxParam.getName()%>_value = context_<%=ctxParam.getName()%>_value.substring(context_<%=ctxParam.getName()%>_pos + 1);
}
private void processContext_<%=lastMethodNumber%>() {
<%
} //endIf
%>
context.setContextType("<%=ctxParam.getName()%>", "<%=ctxParam.getType()%>");
<%
if (ctxParam.getType().equals("id_Password")) {
%>
String pwd_<%=ctxParam.getName()%>_value = context.getProperty("<%=ctxParam.getName()%>");
context.<%=ctxParam.getName()%> = null;
if(pwd_<%=ctxParam.getName()%>_value!=null) {
if(context_param.containsKey("<%=ctxParam.getName()%>")) {//no need to decrypt if it come from program argument or parent job runtime
context.<%=ctxParam.getName()%> = pwd_<%=ctxParam.getName()%>_value;
} else if (!pwd_<%=ctxParam.getName()%>_value.isEmpty()) {
try {
context.<%=ctxParam.getName()%> = routines.system.PasswordEncryptUtil.decryptPassword(pwd_<%=ctxParam.getName()%>_value);
context.put("<%=ctxParam.getName()%>",context.<%=ctxParam.getName()%>);
} catch (java.lang.RuntimeException e) {
//do nothing
}
}
}
<%
continue;
}
String typeToGenerate ="String";
if( !(ctxParam.getType().equals("id_File") || ctxParam.getType().equals("id_Directory") ||ctxParam.getType().equals("id_List Of Value"))) {
typeToGenerate=JavaTypesManager.getTypeToGenerate(ctxParam.getType(),true);
}
if(typeToGenerate.equals("java.util.Date")) {
%>
try{
String context_<%=ctxParam.getName()%>_value = context.getProperty("<%=ctxParam.getName()%>");
if (context_<%=ctxParam.getName()%>_value == null){
context_<%=ctxParam.getName()%>_value = "";
}
int context_<%=ctxParam.getName()%>_pos = context_<%=ctxParam.getName()%>_value.indexOf(";");
String context_<%=ctxParam.getName()%>_pattern = "yyyy-MM-dd HH:mm:ss";
if(context_<%=ctxParam.getName()%>_pos > -1){
context_<%=ctxParam.getName()%>_pattern = context_<%=ctxParam.getName()%>_value.substring(0, context_<%=ctxParam.getName()%>_pos);
context_<%=ctxParam.getName()%>_value = context_<%=ctxParam.getName()%>_value.substring(context_<%=ctxParam.getName()%>_pos + 1);
}
context.<%=ctxParam.getName()%>=(java.util.Date)(new java.text.SimpleDateFormat(context_<%=ctxParam.getName()%>_pattern).parse(context_<%=ctxParam.getName()%>_value));
context.<%=ctxParam.getName()%>=(java.util.Date)(new java.text.SimpleDateFormat(context_<%=ctxParam.getName()%>_pattern).parse(context_<%=ctxParam.getName()%>_value));
}catch(ParseException e)
{
context.<%=ctxParam.getName()%>=null;
}
<%
}else if(typeToGenerate.equals("Object")||typeToGenerate.equals("String")||typeToGenerate.equals("java.lang.String") )
{
%>
context.<%=ctxParam.getName()%>=(<%=typeToGenerate%>) context.getProperty("<%=ctxParam.getName()%>");
<%
}else if(typeToGenerate.equals("Character")&&ctxParam.getName()!=null)
{
%>
context.<%=ctxParam.getName()%>= new java.text.StringCharacterIterator(context.getProperty("<%=ctxParam.getName()%>")).first();
<%
} else {
%>
try{
context.<%=ctxParam.getName()%>=routines.system.ParserUtils.parseTo_<%=typeToGenerate%> (context.getProperty("<%=ctxParam.getName()%>"));
}catch(NumberFormatException e){
context.<%=ctxParam.getName()%>=null;
}
<%
} catch(ParseException e) {
context.<%=ctxParam.getName()%>=null;
}
<%
} else if(typeToGenerate.equals("Object")||typeToGenerate.equals("String")||typeToGenerate.equals("java.lang.String")) {
%>
context.<%=ctxParam.getName()%>=(<%=typeToGenerate%>) context.getProperty("<%=ctxParam.getName()%>");
<%
} else if(typeToGenerate.equals("Character")&&ctxParam.getName()!=null) {
%>
context.<%=ctxParam.getName()%>= new java.text.StringCharacterIterator(context.getProperty("<%=ctxParam.getName()%>")).first();
<%
} else {
%>
try{
context.<%=ctxParam.getName()%>=routines.system.ParserUtils.parseTo_<%=typeToGenerate%> (context.getProperty("<%=ctxParam.getName()%>"));
} catch(NumberFormatException e){
context.<%=ctxParam.getName()%>=null;
}
<%
}
}
%>
} <%/*close last method*/%>
public void processAllContext() {
<%
for (int i = 0; i <=lastMethodNumber; i++) {
%>
processContext_<%=i %>();
<%
}
%>
}
}
%>
new ContextProcessing().processAllContext();
} catch (java.io.IOException ie) {
System.err.println("Could not load context "+contextStr);
ie.printStackTrace();
}
// get context value from parent directly
if (parentContextMap != null && !parentContextMap.isEmpty()) {
<% for (IContextParameter ctxParam :params){

View File

@@ -516,6 +516,8 @@ private class TalendException extends Exception {
if(!(e instanceof TDieException)){
<%
boolean needCatchTalendException = false;
if (process.getNodesOfType("tLogCatcher").size() > 0) {
List<INode> logCatchers = (List<INode>)process.getNodesOfType("tLogCatcher");
for (INode logCatcher : logCatchers) {
@@ -547,6 +549,7 @@ private class TalendException extends Exception {
if (ElementParameterParser.getValue(logCatcher, "__CATCH_JAVA_EXCEPTION__").equals("true")) {
if(logCatcher!=virtualNCatchNode){
// 2) launch logCatcher subProcess
needCatchTalendException = true;
%>
<%=logCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
<%
@@ -559,6 +562,7 @@ private class TalendException extends Exception {
<%
}
if(virtualNCatchNode!=null){
needCatchTalendException = true;
%>
<%=virtualNCatchNode.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
<%
@@ -572,31 +576,12 @@ private class TalendException extends Exception {
%>
}
<%
boolean needCatchTalendException = false;
if (process.getNodesOfType("tLogCatcher").size() > 0) {
for(INode node:process.getNodesOfType("tLogCatcher")){
if(ElementParameterParser.getValue(node, "__CATCH_JAVA_EXCEPTION__").equals("true")){
needCatchTalendException = true;
break;
}
}
}
if ((!needCatchTalendException) && (process.getNodesOfType("tAssertCatcher").size() > 0)) {
for(INode node:process.getNodesOfType("tAssertCatcher")){
if(ElementParameterParser.getValue(node, "__CATCH_JAVA_EXCEPTION__").equals("true")){
needCatchTalendException = true;
break;
}
}
}
if(needCatchTalendException) {
if (process.getNodesOfType("tLogCatcher").size() > 0) {
%>
} catch (TalendException e) {
// do nothing
<%
}
//TODO should catch the TalendException in assertcatcher process? but before should not work too as use java reflect, this exception can't catch
%>
} catch (TalendException e) {
// do nothing
<%
}
%>
} catch (Exception e) {

View File

@@ -1,7 +1,7 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>checkArchive-1.1-20171121</artifactId>
<artifactId>checkArchive-1.1-20181130</artifactId>
<version>6.0.0</version>
<name>checkArchive</name>
<description>Dependence for tFileArchive and tFileUnAchive</description>

View File

@@ -5,10 +5,6 @@ import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.GZIPInputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;
import java.util.zip.ZipInputStream;
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
@@ -17,7 +13,10 @@ import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.PBEKeySpec;
import javax.crypto.spec.PBEParameterSpec;
import org.apache.commons.compress.archivers.ArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
import org.apache.commons.compress.archivers.zip.ZipArchiveInputStream;
import org.apache.commons.compress.archivers.zip.ZipFile;
// import javax.crypto.Cipher;
@@ -36,8 +35,6 @@ public class IntegrityUtil {
zipFile = new ZipFile(file);
return true;
} catch (ZipException e) {
return false;
} catch (IOException e) {
return false;
} finally {
@@ -58,7 +55,6 @@ public class IntegrityUtil {
try {
zipFile = new ZipFile(file);
} catch (ZipException e) {
} catch (IOException e) {
} finally {
try {
@@ -72,27 +68,6 @@ public class IntegrityUtil {
}
}
/**
* Check the zip entry is valid
*
* @param input
* @return
*/
public static boolean isZipEntryValid(final ZipInputStream input) {
try {
ZipEntry entry = input.getNextEntry();
return true;
} catch (ZipException e) {
// TODO Auto-generated catch block
return false;
} catch (IOException e) {
return false;
} catch (Exception e) {
return false;
}
}
/**
* To check if the encrpted zip file is corrupted or not
*
@@ -101,18 +76,15 @@ public class IntegrityUtil {
* @return
*/
public static boolean isEncryptedZipValid(final File file, String password) {
ZipInputStream input = null;
ZipArchiveInputStream input = null;
InputStream target = null;
try {
target = new FileInputStream(file);
target = new CipherInputStream(target, createCipher(Cipher.DECRYPT_MODE, password));
input = new ZipInputStream(target);
ZipEntry entry = input.getNextEntry();
input = new ZipArchiveInputStream(target);
ArchiveEntry entry = input.getNextEntry();
return true;
} catch (ZipException e) {
// TODO Auto-generated catch block
return false;
} catch (IOException e) {
return false;
} catch (Exception e) {

View File

@@ -0,0 +1,50 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>commons-net-ftps-proxy</artifactId>
<version>3.6.1-talend-20190128</version>
<name>commons-net-talend</name>
<dependencies>
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>3.6</version>
</dependency>
</dependencies>
<properties>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
</properties>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
<snapshots><enabled>true</enabled></snapshots>
<releases><enabled>false</enabled></releases>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</distributionManagement>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>

View File

@@ -0,0 +1,309 @@
package org.talend.ftp;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.Inet6Address;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLSocket;
import org.apache.commons.net.ftp.FTPCmd;
import org.apache.commons.net.ftp.FTPReply;
import org.apache.commons.net.util.Base64;
public class HTTPProxyFTPSClient extends SSLSessionReuseFTPSClient {
/** Default PROT Command */
private static final String DEFAULT_PROT = "C";
private final String proxyHost;
private final int proxyPort;
private final String proxyUsername;
private final String proxyPassword;
private SSLContext context;
private String protectionLevel;
private String tunnelHost; // Save the host when setting up a tunnel (needed for EPSV)
private static final byte[] CRLF = { '\r', '\n' };
private final Base64 base64 = new Base64();
public HTTPProxyFTPSClient(boolean isImplicit, SSLContext context, String proxyHost, int proxyPort, String proxyUser,
String proxyPass) {
super(isImplicit, context);
this.proxyHost = proxyHost;
this.proxyPort = proxyPort;
this.proxyUsername = proxyUser;
this.proxyPassword = proxyPass;
this.tunnelHost = null;
this.context = context;
this.protectionLevel = DEFAULT_PROT;
}
/**
* Call execPROT from Parent class and stores current protection level
*
* @see org.apache.commons.net.ftp.FTPSClient#execPROT(String)
*/
@Override
public void execPROT(String prot) throws SSLException, IOException {
super.execPROT(prot);
this.protectionLevel = prot;
}
/**
* @see org.apache.commons.net.SocketClient#connect(java.lang.String, int)
*/
@Override
public void connect(String host, int port) throws IOException {
_socket_ = new Socket(proxyHost, proxyPort);
_input_ = _socket_.getInputStream();
_output_ = _socket_.getOutputStream();
try {
tunnelHandshake(host, port, _input_, _output_);
} catch (Exception e) {
throw new IOException("Could not connect to " + host + " using port " + port, e);
}
super._connectAction_();
}
/**
* Open ssl socket (if private protection level selected) using tunnel socket
*
* @see org.apache.commons.net.ftp.FTPSClient#_openDataConnection_(java.lang.String, java.lang.String)
*/
@Override
protected Socket _openDataConnection_(String command, String arg) throws IOException {
//Force local passive mode, active mode not supported by through proxy
if (getDataConnectionMode() != PASSIVE_LOCAL_DATA_CONNECTION_MODE) {
throw new IllegalStateException("Only passive connection mode supported using proxy");
}
final boolean isInet6Address = getRemoteAddress() instanceof Inet6Address;
String passiveHost;
// Try EPSV command first on IPv6 - and IPv4 if enabled.
// When using IPv4 with NAT it has the advantage
// to work with more rare configurations.
// E.g. if FTP server has a static PASV address (external network)
// and the client is coming from another internal network.
// In that case the data connection after PASV command would fail,
// while EPSV would make the client succeed by taking just the port.
boolean attemptEPSV = isUseEPSVwithIPv4() || isInet6Address;
if (attemptEPSV && epsv() == FTPReply.ENTERING_EPSV_MODE) {
_parseExtendedPassiveModeReply(_replyLines.get(0));
passiveHost = this.tunnelHost;
} else {
if (isInet6Address) {
return null; // Must use EPSV for IPV6
}
// If EPSV failed on IPV4, revert to PASV
if (pasv() != FTPReply.ENTERING_PASSIVE_MODE) {
return null;
}
_parsePassiveModeReply(_replyLines.get(0));
passiveHost = this.getPassiveHost();
}
return DEFAULT_PROT.equals(protectionLevel) ?
openPlainDataConnection(passiveHost, command, arg) : openEncryptedDataConnection(passiveHost, command, arg);
}
/**
* Create, configure and open SSLSocket via plain tunnel Socket
* @param passiveHost host returned from ftp server after sending PASV command
* @param command to execute
* @param arg for command
* @return SSLSocket using to Data transmit
* @throws IOException
*/
private Socket openEncryptedDataConnection(String passiveHost, String command, String arg) throws IOException {
Socket proxySocket = createTunnelSocket();
proxySocket.connect(new InetSocketAddress(proxyHost, proxyPort), getConnectTimeout());
tunnelHandshake(passiveHost, this.getPassivePort(), proxySocket.getInputStream(),
proxySocket.getOutputStream());
Socket socket = context.getSocketFactory().createSocket(proxySocket, passiveHost, this.getPassivePort(), true);
_prepareDataSocket_(socket);
configureDataSocket(socket);
if (isRestartOffsetIncorrect()) {
proxySocket.close();
socket.close();
return null;
}
if (!FTPReply.isPositivePreliminary(sendCommand(command, arg))) {
proxySocket.close();
socket.close();
return null;
}
if (socket instanceof SSLSocket) {
SSLSocket sslSocket = (SSLSocket) socket;
configureSSLDataSocket(sslSocket);
sslSocket.startHandshake();
}
return socket;
}
private Socket createTunnelSocket() throws IOException {
Socket proxySocket = new Socket();
configureDataSocket(proxySocket);
return proxySocket;
}
private void configureSSLDataSocket(SSLSocket sslSocket) {
sslSocket.setUseClientMode(getUseClientMode());
sslSocket.setEnableSessionCreation(getEnableSessionCreation());
// server mode
if (!getUseClientMode()) {
sslSocket.setNeedClientAuth(getNeedClientAuth());
sslSocket.setWantClientAuth(getWantClientAuth());
}
if (getEnabledCipherSuites() != null) {
sslSocket.setEnabledCipherSuites(getEnabledCipherSuites());
}
if (getEnabledProtocols() != null) {
sslSocket.setEnabledProtocols(getEnabledProtocols());
}
}
private void configureDataSocket(Socket socket) throws IOException {
if (getReceiveDataSocketBufferSize() > 0) {
socket.setReceiveBufferSize(getReceiveDataSocketBufferSize());
}
if (getSendDataSocketBufferSize() > 0) {
socket.setSendBufferSize(getSendDataSocketBufferSize());
}
if (getPassiveLocalIPAddress() != null) {
socket.bind(new InetSocketAddress(getPassiveLocalIPAddress(), 0));
}
}
/**
* @see org.apache.commons.net.ftp.FTPHTTPClient#_openDataConnection_(FTPCmd, String)
* @return Plain Socket using HTTP proxy
*/
private Socket openPlainDataConnection(String passiveHost, String command, String arg) throws IOException {
Socket socket = preparePlainDataTunnelSocket(passiveHost);
if (isRestartOffsetIncorrect()) {
socket.close();
return null;
}
if (!FTPReply.isPositivePreliminary(sendCommand(command, arg))) {
socket.close();
return null;
}
return socket;
}
/**
* Create, prepare and connect plain data transmit socket via tunnel proxy
*/
private Socket preparePlainDataTunnelSocket(String passiveHost) throws IOException {
Socket socket = _socketFactory_.createSocket(proxyHost, proxyPort);
InputStream is = socket.getInputStream();
OutputStream os = socket.getOutputStream();
tunnelHandshake(passiveHost, this.getPassivePort(), is, os);
return socket;
}
/**
* HTTP handshake for proxy
*/
private void tunnelHandshake(String host, int port, InputStream input, OutputStream output) throws IOException {
final String connectString = "CONNECT " + host + ":" + port + " HTTP/1.1";
final String hostString = "Host: " + host + ":" + port;
this.tunnelHost = host;
output.write(connectString.getBytes(StandardCharsets.UTF_8));
output.write(CRLF);
output.write(hostString.getBytes(StandardCharsets.UTF_8));
output.write(CRLF);
if (proxyUsername != null && proxyPassword != null) {
final String auth = proxyUsername + ":" + proxyPassword;
final String header =
"Proxy-Authorization: Basic " + base64.encodeToString(auth.getBytes(StandardCharsets.UTF_8));
output.write(header.getBytes(StandardCharsets.UTF_8));
}
output.write(CRLF);
output.flush();
List<String> response = new ArrayList<>();
BufferedReader reader = new BufferedReader(new InputStreamReader(input, getCharset()));
String responseLine;
while ((responseLine = reader.readLine()) != null && responseLine.length() > 0) {
response.add(responseLine);
}
int size = response.size();
if (size == 0) {
throw new IOException("No response from proxy");
}
String code;
String resp = response.get(0);
if (resp.startsWith("HTTP/") && resp.length() >= 12) {
code = resp.substring(9, 12);
} else {
throw new IOException("Invalid response from proxy: " + resp);
}
if (!"200".equals(code)) {
StringBuilder msg = new StringBuilder();
msg.append("HTTPTunnelConnector: connection failed\r\n");
msg.append("Response received from the proxy:\r\n");
for (String line : response) {
msg.append(line);
msg.append("\r\n");
}
throw new IOException(msg.toString());
}
}
private boolean isRestartOffsetIncorrect() throws IOException {
return (getRestartOffset() > 0) && !restart(getRestartOffset());
}
}

View File

@@ -0,0 +1,43 @@
package org.talend.ftp;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.net.Socket;
import java.util.Locale;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSessionContext;
import javax.net.ssl.SSLSocket;
import org.apache.commons.net.ftp.FTPSClient;
public class SSLSessionReuseFTPSClient extends FTPSClient {
public SSLSessionReuseFTPSClient(boolean isImplicit, SSLContext context) {
super(isImplicit, context);
}
@Override
protected void _prepareDataSocket_(final Socket socket) {
if (socket instanceof SSLSocket) {
final SSLSession session = ((SSLSocket) _socket_).getSession();
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
final Method getHostMethod = socket.getClass().getDeclaredMethod("getHost");
getHostMethod.setAccessible(true);
Object host = getHostMethod.invoke(socket);
final String key =
String.format("%s:%s", host, String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT);
putMethod.invoke(cache, key, session);
} catch (Exception e) {
e.printStackTrace();
}
}
}
}

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talend_DB_mssqlUtil-1.2-20171017</artifactId>
<artifactId>talend_DB_mssqlUtil-1.3-20190523</artifactId>
<version>6.0.0</version>
<packaging>jar</packaging>
@@ -50,9 +50,9 @@
<dependencies>
<!-- https://mvnrepository.com/artifact/net.sourceforge.jtds/jtds -->
<dependency>
<groupId>org.talend.libraries</groupId>
<artifactId>jtds-1.3.1-patch</artifactId>
<version>6.1.1</version>
<groupId>net.sourceforge.jtds</groupId>
<artifactId>jtds</artifactId>
<version>1.3.1-patch-20190523</version>
</dependency>
</dependencies>

View File

@@ -0,0 +1,139 @@
<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
>
<modelVersion>4.0.0</modelVersion>
<groupId>net.sourceforge.jtds</groupId>
<artifactId>jtds</artifactId>
<version>1.3.1-patch-20190523</version>
<packaging>jar</packaging>
<name>talend_jtds</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<talend.nexus.url>https://artifacts-oss.talend.com</talend.nexus.url>
<java.source.version>1.7</java.source.version>
</properties>
<repositories>
<repository>
<id>org.talend.studio</id>
<name>talend-update</name>
<url>https://talend-update.talend.com/nexus/content/repositories/libraries/</url>
</repository>
</repositories>
<distributionManagement>
<snapshotRepository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceSnapshot/</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</snapshotRepository>
<repository>
<id>talend_nexus_deployment</id>
<url>${talend.nexus.url}/nexus/content/repositories/TalendOpenSourceRelease/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository>
</distributionManagement>
<dependencies>
<!-- https://mvnrepository.com/artifact/net.sourceforge.jtds/jtds -->
<dependency>
<groupId>org.talend.libraries</groupId>
<artifactId>jtds-1.3.1-patch</artifactId>
<version>6.1.1</version>
</dependency>
</dependencies>
<build>
<resources>
<resource>
<directory>src/main/java</directory>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>unpack</id>
<phase>package</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.talend.libraries</groupId>
<artifactId>jtds-1.3.1-patch</artifactId>
<version>6.1.1</version>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>target/classes</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<id>default</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
<includes>
<include>**</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>${java.source.version}</source>
<target>${java.source.version}</target>
<showDeprecation>true</showDeprecation>
<showWarnings>true</showWarnings>
<compilerArgument>-XDignore.symbol.file</compilerArgument>
<fork>true</fork>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<configuration>
<archive>
<addMavenDescriptor>false</addMavenDescriptor>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>

View File

@@ -55,8 +55,13 @@ class BigQueryUtil_<%=cid%> {
}
public com.google.api.services.bigquery.model.Job executeQuery(String query, boolean useLargeResult) throws Exception{
return executeQuery(query, useLargeResult, true);
}
public com.google.api.services.bigquery.model.Job executeQuery(String query, boolean useLargeResult, boolean useLegacySql) throws Exception{
com.google.api.services.bigquery.model.JobConfigurationQuery queryConfig = new com.google.api.services.bigquery.model.JobConfigurationQuery();
queryConfig.setQuery(query);
queryConfig.setUseLegacySql(useLegacySql);
if(useLargeResult){
this.useLargeResult = true;
tempDataset = genTempName("dataset");

View File

@@ -132,7 +132,7 @@
<IMPORT NAME="google-oauth-client-1.19.0.jar" MODULE="google-oauth-client-1.19.0.jar" MVN="mvn:org.talend.libraries/google-oauth-client-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-oauth-client-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="google-http-client-jackson2-1.19.0.jar" MODULE="google-http-client-jackson2-1.19.0.jar" MVN="mvn:org.talend.libraries/google-http-client-jackson2-1.19.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.google/lib/google-http-client-jackson2-1.19.0.jar" REQUIRED="true" />
<IMPORT NAME="guava-jdk5-13.0.jar" MODULE="guava-jdk5-13.0.jar" MVN="mvn:org.talend.libraries/guava-jdk5-13.0/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.guava/lib/guava-jdk5-13.0.jar" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.4.jar" MODULE="jackson-core-2.9.4.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.4" UrlPath="platform:/plugin/org.talend.libraries.jackson/lib/jackson-core-2.9.4.jar" REQUIRED="true" />
<IMPORT NAME="jackson-core-2.9.5.jar" MODULE="jackson-core-2.9.5.jar" MVN="mvn:com.fasterxml.jackson.core/jackson-core/2.9.5" UrlPath="platform:/plugin/org.talend.libraries.jackson/lib/jackson-core-2.9.5.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -639,17 +639,18 @@
NUM_ROW="50"
REPOSITORY_VALUE="DB_VERSION"
SHOW_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false')">
<ITEMS DEFAULT="VERTICA_3">
<ITEMS DEFAULT="VERTICA_9_0">
<ITEM NAME="VERTICA_3" VALUE="vertica_3.0_jdk_5.jar" />
<ITEM NAME="VERTICA_3.5" VALUE="vertica_3.5_jdk_5.jar" />
<ITEM NAME="VERTICA_4" VALUE="vertica_4.0_jdk_5.jar" />
<ITEM NAME="VERTICA_4.1" VALUE="vertica_4.1.7_jdk_5.jar" />
<ITEM NAME="VERTICA_5.0" VALUE="vertica_4.1.14_jdk_5.jar" />
<ITEM NAME="VERTICA_5_1" VALUE="VERTICA_5_1" />
<ITEM NAME="VERTICA_5_1" VALUE="VERTICA_5_1" />
<ITEM NAME="VERTICA_6_0" VALUE="VERTICA_6_0" />
<ITEM NAME="VERTICA_6_1_X" VALUE="VERTICA_6_1_X" />
<ITEM NAME="VERTICA_7_0_X" VALUE="VERTICA_7_0_X" />
<ITEM NAME="VERTICA_7_1_X" VALUE="VERTICA_7_1_X" />
<ITEM NAME="VERTICA_9_0" VALUE="VERTICA_9_0" />
</ITEMS>
</PARAMETER>
<PARAMETER
@@ -776,7 +777,7 @@
<IMPORT NAME="Driver-DB2-LICENSE-CISUZ" MODULE="db2jcc_license_cisuz.jar" MVN="mvn:org.talend.libraries/db2jcc_license_cisuz/6.0.0" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='DB2')" />
<IMPORT NAME="Driver-DB2-LICENSE-CU" MODULE="db2jcc_license_cu.jar" MVN="mvn:org.talend.libraries/db2jcc_license_cu/6.0.0" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='DB2')" />
<IMPORT NAME="Driver_MaxDB" MODULE="sapdbc.jar" MVN="mvn:org.talend.libraries/sapdbc/6.0.0" REQUIRED_IF="DBTYPE=='MAXDB'" />
<IMPORT NAME="Driver-MSSQL" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='MSSQL') AND (MSSQL_DRIVER=='JTDS')" />
<IMPORT NAME="Driver-MSSQL" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='MSSQL') AND (MSSQL_DRIVER=='JTDS')" />
<IMPORT NAME="Driver-MSSQL-prop" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='MSSQL') AND (MSSQL_DRIVER=='MSSQL_PROP')" />
<IMPORT NAME="Driver-MYSQL4" MODULE="mysql-connector-java-3.1.14-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-3.1.14-bin/6.0.0" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') and (DBTYPE=='MYSQL') and (DB_MYSQL_VERSION=='MYSQL_4')"/>
<IMPORT NAME="Driver-MYSQL5" MODULE="mysql-connector-java-5.1.30-bin.jar" MVN="mvn:org.talend.libraries/mysql-connector-java-5.1.30-bin/6.0.0" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') and (DBTYPE=='MYSQL') and (DB_MYSQL_VERSION=='MYSQL_5')"/>
@@ -808,7 +809,6 @@
<IMPORT NAME="Driver-Teradata_jdbc" MODULE="terajdbc4-16.20.00.02.jar" MVN="mvn:com.teradata/terajdbc4/16.20.00.02" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='TERADATA')" />
<IMPORT NAME="Driver-Teradata_config" MODULE="tdgssconfig-16.20.00.02.jar" MVN="mvn:com.teradata/tdgssconfig/16.20.00.02" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='TERADATA')" />
<IMPORT NAME="Driver-Netezza" MODULE="nzjdbc.jar" MVN="mvn:org.talend.libraries/nzjdbc/6.0.0" REQUIRED_IF="(USE_EXISTING_CONNECTION == 'false') AND (DBTYPE=='NETEZZA')" />
<IMPORT NAME="Driver-VERTICA" MODULE="vertica_3.0_jdk_5.jar" MVN="mvn:org.talend.libraries/vertica_3.0_jdk_5/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_3')" />
<IMPORT NAME="Driver-VERTICA_3.5" MODULE="vertica_3.5_jdk_5.jar" MVN="mvn:org.talend.libraries/vertica_3.5_jdk_5/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_3.5')" />
<IMPORT NAME="Driver-VERTICA_4" MODULE="vertica_4.0_jdk_5.jar" MVN="mvn:org.talend.libraries/vertica_4.0_jdk_5/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_4')" />
@@ -819,10 +819,11 @@
<IMPORT NAME="Driver-VERTICA_6.1.2" MODULE="vertica-jdk5-6.1.2-0.jar" MVN="mvn:org.talend.libraries/vertica-jdk5-6.1.2-0/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_6_1_X')" />
<IMPORT NAME="Driver-VERTICA_7.0.1" MODULE="vertica-jdbc-7.0.1-0.jar" MVN="mvn:org.talend.libraries/vertica-jdbc-7.0.1-0/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_7_0_X')" />
<IMPORT NAME="Driver-VERTICA_7.1.2" MODULE="vertica-jdbc-7.1.2-0.jar" MVN="mvn:org.talend.libraries/vertica-jdbc-7.1.2-0/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_7_1_X')" />
<IMPORT NAME="Driver-EXASolution" MODULE="exajdbc-6.0.9302.jar" MVN="mvn:org.talend.libraries/exajdbc-6.0.9302/6.3.0" REQUIRED_IF="(DBTYPE=='EXASOL') and (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-Redshift" MODULE="RedshiftJDBC41-1.2.10.1009.jar" MVN="mvn:org.talend.libraries/RedshiftJDBC41-1.2.10.1009/6.0.0" REQUIRED_IF="(DBTYPE=='REDSHIFT') and (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-VERTICA_9.0" MODULE="vertica-jdbc-9.0.0-0.jar" MVN="mvn:org.talend.libraries/vertica-jdbc-9.0.0-0/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_9_0')"/>
</IMPORTS>
</CODEGENERATION>

View File

@@ -1683,7 +1683,7 @@ class VerticaManager extends Manager {
this.connection = connection;
}
protected String getDriver() {
if("VERTICA_7_1_X".equals(db_version) || "VERTICA_7_0_X".equals(db_version) || "VERTICA_6_1_X".equals(db_version) || "VERTICA_6_0".equals(db_version) || "VERTICA_5_1".equals(db_version)){
if("VERTICA_9_0".equals(db_version) || "VERTICA_7_1_X".equals(db_version) || "VERTICA_7_0_X".equals(db_version) || "VERTICA_6_1_X".equals(db_version) || "VERTICA_6_0".equals(db_version) || "VERTICA_5_1".equals(db_version)){
return "com.vertica.jdbc.Driver";
}else{
return "com.vertica.Driver";

View File

@@ -112,6 +112,7 @@ DB_VERTICA_VERSION.ITEM.VERTICA_6_0=Vertica 6.0
DB_VERTICA_VERSION.ITEM.VERTICA_6_1_X=Vertica 6.1.X
DB_VERTICA_VERSION.ITEM.VERTICA_7_0_X=Vertica 7.0.X
DB_VERTICA_VERSION.ITEM.VERTICA_7_1_X=Vertica 7.1.X
DB_VERTICA_VERSION.ITEM.VERTICA_9_0=Vertica 9
DB_POSTGRE_VERSION.NAME=DB Version
DB_POSTGRE_VERSION.ITEM.PRIOR_TO_V9=Prior to v9

View File

@@ -188,7 +188,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -461,7 +461,7 @@ pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(deleteQuery_<%=cid %>);
StringBuilder mergeInsertName = new StringBuilder() ;
StringBuilder mergeInsertValue = new StringBuilder() ;
%>
String mergeQuery = "MERGE INTO "+ tableName_<%=cid%> +" target" + " USING (" + select_query_<%=cid %> + ") source ON (" +
String mergeQuery_<%=cid %> = "MERGE INTO "+ tableName_<%=cid%> +" target" + " USING (" + select_query_<%=cid %> + ") source ON (" +
"<%=mergeCondition%>" + ")" ;
@@ -484,28 +484,28 @@ pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(deleteQuery_<%=cid %>);
}
}
%>
mergeQuery += " WHEN MATCHED THEN UPDATE SET <%=mergeUpdate%> " ;
mergeQuery_<%=cid %> += " WHEN MATCHED THEN UPDATE SET <%=mergeUpdate%> " ;
<%if (useUpdateWhere) {
String useUpdateWhereString = ElementParameterParser.getValue(node,"__MERGE_UPDATE_WHERE_STRING__");
if(useUpdateWhereString == null || useUpdateWhereString.length() == 0){
%>
mergeQuery += "WHERE " + "";
mergeQuery_<%=cid %> += "WHERE " + "";
<%
}
else {%>
mergeQuery += "WHERE " + <%=useUpdateWhereString %>;
mergeQuery_<%=cid %> += "WHERE " + <%=useUpdateWhereString %>;
<% }
}%>
<%if (useUpdateDeleteWhere) {
String useUpdateDeleteWhereString = ElementParameterParser.getValue(node,"__MERGE_UPDATE_DELETE_WHERE_STRING__");
if(useUpdateDeleteWhereString == null || useUpdateDeleteWhereString.length() == 0){
%>
mergeQuery += " DELETE WHERE " + "" ;
mergeQuery_<%=cid %> += " DELETE WHERE " + "" ;
<%
}
else {%>
mergeQuery += " DELETE WHERE " + <%=useUpdateDeleteWhereString%> ;
mergeQuery_<%=cid %> += " DELETE WHERE " + <%=useUpdateDeleteWhereString%> ;
<% }
}%>
<%
@@ -531,21 +531,21 @@ pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(deleteQuery_<%=cid %>);
}
}
%>
mergeQuery += " WHEN NOT MATCHED THEN INSERT ( <%=mergeInsertName%>) VALUES ( <%=mergeInsertValue%>) ";
mergeQuery_<%=cid %> += " WHEN NOT MATCHED THEN INSERT ( <%=mergeInsertName%>) VALUES ( <%=mergeInsertValue%>) ";
<%
}
if (useInsertWhere) {
String useInsertWhereString = ElementParameterParser.getValue(node,"__MERGE_INSERT_WHERE_STRING__");
if(useInsertWhereString == null || useInsertWhereString.length() == 0){
%>
mergeQuery += " WHERE " + "";
mergeQuery_<%=cid %> += " WHERE " + "";
<%
} else {%>
mergeQuery += " WHERE " + <%=useInsertWhereString%> ;
mergeQuery_<%=cid %> += " WHERE " + <%=useInsertWhereString%> ;
<%}
}
%>
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(mergeQuery);
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(mergeQuery_<%=cid %>);
<%
}
%>
@@ -588,8 +588,17 @@ nb_line_updated_<%=cid%> = pstmt_<%=cid %>.executeUpdate();
<%
}else if (("MERGE").equals(dataAction)){
%>
<%if(!isLog4jEnabled){%>System.out.println("Merge with : \n" + mergeQuery +"\n");<%}%>
<%log4jCodeGenerateUtil.logInfo(node,"info",cid+" - Executing SQL:\"+mergeQuery+\"");%>
<%
if(!isLog4jEnabled){
%>
System.out.println("Merge with : \n" + mergeQuery_<%=cid %> +"\n");
<%
} else {
%>
log.info("<%=cid %> - Executing SQL: " + mergeQuery_<%=cid %>);
<%
}
%>
nb_line_merged_<%=cid%> = pstmt_<%=cid %>.executeUpdate();
<%log4jCodeGenerateUtil.logInfo(node,"info",cid+" - Executed successfully.");%>
<%if(!isLog4jEnabled){%>System.out.println("--> " + nb_line_merged_<%=cid%> + " rows merged. \n");<%}%>

View File

@@ -8,8 +8,9 @@ String loopPath_<%=cid%> = <%=loopQuery%>;
java.util.List<Object> resultset_<%=cid%> = new java.util.ArrayList<Object>();
boolean isStructError_<%=cid %> = true;
com.jayway.jsonpath.ReadContext document_<%=cid%> = null;
try {
com.jayway.jsonpath.ReadContext document_<%=cid%> = com.jayway.jsonpath.JsonPath.parse(jsonStr_<%=cid%>);
document_<%=cid%> = com.jayway.jsonpath.JsonPath.parse(jsonStr_<%=cid%>);
com.jayway.jsonpath.JsonPath compiledLoopPath_<%=cid%> = jsonPathCache_<%=cid%>.getCompiledJsonPath(loopPath_<%=cid%>);
Object result_<%=cid%> = document_<%=cid%>.read(compiledLoopPath_<%=cid%>,net.minidev.json.JSONObject.class);
if (result_<%=cid%> instanceof net.minidev.json.JSONArray) {
@@ -64,6 +65,7 @@ com.jayway.jsonpath.JsonPath compiledJsonPath_<%=cid%> = null;
Object value_<%=cid%> = null;
Object root_<%=cid%> = null;
for(int i_<%=cid %>=0; isStructError_<%=cid %> || (i_<%=cid %> < resultset_<%=cid %>.size());i_<%=cid %>++){
if(!isStructError_<%=cid %>){
Object row_<%=cid%> = resultset_<%=cid %>.get(i_<%=cid %>);
@@ -82,6 +84,7 @@ if(outConns!=null){
nb_line_<%=cid%>++;
try {
<%
boolean useLoopAsRoot = "true".equals(ElementParameterParser.getValue(node, "__USE_LOOP_AS_ROOT__"));
boolean splitList = "true".equals(ElementParameterParser.getValue(node, "__SPLIT_LIST__"));
for (int i=0;i<mapping.size();i++) {
for(IMetadataColumn column : outConn.getMetadataTable().getListColumns()) {
@@ -113,8 +116,23 @@ for (int i=0;i<mapping.size();i++) {
compiledJsonPath_<%=cid%> = jsonPathCache_<%=cid%>.getCompiledJsonPath(jsonPath_<%=cid%>);
try {
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(row_<%=cid%>);
<%
if(useLoopAsRoot){
%>
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(row_<%=cid%>);
<%
}else{
%>
if(jsonPath_<%=cid%>.startsWith("$")){
if(root_<%=cid%> == null){
root_<%=cid%> = document_<%=cid%>.read(jsonPathCache_<%=cid%>.getCompiledJsonPath("$"));
}
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(root_<%=cid%>);
}else{
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(row_<%=cid%>);
}
<%
}
if(javaType == JavaTypesManager.STRING){
%>
<%=firstConnName%>.<%=column.getLabel() %> = value_<%=cid%> == null ? <%@ include file="./set_default_value.javajet"%> : value_<%=cid%>.toString();

View File

@@ -163,6 +163,11 @@
REQUIRED="true" SHOW_IF="READ_BY=='XPATH'">
<DEFAULT>"UTF-8"</DEFAULT>
</PARAMETER>
<PARAMETER NAME="USE_LOOP_AS_ROOT" FIELD="CHECK" NUM_ROW="25" SHOW_IF="READ_BY=='JSONPATH'" REPOSITORY_VALUE="USE_LOOP_AS_ROOT">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<!-- For migration only -->
<PARAMETER NAME="SPLIT_LIST" FIELD="CHECK" NUM_ROW="100" SHOW="false">
<DEFAULT>true</DEFAULT>

View File

@@ -24,4 +24,6 @@ JSON_PATH_VERSION.ITEM.1_2_0=1.2.0
MAPPING_4_JSONPATH.NAME=Mapping
MAPPING_4_JSONPATH.ITEM.QUERY=Json query
JSON_LOOP_QUERY.NAME=Loop Jsonpath query
JSON_LOOP_QUERY.NAME=Loop Jsonpath query
USE_LOOP_AS_ROOT.NAME=Use the loop node as root

View File

@@ -189,7 +189,7 @@ if (!sftp && !ftps) { // *** ftp *** //
} else { // *** ftps *** //
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -202,7 +202,17 @@ if (!sftp && !ftps) { // *** ftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=sEncoding %>);
<%if(isLog4jEnabled){%>
@@ -227,8 +237,19 @@ if (!sftp && !ftps) { // *** ftp *** //
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -234,6 +234,7 @@
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar" MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" REQUIRED_IF="(FTPS == 'true')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -211,7 +211,7 @@ int nb_file_<%=cid%> = 0;
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -224,7 +224,17 @@ int nb_file_<%=cid%> = 0;
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -248,8 +258,19 @@ int nb_file_<%=cid%> = 0;
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -291,6 +291,9 @@
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -197,7 +197,7 @@ if (sftp) { // *** sftp *** //
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -210,7 +210,17 @@ if (sftp) { // *** sftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -222,19 +232,30 @@ if (sftp) { // *** sftp *** //
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%
passwordFieldName = "__PASSWORD__";
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -274,6 +274,9 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -276,7 +276,7 @@ if (sftp) {// *** sftp *** //
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -289,7 +289,17 @@ if (sftp) {// *** sftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -315,8 +325,19 @@ if (sftp) {// *** sftp *** //
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -256,6 +256,9 @@
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -246,7 +246,7 @@ if (sftp) { // *** sftp *** //
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -258,7 +258,17 @@ if (sftp) { // *** sftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -282,8 +292,19 @@ if (sftp) { // *** sftp *** //
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE); //to calculate md5 for binary files correctly
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -253,6 +253,9 @@
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS />

View File

@@ -692,7 +692,7 @@ if (sftp) { // *** sftp *** //
if ("false".equals(useExistingConn)){
%>
<% /* create connection */ %>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid%> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid%> = null;
@@ -706,7 +706,17 @@ if (sftp) { // *** sftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid%>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -731,8 +741,19 @@ if (sftp) { // *** sftp *** //
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -297,6 +297,9 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -325,7 +325,7 @@ int nb_file_<%=cid%> = 0;
if(("false").equals(useExistingConn)){
%>
<% /* create connection */ %>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid%> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid%> = null;
@@ -339,7 +339,17 @@ int nb_file_<%=cid%> = 0;
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid%>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -364,8 +374,19 @@ int nb_file_<%=cid%> = 0;
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -337,6 +337,9 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -217,7 +217,7 @@ if (sftp) { // *** sftp *** //
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -229,7 +229,17 @@ if (sftp) { // *** sftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -252,8 +262,19 @@ if (sftp) { // *** sftp *** //
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -259,6 +259,9 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -212,7 +212,7 @@ if (sftp) {// *** sftp *** //
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpstlsResumption.javajet"%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
@@ -225,7 +225,17 @@ if (sftp) {// *** sftp *** //
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
ftp_<%=cid %> = new SSLSessionReuseFTPSClient_<%=cid %>(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
@@ -250,9 +260,19 @@ if (sftp) {// *** sftp *** //
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());

View File

@@ -248,6 +248,9 @@ COMPATIBILITY="ALL"
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.53.jar" MVN="mvn:org.talend.libraries/jsch-0.1.53/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/jsch-0.1.53.jar" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
MVN="mvn:org.talend.libraries/commons-net-ftps-proxy/3.6.1-talend-20190128" UrlPath="platform:/plugin/org.talend.designer.components.localprovider/components/tFTPConnection/commons-net-ftps-proxy-3.6.1-talend-20190128.jar"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -132,7 +132,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="COMMONS-COMPRESS-1.10" MODULE="commons-compress-1.10.jar" MVN="mvn:org.talend.libraries/commons-compress-1.10/6.1.0" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-compress-1.10.jar" REQUIRED="true" />
<IMPORT NAME="Encrypt-Zip" MODULE="checkArchive-1.1-20171121.jar" MVN="mvn:org.talend.libraries/checkArchive-1.1-20171121/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/checkArchive-1.1-20171121.jar" REQUIRED="true" />
<IMPORT NAME="Encrypt-Zip" MODULE="checkArchive-1.1-20181130.jar" MVN="mvn:org.talend.libraries/checkArchive-1.1-20181130/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/checkArchive-1.1-20181130.jar" REQUIRED="true" />
<IMPORT NAME="zip4j_1.3.1.jar" MODULE="zip4j_1.3.1.jar" MVN="mvn:org.talend.libraries/zip4j_1.3.1/6.0.0" REQUIRED="true" />
<IMPORT NAME="talendzip.jar" MODULE="talendzip.jar" MVN="mvn:org.talend.libraries/talendzip/6.1.0" REQUIRED="true" />
</IMPORTS>

View File

@@ -170,8 +170,8 @@ if ("http".equals(protocol) || "https".equals(protocol)) {
}
}
org.apache.commons.httpclient.protocol.Protocol myhttps = new org.apache.commons.httpclient.protocol.Protocol("https", new SocketFactory_<%=cid%>(), 443);
org.apache.commons.httpclient.protocol.Protocol.registerProtocol("https", myhttps);
org.apache.commons.httpclient.protocol.Protocol myhttps_<%=cid%> = new org.apache.commons.httpclient.protocol.Protocol("https", new SocketFactory_<%=cid%>(), 443);
org.apache.commons.httpclient.protocol.Protocol.registerProtocol("https", myhttps_<%=cid%>);
<%
}
%>

View File

@@ -515,7 +515,7 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
int fieldCount_<%=cid %> = row_<%= cid %>.length;
dynamic_column_count_<%=cid%> = dynamic_<%=cid %>.getColumnCount();
for (int i = 0; i < dynamic_column_count_<%=cid%> ; i++) {
if ((<%=dynamic_index%>+i) < fieldCount_<%=cid %>){
if ((<%=dynamic_index%>+i) <= fieldCount_<%=cid %>){
if(temp_row_<%= cid %>[<%=dynamic_index%>+i]<%=(isTrimAll || (!trimSelects.isEmpty() && ("true").equals(trimSelects.get(i).get("TRIM"))))?".trim()":"" %>.length() < 1){
emptyColumnCount_<%= cid %>++;
}
@@ -1081,7 +1081,7 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
}
dynamic_column_count_<%=cid%> = dynamic_<%=cid %>.getColumnCount();
for (int i = 0; i < dynamic_column_count_<%=cid%> ; i++) {
if ((<%=dynamic_index%>+i) < fieldCount_<%=cid %>){
if ((<%=dynamic_index%>+i) <= fieldCount_<%=cid %>){
if(temp_row_<%= cid %>[<%=dynamic_index%>+i]<%=(isTrimAll || (!trimSelects.isEmpty() && ("true").equals(trimSelects.get(i).get("TRIM"))))?".trim()":"" %>.length() < 1){
emptyColumnCount_<%= cid %>++;
}
@@ -1424,7 +1424,7 @@ if ((metadatas!=null)&&(metadatas.size()>0)) {
int fieldCount_<%=cid %> = row_<%= cid %>.size();
dynamic_column_count_<%=cid%> = dynamic_<%=cid %>.getColumnCount();
for (int i = 0; i < dynamic_column_count_<%=cid%> ; i++) {
if ((<%=dynamic_index%>+i) < fieldCount_<%=cid %>){
if ((<%=dynamic_index%>+i) <= fieldCount_<%=cid %>){
if(temp_row_<%= cid %>[<%=dynamic_index%>+i]<%=(isTrimAll || (!trimSelects.isEmpty() && ("true").equals(trimSelects.get(i).get("TRIM"))))?".trim()":"" %>.length() < 1){
emptyColumnCount_<%= cid %>++;
}

View File

@@ -7,6 +7,7 @@ String firstConnName = "";
List<IMetadataTable> metadatas = node.getMetadataList();
String loopPath = ElementParameterParser.getValue(node, "__JSON_LOOP_QUERY__");
boolean useLoopAsRoot = "true".equals(ElementParameterParser.getValue(node, "__USE_LOOP_AS_ROOT__"));
%>
class JsonPathCache_<%=cid%> {
final java.util.Map<String,com.jayway.jsonpath.JsonPath> jsonPathString2compiledJsonPath = new java.util.HashMap<String,com.jayway.jsonpath.JsonPath>();
@@ -51,6 +52,7 @@ Object filenameOrStream_<%=cid %> = null;
%>
}
com.jayway.jsonpath.ReadContext document_<%=cid%> = null;
try {
if(filenameOrStream_<%=cid %> instanceof java.io.InputStream){
is_<%=cid%> = (java.io.InputStream)filenameOrStream_<%=cid %>;
@@ -65,7 +67,7 @@ try {
}
com.jayway.jsonpath.ReadContext document_<%=cid%> = parseContext_<%=cid%>.parse(is_<%=cid%>,<%=encoding %>);
document_<%=cid%> = parseContext_<%=cid%>.parse(is_<%=cid%>,<%=encoding %>);
com.jayway.jsonpath.JsonPath compiledLoopPath_<%=cid%> = jsonPathCache_<%=cid%>.getCompiledJsonPath(loopPath_<%=cid%>);
Object result_<%=cid%> = document_<%=cid%>.read(compiledLoopPath_<%=cid%>,net.minidev.json.JSONObject.class);
if (result_<%=cid%> instanceof net.minidev.json.JSONArray) {
@@ -100,6 +102,7 @@ Object value_<%=cid%> = null;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Retrieving records from data.");
<%}%>
Object root_<%=cid%> = null;
for(Object row_<%=cid%> : resultset_<%=cid%>) {
nb_line_<%=cid%>++;
<%
@@ -173,8 +176,23 @@ for(Object row_<%=cid%> : resultset_<%=cid%>) {
compiledJsonPath_<%=cid%> = jsonPathCache_<%=cid%>.getCompiledJsonPath(jsonPath_<%=cid%>);
try {
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(row_<%=cid%>);
<%
if(useLoopAsRoot){
%>
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(row_<%=cid%>);
<%
}else{
%>
if(jsonPath_<%=cid%>.startsWith("$")){
if(root_<%=cid%> == null){
root_<%=cid%> = document_<%=cid%>.read(jsonPathCache_<%=cid%>.getCompiledJsonPath("$"));
}
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(root_<%=cid%>);
}else{
value_<%=cid%> = compiledJsonPath_<%=cid%>.read(row_<%=cid%>);
}
<%
}
if(javaType == JavaTypesManager.STRING){
%>
<%=firstConnName%>.<%=column.getLabel() %> = value_<%=cid%> == null ? <%@ include file="./set_default_value.javajet"%> : value_<%=cid%>.toString();

View File

@@ -141,6 +141,10 @@
<PARAMETER NAME="CHECK_DATE" FIELD="CHECK" NUM_ROW="20" SHOW_IF="READ_BY=='XPATH'">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="USE_LOOP_AS_ROOT" FIELD="CHECK" NUM_ROW="25" SHOW_IF="READ_BY=='JSONPATH'" REPOSITORY_VALUE="USE_LOOP_AS_ROOT">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="ENCODING" FIELD="ENCODING_TYPE" NUM_ROW="30"
REQUIRED="true" REPOSITORY_VALUE="ENCODING">

View File

@@ -35,4 +35,5 @@ JSON_PATH_VERSION.ITEM.1_2_0=1.2.0
JSON_LOOP_QUERY.NAME=Loop Json query
MAPPING_JSONPATH.NAME=Mapping
MAPPING_JSONPATH.ITEM.QUERY=Json query
MAPPING_JSONPATH.ITEM.QUERY=Json query
USE_LOOP_AS_ROOT.NAME=Use the loop node as root

View File

@@ -777,7 +777,11 @@ if(!hasDynamic && isIncludeHeader){
xlsxTool_<%=cid%>.setRecalculateFormula(<%=recalculateFormula%>);
xlsxTool_<%=cid%>.setXY(<%=firstCellYAbsolute%>,<%=firstCellXStr%>,<%=firstCellYStr%>,<%=keepCellFormating%>);
<%if(!useStream){%>
xlsxTool_<%=cid%>.prepareXlsxFile(fileName_<%=cid%>);
java.util.concurrent.ConcurrentHashMap<java.lang.Object, java.lang.Object> chm_<%=cid%> = (java.util.concurrent.ConcurrentHashMap<java.lang.Object, java.lang.Object>) globalMap.get("concurrentHashMap");
java.lang.Object lockObj_<%=cid%> = chm_<%=cid%>.computeIfAbsent("EXCEL_OUTPUT_LOCK_OBJ_<%=cid%>", k -> new Object());
synchronized (lockObj_<%=cid%>) {
xlsxTool_<%=cid%>.prepareXlsxFile(fileName_<%=cid%>);
}
<%}else{%>
xlsxTool_<%=cid%>.prepareStream();
<%}%>

View File

@@ -88,7 +88,7 @@
<IMPORTS>
<IMPORT NAME="COMMONS-COMPRESS-1.10" MODULE="commons-compress-1.10.jar" MVN="mvn:org.talend.libraries/commons-compress-1.10/6.1.0" UrlPath="platform:/plugin/org.talend.libraries.apache.common/lib/commons-compress-1.10.jar" REQUIRED="true" />
<IMPORT NAME="Ant-Zip" MODULE="ant-1.10.1.jar" MVN="mvn:org.apache.ant/ant/1.10.1" REQUIRED="true" />
<IMPORT NAME="Check-Archive" MODULE="checkArchive-1.1-20171121.jar" MVN="mvn:org.talend.libraries/checkArchive-1.1-20171121/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/checkArchive-1.1-20171121.jar" REQUIRED="true" />
<IMPORT NAME="Check-Archive" MODULE="checkArchive-1.1-20181130.jar" MVN="mvn:org.talend.libraries/checkArchive-1.1-20181130/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/checkArchive-1.1-20181130.jar" REQUIRED="true" />
<IMPORT NAME="zip4j_1.3.1.jar" MODULE="zip4j_1.3.1.jar" MVN="mvn:org.talend.libraries/zip4j_1.3.1/6.0.0" REQUIRED="true" />
<IMPORT NAME="talendzip.jar" MODULE="talendzip.jar" MVN="mvn:org.talend.libraries/talendzip/6.1.0" REQUIRED="true" />
</IMPORTS>

View File

@@ -354,7 +354,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -199,7 +199,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="DRIVER=='JTDS'" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="DRIVER=='JTDS'" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="DRIVER=='MSSQL_PROP'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -258,9 +258,9 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Talend_MssqlUtil" MODULE="talend_DB_mssqlUtil-1.2-20171017.jar" MVN="mvn:org.talend.libraries/talend_DB_mssqlUtil-1.2-20171017/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_DB_mssqlUtil-1.2-20171017.jar" REQUIRED="true" />
<IMPORT NAME="Talend_MssqlUtil" MODULE="talend_DB_mssqlUtil-1.3-20190523.jar" MVN="mvn:org.talend.libraries/talend_DB_mssqlUtil-1.3-20190523/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_DB_mssqlUtil-1.3-20190523.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -275,7 +275,10 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
}
}else{
<%=prefix+cid%>.setNull(count<%=cid%>,java.sql.Types.DATE);
<%
String typeToSetNull = "TIME".equals(dbType) ? "java.sql.Types.TIME" : "java.sql.Types.DATE";
%>
<%=prefix+cid%>.setNull(count<%=cid%>,<%=typeToSetNull%>);
}
<%
}else{

View File

@@ -332,7 +332,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar"
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523"
REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>

View File

@@ -273,8 +273,10 @@ skeleton="../templates/db_output_bulk.skeleton"
%>
}
}else{
<%=prefix+cid%>.setNull(counter<%=cid%>,java.sql.Types.DATE);
<%
String typeToSetNull = "TIME".equals(dbType) ? "java.sql.Types.TIME" : "java.sql.Types.DATE";
%>
<%=prefix+cid%>.setNull(counter<%=cid%>,<%=typeToSetNull %>);
}
<%

View File

@@ -315,7 +315,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(DRIVER=='JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(DRIVER=='JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER=='MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -513,9 +513,9 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Talend_MssqlUtil" MODULE="talend_DB_mssqlUtil-1.2-20171017.jar" MVN="mvn:org.talend.libraries/talend_DB_mssqlUtil-1.2-20171017/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_DB_mssqlUtil-1.2-20171017.jar" REQUIRED="true" />
<IMPORT NAME="Talend_MssqlUtil" MODULE="talend_DB_mssqlUtil-1.3-20190523.jar" MVN="mvn:org.talend.libraries/talend_DB_mssqlUtil-1.3-20190523/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_DB_mssqlUtil-1.3-20190523.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -254,7 +254,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-jTDS" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED_IF="(DRIVER == 'JTDS') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -336,7 +336,7 @@
<IMPORT NAME="Driver-DB2-LICENSE-CISUZ" MODULE="db2jcc_license_cisuz.jar" MVN="mvn:org.talend.libraries/db2jcc_license_cisuz/6.0.0" REQUIRED="true" />
<IMPORT NAME="Driver-DB2-LICENSE-CU" MODULE="db2jcc_license_cu.jar" MVN="mvn:org.talend.libraries/db2jcc_license_cu/6.0.0" REQUIRED="true" />
<IMPORT NAME="Driver_MaxDB" MODULE="sapdbc.jar" MVN="mvn:org.talend.libraries/sapdbc/6.0.0" REQUIRED="true" />
<IMPORT NAME="Driver-MSSQL" MODULE="jtds-1.3.1-patch.jar" MVN="mvn:org.talend.libraries/jtds-1.3.1-patch/6.1.1" UrlPath="platform:/plugin/org.talend.libraries.jdbc.jtds/lib/jtds-1.3.1-patch.jar" REQUIRED="true" />
<IMPORT NAME="Driver-MSSQL" MODULE="jtds-1.3.1-patch-20190523.jar" MVN="mvn:net.sourceforge.jtds/jtds/1.3.1-patch-20190523" REQUIRED="true" />
<IMPORT NAME="Driver-Oracle11g-2" MODULE="ojdbc6.jar" MVN="mvn:org.talend.libraries/ojdbc6/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.jdbc.oracle/lib/ojdbc6.jar" REQUIRED="true" />
<IMPORT NAME="Driver-POSTGRESQL" MODULE="postgresql-8.3-603.jdbc3.jar" MVN="mvn:org.talend.libraries/postgresql-8.3-603.jdbc3/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.jdbc.postgresql/lib/postgresql-8.3-603.jdbc3.jar" REQUIRED="true" />
<IMPORT NAME="Driver-FIREBIRD" MODULE="jaybird-full-2.1.1.jar" MVN="mvn:org.talend.libraries/jaybird-full-2.1.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.jdbc.firebird/lib/jaybird-full-2.1.1.jar" REQUIRED="true" />

View File

@@ -398,10 +398,15 @@ skeleton="../templates/db_output_bulk.skeleton"
// added "print command" feature
<% if (printCommand == true) { %>
System.out.println("INFO: executing nzload command:");
for( String cmd : cmds ) {
System.out.print(cmd + " ");
}
System.out.println("INFO: executing nzload command:");
for(int i = 0; i < cmds.length; i++) {
if ("-pw".equals(cmds[i]) && decryptedPwd_<%=cid %>.equals(cmds[i+1])) {
System.out.print(cmds[i] + " " + routines.system.PasswordEncryptUtil.PASSWORD_FOR_LOGS_VALUE + " ");
i++;
} else {
System.out.print(cmds[i] + " ");
}
}
<% } %>
Runtime rt = Runtime.getRuntime();

View File

@@ -12,20 +12,30 @@ imports="
public void createURL(INode node) {
super.createURL(node);
boolean useLogFile= "true".equals(ElementParameterParser.getValue(node, "__USE_LOG_FILE__"));
String logLevel= ElementParameterParser.getValue(node, "__LOG_LEVEL__");
String logFile= ElementParameterParser.getValue(node, "__LOG_FILE__");
if (!logLevel.equals("0")) {
if (useLogFile) {
%>
java.io.Writer output_<%=cid %> = new java.io.FileWriter(<%= logFile %>, true);
java.io.PrintWriter logWriter_<%=cid %> = new java.io.PrintWriter(output_<%=cid %>,true);
globalMap.put("logWriter_<%=cid %>", logWriter_<%=cid %>);
java.sql.DriverManager.setLogWriter(logWriter_<%=cid %>);
<%
}else{%>
java.io.PrintWriter logWriter_<%=cid %> = new java.io.PrintWriter(System.out,true);
<%
}
%>
globalMap.put("logWriter_<%=cid %>", logWriter_<%=cid %>);
java.sql.DriverManager.setLogWriter(logWriter_<%=cid %>);
<%}%>
StringBuilder sbuilder_<%=cid%> = new StringBuilder();
sbuilder_<%=cid%>.append("jdbc:redshift://").append(<%=dbhost%>).append(":").append(<%=dbport%>).append("/").append(<%=dbname%>).append("?loglevel=").append(<%= logLevel %>);
<%
if(dbproperties != null && !"\"\"".equals(dbproperties) && !"".equals(dbproperties)) {
%>

View File

@@ -152,11 +152,19 @@
</PARAMETERS>
<ADVANCED_PARAMETERS>
<PARAMETER
NAME="USE_LOG_FILE"
FIELD="CHECK"
NUM_ROW="10"
REQUIRED="true"
>
</PARAMETER>
<PARAMETER
NAME="LOG_FILE"
FIELD="FILE"
NUM_ROW="10"
REQUIRED="true"
SHOW_IF="USE_LOG_FILE == 'true'"
>
<DEFAULT>"__COMP_DEFAULT_FILE_DIR__/redshift-jdbc.log"</DEFAULT>
</PARAMETER>

View File

@@ -16,6 +16,7 @@ AUTO_COMMIT.NAME = Auto Commit
TYPE.NAME=Database Driver
PROPERTY.NAME=Property Type
USE_LOG_FILE.NAME=Use Log file
LOG_FILE.NAME=Log file
LOG_LEVEL.NAME=Logging level
LOG_LEVEL.ITEM.NONE=None

View File

@@ -19,20 +19,31 @@ imports="
public void setURL(INode node) {
boolean useLogFile= "true".equals(ElementParameterParser.getValue(node, "__USE_LOG_FILE__"));
String logLevel= ElementParameterParser.getValue(node, "__LOG_LEVEL__");
String logFile= ElementParameterParser.getValue(node, "__LOG_FILE__");
if (!logLevel.equals("0")) {
if (useLogFile) {
%>
java.io.Writer output_<%=cid %> = new java.io.FileWriter(<%= logFile %>, true);
java.io.PrintWriter logWriter_<%=cid %> = new java.io.PrintWriter(output_<%=cid %>,true);
java.sql.DriverManager.setLogWriter(logWriter_<%=cid %>);
<%
}else{%>
java.io.PrintWriter logWriter_<%=cid %> = new java.io.PrintWriter(System.out,true);
<%
}
%>
globalMap.put("logWriter_<%=cid %>", logWriter_<%=cid %>);
java.sql.DriverManager.setLogWriter(logWriter_<%=cid %>);
<%}%>
StringBuilder sbuilder_<%=cid%> = new StringBuilder();
sbuilder_<%=cid%>.append("jdbc:redshift://").append(<%=dbhost%>).append(":").append(<%=dbport%>).append("/").append(<%=dbname%>).append("?loglevel=").append(<%= logLevel %>);
<%
if(dbproperties != null && !"\"\"".equals(dbproperties) && !"".equals(dbproperties)) {
%>
sbuilder_<%=cid%>.append("&").append(<%=dbproperties%>);

View File

@@ -198,12 +198,21 @@
</PARAMETERS>
<ADVANCED_PARAMETERS>
<PARAMETER
NAME="USE_LOG_FILE"
FIELD="CHECK"
NUM_ROW="10"
SHOW_IF="USE_EXISTING_CONNECTION == 'false'"
>
</PARAMETER>
<PARAMETER
NAME="LOG_FILE"
FIELD="FILE"
NUM_ROW="10"
REQUIRED="true"
SHOW_IF="USE_EXISTING_CONNECTION == 'false'"
SHOW_IF="(USE_EXISTING_CONNECTION == 'false') AND (USE_LOG_FILE == 'true')"
>
<DEFAULT>"__COMP_DEFAULT_FILE_DIR__/redshift-jdbc.log"</DEFAULT>
</PARAMETER>

View File

@@ -36,6 +36,7 @@ TRIM_COLUMN.ITEM.TRIM=Trim
TABLE.NAME=Table Name
MAPPING.NAME=Mapping
USE_LOG_FILE.NAME=Use Log file
LOG_FILE.NAME=Log file
LOG_LEVEL.NAME=Logging level
LOG_LEVEL.ITEM.NONE=None

View File

@@ -65,7 +65,7 @@ if (metadatas != null && metadatas.size() > 0) {
%>
java.util.Map<<%=lookup %>Struct, <%=lookup %>Struct> tMap_<%=lookup%>_<%=cid %> = (java.util.Map<<%=lookup %>Struct, <%=lookup %>Struct>) globalMap.get("tHash_<%=lookup %>");
java.util.Map<<%=typeToGenerate_lookup%>,<%=typeToGenerate_replacement%>> replace_<%=cid%> = new java.util.HashMap<<%=typeToGenerate_lookup%>,<%=typeToGenerate_replacement%>>();
java.util.Map<<%=typeToGenerate_lookup%>,<%=typeToGenerate_replacement%>> replace_<%=cid%> = new java.util.LinkedHashMap<<%=typeToGenerate_lookup%>,<%=typeToGenerate_replacement%>>();
for (Object o : tMap_<%=lookup%>_<%=cid %>.keySet()) {

View File

@@ -311,16 +311,51 @@ String inputConnName = null;
}
%>
<%=localContext%>.synchronizeContext();
<%
<%
List<IContextParameter> params = currentProcess.getContextManager().getDefaultContext().getContextParameterList();
for (IContextParameter ctxParam :params){
String ctxParamName = ctxParam.getName();
%>
parentContextMap_<%=cid %>.put("<%=ctxParamName %>", <%=localContext%>.<%=ctxParamName %>);
paraList_<%=cid %>.add("--context_type " + "<%=ctxParam.getName()%>" + "=" + "<%=ctxParam.getType()%>");
<%
}
%>
/*Create local class to avoid 64kB method problem when huge amount of context variables declared*/
if (!params.isEmpty()) {
final int defaultInnerMethodLenght = 500;
int lastMethodNumber = 0;
%>
class ContextProcessor_<%=cid %> {
private void transmitContext_0() {
<%
for (int i = 1; i <= params.size(); i++) {
IContextParameter ctxParam = params.get(i-1);
String ctxParamName = ctxParam.getName();
String ctxParameterType = ctxParam.getType();
if (i % defaultInnerMethodLenght == 0) {
/* close previous method and declare new */
lastMethodNumber++;
%>
}
private void transmitContext_<%=lastMethodNumber%>() {
<%
} //endIf
%>
parentContextMap_<%=cid %>.put("<%=ctxParamName %>", <%=localContext%>.<%=ctxParamName %>);
paraList_<%=cid %>.add("--context_type " + "<%=ctxParamName%>" + "=" + "<%=ctxParam.getType()%>");
<%
} //endFor
%>
<%/*close last method*/%>
}
public void transmitAllContext() {
<%
for (int i = 0; i <=lastMethodNumber; i++) {
%>
transmitContext_<%=i %>();
<%
}
%>
}
}
new ContextProcessor_<%=cid %>().transmitAllContext();
<%
}
%>
java.util.Enumeration<?> propertyNames_<%=cid %> = <%=localContext%>.propertyNames();
while (propertyNames_<%=cid %>.hasMoreElements()) {
String key_<%=cid %> = (String) propertyNames_<%=cid %>.nextElement();
@@ -426,7 +461,8 @@ String inputConnName = null;
<%if (dieOnError) { %>
if (childJob_<%=cid %>.getErrorCode() != null || ("failure").equals(childJob_<%=cid %>.getStatus())) {
throw new RuntimeException("Child job running failed.\n"+childJob_<%=cid %>.getException().getClass().getName() + ": " + childJob_<%=cid %>.getException().getMessage());
java.lang.Exception ce_<%=cid%> = childJob_<%=cid %>.getException();
throw new RuntimeException("Child job running failed.\n" + ((ce_<%=cid%>!=null) ? (ce_<%=cid%>.getClass().getName() + ": " + ce_<%=cid%>.getMessage()) : ""));
}
<%}
}%>
@@ -507,66 +543,92 @@ String inputConnName = null;
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// use independent process to run subjob
%>
Runtime runtime_<%=cid %> = Runtime.getRuntime();
final Process ps_<%=cid %>;
ps_<%=cid %> = runtime_<%=cid %>.exec((String[])paraList_<%=cid %>.toArray(new String[paraList_<%=cid %>.size()]));
Thread normal_<%=cid %> = new Thread() {
public void run() {
try {
java.io.BufferedReader reader = new java.io.BufferedReader(new java.io.InputStreamReader(ps_<%=cid %>.getInputStream()));
String line = "";
try {
while((line = reader.readLine()) != null) {
System.out.println(line);
}
} finally {
reader.close();
class ConsoleHelper_<%=cid %> {
private Thread getNormalThread(Process process) {
return new Thread() {
public void run() {
try {
java.io.BufferedReader reader = new java.io.BufferedReader(
new java.io.InputStreamReader(
process.getInputStream()));
String line = "";
try {
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
} finally {
reader.close();
}
} catch (java.io.IOException ioe) {
<%if(isLog4jEnabled){%>
log.error("<%=cid %> - " + ioe.getMessage());
<%}%>
ioe.printStackTrace();
}
}
};
}
private Thread getErrorThread(Process process, StringBuffer sb) {
return new Thread() {
public void run() {
try {
java.io.BufferedReader reader = new java.io.BufferedReader(
new java.io.InputStreamReader(
process.getErrorStream()));
String line = "";
try {
while ((line = reader.readLine()) != null) {
sb.append(line)
.append("\n");
}
} finally {
reader.close();
}
} catch (java.io.IOException ioe) {
<%if(isLog4jEnabled){%>
log.error("<%=cid %> - " + ioe.getMessage());
<%}%>
ioe.printStackTrace();
}
}
};
}
} catch(java.io.IOException ioe) {
<%if(isLog4jEnabled){%>
log.error("<%=cid %> - " + ioe.getMessage());
<%}%>
ioe.printStackTrace();
}
}
};
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - The child job '<%if(!useDynamicJob){%><%=childJob %><%}else{%>"+<%=dynamicJobName%>+"<%}%>' starts on the version '<%=version%>' with the context '<%=context%>'.");
<%}%>
normal_<%=cid %>.start();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - The child job '<%if(!useDynamicJob){%><%=childJob %><%}else{%>"+<%=dynamicJobName%>+"<%}%>' is done.");
<%}%>
ConsoleHelper_<%=cid %> consoleHelper_<%=cid %> = new ConsoleHelper_<%=cid %>();
final StringBuffer errorMsg_<%=cid %> = new StringBuffer();
Thread error_<%=cid %> = new Thread() {
public void run() {
try {
java.io.BufferedReader reader = new java.io.BufferedReader(new java.io.InputStreamReader(ps_<%=cid %>.getErrorStream()));
String line = "";
try {
while((line = reader.readLine()) != null) {
errorMsg_<%=cid %>.append(line).append("\n");
}
} finally {
reader.close();
}
} catch(java.io.IOException ioe) {
<%if(isLog4jEnabled){%>
log.error("<%=cid %> - " + ioe.getMessage());
<%}%>
ioe.printStackTrace();
}
}
};
error_<%=cid %>.start();
Runtime runtime_<%=cid %> = Runtime.getRuntime();
Process ps_<%=cid %> = null;
//0 indicates normal termination
int result_<%=cid %>;
StringBuffer errorMsg_<%=cid %> = new StringBuffer();
try {
ps_<%=cid %> = runtime_<%=cid %>.exec((String[])paraList_<%=cid %>.toArray(new String[paraList_<%=cid %>.size()]));
Thread normal_<%=cid %> = consoleHelper_<%=cid %>.getNormalThread(ps_<%=cid %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - The child job '<%if(!useDynamicJob){%><%=childJob %><%}else{%>"+<%=dynamicJobName%>+"<%}%>' starts on the version '<%=version%>' with the context '<%=context%>'.");
<%}%>
normal_<%=cid %>.start();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - The child job '<%if(!useDynamicJob){%><%=childJob %><%}else{%>"+<%=dynamicJobName%>+"<%}%>' is done.");
<%}%>
Thread error_<%=cid %> = consoleHelper_<%=cid %>.getErrorThread(ps_<%=cid %>, errorMsg_<%=cid %>);
error_<%=cid %>.start();
result_<%=cid %> = ps_<%=cid %>.waitFor();
normal_<%=cid %>.join(10000);
error_<%=cid %>.join(10000);
} catch (ThreadDeath tde) {
<%if(isLog4jEnabled){%>
log.error("<%=cid %> - thread was terminated.");
<%}%>
ps_<%=cid %>.destroy();
throw tde;
}
//0 indicates normal termination
int result_<%=cid %> = ps_<%=cid %>.waitFor();
normal_<%=cid %>.join(10000);
error_<%=cid %>.join(10000);
globalMap.put("<%=cid %>_CHILD_RETURN_CODE",result_<%=cid %>);
if(result_<%=cid %> != 0){
globalMap.put("<%=cid %>_CHILD_EXCEPTION_STACKTRACE",errorMsg_<%=cid %>.toString());

View File

@@ -0,0 +1,54 @@
<%@ jet
imports="
org.talend.core.model.process.INode
org.talend.core.model.process.ElementParameterParser
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.utils.NodeUtil
java.util.List
java.util.Map
"
%>
<%@ include file="../templates/DB/Input/HelpClass.javajet"%>
<%
class DBInputBeginUtil extends DefaultDBInputUtil{
public void setURL(INode node) {
String jdbcUrl = ElementParameterParser.getValue(node, "__URL__");
%>
String url_<%=cid %> = <%=jdbcUrl%>;
<%
}
public String getDirverClassName(INode node){
return "org.jdbc4olap.jdbc.OlapDriver";
}
//-----------according schema type to generate ResultSet
public void generateTimestampResultSet(String firstConnName, IMetadataColumn column, int currentColNo) {
%>
java.util.Date date_<%=cid%> = null;
try{
date_<%=cid%> = rs_<%=cid %>.getTimestamp(<%if(isDynamic){%>column_index_<%=cid%><%}else{%><%=currentColNo%><%}%>);
}catch(java.lang.Exception e){
date_<%=cid%> = rs_<%=cid %>.getDate(<%if(isDynamic){%>column_index_<%=cid%><%}else{%><%=currentColNo%><%}%>);
}
<%=firstConnName%>.<%=column.getLabel()%> = date_<%=cid%>;
<%
}
//---------end according schema type to generate ResultSet
}//end class
dbInputBeginUtil = new DBInputBeginUtil();
%>
<%@ include file="../templates/DB/Input/AbstractDBInputBegin.javajet"%>

View File

@@ -0,0 +1,31 @@
<%@ jet
imports="
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.process.ElementParameterParser
"
%>
<%@ include file="../templates/Log4j/Log4jDBConnUtil.javajet"%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode) codeGenArgument.getArgument();
String cid = node.getUniqueName();
%>
}
}finally{
if(rs_<%=cid%> !=null){
rs_<%=cid %>.close();
}
stmt_<%=cid%>.close();
if(conn_<%=cid%> != null && !conn_<%=cid%>.isClosed()) {
<%log4jCodeGenerateUtil.close(node);%>
}
}
globalMap.put("<%=cid %>_NB_LINE", nb_line_<%=cid%>);
<%
log4jCodeGenerateUtil.retrieveRecordsCount(node);
%>

View File

@@ -41,16 +41,6 @@
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="DRIVER_CLASS" FIELD="TEXT" NUM_ROW="50"
REQUIRED="true" SHOW="false">
<DEFAULT>"org.jdbc4olap.jdbc.OlapDriver"</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CODE" FIELD="MEMO_JAVA" RAW="true" REQUIRED="false"
NUM_ROW="60" SHOW="false">
<DEFAULT></DEFAULT>
</PARAMETER>
<PARAMETER NAME="TABLE" FIELD="DBTABLE" NUM_ROW="70">
<DEFAULT>""</DEFAULT>
</PARAMETER>
@@ -68,11 +58,6 @@
</PARAMETERS>
<ADVANCED_PARAMETERS>
<PARAMETER NAME="ENCODING" FIELD="ENCODING_TYPE" NUM_ROW="10"
REQUIRED="true" SHOW="false">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="TRIM_ALL_COLUMN" FIELD="CHECK" NUM_ROW="20">
<DEFAULT>false</DEFAULT>
</PARAMETER>
@@ -86,28 +71,6 @@
</ADVANCED_PARAMETERS>
<CODEGENERATION>
<TEMPLATES INPUT="In" OUTPUT="Out">
<TEMPLATE NAME="In" COMPONENT="tJava">
<LINK_TO NAME="Out" CTYPE="FLOW" />
</TEMPLATE>
<TEMPLATE NAME="Out" COMPONENT="tJDBCInput" />
<TEMPLATE_PARAM SOURCE="self.CODE" TARGET="In.CODE" />
<TEMPLATE_PARAM SOURCE="self.URL" TARGET="Out.URL" />
<TEMPLATE_PARAM SOURCE="self.USER" TARGET="Out.USER" />
<TEMPLATE_PARAM SOURCE="self.PASS" TARGET="Out.PASS" />
<TEMPLATE_PARAM SOURCE="self.SCHEMA" TARGET="Out.SCHEMA" />
<TEMPLATE_PARAM SOURCE="self.DRIVER_CLASS" TARGET="Out.DRIVER_CLASS" />
<TEMPLATE_PARAM SOURCE="self.TABLE" TARGET="Out.TABLE" />
<TEMPLATE_PARAM SOURCE="self.QUERYSTORE" TARGET="Out.QUERYSTORE" />
<TEMPLATE_PARAM SOURCE="self.QUERY" TARGET="Out.QUERY" />
<TEMPLATE_PARAM SOURCE="self.ENCODING" TARGET="Out.ENCODING" />
<TEMPLATE_PARAM SOURCE="self.TRIM_ALL_COLUMN" TARGET="Out.TRIM_ALL_COLUMN" />
<TEMPLATE_PARAM SOURCE="self.TRIM_COLUMN" TARGET="Out.TRIM_COLUMN" />
</TEMPLATES>
<IMPORTS>
<IMPORT NAME="Driver-OLAP" MODULE="jdbc4olap-1.1.4.jar" MVN="mvn:org.jdbc4olap/jdbc4olap/1.1.4"
REQUIRED="true" />

View File

@@ -9,6 +9,7 @@ imports="
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.metadata.types.JavaType
org.talend.core.model.utils.TalendTextUtils
java.util.List
"
%>
@@ -82,7 +83,7 @@ if (inputConnName != null){
}
if(column.getComment()!=null || column.getComment().length()>0){
%>
spVar_<%=i%>.setLabel("<%=column.getComment()%>");
spVar_<%=i%>.setLabel("<%=TalendTextUtils.escapeJavaText(column.getComment())%>");
<%
}
%>

View File

@@ -259,7 +259,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Driver-mssql-jdbc" MODULE="mssql-jdbc.jar" MVN="mvn:org.talend.libraries/mssql-jdbc/6.0.0" REQUIRED_IF="(DRIVER == 'MSSQL_PROP') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Talend_MssqlUtil" MODULE="talend_DB_mssqlUtil-1.2-20171017.jar" MVN="mvn:org.talend.libraries/talend_DB_mssqlUtil-1.2-20171017/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_DB_mssqlUtil-1.2-20171017.jar" REQUIRED="true" />
<IMPORT NAME="Talend_MssqlUtil" MODULE="talend_DB_mssqlUtil-1.3-20190523.jar" MVN="mvn:org.talend.libraries/talend_DB_mssqlUtil-1.3-20190523/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_DB_mssqlUtil-1.3-20190523.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -6,6 +6,7 @@ imports="
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.process.IProcess
org.talend.core.model.process.INode
org.talend.core.model.utils.TalendTextUtils
java.io.File
java.io.BufferedWriter
java.io.FileOutputStream
@@ -139,7 +140,7 @@ imports="
}
if(imetadataColumn.getComment() != null && !"".equals(imetadataColumn.getComment())){
%>
.comment("<%= imetadataColumn.getComment()%>")
.comment("<%= TalendTextUtils.escapeJavaText(imetadataColumn.getComment())%>")
<%
}
%>

View File

@@ -1,548 +1,554 @@
<%@ jet
imports="
java.util.ArrayList
java.util.List
java.util.Map
<%@ jet
imports="
java.util.ArrayList
java.util.List
java.util.Map
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.metadata.types.JavaType
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.process.ElementParameterParser
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.utils.NodeUtil
"
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.metadata.types.JavaType
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.process.ElementParameterParser
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.utils.NodeUtil
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
final INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
final INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
/*in shema:*/
List<? extends IConnection> listInConns = node.getIncomingConnections();
String sInConnName = null;
IConnection inConn = null;
List<IMetadataColumn> listInColumns = null;
/*in shema:*/
List<? extends IConnection> listInConns = node.getIncomingConnections();
String sInConnName = null;
IConnection inConn = null;
List<IMetadataColumn> listInColumns = null;
if (listInConns != null && listInConns.size() > 0) {
IConnection inConnTemp = listInConns.get(0);
sInConnName = inConnTemp.getName();
if(inConnTemp.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)){
inConn = inConnTemp;
listInColumns = inConnTemp.getMetadataTable().getListColumns();
}
}
if (listInConns != null && listInConns.size() > 0) {
IConnection inConnTemp = listInConns.get(0);
sInConnName = inConnTemp.getName();
if(inConnTemp.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)){
inConn = inConnTemp;
listInColumns = inConnTemp.getMetadataTable().getListColumns();
}
}
/* get the schema of itself (maybe no output flow)*/
List<IMetadataColumn> listColumsToTest = node.getMetadataList().get(0).getListColumns();
String anotherChecked = ElementParameterParser.getValue(node, "__CHECK_ANOTHER__");
String checkAll = ElementParameterParser.getValue(node, "__CHECK_ALL__");
final boolean bIsTrim = "true".equals(ElementParameterParser.getValue(node, "__SUB_STRING__"));
final boolean useFasteDateChecker = "true".equals(ElementParameterParser.getValue(node, "__FAST_DATE_CHECK__"));
final boolean emptyIsNull = "true".equals(ElementParameterParser.getValue(node, "__EMPTY_IS_NULL__"));
final boolean allEmptyAreNull = "true".equals(ElementParameterParser.getValue(node, "__ALL_EMPTY_ARE_NULL__"));
List<Map<String, String>> list = (List<Map<String, String>>)ElementParameterParser.getObjectValue(node, "__EMPTY_NULL_TABLE__");
final List<String> listEmptyAsNull = new ArrayList<String>();
for(Map<String, String> map : list){
if("true".equals(map.get("EMPTY_NULL"))){
listEmptyAsNull.add(map.get("SCHEMA_COLUMN"));
}
}
/* get the schema of itself (maybe no output flow)*/
List<IMetadataColumn> listColumsToTest = node.getMetadataList().get(0).getListColumns();
class SchemaChecker { //CLASS SCHEMACHECKER START
boolean anotherChecked = "true".equals(ElementParameterParser.getValue(node, "__CHECK_ANOTHER__"));
boolean ignoreTimeZone = "true".equals(ElementParameterParser.getValue(node, "__IGNORE_TIMEZONE__"));
String anotherChecked = ElementParameterParser.getValue(node, "__CHECK_ANOTHER__");
String checkAll = ElementParameterParser.getValue(node, "__CHECK_ALL__");
final boolean bIsTrim = "true".equals(ElementParameterParser.getValue(node, "__SUB_STRING__"));
final boolean useFasteDateChecker = "true".equals(ElementParameterParser.getValue(node, "__FAST_DATE_CHECK__"));
final boolean useStrictDateChecker = "true".equals(ElementParameterParser.getValue(node, "__STRICT_DATE_CHECK__"));
final boolean emptyIsNull = "true".equals(ElementParameterParser.getValue(node, "__EMPTY_IS_NULL__"));
final boolean allEmptyAreNull = "true".equals(ElementParameterParser.getValue(node, "__ALL_EMPTY_ARE_NULL__"));
public void testDataType(boolean _bNullable, String _sInConnName, IMetadataColumn metadataColumn, String typeSelected, String cid) { //METHOD_TESTDATATYPE START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType( javaType, metadataColumn.isNullable());
String colName = metadataColumn.getLabel();
List<Map<String, String>> list = (List<Map<String, String>>)ElementParameterParser.getObjectValue(node, "__EMPTY_NULL_TABLE__");
final List<String> listEmptyAsNull = new ArrayList<String>();
for(Map<String, String> map : list){
if("true".equals(map.get("EMPTY_NULL"))){
listEmptyAsNull.add(map.get("SCHEMA_COLUMN"));
}
}
if (javaType == JavaTypesManager.OBJECT || javaType == JavaTypesManager.STRING) { //CONDITION_00100 START
%>
try {
if(
<%
if (_bNullable){ //CONDITION_00110 START
%>
<%=_sInConnName%>.<%=colName%> != null
<%
if(allEmptyAreNull || listEmptyAsNull.contains(metadataColumn.getLabel())) {
%>
&& (!"".equals(<%=_sInConnName%>.<%=colName%>))
<%
}
}else if(!isPrimitive){ //CONDITION_00110 ELSE IF
%>
<%=_sInConnName%>.<%=colName%> != null
<%
}else{ //CONDITION_00110 ELSE
%>
true
<%
} //CONDITION_00110 STOP
%>
) {
<%
if(typeSelected.equals("Boolean") ) { //CONDITION_00120 START
%>
if(!("true".equals(<%=_sInConnName%>.<%=colName%>) || "false".equals(<%=_sInConnName%>.<%=colName%>))){
throw new java.lang.Exception("Wrong Boolean type!");
}
<%
} else if(typeSelected.equals("Character")) { //CONDITION_00120 ELSE IF
%>
if(<%=_sInConnName%>.<%=colName%>.toCharArray().length != 1){
throw new java.lang.Exception("Wrong Character type!");
}
<%
} else if(typeSelected.equals("BigDecimal")) { //CONDITION_00120 ELSE IF
%>
<%=typeSelected%> tester_<%=cid%> = new <%=typeSelected%>(<%=_sInConnName%>.<%=colName%>);
<%
} else if(typeSelected.equals("Object")){ //CONDITION_00120 ELSE IF
%>
<%=typeSelected%> tester_<%=cid%> = new <%=typeSelected%>();
<%
} else { //CONDITION_00120 ELSE
%>
<%=typeSelected%> tester_<%=cid%> = <%=typeSelected%>.valueOf(<%=_sInConnName%>.<%=colName%>);
<%
} //CONDITION_00120 STOP
%>
}
} catch(java.lang.Exception e) {
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|wrong type";
}
<%
} //CONDITION_00100 STOP
} //METHOD_TESTDATATYPE STOP
class SchemaChecker { //CLASS SCHEMACHECKER START
boolean anotherChecked = "true".equals(ElementParameterParser.getValue(node, "__CHECK_ANOTHER__"));
boolean ignoreTimeZone = "true".equals(ElementParameterParser.getValue(node, "__IGNORE_TIMEZONE__"));
public void testPrecision(int _maxLength, int iPrecision, String _sInConnName, IMetadataColumn metadataColumn, String typeSelected, String cid) { //METHOD_TESTPRECISION START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
String colName = metadataColumn.getLabel();
boolean needCheck = false;
if(anotherChecked) {
if("BigDecimal".equalsIgnoreCase(typeSelected)) {
needCheck = true;
}
} else if (javaType == JavaTypesManager.BIGDECIMAL) {
/* NULLable, in case input value is Null, do nothing...
Non-NULLable,
(1) in case input value is Non-null, go into...;
(2) in case input value is Null, do nothing and warning by NULL-CHECKER.
*/
/*
if precision value is not empty or Null, checking "Precision" at first, if passed then checking "Length"
*/
needCheck = true;
}
if(needCheck) { //CONDITION_00130 START
%>
if (<%=_sInConnName%>.<%=colName%> != null){
<%
if(javaType == JavaTypesManager.BIGDECIMAL) { //CONDITION_00131 START
%>
handleBigdecimalPrecision((<%=_sInConnName%>.<%=colName%>).toPlainString(), <%=iPrecision%>, <%=_maxLength%>);
<%
} else { //CONDITION_00131 ELSE
%>
handleBigdecimalPrecision(String.valueOf(<%=_sInConnName%>.<%=colName%>), <%=iPrecision%>, <%=_maxLength%>);
<%
} //CONDITION_00131 STOP
%>
ifPassedThrough = ifPassedThrough?ifPassed:false;
errorCodeThrough += errorCode;
errorMessageThrough += errorMessage;
}
<%
} //CONDITION_00130 STOP
} //METHOD_TESTPRECISION STOP
public void testDataType(boolean _bNullable, String _sInConnName, IMetadataColumn metadataColumn, String typeSelected, String cid) { //METHOD_TESTDATATYPE START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType( javaType, metadataColumn.isNullable());
String colName = metadataColumn.getLabel();
public void testDataLength(boolean _bNullable, String _sInConnName,IMetadataColumn inColumn, IMetadataColumn metadataColumn, int maxLength, String cid) { //METHOD_TESTDATALENGTH START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType(javaType, metadataColumn.isNullable());
boolean bIsStringType = (javaType == JavaTypesManager.STRING), bIsIntegerType = (javaType == JavaTypesManager.INTEGER);
String colName = inColumn.getLabel();
if (javaType == JavaTypesManager.OBJECT || javaType == JavaTypesManager.STRING) { //CONDITION_00100 START
%>
try {
if(
<%
if (_bNullable){ //CONDITION_00110 START
%>
<%=_sInConnName%>.<%=colName%> != null
<%
if(allEmptyAreNull || listEmptyAsNull.contains(metadataColumn.getLabel())) {
%>
&& (!"".equals(<%=_sInConnName%>.<%=colName%>))
<%
}
}else if(!isPrimitive){ //CONDITION_00110 ELSE IF
%>
<%=_sInConnName%>.<%=colName%> != null
<%
}else{ //CONDITION_00110 ELSE
%>
true
<%
} //CONDITION_00110 STOP
%>
) {
<%
if(typeSelected.equals("Boolean") ) { //CONDITION_00120 START
%>
if(!("true".equals(<%=_sInConnName%>.<%=colName%>) || "false".equals(<%=_sInConnName%>.<%=colName%>))){
throw new java.lang.Exception("Wrong Boolean type!");
}
<%
} else if(typeSelected.equals("Character")) { //CONDITION_00120 ELSE IF
%>
if(<%=_sInConnName%>.<%=colName%>.toCharArray().length != 1){
throw new java.lang.Exception("Wrong Character type!");
}
<%
} else if(typeSelected.equals("BigDecimal")) { //CONDITION_00120 ELSE IF
%>
<%=typeSelected%> tester_<%=cid%> = new <%=typeSelected%>(<%=_sInConnName%>.<%=colName%>);
<%
} else if(typeSelected.equals("Object")){ //CONDITION_00120 ELSE IF
%>
<%=typeSelected%> tester_<%=cid%> = new <%=typeSelected%>();
<%
} else { //CONDITION_00120 ELSE
%>
<%=typeSelected%> tester_<%=cid%> = <%=typeSelected%>.valueOf(<%=_sInConnName%>.<%=colName%>);
<%
} //CONDITION_00120 STOP
%>
}
} catch(java.lang.Exception e) {
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|wrong type";
}
<%
} //CONDITION_00100 STOP
} //METHOD_TESTDATATYPE STOP
if (maxLength > 0 && ( bIsStringType || bIsIntegerType )){ //CONDITION_00140 START
public void testPrecision(int _maxLength, int iPrecision, String _sInConnName, IMetadataColumn metadataColumn, String typeSelected, String cid) { //METHOD_TESTPRECISION START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
String colName = metadataColumn.getLabel();
boolean needCheck = false;
if(anotherChecked) {
if("BigDecimal".equalsIgnoreCase(typeSelected)) {
needCheck = true;
}
} else if (javaType == JavaTypesManager.BIGDECIMAL) {
/* NULLable, in case input value is Null, do nothing...
Non-NULLable,
(1) in case input value is Non-null, go into...;
(2) in case input value is Null, do nothing and warning by NULL-CHECKER.
*/
/*
if precision value is not empty or Null, checking "Precision" at first, if passed then checking "Length"
*/
needCheck = true;
}
if(needCheck) { //CONDITION_00130 START
%>
if (
if (<%=_sInConnName%>.<%=colName%> != null){
<%
if (_bNullable){ //CONDITION_00141 START
if(javaType == JavaTypesManager.BIGDECIMAL) { //CONDITION_00131 START
%>
<%=_sInConnName%>.<%=colName%> != null
handleBigdecimalPrecision((<%=_sInConnName%>.<%=colName%>).toPlainString(), <%=iPrecision%>, <%=_maxLength%>);
<%
if(allEmptyAreNull || listEmptyAsNull.contains(metadataColumn.getLabel())) {
} else { //CONDITION_00131 ELSE
%>
&& (!"".equals(<%=_sInConnName%>.<%=colName%>))
handleBigdecimalPrecision(String.valueOf(<%=_sInConnName%>.<%=colName%>), <%=iPrecision%>, <%=_maxLength%>);
<%
}
}else if (!isPrimitive){ //CONDITION_00141 ELSE IF
%>
<%=_sInConnName%>.<%=colName%> != null
<%
}else { //CONDITION_00141 ELSE
%>
true
<%
} //CONDITION_00141 STOP
} //CONDITION_00131 STOP
%>
) {
ifPassedThrough = ifPassedThrough?ifPassed:false;
errorCodeThrough += errorCode;
errorMessageThrough += errorMessage;
}
<%
if ( bIsTrim ){ //CONDITION_00142 START
if (bIsStringType) { //CONDITION_001421 START
} //CONDITION_00130 STOP
} //METHOD_TESTPRECISION STOP
public void testDataLength(boolean _bNullable, String _sInConnName,IMetadataColumn inColumn, IMetadataColumn metadataColumn, int maxLength, String cid) { //METHOD_TESTDATALENGTH START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType(javaType, metadataColumn.isNullable());
boolean bIsStringType = (javaType == JavaTypesManager.STRING), bIsIntegerType = (javaType == JavaTypesManager.INTEGER);
String colName = inColumn.getLabel();
if (maxLength > 0 && ( bIsStringType || bIsIntegerType )){ //CONDITION_00140 START
%>
if( <%=_sInConnName%>.<%=colName%>.length() > <%=maxLength%> )
<%=_sInConnName%>.<%=colName%> = <%=_sInConnName%>.<%=colName%>.substring(0, <%=maxLength%>);
if (
<%
} else if ( bIsIntegerType ){//CONDITION_001421 ELSE IF
String generatedType = JavaTypesManager.getTypeToGenerate(metadataColumn.getTalendType(), metadataColumn.isNullable());
if ("int".equals(generatedType)) { //CONDITION_0014211 START
if (_bNullable){ //CONDITION_00141 START
%>
tmpContentThrough = String.valueOf(<%=_sInConnName%>.<%=colName%>);
<%=_sInConnName%>.<%=colName%> != null
<%
} else{ //CONDITION_0014211 ELSE
if(allEmptyAreNull || listEmptyAsNull.contains(metadataColumn.getLabel())) {
%>
tmpContentThrough = <%=_sInConnName%>.<%=colName%>.toString();
&& (!"".equals(<%=_sInConnName%>.<%=colName%>))
<%
} //CONDITION_0014211 STOP
}
}else if (!isPrimitive){ //CONDITION_00141 ELSE IF
%>
if (tmpContentThrough.length() > <%=maxLength%>)
<%=_sInConnName%>.<%=colName%> = <%=_sInConnName%>.<%=colName%>.substring(0, <%=maxLength%>);
<%=_sInConnName%>.<%=colName%> != null
<%
} //CONDITION_001421 STOP
} else{ //CONDITION_00142 ELSE
if (bIsStringType) { //CONDITION_001422 START
}else { //CONDITION_00141 ELSE
%>
if (<%=_sInConnName%>.<%=colName%>.length() > <%=maxLength%>) {
ifPassedThrough = false;
errorCodeThrough += 8;
errorMessageThrough += "|exceed max length";
}
true
<%
} else if (bIsIntegerType) { //CONDITION_001422 ELSE IF
String generatedType = JavaTypesManager.getTypeToGenerate(metadataColumn.getTalendType(), metadataColumn.isNullable());
if ("int".equals(generatedType)) { //CONDITION_0014221 START
} //CONDITION_00141 STOP
%>
tmpContentThrough = String.valueOf(<%=_sInConnName%>.<%=colName%>);
) {
<%
} else { //CONDITION_0014221 ELSE
if ( bIsTrim ){ //CONDITION_00142 START
if (bIsStringType) { //CONDITION_001421 START
%>
tmpContentThrough = <%=_sInConnName%>.<%=colName%>.toString();
if( <%=_sInConnName%>.<%=colName%>.length() > <%=maxLength%> )
<%=_sInConnName%>.<%=colName%> = <%=_sInConnName%>.<%=colName%>.substring(0, <%=maxLength%>);
<%
} //CONDITION_0014221 STOP
} else if ( bIsIntegerType ){//CONDITION_001421 ELSE IF
String generatedType = JavaTypesManager.getTypeToGenerate(metadataColumn.getTalendType(), metadataColumn.isNullable());
if ("int".equals(generatedType)) { //CONDITION_0014211 START
%>
tmpContentThrough = String.valueOf(<%=_sInConnName%>.<%=colName%>);
<%
} else{ //CONDITION_0014211 ELSE
%>
tmpContentThrough = <%=_sInConnName%>.<%=colName%>.toString();
<%
} //CONDITION_0014211 STOP
%>
if (tmpContentThrough.length() > <%=maxLength%>)
<%=_sInConnName%>.<%=colName%> = <%=_sInConnName%>.<%=colName%>.substring(0, <%=maxLength%>);
<%
} //CONDITION_001421 STOP
} else{ //CONDITION_00142 ELSE
if (bIsStringType) { //CONDITION_001422 START
%>
if (<%=_sInConnName%>.<%=colName%>.length() > <%=maxLength%>) {
ifPassedThrough = false;
errorCodeThrough += 8;
errorMessageThrough += "|exceed max length";
}
<%
} else if (bIsIntegerType) { //CONDITION_001422 ELSE IF
String generatedType = JavaTypesManager.getTypeToGenerate(metadataColumn.getTalendType(), metadataColumn.isNullable());
if ("int".equals(generatedType)) { //CONDITION_0014221 START
%>
tmpContentThrough = String.valueOf(<%=_sInConnName%>.<%=colName%>);
<%
} else { //CONDITION_0014221 ELSE
%>
tmpContentThrough = <%=_sInConnName%>.<%=colName%>.toString();
<%
} //CONDITION_0014221 STOP
%>
if (tmpContentThrough.length() > <%=maxLength%>) {
ifPassedThrough = false;
errorCodeThrough += 8;
errorMessageThrough += "|exceed max length";
}
if (tmpContentThrough.length() > <%=maxLength%>) {
ifPassedThrough = false;
errorCodeThrough += 8;
errorMessageThrough += "|exceed max length";
}
<%
}//CONDITION_001422 STOP
} //CONDITION_00142 STOP
}//CONDITION_001422 STOP
} //CONDITION_00142 STOP
%>
}
<%
} //CONDITION_00140 STOP
} //METHOD_TESTDATALENGTH STOP
}
<%
} //CONDITION_00140 STOP
} //METHOD_TESTDATALENGTH STOP
public void testDate(boolean _bNullable, String _sInConnName, IMetadataColumn metadataColumn, String pattern, String cid) { //METHOD_TESTDATE START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType( javaType, metadataColumn.isNullable());
String colName = metadataColumn.getLabel();
public void testDate(boolean _bNullable, String _sInConnName, IMetadataColumn metadataColumn, String pattern, String cid) { //METHOD_TESTDATE START
JavaType javaType = JavaTypesManager.getJavaTypeFromId(metadataColumn.getTalendType());
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType( javaType, metadataColumn.isNullable());
String colName = metadataColumn.getLabel();
if ("".equals(pattern)){ //CONDITION_00150 START
if ("".equals(pattern)){ //CONDITION_00150 START
%>
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|Date format not defined";
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|Date format not defined";
<%
} else { //CONDITION_00150 ELSE
if (javaType == JavaTypesManager.OBJECT || javaType == JavaTypesManager.STRING) { //CONDITION_00151 START
} else { //CONDITION_00150 ELSE
if (javaType == JavaTypesManager.OBJECT || javaType == JavaTypesManager.STRING) { //CONDITION_00151 START
%>
try{
if (
try{
if (
<%
if (_bNullable){ //CONDITION_001511 START
if (_bNullable){ //CONDITION_001511 START
%>
<%=_sInConnName%>.<%=colName%> != null
<%=_sInConnName%>.<%=colName%> != null
<%
if(allEmptyAreNull || listEmptyAsNull.contains(metadataColumn.getLabel())) {
if(allEmptyAreNull || listEmptyAsNull.contains(metadataColumn.getLabel())) {
%>
&& (!"".equals(<%=_sInConnName%>.<%=colName%>))
&& (!"".equals(<%=_sInConnName%>.<%=colName%>))
<%
}
}else if (!isPrimitive){ //CONDITION_001511 ELSE IF
%>
<%=_sInConnName%>.<%=colName%> != null
<%
}else { //CONDITION_001511 ELSE
%>
true
<%
} //CONDITION_001511 STOP
}
}else if (!isPrimitive){ //CONDITION_001511 ELSE IF
%>
){
<%=_sInConnName%>.<%=colName%> != null
<%
if (!useFasteDateChecker) { //CONDITION_001512 START
}else { //CONDITION_001511 ELSE
%>
if (!TalendDate.isDate((<%=_sInConnName%>.<%=colName%>).toString(), <%=pattern%>,<%if(ignoreTimeZone){%>true<%}else{%>false<%}%>))
throw new IllegalArgumentException("Data format not matches");
true
<%
} else { //CONDITION_001512 ELSE
} //CONDITION_001511 STOP
%>
FastDateParser.getInstance(<%=pattern%>, false).parse(<%=_sInConnName%>.<%=colName%>);
){
<%
} //CONDITION_001512 STOP
if (!useFasteDateChecker && !useStrictDateChecker) { //CONDITION_001512 START
%>
}
} catch(java.lang.Exception e){
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|wrong DATE pattern or wrong DATE data";
}
if (!TalendDate.isDate((<%=_sInConnName%>.<%=colName%>).toString(), <%=pattern%>,<%if(ignoreTimeZone){%>true<%}else{%>false<%}%>))
throw new IllegalArgumentException("Data format not matches");
<%
// date type need check also (some inputting data not legal, beacause original data is not suite with pattern and has be converted)
} else if (javaType == JavaTypesManager.DATE){ //CONDITION_00151 ELSE IF
if (!metadataColumn.getPattern().equals(pattern)){ //CONDITION_001513 START
} else if (useFasteDateChecker){ //CONDITION_001512 ELSE
%>
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|wrong DATE pattern or wrong DATE data";
FastDateParser.getInstance(<%=pattern%>, false).parse(<%=_sInConnName%>.<%=colName%>);
<%
} //CONDITION_001513 STOP
} else{ //CONDITION_00151 ELSE
} else {//CONDITION_001512 STOP
%>
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|The TYPE of inputting data is error. (one of OBJECT, STRING, DATE)";
if(!TalendDate.isDateStrict((<%=_sInConnName%>.<%=colName%>).toString(), <%=pattern%>))
throw new IllegalArgumentException("Data format not matches");
<%
} //CONDITION_00151 STOP
} //CONDITION_00150 STOP
} //METHOD_TESTDATE STOP
}
%>
}
} catch(java.lang.Exception e){
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|wrong DATE pattern or wrong DATE data";
}
<%
// date type need check also (some inputting data not legal, beacause original data is not suite with pattern and has be converted)
} else if (javaType == JavaTypesManager.DATE){ //CONDITION_00151 ELSE IF
if (!metadataColumn.getPattern().equals(pattern)){ //CONDITION_001513 START
%>
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|wrong DATE pattern or wrong DATE data";
<%
} //CONDITION_001513 STOP
} else{ //CONDITION_00151 ELSE
%>
ifPassedThrough = false;
errorCodeThrough += 2;
errorMessageThrough += "|The TYPE of inputting data is error. (one of OBJECT, STRING, DATE)";
<%
} //CONDITION_00151 STOP
} //CONDITION_00150 STOP
} //METHOD_TESTDATE STOP
public void testNull(String _sInConnName, IMetadataColumn metadataColumn, String cid){ //METHOD_TESTNULL START
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType(metadataColumn.getTalendType(), metadataColumn.isNullable());
if (!isPrimitive){ //CONDITION_00160 START
if(emptyIsNull && !allEmptyAreNull){ //CONDITION_001601 START - for the migration task
if(listEmptyAsNull.contains(metadataColumn.getLabel())){ //CONDITION_0016011 START
public void testNull(String _sInConnName, IMetadataColumn metadataColumn, String cid){ //METHOD_TESTNULL START
boolean isPrimitive = JavaTypesManager.isJavaPrimitiveType(metadataColumn.getTalendType(), metadataColumn.isNullable());
if (!isPrimitive){ //CONDITION_00160 START
if(emptyIsNull && !allEmptyAreNull){ //CONDITION_001601 START - for the migration task
if(listEmptyAsNull.contains(metadataColumn.getLabel())){ //CONDITION_0016011 START
%>
// validate nullable (empty as null)
if ((<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) || ("".equals(<%=_sInConnName%>.<%=metadataColumn.getLabel()%>))) {
// validate nullable (empty as null)
if ((<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) || ("".equals(<%=_sInConnName%>.<%=metadataColumn.getLabel()%>))) {
<%
}else{ //CONDITION_0016011 ELSE
}else{ //CONDITION_0016011 ELSE
%>
// validate nullable
if (<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) {
// validate nullable
if (<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) {
<%
} //CONDITION_0016011 STOP
}else{ //CONDITION_001601 ELSE
if(allEmptyAreNull){ //CONDITION_0016012 START
} //CONDITION_0016011 STOP
}else{ //CONDITION_001601 ELSE
if(allEmptyAreNull){ //CONDITION_0016012 START
%>
// validate nullable (empty as null)
if ((<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) || ("".equals(<%=_sInConnName%>.<%=metadataColumn.getLabel()%>))) {
// validate nullable (empty as null)
if ((<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) || ("".equals(<%=_sInConnName%>.<%=metadataColumn.getLabel()%>))) {
<%
}else if(listEmptyAsNull.contains(metadataColumn.getLabel())){ //CONDITION_0016012 ELSE IF
}else if(listEmptyAsNull.contains(metadataColumn.getLabel())){ //CONDITION_0016012 ELSE IF
%>
// validate nullable (empty as null)
if ((<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) || ("".equals(<%=_sInConnName%>.<%=metadataColumn.getLabel()%>))) {
// validate nullable (empty as null)
if ((<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) || ("".equals(<%=_sInConnName%>.<%=metadataColumn.getLabel()%>))) {
<%
}else{ //CONDITION_0016012 ELSE
}else{ //CONDITION_0016012 ELSE
%>
// validate nullable
if (<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) {
// validate nullable
if (<%=_sInConnName%>.<%=metadataColumn.getLabel()%> == null) {
<%
} //CONDITION_0016012 STOP
} //CONDITION_001601 STOP
} //CONDITION_0016012 STOP
} //CONDITION_001601 STOP
%>
ifPassedThrough = false;
errorCodeThrough += 4;
errorMessageThrough += "|empty or null";
}
ifPassedThrough = false;
errorCodeThrough += 4;
errorMessageThrough += "|empty or null";
}
<%
} //CONDITION_00160 STOP
} //METHOD_TESTNULL STOP
} //CLASS SCHEMACHECKER STOP
} //CONDITION_00160 STOP
} //METHOD_TESTNULL STOP
} //CLASS SCHEMACHECKER STOP
SchemaChecker checker = new SchemaChecker();
List<Map<String, String>> listCheckedColumns = (List<Map<String, String>>)ElementParameterParser.getObjectValue(node, "__CHECKCOLS__");
boolean bNeedReferSchema = false;
SchemaChecker checker = new SchemaChecker();
List<Map<String, String>> listCheckedColumns = (List<Map<String, String>>)ElementParameterParser.getObjectValue(node, "__CHECKCOLS__");
boolean bNeedReferSchema = false;
if ("true".equals(anotherChecked)){
if (node.getMetadataFromConnector("OTHER") != null)
listColumsToTest = node.getMetadataFromConnector("OTHER").getListColumns();
} else if ("true".equals(checkAll)){
;
} else{
bNeedReferSchema = true;
}
if ("true".equals(anotherChecked)){
if (node.getMetadataFromConnector("OTHER") != null)
listColumsToTest = node.getMetadataFromConnector("OTHER").getListColumns();
} else if ("true".equals(checkAll)){
;
} else{
bNeedReferSchema = true;
}
%>
class RowSetValueUtil_<%=cid %> {
class RowSetValueUtil_<%=cid %> {
boolean ifPassedThrough = true;
int errorCodeThrough = 0;
String errorMessageThrough = "";
int resultErrorCodeThrough = 0;
String resultErrorMessageThrough = "";
String tmpContentThrough = null;
boolean ifPassedThrough = true;
int errorCodeThrough = 0;
String errorMessageThrough = "";
int resultErrorCodeThrough = 0;
String resultErrorMessageThrough = "";
String tmpContentThrough = null;
boolean ifPassed = true;
int errorCode = 0;
String errorMessage = "";
boolean ifPassed = true;
int errorCode = 0;
String errorMessage = "";
void handleBigdecimalPrecision(String data, int iPrecision, int maxLength){
//number of digits before the decimal point(ignoring frontend zeroes)
int len1 = 0;
int len2 = 0;
ifPassed = true;
errorCode = 0;
errorMessage = "";
if(data.startsWith("-")){
data = data.substring(1);
}
data = org.apache.commons.lang.StringUtils.stripStart(data, "0");
void handleBigdecimalPrecision(String data, int iPrecision, int maxLength){
//number of digits before the decimal point(ignoring frontend zeroes)
int len1 = 0;
int len2 = 0;
ifPassed = true;
errorCode = 0;
errorMessage = "";
if(data.startsWith("-")){
data = data.substring(1);
}
data = org.apache.commons.lang.StringUtils.stripStart(data, "0");
if(data.indexOf(".") >= 0){
len1 = data.indexOf(".");
data = org.apache.commons.lang.StringUtils.stripEnd(data, "0");
len2 = data.length() - (len1 + 1);
}else{
len1 = data.length();
}
if(data.indexOf(".") >= 0){
len1 = data.indexOf(".");
data = org.apache.commons.lang.StringUtils.stripEnd(data, "0");
len2 = data.length() - (len1 + 1);
}else{
len1 = data.length();
}
if (iPrecision < len2) {
ifPassed = false;
errorCode += 8;
errorMessage += "|precision Non-matches";
} else if (maxLength < len1 + iPrecision) {
ifPassed = false;
errorCode += 8;
errorMessage += "|invalid Length setting is unsuitable for Precision";
}
}
if (iPrecision < len2) {
ifPassed = false;
errorCode += 8;
errorMessage += "|precision Non-matches";
} else if (maxLength < len1 + iPrecision) {
ifPassed = false;
errorCode += 8;
errorMessage += "|invalid Length setting is unsuitable for Precision";
}
}
int handleErrorCode(int errorCode, int resultErrorCode){
if (errorCode > 0) {
if (resultErrorCode > 0) {
resultErrorCode = 16;
} else {
resultErrorCode = errorCode;
}
}
return resultErrorCode;
}
int handleErrorCode(int errorCode, int resultErrorCode){
if (errorCode > 0) {
if (resultErrorCode > 0) {
resultErrorCode = 16;
} else {
resultErrorCode = errorCode;
}
}
return resultErrorCode;
}
String handleErrorMessage(String errorMessage, String resultErrorMessage, String columnLabel){
if (errorMessage.length() > 0) {
if (resultErrorMessage.length() > 0) {
resultErrorMessage += ";"+ errorMessage.replaceFirst("\\|", columnLabel);
} else {
resultErrorMessage = errorMessage.replaceFirst("\\|", columnLabel);
}
}
return resultErrorMessage;
}
String handleErrorMessage(String errorMessage, String resultErrorMessage, String columnLabel){
if (errorMessage.length() > 0) {
if (resultErrorMessage.length() > 0) {
resultErrorMessage += ";"+ errorMessage.replaceFirst("\\|", columnLabel);
} else {
resultErrorMessage = errorMessage.replaceFirst("\\|", columnLabel);
}
}
return resultErrorMessage;
}
void reset(){
ifPassedThrough = true;
errorCodeThrough = 0;
errorMessageThrough = "";
resultErrorCodeThrough = 0;
resultErrorMessageThrough = "";
tmpContentThrough = null;
void reset(){
ifPassedThrough = true;
errorCodeThrough = 0;
errorMessageThrough = "";
resultErrorCodeThrough = 0;
resultErrorMessageThrough = "";
tmpContentThrough = null;
ifPassed = true;
errorCode = 0;
errorMessage = "";
}
ifPassed = true;
errorCode = 0;
errorMessage = "";
}
<%
for (IMetadataColumn inColumn : listInColumns) { //LOOP_00100 START
int iInColIndex = listInColumns.indexOf(inColumn);
if(iInColIndex % 100 == 0){ //CONDITION_00170 START
for (IMetadataColumn inColumn : listInColumns) { //LOOP_00100 START
int iInColIndex = listInColumns.indexOf(inColumn);
if(iInColIndex % 100 == 0){ //CONDITION_00170 START
%>
void setRowValue_<%=(iInColIndex/100) %>(<%=NodeUtil.getPrivateConnClassName(inConn) %>Struct <%=sInConnName %>) {
void setRowValue_<%=(iInColIndex/100) %>(<%=NodeUtil.getPrivateConnClassName(inConn) %>Struct <%=sInConnName %>) {
<%
} //CONDITION_00170 STOP
// when using another schema, it's size may less than listInColumns
if (iInColIndex >= listColumsToTest.size()){
break;
}
} //CONDITION_00170 STOP
// when using another schema, it's size may less than listInColumns
if (iInColIndex >= listColumsToTest.size()){
break;
}
Object pre_iPrecision = null;
String sInColumnName = inColumn.getLabel(), sTestColName = null, sTestColType = null, sTestColPattern = null;
boolean bNullable = true, bMaxLenLimited = true;
/* use setting of tSchemaComplianceCheck schema (it is synchronize with inputting schema, but length value can be different) */
Object pre_maxLength = listColumsToTest.get(iInColIndex).getLength();
int maxLength = (pre_maxLength == null) ? 0 : Integer.parseInt(pre_maxLength.toString());
IMetadataColumn schemaColumn = null;
Object pre_iPrecision = null;
String sInColumnName = inColumn.getLabel(), sTestColName = null, sTestColType = null, sTestColPattern = null;
boolean bNullable = true, bMaxLenLimited = true;
/* use setting of tSchemaComplianceCheck schema (it is synchronize with inputting schema, but length value can be different) */
Object pre_maxLength = listColumsToTest.get(iInColIndex).getLength();
int maxLength = (pre_maxLength == null) ? 0 : Integer.parseInt(pre_maxLength.toString());
IMetadataColumn schemaColumn = null;
if (bNeedReferSchema) {
Map<String, String> checkedColumn = listCheckedColumns.get(iInColIndex);
sTestColName = checkedColumn.get("SCHEMA_COLUMN");
sTestColType = checkedColumn.get("SELECTED_TYPE");
sTestColPattern = checkedColumn.get("DATEPATTERN");
bNullable = "true".equals(checkedColumn.get("NULLABLE"));
bMaxLenLimited = "true".equals(checkedColumn.get("MAX_LENGTH"));
} else{
schemaColumn = listColumsToTest.get(iInColIndex);
sTestColName = schemaColumn.getLabel();
sTestColType = JavaTypesManager.getTypeToGenerate(schemaColumn.getTalendType(), true);
sTestColPattern = schemaColumn.getPattern();
bNullable = schemaColumn.isNullable();
pre_iPrecision = schemaColumn.getPrecision();
}
if (bNeedReferSchema) {
Map<String, String> checkedColumn = listCheckedColumns.get(iInColIndex);
sTestColName = checkedColumn.get("SCHEMA_COLUMN");
sTestColType = checkedColumn.get("SELECTED_TYPE");
sTestColPattern = checkedColumn.get("DATEPATTERN");
bNullable = "true".equals(checkedColumn.get("NULLABLE"));
bMaxLenLimited = "true".equals(checkedColumn.get("MAX_LENGTH"));
} else{
schemaColumn = listColumsToTest.get(iInColIndex);
sTestColName = schemaColumn.getLabel();
sTestColType = JavaTypesManager.getTypeToGenerate(schemaColumn.getTalendType(), true);
sTestColPattern = schemaColumn.getPattern();
bNullable = schemaColumn.isNullable();
pre_iPrecision = schemaColumn.getPrecision();
}
// NULL checking
if (!bNullable){
checker.testNull(sInConnName, inColumn, cid);
}
// NULL checking
if (!bNullable){
checker.testNull(sInConnName, inColumn, cid);
}
// type checking
if (sTestColType != null){
if (sTestColType.indexOf("Date") >= 0){
checker.testDate(bNullable, sInConnName, inColumn, sTestColPattern, cid);
} else{
checker.testDataType(bNullable, sInConnName, inColumn, sTestColType, cid);
}
}
// type checking
if (sTestColType != null){
if (sTestColType.indexOf("Date") >= 0){
checker.testDate(bNullable, sInConnName, inColumn, sTestColPattern, cid);
} else{
checker.testDataType(bNullable, sInConnName, inColumn, sTestColType, cid);
}
}
// length checking
if (bMaxLenLimited){
checker.testDataLength(bNullable, sInConnName,inColumn, "true".equals(anotherChecked)?schemaColumn:inColumn, maxLength, cid);
}
// length checking
if (bMaxLenLimited){
checker.testDataLength(bNullable, sInConnName,inColumn, "true".equals(anotherChecked)?schemaColumn:inColumn, maxLength, cid);
}
// precision checking
if (pre_iPrecision != null){
checker.testPrecision(maxLength, Integer.parseInt(pre_iPrecision.toString()), sInConnName, inColumn, sTestColType, cid);
}
// precision checking
if (pre_iPrecision != null){
checker.testPrecision(maxLength, Integer.parseInt(pre_iPrecision.toString()), sInConnName, inColumn, sTestColType, cid);
}
%>
resultErrorCodeThrough = handleErrorCode(errorCodeThrough,resultErrorCodeThrough);
errorCodeThrough = 0;
resultErrorMessageThrough = handleErrorMessage(errorMessageThrough,resultErrorMessageThrough,"<%=inColumn.getLabel()%>:");
errorMessageThrough = "";
resultErrorCodeThrough = handleErrorCode(errorCodeThrough,resultErrorCodeThrough);
errorCodeThrough = 0;
resultErrorMessageThrough = handleErrorMessage(errorMessageThrough,resultErrorMessageThrough,"<%=inColumn.getLabel()%>:");
errorMessageThrough = "";
<%
if((iInColIndex + 1) % 100 == 0){ //CONDITION_00171 START
if((iInColIndex + 1) % 100 == 0){ //CONDITION_00171 START
%>
}
}
<%
} //CONDITION_00171 STOP
} //LOOP_00100 STOP
if(listInColumns.size() > 0 && listInColumns.size() % 100 > 0){ //CONDITION_00180 START
} //CONDITION_00171 STOP
} //LOOP_00100 STOP
if(listInColumns.size() > 0 && listInColumns.size() % 100 > 0){ //CONDITION_00180 START
%>
}
}
<%
} //CONDITION_00180 STOP
} //CONDITION_00180 STOP
%>
}
RowSetValueUtil_<%=cid %> rsvUtil_<%=cid %> = new RowSetValueUtil_<%=cid %>();
}
RowSetValueUtil_<%=cid %> rsvUtil_<%=cid %> = new RowSetValueUtil_<%=cid %>();

View File

@@ -107,28 +107,31 @@
</PARAMETER>
</PARAMETERS>
<ADVANCED_PARAMETERS>
<PARAMETER NAME="FAST_DATE_CHECK" FIELD="CHECK" NUM_ROW="1" REQUIRED="true">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="NOTE" FIELD="LABEL" NUM_ROW="1">
<DEFAULT>(Not compatible with multi-threaded jobs)</DEFAULT>
</PARAMETER>
<PARAMETER NAME="IGNORE_TIMEZONE" FIELD="CHECK" NUM_ROW="3" REQUIRED="true" SHOW_IF="(CUSTOMER == 'true' OR CHECK_ANOTHER == 'true') AND (FAST_DATE_CHECK == 'false')">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="EMPTY_IS_NULL" FIELD="CHECK" NUM_ROW="9" REQUIRED="true" SHOW="false">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="ALL_EMPTY_ARE_NULL" FIELD="CHECK" NUM_ROW="10" REQUIRED="true">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="EMPTY_NULL_TABLE" FIELD="TABLE" REQUIRED="true" NUM_ROW="11" NB_LINES="3" SHOW_IF="ALL_EMPTY_ARE_NULL == 'false'">
<ITEMS BASED_ON_SCHEMA="true">
<ITEM NAME="EMPTY_NULL" FIELD="CHECK" VALUE="false" />
</ITEMS>
</PARAMETER>
</ADVANCED_PARAMETERS>
<ADVANCED_PARAMETERS>
<PARAMETER NAME="FAST_DATE_CHECK" FIELD="CHECK" NUM_ROW="1" REQUIRED="true" SHOW_IF="STRICT_DATE_CHECK == 'false'">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="NOTE" FIELD="LABEL" NUM_ROW="1" SHOW_IF="STRICT_DATE_CHECK == 'false'">
<DEFAULT>(Not compatible with multi-threaded jobs)</DEFAULT>
</PARAMETER>
<PARAMETER NAME="STRICT_DATE_CHECK" FIELD="CHECK" NUM_ROW="3" REQUIRED="true" SHOW_IF="FAST_DATE_CHECK == 'false'">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="IGNORE_TIMEZONE" FIELD="CHECK" NUM_ROW="5" REQUIRED="true" SHOW_IF="(CUSTOMER == 'true' OR CHECK_ANOTHER == 'true') AND (FAST_DATE_CHECK == 'false') AND (STRICT_DATE_CHECK == 'false')">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="EMPTY_IS_NULL" FIELD="CHECK" NUM_ROW="11" REQUIRED="true" SHOW="false">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="ALL_EMPTY_ARE_NULL" FIELD="CHECK" NUM_ROW="12" REQUIRED="true">
<DEFAULT>true</DEFAULT>
</PARAMETER>
<PARAMETER NAME="EMPTY_NULL_TABLE" FIELD="TABLE" REQUIRED="true" NUM_ROW="13" NB_LINES="3" SHOW_IF="ALL_EMPTY_ARE_NULL == 'false'">
<ITEMS BASED_ON_SCHEMA="true">
<ITEM NAME="EMPTY_NULL" FIELD="CHECK" VALUE="false" />
</ITEMS>
</PARAMETER>
</ADVANCED_PARAMETERS>
<CODEGENERATION>
<IMPORTS>

View File

@@ -58,6 +58,7 @@ DATE_FORMAT.ITEM.US=US format : 12/10 is Dec. 10
DATE_FORMAT.ITEM.NON-US=Other format : 12/10 is Oct. 12
#advance options
FAST_DATE_CHECK.NAME=Use Fastest Date check
STRICT_DATE_CHECK.NAME=Use Strict Date check
ALL_EMPTY_ARE_NULL.NAME=Treat all empty string as NULL
EMPTY_NULL_TABLE.NAME=Choose Column(s)
EMPTY_NULL_TABLE.ITEM.COLUMN=Column

View File

@@ -0,0 +1,29 @@
<%@ jet
%>
class MyTrust_<%=cid %> {
private javax.net.ssl.TrustManager[] getTrustManagers()
throws java.security.KeyStoreException, java.security.NoSuchAlgorithmException,
java.security.cert.CertificateException, java.security.UnrecoverableKeyException,
java.io.IOException {
java.security.KeyStore ks = java.security.KeyStore.getInstance("JKS");
<%
passwordFieldName = "__KEYSTORE_PASS__";
if (ElementParameterParser.canEncrypt(node, passwordFieldName)) {
%>
String decryptedKeyStorePassword_<%=cid%> = routines.system.PasswordEncryptUtil.decryptPassword(<%=ElementParameterParser.getEncryptedValue(node, passwordFieldName)%>);
<%
} else {
%>
String decryptedKeyStorePassword_<%=cid%> = <%= ElementParameterParser.getValue(node, passwordFieldName)%>;
<%
}
%>
ks.load(new java.io.FileInputStream(<%=keystoreFile%>), decryptedKeyStorePassword_<%=cid%>.toCharArray());
javax.net.ssl.TrustManagerFactory tmf = javax.net.ssl.TrustManagerFactory.getInstance(javax.net.ssl.KeyManagerFactory.getDefaultAlgorithm());
tmf.init(ks);
return tmf.getTrustManagers();
}
}

View File

@@ -1,66 +0,0 @@
<%@ jet
%>
class MyTrust_<%=cid %> {
private javax.net.ssl.TrustManager[] getTrustManagers()
throws java.security.KeyStoreException, java.security.NoSuchAlgorithmException,
java.security.cert.CertificateException, java.security.UnrecoverableKeyException,
java.io.IOException {
java.security.KeyStore ks = java.security.KeyStore.getInstance("JKS");
<%
passwordFieldName = "__KEYSTORE_PASS__";
if (ElementParameterParser.canEncrypt(node, passwordFieldName)) {
%>
String decryptedKeyStorePassword_<%=cid%> = routines.system.PasswordEncryptUtil.decryptPassword(<%=ElementParameterParser.getEncryptedValue(node, passwordFieldName)%>);
<%
} else {
%>
String decryptedKeyStorePassword_<%=cid%> = <%= ElementParameterParser.getValue(node, passwordFieldName)%>;
<%
}
%>
ks.load(new java.io.FileInputStream(<%=keystoreFile%>), decryptedKeyStorePassword_<%=cid%>.toCharArray());
javax.net.ssl.TrustManagerFactory tmf = javax.net.ssl.TrustManagerFactory.getInstance(javax.net.ssl.KeyManagerFactory.getDefaultAlgorithm());
tmf.init(ks);
return tmf.getTrustManagers();
}
}
class SSLSessionReuseFTPSClient_<%=cid %> extends org.apache.commons.net.ftp.FTPSClient {
public SSLSessionReuseFTPSClient_<%=cid %>(boolean isImplicit, javax.net.ssl.SSLContext context) {
super(isImplicit, context);
}
// changed to support TLS session resumption
@Override
protected void _prepareDataSocket_(final java.net.Socket socket) throws IOException {
if(socket instanceof javax.net.ssl.SSLSocket) {
final javax.net.ssl.SSLSession session = ((javax.net.ssl.SSLSocket) _socket_).getSession();
final javax.net.ssl.SSLSessionContext context = session.getSessionContext();
try {
final java.lang.reflect.Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final java.lang.reflect.Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
final java.lang.reflect.Method getHostMethod = socket.getClass().getDeclaredMethod("getHost");
getHostMethod.setAccessible(true);
Object host = getHostMethod.invoke(socket);
final String key = String.format("%s:%s", host, String.valueOf(socket.getPort())).toLowerCase(java.util.Locale.ROOT);
putMethod.invoke(cache, key, session);
} catch(Exception e) {
<%
if (isLog4jEnabled) {
%>
log.error("Can't adapt ftps client to support TLS session resumption");
<%
}
%>
throw new RuntimeException(e);
}
}
}
}

View File

@@ -2011,7 +2011,11 @@ public class CLASS {
setStmt.append(");\r\n");
}
setStmt.append("} else {\r\n");
setStmt.append(prefix + cid + ".setNull(" + index + dynamic + ", java.sql.Types.DATE);\r\n");
if ("TIME".equals(dbType)) {
setStmt.append(prefix + cid + ".setNull(" + index + dynamic + ", java.sql.Types.TIME);\r\n");
} else {
setStmt.append(prefix + cid + ".setNull(" + index + dynamic + ", java.sql.Types.DATE);\r\n");
}
setStmt.append("}\r\n");
} else {
setStmt.append(prefix + cid + ".set" + typeToGenerate + "(" + index + dynamic + ", " + incomingConnName + "." + column.getName() + ");\r\n");

View File

@@ -16,6 +16,8 @@
<webservice.dir>${project.basedir}/components/tWebServiceInput</webservice.dir>
<thashfile.dir>${project.basedir}/components/tHashInput</thashfile.dir>
<thashfile.version>3.0-20170711</thashfile.version>
<ftpProxy.dir>${project.basedir}/components/tFTPConnection</ftpProxy.dir>
<ftpProxy.version>3.6.1-talend-20190128</ftpProxy.version>
</properties>
<build>
@@ -111,6 +113,14 @@
<overWrite>true</overWrite>
<outputDirectory>${thashfile.dir}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.talend.libraries</groupId>
<artifactId>commons-net-ftps-proxy</artifactId>
<version>${ftpProxy.version}</version>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${ftpProxy.dir}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>

View File

@@ -104,7 +104,7 @@
</projecttask>
<projecttask
beforeLogon="false"
breaks="7.0.1"
breaks="7.0.0"
class="org.talend.designer.core.generic.model.migration.NewGoogleDriveMigrationTask"
description="Migration task for jobs using Google Drive components to TCOMP"
id="org.talend.designer.core.generic.model.migration.NewGoogleDriveMigrationTask"

View File

@@ -1469,9 +1469,11 @@ public class Component extends AbstractBasicComponent {
}
if (GenericTypeUtils.isStringType(property)
&& property.getTaggedValue(IGenericConstants.LINE_SEPARATOR_REPLACED_TO) != null) {
String replacedTo = String.valueOf(property.getTaggedValue(IGenericConstants.LINE_SEPARATOR_REPLACED_TO));
// "Win", "Linux/Unix", "Mac"
return value.replaceAll("\r\n", replacedTo).replaceAll("\n", replacedTo).replaceAll("\r", replacedTo);
//process for the sql field for jdbc, snowflake, salesforce, LINE_SEPARATOR_REPLACED_TO key can tell us which a sql type field,
//as sql type value may have newline and return characters, which make compiler issue in java code,
//so have to convert the newline characters to visible "\r", "\n" for pass the compiler issue and can't only convert them to white space as TDI-41898
//jdbc drivers, salesforce driver can work like that sql : select * \nfrom Account, so it is ok
return NodeUtil.replaceCRLFInMEMO_SQL(value);
}
if (GenericTypeUtils.isSchemaType(property)) {
// Handles embedded escaped quotes which might occur

View File

@@ -6,12 +6,39 @@ import java.util.Date;
import java.util.GregorianCalendar;
import java.util.Properties;
import org.talend.core.model.properties.Item;
import org.talend.core.utils.TalendQuoteUtils;
import org.talend.designer.core.generic.utils.ParameterUtilTool;
import org.talend.designer.core.model.utils.emf.talendfile.ElementParameterType;
import org.talend.designer.core.model.utils.emf.talendfile.NodeType;
import org.talend.designer.core.model.utils.emf.talendfile.ProcessType;
public class NewGoogleDriveMigrationTask extends org.talend.designer.core.generic.model.migration.NewComponentFrameworkMigrationTask {
public static final String GOOGLE_DRIVE_PREFIX = "tGoogleDrive";
public static final String JAVAJET_VERSION = "0.102";
@Override
public ExecutionResult execute(Item item) {
ProcessType processType = getProcessType(item);
if (processType != null) {
for (Object obj : processType.getNode()) {
if (obj != null && obj instanceof NodeType) {
String componentName = ((NodeType) obj).getComponentName();
String version = ((NodeType) obj).getComponentVersion();
if (componentName != null && componentName.startsWith(GOOGLE_DRIVE_PREFIX)) {
if (!JAVAJET_VERSION.equals(version)) {
// not a javajet component (tcompv0), we skip this migration for not corrupting comps.
return ExecutionResult.NOTHING_TO_DO;
}
}
}
}
}
return super.execute(item);
}
@Override
public Date getOrder() {
return new GregorianCalendar(2017, 10, 23, 10, 15, 0).getTime();
@@ -33,44 +60,43 @@ public class NewGoogleDriveMigrationTask extends org.talend.designer.core.generi
@Override
protected ElementParameterType getParameterType(NodeType node, String paramName) {
// sanitize inexistent props
ElementParameterType oAuthMethod = ParameterUtilTool.findParameterType(node, "OAUTH_METHOD");
if (oAuthMethod == null) {
ParameterUtilTool.addParameterType(node, "TEXT", "OAUTH_METHOD", "AccessToken");
}
ElementParameterType dsPath = ParameterUtilTool.findParameterType(node, "DATASTORE_PATH");
if (dsPath == null) {
String datastorePath = (System.getProperty("user.home", ".") + "/.credentials/talend-googledrive")
.replace("\\", "/");
ParameterUtilTool.addParameterType(node, "TEXT", "DATASTORE_PATH",
TalendQuoteUtils.addQuotesIfNotExist(datastorePath));
}
ElementParameterType paramType = ParameterUtilTool.findParameterType(node, paramName);
if (node != null && paramType != null) {
Object value = ParameterUtilTool.convertParameterValue(paramType);
String componentName = node.getComponentName();
//
if ("tGoogleDriveCopy".equals(componentName)) {
if ("FILE_MODE".equals(paramName)) {
if ("true".equals(String.valueOf(value))) {
paramType.setValue("File");
} else {
paramType.setValue("Folder");
}
}
if ("FILE_NAME".equals(paramName)) {
ElementParameterType fileMode = ParameterUtilTool.findParameterType(node, "FILE_MODE");
Object fileModeValue = ParameterUtilTool.convertParameterValue(fileMode);
if ("true".equals(String.valueOf(fileModeValue))) {
paramType.setValue(String.valueOf(value));
} else {
ElementParameterType folderName = ParameterUtilTool.findParameterType(node, "FOLDER_NAME");
Object folderNameValue = ParameterUtilTool.convertParameterValue(folderName);
paramType.setValue(String.valueOf(folderNameValue));
}
Object copyModeParam = ParameterUtilTool.findParameterType(node, "COPY_MODE");
if (copyModeParam == null) {
Object fileMode = ParameterUtilTool.getParameterValue(node, "FILE_MODE");
Object fileName = ParameterUtilTool.getParameterValue(node, "FILE_NAME");
Object folderName = ParameterUtilTool.getParameterValue(node, "FOLDER_NAME");
String copyMode = "true".equals(fileMode) ? "File" : "Folder";
String source = "File".equals(copyMode) ? String.valueOf(fileName) : String.valueOf(folderName);
ParameterUtilTool.addParameterType(node, "TEXT", "COPY_MODE", copyMode);
ParameterUtilTool.addParameterType(node, "TEXT", "SOURCE", source);
}
}
if ("tGoogleDrivePut".equals(componentName)) {
if ("READ_CONTENT_FROM_INPUT".equals(paramName)) {
if ("true".equals(String.valueOf(value))) {
paramType.setValue("READ_CONTENT_FROM_INPUT");
} else {
ElementParameterType upl = ParameterUtilTool.findParameterType(node, "UPLOAD_LOCAL_FILE");
Object uplv = ParameterUtilTool.convertParameterValue(upl);
if ("true".equals(String.valueOf(uplv))) {
paramType.setValue("UPLOAD_LOCAL_FILE");
} else {
paramType.setValue("EXPOSE_OUTPUT_STREAM");
}
Object uploadModeParam = ParameterUtilTool.findParameterType(node, "UPLOAD_MODE");
if (uploadModeParam == null) {
Object isReadContentFromInput = ParameterUtilTool.getParameterValue(node, "READ_CONTENT_FROM_INPUT");
Object uploadLocalFile = ParameterUtilTool.getParameterValue(node, "UPLOAD_LOCAL_FILE");
String uploadMode = "READ_CONTENT_FROM_INPUT";
if (!"true".equals(isReadContentFromInput)) {
uploadMode = "true".equals(uploadLocalFile) ? "UPLOAD_LOCAL_FILE" : "EXPOSE_OUTPUT_STREAM";
}
ParameterUtilTool.addParameterType(node, "TEXT", "UPLOAD_MODE", uploadMode);
}
}
}

View File

@@ -9,11 +9,25 @@ tGoogleDriveConnection.useSSL=USE_SSL
tGoogleDriveConnection.sslAlgorithm=SSL_ALGORITHM
tGoogleDriveConnection.sslTrustStore=SSL_TRUST_STORE
tGoogleDriveConnection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDriveConnection.oAuthMethod=OAUTH_METHOD
tGoogleDriveConnection.datastorePath=DATASTORE_PATH
# tGoogleDriveCopy
tGoogleDriveCopy=tGoogleDriveCopy
tGoogleDriveCopy.copyMode=FILE_MODE
tGoogleDriveCopy.source=FILE_NAME
tGoogleDriveCopy.connection.referencedComponent=CONNECTION
tGoogleDriveCopy.connection.applicationName=APPLICATION_NAME
tGoogleDriveCopy.connection.accessToken=ACCESS_TOKEN
tGoogleDriveCopy.connection.useProxy=USE_PROXY
tGoogleDriveCopy.connection.proxyHost=PROXY_HOST
tGoogleDriveCopy.connection.proxyPort=PROXY_PORT
tGoogleDriveCopy.connection.useSSL=USE_SSL
tGoogleDriveCopy.connection.sslAlgorithm=SSL_ALGORITHM
tGoogleDriveCopy.connection.sslTrustStore=SSL_TRUST_STORE
tGoogleDriveCopy.connection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDriveCopy.connection.oAuthMethod=OAUTH_METHOD
tGoogleDriveCopy.connection.datastorePath=DATASTORE_PATH
tGoogleDriveCopy.copyMode=COPY_MODE
tGoogleDriveCopy.source=SOURCE
tGoogleDriveCopy.destinationFolder=DESTINATION_FOLDER_NAME
tGoogleDriveCopy.rename=RENAME
tGoogleDriveCopy.newName=DESTINATION_FILE_NAME
@@ -21,16 +35,52 @@ tGoogleDriveCopy.deleteSourceFile=REMOVE_SOURCE_FILE
# tGoogleDriveCreate
tGoogleDriveCreate=tGoogleDriveCreate
tGoogleDriveCreate.connection.referencedComponent=CONNECTION
tGoogleDriveCreate.connection.applicationName=APPLICATION_NAME
tGoogleDriveCreate.connection.accessToken=ACCESS_TOKEN
tGoogleDriveCreate.connection.useProxy=USE_PROXY
tGoogleDriveCreate.connection.proxyHost=PROXY_HOST
tGoogleDriveCreate.connection.proxyPort=PROXY_PORT
tGoogleDriveCreate.connection.useSSL=USE_SSL
tGoogleDriveCreate.connection.sslAlgorithm=SSL_ALGORITHM
tGoogleDriveCreate.connection.sslTrustStore=SSL_TRUST_STORE
tGoogleDriveCreate.connection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDriveCreate.connection.oAuthMethod=OAUTH_METHOD
tGoogleDriveCreate.connection.datastorePath=DATASTORE_PATH
tGoogleDriveCreate.parentFolder=FOLDER_NAME
tGoogleDriveCreate.newFolder=NEW_FOLDER_NAME
# tGoogleDriveDelete
tGoogleDriveDelete=tGoogleDriveDelete
tGoogleDriveDelete.connection.referencedComponent=CONNECTION
tGoogleDriveDelete.connection.applicationName=APPLICATION_NAME
tGoogleDriveDelete.connection.accessToken=ACCESS_TOKEN
tGoogleDriveDelete.connection.useProxy=USE_PROXY
tGoogleDriveDelete.connection.proxyHost=PROXY_HOST
tGoogleDriveDelete.connection.proxyPort=PROXY_PORT
tGoogleDriveDelete.connection.useSSL=USE_SSL
tGoogleDriveDelete.connection.sslAlgorithm=SSL_ALGORITHM
tGoogleDriveDelete.connection.sslTrustStore=SSL_TRUST_STORE
tGoogleDriveDelete.connection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDriveDelete.connection.oAuthMethod=OAUTH_METHOD
tGoogleDriveDelete.connection.datastorePath=DATASTORE_PATH
tGoogleDriveDelete.file=FILE_NAME
tGoogleDriveDelete.useTrash=USE_TRASH
# tGoogleDriveGet
tGoogleDriveGet=tGoogleDriveGet
tGoogleDriveGet.connection.referencedComponent=CONNECTION
tGoogleDriveGet.connection.applicationName=APPLICATION_NAME
tGoogleDriveGet.connection.accessToken=ACCESS_TOKEN
tGoogleDriveGet.connection.useProxy=USE_PROXY
tGoogleDriveGet.connection.proxyHost=PROXY_HOST
tGoogleDriveGet.connection.proxyPort=PROXY_PORT
tGoogleDriveGet.connection.useSSL=USE_SSL
tGoogleDriveGet.connection.sslAlgorithm=SSL_ALGORITHM
tGoogleDriveGet.connection.sslTrustStore=SSL_TRUST_STORE
tGoogleDriveGet.connection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDriveGet.connection.oAuthMethod=OAUTH_METHOD
tGoogleDriveGet.connection.datastorePath=DATASTORE_PATH
tGoogleDriveGet.file=FILE_NAME
tGoogleDriveGet.storeToLocal=STORE_TO_LOCAL
tGoogleDriveGet.outputFileName=OUTPUT
@@ -43,6 +93,18 @@ tGoogleDriveGet.mainSchema.schema.connector=FLOW->FLOW
# tGoogleDriveList
tGoogleDriveList=tGoogleDriveList
tGoogleDriveList.connection.referencedComponent=CONNECTION
tGoogleDriveList.connection.applicationName=APPLICATION_NAME
tGoogleDriveList.connection.accessToken=ACCESS_TOKEN
tGoogleDriveList.connection.useProxy=USE_PROXY
tGoogleDriveList.connection.proxyHost=PROXY_HOST
tGoogleDriveList.connection.proxyPort=PROXY_PORT
tGoogleDriveList.connection.useSSL=USE_SSL
tGoogleDriveList.connection.sslAlgorithm=SSL_ALGORITHM
tGoogleDriveList.connection.sslTrustStore=SSL_TRUST_STORE
tGoogleDriveList.connection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDriveList.connection.oAuthMethod=OAUTH_METHOD
tGoogleDriveList.connection.datastorePath=DATASTORE_PATH
tGoogleDriveList.folder=FOLDER_NAME
tGoogleDriveList.includeSubDirectories=INCLUDSUBDIR
tGoogleDriveList.listMode=LIST_MODE
@@ -50,7 +112,19 @@ tGoogleDriveList.includeTrashedFiles=INCLUDTRASHFILES
# tGoogleDrivePut
tGoogleDrivePut=tGoogleDrivePut
tGoogleDrivePut.uploadMode=READ_CONTENT_FROM_INPUT
tGoogleDrivePut.connection.referencedComponent=CONNECTION
tGoogleDrivePut.connection.applicationName=APPLICATION_NAME
tGoogleDrivePut.connection.accessToken=ACCESS_TOKEN
tGoogleDrivePut.connection.useProxy=USE_PROXY
tGoogleDrivePut.connection.proxyHost=PROXY_HOST
tGoogleDrivePut.connection.proxyPort=PROXY_PORT
tGoogleDrivePut.connection.useSSL=USE_SSL
tGoogleDrivePut.connection.sslAlgorithm=SSL_ALGORITHM
tGoogleDrivePut.connection.sslTrustStore=SSL_TRUST_STORE
tGoogleDrivePut.connection.sslTrustStorePassword=SSL_TRUST_STORE_PASSWORD
tGoogleDrivePut.connection.oAuthMethod=OAUTH_METHOD
tGoogleDrivePut.connection.datastorePath=DATASTORE_PATH
tGoogleDrivePut.uploadMode=UPLOAD_MODE
tGoogleDrivePut.fileName=FILE_TITLE
tGoogleDrivePut.destinationFolder=DESTINATION_FOLDER_NAME
tGoogleDrivePut.overwrite=OVERWRITE

View File

@@ -17,19 +17,25 @@ import java.util.Date;
import java.util.GregorianCalendar;
import java.util.List;
import org.apache.commons.lang.StringUtils;
import org.eclipse.emf.common.util.EList;
import org.talend.commons.ui.runtime.exception.ExceptionHandler;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.model.context.ContextUtils;
import org.talend.core.model.metadata.builder.connection.DatabaseConnection;
import org.talend.core.model.migration.AbstractJobMigrationTask;
import org.talend.core.model.properties.ContextItem;
import org.talend.core.model.properties.DatabaseConnectionItem;
import org.talend.core.model.properties.Item;
import org.talend.core.model.repository.ERepositoryObjectType;
import org.talend.core.model.utils.ContextParameterUtils;
import org.talend.core.repository.model.ProxyRepositoryFactory;
import org.talend.core.runtime.services.IGenericDBService;
import org.talend.daikon.properties.Properties;
import org.talend.daikon.properties.property.Property;
import org.talend.designer.core.generic.constants.IGenericConstants;
import org.talend.migration.IMigrationTask.ExecutionResult;
import org.talend.designer.core.model.utils.emf.talendfile.ContextParameterType;
import org.talend.designer.core.model.utils.emf.talendfile.ContextType;
/**
* DOC hwang class global comment. Detailled comment
@@ -107,6 +113,16 @@ public class NewJDBCConnectionMigrationTask extends AbstractJobMigrationTask{
setDrivers(dirJar, connection.getDriverJarPath(), isContextMode);
connection.setCompProperties(properties.toSerialized());
try {
if (isContextMode) {
// for context mode JDBC connection, the value of DriverJar context parameter need to be changed
// <jarName>.jar => "mvn:org.talend.libraries/<jarName>/6.0.0-SNAPSHOT/jar"
String contextId = dbConnection.getContextId();
ContextItem contextItem = ContextUtils.getContextItemById2(contextId);
if (contextItem != null) {
setContextDriversValue(contextItem, connection.getDriverJarPath());
factory.save(contextItem, true);
}
}
factory.save(item, true);
return ExecutionResult.SUCCESS_WITH_ALERT;
} catch (Exception e) {
@@ -139,4 +155,35 @@ public class NewJDBCConnectionMigrationTask extends AbstractJobMigrationTask{
dirJar.setTaggedValue(IGenericConstants.IS_CONTEXT_MODE, isContextMode);
}
private void setContextDriversValue(ContextItem contextItem, String paramName) {
IGenericDBService dbService = null;
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericDBService.class)) {
dbService = (IGenericDBService) GlobalServiceRegister.getDefault().getService(IGenericDBService.class);
}
boolean containContextParam = ContextParameterUtils.isContainContextParam(paramName);
if (!containContextParam || dbService == null) {
return;
}
paramName = ContextParameterUtils.getContextString(paramName);
EList contexts = contextItem.getContext();
for (Object context : contexts) {
if (context instanceof ContextType) {
ContextParameterType contextParam = ContextUtils.getContextParameterTypeByName((ContextType) context, paramName);
if (contextParam == null || StringUtils.isBlank(contextParam.getValue())) {
continue;
}
StringBuffer jarBuffer = new StringBuffer();
String[] jars = contextParam.getValue().split(";");
for (int i = 0; i < jars.length; i++) {
if (i != 0) {
jarBuffer.append(";");
}
jarBuffer.append(dbService.getMVNPath(jars[i]));
}
contextParam.setValue(jarBuffer.toString());
}
}
}
}

View File

@@ -19,8 +19,13 @@ import java.util.Map;
import java.util.Set;
import org.talend.components.api.component.ComponentDefinition;
import org.talend.components.api.properties.ComponentProperties;
import org.talend.components.api.properties.ComponentReferenceProperties;
import org.talend.core.model.process.EParameterFieldType;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.INode;
import org.talend.core.runtime.services.IGenericService;
import org.talend.daikon.properties.Properties;
import org.talend.designer.core.generic.model.GenericElementParameter;
import org.talend.designer.core.generic.utils.ComponentsUtils;
@@ -52,5 +57,56 @@ public class GenericService implements IGenericService {
}
return genericComponents;
}
@Override
public void resetReferenceValue(INode curNode, String oldConnectionName, String newConnectionName) {
ComponentReferenceProperties comPro = null;
IElementParameter refPara = curNode.getElementParameterFromField(EParameterFieldType.COMPONENT_REFERENCE);
if(refPara != null){
comPro = ComponentsUtils.getReferencedComponent(refPara);
}
if(comPro != null){
Object sValue = comPro.componentInstanceId.getStoredValue();
if (oldConnectionName.equals(sValue)) {
comPro.componentInstanceId.setValue(newConnectionName);
return;
}
if (sValue != null && ((String)sValue).startsWith(oldConnectionName + "_")) { //$NON-NLS-1$
comPro.componentInstanceId.setValue(((String)sValue).replaceFirst(oldConnectionName + "_", newConnectionName + "_")); //$NON-NLS-1$
return;
}
}
comPro = getComponentReferenceProperties(curNode);
if(comPro == null){
return;
}
Object sValue = comPro.componentInstanceId.getStoredValue();
if (oldConnectionName.equals(sValue)) {
comPro.componentInstanceId.setValue(newConnectionName);
} else if (sValue != null && ((String)sValue).startsWith(oldConnectionName + "_")) { //$NON-NLS-1$
comPro.componentInstanceId.setValue(((String)sValue).replaceFirst(oldConnectionName + "_", newConnectionName + "_")); //$NON-NLS-1$
}
}
private ComponentReferenceProperties getComponentReferenceProperties(INode curNode){
ComponentProperties pros = curNode.getComponentProperties();
if(pros == null){
return null;
}
Properties ps = pros.getProperties("referencedComponent"); //$NON-NLS-1$
if(ps == null){
Properties conn = pros.getProperties("connection"); //$NON-NLS-1$
if(conn != null){
ps = conn.getProperties("referencedComponent"); //$NON-NLS-1$
}
}
if(ps == null){
return null;
}
if(ps instanceof ComponentReferenceProperties){
return (ComponentReferenceProperties)ps;
}
return null;
}
}

View File

@@ -837,6 +837,20 @@ public class ComponentsUtils {
}
return nals;
}
public static ComponentReferenceProperties getReferencedComponent(IElementParameter refPara){
if (!(refPara instanceof GenericElementParameter)) {
return null;
}
Widget widget = ((GenericElementParameter) refPara).getWidget();
NamedThing widgetProperty = widget.getContent();
if (widgetProperty instanceof ComponentReferenceProperties
&& Widget.COMPONENT_REFERENCE_WIDGET_TYPE.equals(widget.getWidgetType())) {
ComponentReferenceProperties props = (ComponentReferenceProperties) widgetProperty;
return props;
}
return null;
}
public static void initReferencedComponent(IElementParameter refPara, String newValue) {

View File

@@ -506,6 +506,7 @@ Node.ELTDBMap.canNotHaveMultiStartNode=The Link type between "{0}" and "{1}" sho
Node.checkHasMultiPrejobOrPostJobComponents="{0}" component found in this job,there must be only one totally,Please check it.
Node.checkTRunjobwithMRProcess=The {1} job is only compatible with independant execution({0}).
Node.checkMultipleTRunjobVersion=All tRunJob of the same process must be of the same version. Please check.
Node.checkDynamicJobUsageWarning=Usage of dynamic job is not discouraged in joblets, not compatible with continuous integration.
Node.checkSparkMLComponentVersion14=This component requires Spark 1.4 or higher.
Node.checkSparkMLComponentVersion15=This component requires Spark 1.5 or higher.
Node.notSupportDepartition=this component not support departition row now.

View File

@@ -66,6 +66,7 @@ import org.talend.core.model.process.IElement;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.IExternalData;
import org.talend.core.model.process.IExternalNode;
import org.talend.core.model.process.IGenericElementParameter;
import org.talend.core.model.process.INode;
import org.talend.core.model.process.INodeConnector;
import org.talend.core.model.process.IProcess;
@@ -89,6 +90,7 @@ import org.talend.core.ui.IJobletProviderService;
import org.talend.core.ui.component.ComponentsFactoryProvider;
import org.talend.daikon.properties.Properties;
import org.talend.daikon.properties.PropertiesVisitor;
import org.talend.daikon.properties.property.PropertyValueEvaluator;
import org.talend.designer.core.i18n.Messages;
import org.talend.designer.core.model.components.AbstractBasicComponent;
import org.talend.designer.core.model.components.EParameterName;
@@ -173,7 +175,7 @@ public class DataProcess implements IGeneratingProcess {
shortUniqueNameList = new ArrayList<String>();
}
private void copyElementParametersValue(IElement sourceElement, IElement targetElement) {
protected void copyElementParametersValue(IElement sourceElement, IElement targetElement) {
if (IAdditionalInfo.class.isInstance(sourceElement) && IAdditionalInfo.class.isInstance(targetElement)) {
IAdditionalInfo.class.cast(sourceElement).cloneAddionalInfoTo((IAdditionalInfo) targetElement);
}
@@ -189,6 +191,20 @@ public class DataProcess implements IGeneratingProcess {
targetParam.setContextMode(sourceParam.isContextMode());
targetParam.setValue(sourceParam.getValue());
if (sourceElement instanceof INode && sourceParam instanceof IGenericElementParameter) {
IComponent component = ((INode) sourceElement).getComponent();
if (component instanceof AbstractBasicComponent
&& EComponentType.GENERIC.equals(component.getComponentType())) {
org.talend.daikon.properties.property.Property property = ((IGenericElementParameter) sourceParam)
.getProperty();
if (sourceParam.getFieldType().equals(EParameterFieldType.CLOSED_LIST) && property != null) {
PropertyValueEvaluator evaluator = property.getValueEvaluator();
if (evaluator != null) {
targetParam.setValue(evaluator.evaluate(property, sourceParam.getValue()));
}
}
}
}
if (sourceParam.getValue() instanceof List) {
List sourceList = (List) sourceParam.getValue();
List targetList = new ArrayList();
@@ -3039,6 +3055,10 @@ public class DataProcess implements IGeneratingProcess {
return;
}
INode refNode = buildCheckMap.get(graphicalNode);
// for joblet node not active => not included in buildCheckMap
if (refNode == null) {
return;
}
List<? extends IConnection> connections = refNode.getIncomingConnections(EConnectionType.FLOW_MAIN);
if (connections.size() == 0) {
return;

View File

@@ -737,6 +737,12 @@ public class StatsAndLogsManager {
if (isGeneric) {// reset the show if
resetShowIf(connectionNode);
if (checkUrlContainsAutoCommit(connectionNode)) {
IElementParameter autoCommitParam = connectionNode.getElementParameter("autocommit");//$NON-NLS-1$
if (autoCommitParam != null) {
autoCommitParam.setValue(Boolean.TRUE);
}
}
}
if (connectionComponentName.contains("Oracle")) {//$NON-NLS-1$
@@ -764,13 +770,44 @@ public class StatsAndLogsManager {
}
}
}
DataConnection dataConnec = createDataConnectionForSubJobOK(dataNode, commitNode);
((List<IConnection>) dataNode.getOutgoingConnections()).add(dataConnec);
((List<IConnection>) commitNode.getIncomingConnections()).add(dataConnec);
boolean noCommitNode = false;
if (checkUrlContainsAutoCommit(connectionNode)) {
IElementParameter autoCommitParam = connectionNode.getElementParameter("autocommit");//$NON-NLS-1$
if (autoCommitParam != null && autoCommitParam.getValue() != null) {
noCommitNode = Boolean.parseBoolean(autoCommitParam.getValue().toString());
if (noCommitNode && nodeList.contains(commitNode)) {
nodeList.remove(commitNode);
}
}
}
if (!noCommitNode) {
DataConnection dataConnec = createDataConnectionForSubJobOK(dataNode, commitNode);
((List<IConnection>) dataNode.getOutgoingConnections()).add(dataConnec);
((List<IConnection>) commitNode.getIncomingConnections()).add(dataConnec);
}
return connectionNode;
}
private static boolean checkUrlContainsAutoCommit(DataNode connectionNode) {
if (connectionNode != null) {
boolean isGeneric = connectionNode.getComponent().getComponentType() == EComponentType.GENERIC;
if (isGeneric) {
IElementParameter urlParam = connectionNode
.getElementParameter(EConnectionParameterName.GENERIC_URL.getDisplayName());
if (urlParam != null) {
Object obj = urlParam.getValue();
if (obj != null && obj instanceof String) {
String url = (String) obj;
if (url != null && url.toLowerCase().contains("autocommit=true")) {//$NON-NLS-1$
return true;
}
}
}
}
}
return false;
}
private static void resetShowIf(DataNode connectionNode) {
ComponentProperties tcomp_properties = connectionNode.getComponentProperties();
if (tcomp_properties != null) {

View File

@@ -32,7 +32,6 @@ import org.eclipse.core.runtime.IAdaptable;
import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.core.runtime.Platform;
import org.eclipse.core.runtime.SubProgressMonitor;
import org.eclipse.draw2d.Cursors;
import org.eclipse.draw2d.FigureCanvas;
@@ -204,7 +203,6 @@ import org.talend.designer.core.ui.editor.connections.ConnLabelEditPart;
import org.talend.designer.core.ui.editor.connections.Connection;
import org.talend.designer.core.ui.editor.connections.ConnectionPart;
import org.talend.designer.core.ui.editor.connections.NodeConnectorTool;
import org.talend.designer.core.ui.editor.jobletcontainer.JobletContainer;
import org.talend.designer.core.ui.editor.nodecontainer.NodeContainer;
import org.talend.designer.core.ui.editor.nodes.Node;
import org.talend.designer.core.ui.editor.nodes.NodePart;
@@ -223,6 +221,7 @@ import org.talend.designer.core.ui.editor.process.ProcessPart;
import org.talend.designer.core.ui.editor.process.TalendEditorDropTargetListener;
import org.talend.designer.core.ui.editor.subjobcontainer.SubjobContainer;
import org.talend.designer.core.ui.editor.subjobcontainer.SubjobContainerPart;
import org.talend.designer.core.ui.views.CodeView;
import org.talend.designer.core.ui.views.jobsettings.JobSettings;
import org.talend.designer.core.ui.views.problems.Problems;
import org.talend.designer.core.ui.views.properties.ComponentSettingsView;
@@ -1186,6 +1185,7 @@ public abstract class AbstractTalendEditor extends GraphicalEditorWithFlyoutPale
this.savePreviouslyNeeded = true;
firePropertyChange(IEditorPart.PROP_DIRTY);
}
checkSaveAsEnabled();
} else {
savePreviouslyNeeded = false;
firePropertyChange(IEditorPart.PROP_DIRTY);
@@ -1193,6 +1193,21 @@ public abstract class AbstractTalendEditor extends GraphicalEditorWithFlyoutPale
super.commandStackChanged(event);
}
private void checkSaveAsEnabled() {
if (!isSaveAsAllowed()) {
return;
}
IAction action = getAction(ActionFactory.SAVE_AS.getId());
if (action == null) {
action = ActionFactory.SAVE_AS.create(PlatformUI.getWorkbench().getActiveWorkbenchWindow());
getActionRegistry().registerAction(action);
}
IAction saveAsAction = getAction(ActionFactory.SAVE_AS.getId());
if (saveAsAction != null && !getAction(ActionFactory.SAVE_AS.getId()).isEnabled()) {
getAction(ActionFactory.SAVE_AS.getId()).setEnabled(true);
}
}
protected void removeErrorStatusIfDirty() {
// remove all error status at any change of the job .
IProcess2 process = getProcess();
@@ -1368,6 +1383,7 @@ public abstract class AbstractTalendEditor extends GraphicalEditorWithFlyoutPale
getEditDomain().setDefaultTool(null);
getSelectionSynchronizer().removeViewer(getGraphicalViewer());
getSite().setSelectionProvider(null);
CodeView.refreshCodeView(null);
}
public void gotoMarker(final IMarker marker) {

View File

@@ -41,6 +41,7 @@ import org.eclipse.jdt.internal.corext.refactoring.rename.JavaRenameProcessor;
import org.eclipse.jdt.internal.corext.refactoring.rename.RenameCompilationUnitProcessor;
import org.eclipse.jdt.internal.ui.javaeditor.CompilationUnitEditor;
import org.eclipse.jdt.ui.actions.IJavaEditorActionDefinitionIds;
import org.eclipse.jface.action.IAction;
import org.eclipse.jface.dialogs.MessageDialog;
import org.eclipse.jface.operation.IRunnableWithProgress;
import org.eclipse.jface.text.source.ISourceViewer;
@@ -58,6 +59,7 @@ import org.eclipse.ui.IEditorPart;
import org.eclipse.ui.IEditorSite;
import org.eclipse.ui.PartInitException;
import org.eclipse.ui.PlatformUI;
import org.eclipse.ui.actions.ActionFactory;
import org.eclipse.ui.part.FileEditorInput;
import org.osgi.framework.Bundle;
import org.osgi.framework.FrameworkUtil;
@@ -418,9 +420,29 @@ public class StandAloneTalendJavaEditor extends CompilationUnitEditor implements
if (!isEditable()) {
return false;
}
return super.isDirty();
boolean dirty = super.isDirty();
if (dirty) {
checkSaveAsEnabled();
}
return dirty;
}
private void checkSaveAsEnabled() {
if (!isSaveAsAllowed()) {
return;
}
IAction action = getAction(ActionFactory.SAVE_AS.getId());
if (action == null) {
action = ActionFactory.SAVE_AS.create(PlatformUI.getWorkbench().getActiveWorkbenchWindow());
setAction(ActionFactory.SAVE_AS.getId(), action);
}
IAction saveAsAction = getAction(ActionFactory.SAVE_AS.getId());
if (saveAsAction != null && !getAction(ActionFactory.SAVE_AS.getId()).isEnabled()) {
getAction(ActionFactory.SAVE_AS.getId()).setEnabled(true);
}
}
@Override
public boolean isEditorInputReadOnly() {
return !isEditable();
@@ -730,6 +752,10 @@ public class StandAloneTalendJavaEditor extends CompilationUnitEditor implements
@Override
public void doSaveAs() {
// for the EditorInput of new created RepositoryNode, need to reload
if (this.rEditorInput.getRepositoryNode() == null) {
this.rEditorInput.setRepositoryNode(null);
}
ERepositoryObjectType type = this.rEditorInput.getRepositoryNode().getObject().getRepositoryObjectType();
if (type == ERepositoryObjectType.ROUTINES) {
SaveAsRoutineAction saveAsAction = new SaveAsRoutineAction(this);

View File

@@ -13,7 +13,9 @@
package org.talend.designer.core.ui.editor.cmd;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
@@ -411,6 +413,8 @@ public class ChangeActivateStatusElementCommand extends Command {
for (INode node : jobletandnodeList) {
if (node.isActivate()) {
// MiddleNodes Map<IConnection, Node> connection->Node
// if Node deactivate then connection->null
Map<IConnection, Node> outMiddleNodes = getAllOutMiddleNodes(node);
Map<IConnection, Node> inMiddleNodes = getAllInMiddleNodes(node);
@@ -457,7 +461,8 @@ public class ChangeActivateStatusElementCommand extends Command {
}
}
if (!exist) {
middConnMap.put(nodeList, connList);
List<IConnection> exactConnections = getExactConnectionsBetweenNodes(node, nodeList, connList, true);
middConnMap.put(nodeList, exactConnections);
}
}
@@ -504,11 +509,86 @@ public class ChangeActivateStatusElementCommand extends Command {
}
}
if (!exist) {
middConnMap.put(nodeList, connList);
List<IConnection> exactConnections = getExactConnectionsBetweenNodes(node, nodeList, connList, false);
middConnMap.put(nodeList, exactConnections);
}
}
}
}
// middConnMap=> key(BaseNode, Node1, Node2......);
// BaseNode->(0/some deactivate node)->Node1;BaseNode->(0/some deactivate node)->Node2;
// middConnMap=> value(connections between Node1 and Node2).
return middConnMap;
}
public static List<IConnection> getExactConnectionsBetweenNodes(INode node, List<INode> nodeList, List<IConnection> connList,
boolean isOutgoing) {
List<IConnection> exactList = new ArrayList<IConnection>(connList);
List<INode> targetNodeList = new ArrayList<INode>(nodeList);
targetNodeList.remove(node);
Map<INode, Set<INode>> pathhm = new HashMap<INode, Set<INode>>();
for (INode targetNode : targetNodeList) {
Set<INode> pathNodes = new HashSet<INode>();
pathNodes.add(node);
// Got the passby Node between node and tatgetNode
boolean pathFound = getPathsNodes(node, targetNode, pathNodes, isOutgoing);
if (pathFound) {
pathhm.put(targetNode, pathNodes);
}
}
Collection<Set<INode>> pathSets = pathhm.values();
if (!pathSets.isEmpty()) {
Iterator<IConnection> connectionIte = exactList.iterator();
while (connectionIte.hasNext()) {
IConnection connection = (IConnection) connectionIte.next();
boolean rightIn = false;
Iterator<Set<INode>> pathIte = pathSets.iterator();
while (pathIte.hasNext()) {
Set<INode> nodes = (Set<INode>) pathIte.next();
// if sourceNode and targetNode of the connection don't exist in the NodePath then this connection
// doesn't belong to this flow of BaseNode->(0/some deactivate node)->Node1
// maybe belongs to other flow of BaseNode but all flow deactivate.
if (nodes.contains(connection.getSource()) && nodes.contains(connection.getTarget())) {
rightIn = true;
}
}
if (!rightIn) {
connectionIte.remove();
}
}
}
return exactList;
}
private static boolean getPathsNodes(INode node, INode targetNode, Set<INode> nodes, boolean isOutgoing) {
List<? extends IConnection> connections = null;
if (node.equals(targetNode)) {
return true;
}
if (isOutgoing) {
connections = node.getOutgoingConnections();
} else {
connections = node.getIncomingConnections();
}
for (IConnection connection : connections) {
INode deepNode = null;
if (isOutgoing) {
deepNode = connection.getTarget();
} else {
deepNode = connection.getSource();
}
boolean flag = getPathsNodes(deepNode, targetNode, nodes, isOutgoing);
if (flag) {
nodes.add(deepNode);
return true;
}
}
return false;
}
}

View File

@@ -39,6 +39,7 @@ import org.talend.core.ui.IJobletProviderService;
import org.talend.core.utils.KeywordsValidator;
import org.talend.designer.core.ICamelDesignerCoreService;
import org.talend.designer.core.i18n.Messages;
import org.talend.designer.core.model.components.EParameterName;
import org.talend.designer.core.model.components.ExternalUtilities;
import org.talend.designer.core.model.process.ConnectionManager;
import org.talend.designer.core.ui.dialog.mergeorder.ConnectionTableAndSchemaNameDialog;
@@ -75,6 +76,8 @@ public class ConnectionCreateCommand extends Command {
private static boolean creatingConnection = false;
private boolean insertTMap;
private Map<EParameterName, Object> connectionParameters;
/**
* Initialisation of the creation of the connection with a source and style of connection.
@@ -90,6 +93,9 @@ public class ConnectionCreateCommand extends Command {
this.metaName = (String) listArgs.get(0);
this.connectionName = (String) listArgs.get(1);
newMetadata = (IMetadataTable) listArgs.get(2);
if (listArgs.size() >= 4) {
connectionParameters = (Map<EParameterName, Object>) listArgs.get(3);
}
}
public ConnectionCreateCommand(Node nodeSource, String connectorName, List<Object> listArgs, boolean insertTMap) {
@@ -424,7 +430,7 @@ public class ConnectionCreateCommand extends Command {
monitorConnection);
} else {
this.connection = new Connection(source, target, newLineStyle, connectorName, metaName, connectionName, metaName,
monitorConnection);
monitorConnection, this.connectionParameters);
}
} else { // in case of redo, reuse the same instance
if (newMetadata != null) {

View File

@@ -25,6 +25,7 @@ import org.talend.core.GlobalServiceRegister;
import org.talend.core.ITDQRuleService;
import org.talend.core.database.EDatabase4DriverClassName;
import org.talend.core.database.EDatabaseTypeName;
import org.talend.core.model.components.IMultipleComponentManager;
import org.talend.core.model.metadata.IMetadataTable;
import org.talend.core.model.metadata.QueryUtil;
import org.talend.core.model.metadata.builder.connection.Connection;
@@ -274,13 +275,36 @@ public class QueryGuessCommand extends Command {
IElementParameter connector = node.getElementParameter("CONNECTION");
if (connector != null) {
String connectorValue = connector.getValue().toString();
List<? extends INode> graphicalNodes = process.getGeneratingNodes();
for (INode node : graphicalNodes) {
if (node.getUniqueName().equals(connectorValue)) {
connectionNode = node;
for (INode generatingNode : process.getGraphicalNodes()) {
if (generatingNode.getUniqueName().equals(connectorValue)) {
connectionNode = generatingNode;
break;
}
}
if (connectionNode == null) {
List<? extends INode> graphicalNodes = process.getGeneratingNodes();
for (INode node : graphicalNodes) {
if (node.getUniqueName().equals(connectorValue)) {
connectionNode = node;
break;
}
}
// for visual connection component in joblet
if (connectionNode == null && node instanceof INode) {
List<IMultipleComponentManager> multipleComponentManagers = ((INode) node).getComponent()
.getMultipleComponentManagers();
for (IMultipleComponentManager manager : multipleComponentManagers) {
String inName = manager.getInput().getName();
String componentValue = connectorValue + "_" + inName;
for (INode gnode : process.getGeneratingNodes()) {
if (gnode.getUniqueName().equals(componentValue) && (gnode instanceof INode)) {
connectionNode = gnode;
break;
}
}
}
}
}
}
}
// hywang add for bug 7575

View File

@@ -109,6 +109,8 @@ public class Connection extends Element implements IConnection, IPerformance, IA
private boolean monitorConnection = false;
public boolean setNullColumn = false;
private Map<EParameterName, Object> paramValues = new HashMap<EParameterName, Object>();
/**
* Tells if this connection has a subjob source or not instead of a node.
@@ -132,6 +134,13 @@ public class Connection extends Element implements IConnection, IPerformance, IA
this.uniqueName = uniqueName;
init(source, target, lineStyle, connectorName, metaName, linkName, monitorConnection);
}
public Connection(INode source, INode target, EConnectionType lineStyle, String connectorName, String metaName,
String linkName, String uniqueName, final boolean monitorConnection, final Map<EParameterName, Object> paramValues) {
this.uniqueName = uniqueName;
this.paramValues = paramValues;
init(source, target, lineStyle, connectorName, metaName, linkName, monitorConnection);
}
// used only in ConnectionManager to test if we can connect or not.
public Connection(INode source, INode target, EConnectionType lineStyle, final boolean monitorConnection) {
@@ -265,7 +274,13 @@ public class Connection extends Element implements IConnection, IPerformance, IA
param.setListItemsDisplayCodeName(strList);
param.setNbLines(1);
param.setFieldType(EParameterFieldType.CLOSED_LIST);
param.setValue(supportedLanguages.getValue());
if (supportedLanguages != null) {
param.setValue(supportedLanguages.getValue());
} else {
param.setValue("");
}
param.setShow(true);
param.setNumRow(1);
addElementParameter(param);
@@ -524,9 +539,24 @@ public class Connection extends Element implements IConnection, IPerformance, IA
}
}
}
updateParametersValues();
updateInputConnection();
}
private void updateParametersValues() {
if (paramValues == null) {
return;
}
for (Map.Entry<EParameterName, Object> param : paramValues.entrySet()) {
if (param.getValue() != null) {
setPropertyValue(param.getKey().getName(), param.getValue());
}
}
}
private void updateInputConnection() {
IComponent component = null;
if (this.target != null) {

View File

@@ -1883,7 +1883,15 @@ public class Node extends Element implements IGraphicalNode {
if (id.equals(EParameterName.CONNECTION_FORMAT.getName())) {
connectionToParse = (String) value;
setConnectionName(ElementParameterParser.parse(this, connectionToParse));
boolean isTestCase = false;
if (GlobalServiceRegister.getDefault().isServiceRegistered(ITestContainerProviderService.class)) {
ITestContainerProviderService testContainerService = (ITestContainerProviderService) GlobalServiceRegister
.getDefault().getService(ITestContainerProviderService.class);
isTestCase = getProcess() != null && testContainerService.isTestContainerProcess(getProcess());
}
if (!isTestCase) {
setConnectionName(ElementParameterParser.parse(this, connectionToParse));
}
// to check
// String newValue = ElementParameterParser.parse(this, connectionToParse);
// setConnectionName(newValue);
@@ -3002,6 +3010,7 @@ public class Node extends Element implements IGraphicalNode {
checktAggregateRow(param);
checkDynamicJobUsage(param);
}
checkJobletConnections();
@@ -3125,6 +3134,25 @@ public class Node extends Element implements IGraphicalNode {
}
}
private void checkDynamicJobUsage(IElementParameter param) {
if (!EParameterName.USE_DYNAMIC_JOB.getName().equals(param.getName())) {
return;
}
boolean isSelectUseDynamic = false;
Object paramValue = param.getValue();
if (paramValue != null && paramValue instanceof Boolean) {
isSelectUseDynamic = (Boolean) paramValue;
}
if (isSelectUseDynamic) {
ERepositoryObjectType itemType = ERepositoryObjectType.getItemType(process.getProperty().getItem());
if (ERepositoryObjectType.getAllTypesOfJoblet().contains(itemType)) {
String warningMessage = Messages.getString("Node.checkDynamicJobUsageWarning");
Problems.add(ProblemStatus.WARNING, this, warningMessage);
}
}
}
private void checkJobletConnections() {
if (this.isJoblet()) {
IProcess2 comProcess = (IProcess2) component.getProcess();

View File

@@ -80,6 +80,7 @@ import org.talend.core.language.CodeProblemsChecker;
import org.talend.core.language.ECodeLanguage;
import org.talend.core.language.ICodeProblemsChecker;
import org.talend.core.model.components.EComponentType;
import org.talend.core.model.components.IMultipleComponentManager;
import org.talend.core.model.metadata.IMetadataConnection;
import org.talend.core.model.metadata.IMetadataTable;
import org.talend.core.model.metadata.QueryUtil;
@@ -2181,6 +2182,28 @@ public abstract class AbstractElementPropertySectionController implements Proper
}
}
}
if (connectionNode == null) {
INode node = null;
if (elem instanceof INode) {
node = (INode) elem;
} else { // else instanceof Connection
node = ((IConnection) elem).getSource();
}
if (node != null) {
List<IMultipleComponentManager> multipleComponentManagers = node.getComponent()
.getMultipleComponentManagers();
for (IMultipleComponentManager manager : multipleComponentManagers) {
String inName = manager.getInput().getName();
String componentValue = compValue + "_" + inName;
for (INode gnode : nodes) {
if (gnode.getUniqueName().equals(componentValue) && (gnode instanceof INode)) {
connectionNode = gnode;
break;
}
}
}
}
}
if (connectionNode != null) {
if (isUserExistionConnectionType) {
IElementParameter ele = connectionNode.getElementParameter("CONNECTION_TYPE");

View File

@@ -48,6 +48,7 @@ import org.talend.core.model.process.IElement;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.INode;
import org.talend.core.model.process.IProcess2;
import org.talend.core.runtime.services.IGenericService;
import org.talend.core.ui.IJobletProviderService;
import org.talend.core.ui.process.IGraphicalNode;
import org.talend.core.ui.properties.tab.IDynamicProperty;
@@ -209,6 +210,20 @@ public class ComponentListController extends AbstractElementPropertySectionContr
} else if (value != null && value.startsWith(oldConnectionName + "_")) {
curParam.setValue(value.replaceFirst(oldConnectionName + "_", newConnectionName + "_"));
}
}else if(curParam.getFieldType().equals(EParameterFieldType.COMPONENT_REFERENCE)){
String value = (String)curParam.getValue();
if (oldConnectionName.equals(value)) {
curParam.setValue(newConnectionName);
} else if (value != null && value.startsWith(oldConnectionName + "_")) {
curParam.setValue(value.replaceFirst(oldConnectionName + "_", newConnectionName + "_"));
}
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericService.class)) {
IGenericService genericService = (IGenericService) GlobalServiceRegister.getDefault().getService(
IGenericService.class);
if(genericService != null){
genericService.resetReferenceValue(curNode, oldConnectionName, newConnectionName);
}
}
} else if (curParam.getFieldType().equals(EParameterFieldType.TABLE)) {
final Object[] itemsValue = curParam.getListItemsValue();
for (Object element : itemsValue) {

View File

@@ -23,6 +23,7 @@ import org.talend.core.database.conn.version.EDatabaseVersion4Drivers;
import org.talend.core.language.LanguageManager;
import org.talend.core.model.components.ComponentCategory;
import org.talend.core.model.components.IComponent;
import org.talend.core.model.components.IMultipleComponentManager;
import org.talend.core.model.general.ModuleNeeded;
import org.talend.core.model.process.IContext;
import org.talend.core.model.process.IElementParameter;
@@ -118,13 +119,37 @@ public class GuessSchemaProcess extends AbstractGuessSchemaProcess {
IElementParameter connector = node.getElementParameter("CONNECTION");
if (connector != null) {
String connectorValue = connector.getValue().toString();
List<? extends INode> generatingNodes = originalProcess.getGeneratingNodes();
for (INode generatingNode : generatingNodes) {
for (INode generatingNode : originalProcess.getGraphicalNodes()) {
if (generatingNode.getUniqueName().equals(connectorValue)) {
connectionNode = generatingNode;
break;
}
}
List<? extends INode> generatingNodes = originalProcess.getGeneratingNodes();
if (connectionNode == null) {
// for connection node in joblet
for (INode generatingNode : generatingNodes) {
if (generatingNode.getUniqueName().equals(connectorValue)) {
connectionNode = generatingNode;
break;
}
}
// for visual connection component in joblet
if (connectionNode == null) {
List<IMultipleComponentManager> multipleComponentManagers = node.getComponent()
.getMultipleComponentManagers();
for (IMultipleComponentManager manager : multipleComponentManagers) {
String inName = manager.getInput().getName();
String componentValue = connectorValue + "_" + inName;
for (INode gnode : generatingNodes) {
if (gnode.getUniqueName().equals(componentValue) && (gnode instanceof INode)) {
connectionNode = gnode;
break;
}
}
}
}
}
}
}

View File

@@ -121,7 +121,7 @@ public class StatsAndLogsConstants {
public static final String[] MSSQL_VERSION_CODE = new String[] { "JTDS", "MSSQL_PROP" }; //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
public static final String[] MSSQL_VERSION_DRIVER = new String[] {
"jtds-1.3.1-patch.jar", "mssql-jdbc.jar"};//$NON-NLS-1$ //$NON-NLS-2$
"jtds-1.3.1-patch-20190523.jar", "mssql-jdbc.jar"};//$NON-NLS-1$ //$NON-NLS-2$
public static final String[] DB_VERSION_DISPLAY = new String[] { "Oracle 12", "Oracle 11", "Oracle 10", "Oracle 9",
"Oracle 8", "MySQL 5", "MySQL 4", "MariaDB","Open source JTDS", "Microsoft", "Access 2003", "Access 2007", "Prior to v9", "v9 +" };

Some files were not shown because too many files have changed in this diff Show More