Compare commits

...

51 Commits

Author SHA1 Message Date
mbasiuk
bb04e941fb chore: update timestamp 2021-05-21 10:20:07 +03:00
mbasiuk
f024dac0f4 chore: update timestamp 2021-05-18 12:58:58 +03:00
mbasiuk
bfa7955f7b chore: patch notes 2021-05-14 19:18:59 +03:00
hcyi
c928d6f456 fix(TUP-30043):Add a new option for tELTOracleMap to generate column alias on selection. (#5863)
* fix(TUP-30043):Add a new option for tELTOracleMap to generate column
alias on selection.

* fix(TUP-30043):Add a new option for tELTOracleMap to generate column
alias on selection.
2021-03-08 18:29:14 +08:00
bhe-talendbj
8b10f58b29 fix(TUP-25109): backport TUP-25109 (#5882) (#5914)
* fix(TUP-25109): backport TUP-25109

* fix(TUP-25109): fix guess schema warning

Conflicts:
	main/plugins/org.talend.designer.core/src/main/java/org/talend/designer/core/ui/editor/properties/controllers/GuessSchemaProcess.java
2021-03-08 14:57:40 +08:00
bhe-talendbj
037fe2654e fix(TUP-30360): add an option to disable sharing libraries (#5814) (#5841) (#5889)
* fix(TUP-30360): add an option to disable sharing libraries

* fix(TUP-30360): add an option to disable sharing libraries, default to true
2021-03-02 15:12:16 +08:00
hzhao-talendbj
b7d3d4dd5b TUP-30333 tMap with Lookup model Reload at each row freezes Studio (#5817) (#5877)
* TUP-30333 tMap with Lookup model Reload at each row freezes Studio

* TUP-30333 add comments

* TUP-30333 add condition for refresh background
2021-02-25 19:15:21 +08:00
mbasiuk-talend
c14e9c2e5d chore(TDI-45608): upgrade Snowflake driver version (#5845) 2021-02-17 13:02:57 +02:00
Maksym Basiuk
3ec58bc01a chore: backport migration task for TDI-45455 2021-01-27 11:04:38 +02:00
Maksym Basiuk
fe4f6df6b2 chore: backport TDI-45455 2021-01-27 10:59:03 +02:00
Maksym Basiuk
5cc63fe6d8 chore: backport TDI-45455 2021-01-27 10:53:31 +02:00
SunChaoqun
cae81a6172 TESB-29921:Build/Deploy of a route snapshot with CommandLine has 0 byte bundle in Nexus (#5711) 2021-01-26 16:19:02 +08:00
vdrokov
ee7068dc53 TESB-30317:[CI Builder] Some models throw error "context file doesn't exist" during CITEST compilation phase (#5406) 2021-01-12 14:45:50 +01:00
sbliu
e826fdc490 fix(TUP-29550) fix problem that "rename" Job in property sheet page duplicates item in Pom file (#5664) 2020-12-29 10:59:41 +08:00
hcyi
10509c005c fix(TUP-29072):improve for When renaming link (table) between teltinput (#5638)
and teltmap , the generated SQL query is not updated.
2020-12-17 17:50:38 +08:00
mbasiuk-talend
54971f678f fix(TDI-45260): update components with new library version (#5602) (#5644) 2020-12-16 17:29:25 +02:00
Dmytro Grygorenko
c64a3a276b fix(TDI-45006): correct way to fix. (#5605) 2020-12-08 10:29:01 +02:00
hcyi
bd99714f18 fix(TUP-29072):When renaming link (table) between teltinput and teltmap, (#5576)
the generated SQL query is not updated.
2020-12-04 11:10:14 +08:00
hcyi
cc5b551d84 fix(TUP-29090):improve since broken some junits . (#5580) 2020-11-30 20:15:13 +08:00
hcyi
ce18e0ffd1 fix(TUP-29090):[7.2.1] Extra plus '+' signs in generated SQL (#5559) 2020-11-25 10:53:48 +08:00
Dmytro Grygorenko
d8aad268c2 fix(TDI-45015): removed streams declaration outside try-with-resources (#5445) 2020-10-30 09:53:25 +02:00
Dmytro Grygorenko
b9e74d9f02 fix(TDI-45015): backport fix into "maintenance/7.2" branch (#5421) 2020-10-23 11:24:46 +03:00
Oleksandr Zhelezniak
4539464127 fix(TDI-44935): reuse custom query expression (#5396)
* make query expression idempotent in the generated code
* store the expression in the variable and then read it again
2020-10-20 13:55:33 +03:00
Dmytro Grygorenko
0721ad904b fix(TDI-44880): backport to "maintenance/7.2" branch. (#5306) 2020-10-05 17:20:48 +03:00
wang wei
0de13ccfee fix(TDI-44643): NPE exception on tWaitForSQLData with snowflake
connection(#5006)
2020-08-31 11:13:46 +08:00
apoltavtsev
4468aaa175 fix(TESB-29963) MQ Dependency issues in Runtime 2020-08-24 16:05:45 +02:00
Chao MENG
a6c8578551 Cmeng/maintenance/7.2/tup 28304 backport (#5087)
* fix(TUP-27953): tSalesforceInput component displays in-job (#4996)

tSalesforceConnection incorrectly
https://jira.talendforge.org/browse/TUP-27953

Conflicts:
	main/plugins/org.talend.designer.core.generic/src/main/java/org/talend/designer/core/generic/controller/ComponentRefController.java

* fix(TUP-28295): UI Problem of TCOMPV0 in the recent 741NB and 731
Release + Aug. Nightly Build Patch
https://jira.talendforge.org/browse/TUP-28295
2020-08-19 10:46:34 +08:00
Mike Yan
e7cd94f879 fix(TESB-29210): Build type not updated for consumer job (#5014) 2020-08-03 16:56:23 +08:00
Dmytro Sylaiev
2daa07e10f fix(TDI-44349): Add artifact-copy for talend-proxy lib 2020-07-22 15:40:42 +03:00
msheverda-talend
4418345c3e TESB-29647 Bundle version does not use 'Custom Version' for jobs (#4888)
Co-authored-by: Maksym Sheverda <maksym.sheverda@synapse.com>
2020-07-14 09:59:35 +03:00
Dmytro Sylaiev
4b243ce10e fix(TDI-44349): Backport to 7.2 2020-07-06 21:01:09 +03:00
hzhao-talendbj
e5338b845e Hengzhao/backport72/tup 27070 2 (#4845)
* TUP27070  fix junit failed on 721

* strange code was add to the pr, remove it
2020-06-23 09:19:15 +08:00
hzhao-talendbj
f95639be52 Tup-27070 (#4734) (#4764)
* Tup-27070 (#4734)
backport to 7.2 and fix conflicts

* TUP-27070  fix cross join cases
2020-06-19 15:04:19 +08:00
Maksym Basiuk
15ee342c10 chore: backport for TDI-44165 2020-06-18 18:45:37 +03:00
Maksym Basiuk
606d704249 fix(TDI-43632): use defined pattern by user instead of static value 2020-06-04 11:31:04 +03:00
bhe-talendbj
8db7b77fb0 fix(TUP-27053): fix junit test case (#4676) (#4771) 2020-06-04 14:40:53 +08:00
bhe-talendbj
9edc76339a fix(TUP-27053): backport (#4767) 2020-06-04 11:27:44 +08:00
undx
ae8bf4b59a fix(TDI-44174): Component manager is not thread safe
- this fixes the case when run as independent job is checked and the
  m2.repository automatically defaults to user's m2 in exported job.
- issue on singleton will be fix by component-runtime upgrade.
2020-06-03 10:08:50 +02:00
wang wei
df4ee764ea fix(TDI-43833): tDBBulkExec bug when using PostGress driver (#4701) 2020-05-27 14:23:09 +08:00
Zhiwei Xue
a453e16e66 fix(TUP-27199): remove wrapping space for condition. 2020-05-26 14:47:49 +08:00
hcyi
f25c4badd1 fix(TUP-26924):tELTXXX: link name won't be updated after updating (#4698)
"default table name" in tELTInput.
2020-05-20 14:32:30 +08:00
wang wei
94b67f8772 fix(TDI-44159): Data viewer on tSybaseInput get error in context model with sybase 16 anywhere database(#4661) 2020-05-13 14:45:48 +08:00
Richard Lecomte
31b4ee7333 TDI-43935 : MsSQLRow NString type for prepared statement (#4651) 2020-05-12 13:09:16 +02:00
ypiel
5027b0121b fix(TDI-44147) : Don't explicitly create the table 2020-05-12 11:24:01 +02:00
undx
b6cc2476f9 fix(TCOMP-1681): fix Windows path case 2020-05-07 15:34:54 +02:00
undx
48949763d2 feat(TCOMP-1681): Make ESB Studio job export TaCoKit compliant
- OSGi assembly templates
- TaCoKit Studio part
- OSGi resources
2020-04-24 18:12:04 +02:00
pyzhou
5ef8452927 fix(TDI-44051):add the real value to the db name (#4621)
* fix(TDI-44051):add the real value to the db name

* set dbName as name when db column is null

* workaround

* change back name
2020-04-24 11:31:52 +08:00
sbovsunovskyi
65b6907dcd fix(TDI-44070): Unable to execute the job, receiving "Failed to gener… (#4625)
* fix(TDI-44070): Unable to execute the job, receiving "Failed to generate code." when using tAmazonRedshiftManage component.

* fix(TDI-44070): Unable to execute the job, receiving "Failed to generate code." when using tAmazonRedshiftManage component.

(cherry picked from commit 0fecec9635)
2020-04-21 16:33:53 +03:00
OleksiiNimych
c7218025c6 fix(TDI-43995): Support listing any path and file (#4611)
* fix(TDI-43995): Support listing any path and file

* fix(TDI-43995): Updated Apache Commons Lang library and added diamond operator

* fix(TDI-43995): Remove Apache Commons Lang library
2020-04-16 10:45:00 +03:00
hcyi
4d7fe6a502 fix(TUP-26750):Table name in eltinput component, the default table is (#4623)
taken into consideration
2020-04-16 11:12:08 +08:00
hzhao-talendbj
22e414d893 TUP-26388 fix missing DB context value (#4614) (#4620)
* TUP-26388 fix missing DB context value

* fix missing db value issue

* remove useless isTcompv0
2020-04-14 23:52:04 +08:00
157 changed files with 5204 additions and 2945 deletions

59
PATCH_RELEASE_NOTE.md Normal file
View File

@@ -0,0 +1,59 @@
---
version: 7.2.1
module: https://talend.poolparty.biz/coretaxonomy/42
product:
- https://talend.poolparty.biz/coretaxonomy/23
---
# TPS-4808
| Info | Value |
| ---------------- | ---------------- |
| Patch Name | Patch\_20210521\_TPS-4808\_v1-7.2.1 |
| Release Date | 2021-05-21 |
| Target Version | 20190620_1446-V7.2.1 |
| Product affected | Talend Studio |
## Introduction
This is a self-contained patch.
**NOTE**: For information on how to obtain this patch, reach out to your Support contact at Talend.
## Fixed issues
This patch contains the following fixes:
- TPS-4808 [7.2.1] tSoap components doesn't work and returning a 400 Bad request message(TDI-44893)
## Prerequisites
Consider the following requirements for your system:
- Talend Studio 7.2.1 must be installed.
## Installation
**NOTE**: If the patch is deployed in the approach Installing the patch using Talend Studio, the folder configuration under this patch must be replaced manually.
### Installing the patch using Software update
1) Logon TAC and switch to Configuration->Software Update, then enter the correct values and save referring to the documentation: https://help.talend.com/reader/f7Em9WV_cPm2RRywucSN0Q/j9x5iXV~vyxMlUafnDejaQ
2) Switch to Software update page, where the new patch will be listed. The patch can be downloaded from here into the nexus repository.
3) On Studio Side: Logon Studio with remote mode, on the logon page the Update button is displayed: click this button to install the patch.
### Installing the patch using Talend Studio
1) Create a folder named "patches" under your studio installer directory and copy the patch .zip file to this folder.
2) Restart your studio: a window pops up, then click OK to install the patch, or restart the commandline and the patch will be installed automatically.
### Installing the patch using Commandline
Execute the following commands:
1. Talend-Studio-win-x86_64.exe -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002 --talendDebug
2. initRemote {tac_url} -ul {TAC login username} -up {TAC login password}
3. checkAndUpdate -tu {TAC login username} -tup {TAC login password}

View File

@@ -1290,13 +1290,11 @@ if (execStat) {
} else {
type = "ftp";
}
if ("ftp".equals(type)) { %>
((com.enterprisedt.net.ftp.FTPClient) obj_conn).quit();
<%} else if ("sftp".equals(type)) { %>
if ("sftp".equals(type)) { %>
((com.jcraft.jsch.ChannelSftp) obj_conn).quit();
<%} else { %>
((org.apache.commons.net.ftp.FTPSClient) obj_conn).disconnect();
<%}%>
<%} else { %>
((org.apache.commons.net.ftp.FTPClient) obj_conn).disconnect();
<%}%>
}
<% } %>
} catch (java.lang.Exception e) {

View File

@@ -62,6 +62,8 @@ class IndexedRecordToRowStructGenerator {
%>
boolean <%=codeVarIsDynamicInitialized%> = false;
routines.system.Dynamic <%=codeVarDynamic%> = new routines.system.Dynamic();
//Workaround for TDI-44051, TcimpV0 do not need DbmsID
<%=codeVarDynamic%>.setDbmsId("<%=cid%>");
<%
}
@@ -115,10 +117,11 @@ class IndexedRecordToRowStructGenerator {
for (org.apache.avro.Schema.Field dynamicField_<%=cid%> : dynSchema_<%=cid%>.getFields()){
routines.system.DynamicMetadata dynamicMetadata_<%=cid%> = new routines.system.DynamicMetadata();
org.apache.avro.Schema dynamicFieldSchema_<%=cid%> = dynamicField_<%=cid%>.schema();
String dbName = dynamicField_<%=cid%>.getProp("talend.field.dbColumnName");
// set name
dynamicMetadata_<%=cid%>.setName(dynamicField_<%=cid%>.name());
// set db name
dynamicMetadata_<%=cid%>.setDbName(dynamicField_<%=cid%>.name());
dynamicMetadata_<%=cid%>.setDbName(dbName==null?dynamicField_<%=cid%>.name():dbName);
// set nullable
if (org.talend.daikon.avro.AvroUtils.isNullable(dynamicFieldSchema_<%=cid%>)) {
dynamicMetadata_<%=cid%>.setNullable(true);

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.talend.libraries</groupId>
<artifactId>talendExcel-1.4-20190531</artifactId>
<artifactId>talendExcel-1.6-20210127</artifactId>
<version>6.0.0</version>
<packaging>jar</packaging>

View File

@@ -69,6 +69,10 @@ public class ExcelTool {
private int rowAccessWindowSize = SXSSFWorkbook.DEFAULT_WINDOW_SIZE;// used in auto flush
private boolean isTrackAllColumns = false;
private boolean isTruncateExceedingCharacters = false;
private static final int CELL_CHARACTERS_LIMIT = 32767;
public ExcelTool() {
cellStylesMapping = new HashMap<>();
@@ -274,7 +278,10 @@ public class ExcelTool {
public void addCellValue(String stringValue) {
addCell();
curCell.setCellValue(stringValue);
String value = isTruncateExceedingCharacters && stringValue != null && stringValue.length() > CELL_CHARACTERS_LIMIT
? stringValue.substring(0, CELL_CHARACTERS_LIMIT)
: stringValue;
curCell.setCellValue(value);
curCell.setCellStyle(getNormalCellStyle());
}
@@ -349,4 +356,8 @@ public class ExcelTool {
((SXSSFSheet) sheet).flushRows();
}
}
public void setTruncateExceedingCharacters(boolean isTruncateExceedingCharacters) {
this.isTruncateExceedingCharacters = isTruncateExceedingCharacters;
}
}

View File

@@ -54,11 +54,11 @@ for(IConnection conn : outgoingConns) {
<%
}
}
log4jCodeGenerateUtil.query(node);
%>
query_<%=cid %> = <%=dbquery%>;
whetherReject_<%=cid%> = false;
<%
log4jCodeGenerateUtil.query(node, "query_" + cid);
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
@@ -131,7 +131,7 @@ try {
<%
}
}
log4jCodeGenerateUtil.logInfo(node,"info",cid+" - Execute the query: '\" + "+dbquery +" + \"' has finished.");
log4jCodeGenerateUtil.logInfo(node,"info",cid+" - Execute the query: '\" + query_" + cid + " + \"' has finished.");
%>
<% //feature 0010425
if(usePrepareStatement){

View File

@@ -24,6 +24,9 @@
String user = ElementParameterParser.getValue(node, "__USER__");
String node_type = ElementParameterParser.getValue(node, "__NODE_TYPE__");
String node_count = ElementParameterParser.getValue(node, "__NODE_COUNT__");
if (node_count.isEmpty()) {
node_count = "1";
}
boolean isLog4jEnabled = "true".equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -88,52 +91,46 @@
.withMasterUsername(<%=user%>)
.withMasterUserPassword(decryptedPwd_<%=cid%>)
.withNodeType(<%=node_type%>)
.withNodeType(<%=node_type%>);
<%
if(new Integer(node_count) > 1){
%>
.withNumberOfNodes(<%=node_count%>)
<%
} else {
%>
.withClusterType("single-node")
<%
}
%>
//advanced settings
<%if(parameterUtil.isValid(parameter_group_name)) {%>
.withClusterParameterGroupName(<%=parameter_group_name%>)
<%}%>
<%if(parameterUtil.isValid(subnet_group_name)) {%>
.withClusterSubnetGroupName(<%=subnet_group_name%>)
if (<%=node_count%> > 1) {
request_<%=cid%> = request_<%=cid%>.withNumberOfNodes(<%=node_count%>);
} else {
request_<%=cid%> = request_<%=cid%>.withClusterType("single-node");
}
//advanced settings
<%if(parameterUtil.isValid(parameter_group_name)) {%>
request_<%=cid%> = request_<%=cid%>.withClusterParameterGroupName(<%=parameter_group_name%>);
<%}%>
<%if(parameterUtil.isValid(subnet_group_name)) {%>
request_<%=cid%> = request_<%=cid%>.withClusterSubnetGroupName(<%=subnet_group_name%>);
<%}%>
<%
if(publicly_accessible) {
%>
request_<%=cid%> = request_<%=cid%>.withPubliclyAccessible(true);
<%if(set_public_ip_address) {%>
request_<%=cid%> = request_<%=cid%>.withElasticIp(<%=elastic_ip%>);
<%}%>
<%
if(publicly_accessible) {
%>
.withPubliclyAccessible(true)
<%if(set_public_ip_address) {%>
.withElasticIp(<%=elastic_ip%>)
<%}%>
<%
} else {
%>
.withPubliclyAccessible(false)
<%
}
%>
<%if(parameterUtil.isValid(availability_zone)) {%>
.withAvailabilityZone(<%=availability_zone%>)
<%}%>
<%if(parameterUtil.isValid(vpc_security_groupids)) {%>
.withVpcSecurityGroupIds(<%=vpc_security_groupids%>.split(","))
<%}%>
;
<%
} else {
%>
request_<%=cid%> = request_<%=cid%>.withPubliclyAccessible(false);
<%
}
%>
<%if(parameterUtil.isValid(availability_zone)) {%>
request_<%=cid%> = request_<%=cid%>.withAvailabilityZone(<%=availability_zone%>);
<%}%>
<%if(parameterUtil.isValid(vpc_security_groupids)) {%>
request_<%=cid%> = request_<%=cid%>.withVpcSecurityGroupIds(<%=vpc_security_groupids%>.split(","));
<%}%>
com.amazonaws.services.redshift.model.Cluster result_<%=cid%> = client_<%=cid%>.createCluster(request_<%=cid%>);
<%if(isLog4jEnabled) {%>

View File

@@ -149,7 +149,7 @@
<IMPORTS>
<IMPORT
NAME="Talen File Enhanced"
MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar"
MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar"
REQUIRED="true"
/>
</IMPORTS>

View File

@@ -32,6 +32,8 @@
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
boolean useCustomNullMarker = ElementParameterParser.getBooleanValue(node, "__USE_CUSTOM_NULL_MARKER__");
String nullMarker = useCustomNullMarker ? ElementParameterParser.getValue(node, "__CUSTOM_NULL_MARKER__") : "\"\\\\N\"";
String passwordFieldName = "";
if (authMode.equals("OAUTH")) {
@@ -360,7 +362,7 @@
queryLoad_<%=cid%>.setDestinationTable(destinationTable_<%=cid%>);
queryLoad_<%=cid%>.setSourceUris(java.util.Arrays.asList(<%=ElementParameterParser.getValue(node, "__GS_FILE__")%>));
queryLoad_<%=cid%>.setSkipLeadingRows(<%=ElementParameterParser.getValue(node, "__GS_FILE_HEADER__")%>);
queryLoad_<%=cid%>.setNullMarker("\\N");
queryLoad_<%=cid%>.setNullMarker(<%= nullMarker %>);
config_<%=cid%>.setLoad(queryLoad_<%=cid%>);
job_<%=cid%>.setConfiguration(config_<%=cid%>);
@@ -536,7 +538,6 @@
com.google.cloud.bigquery.Schema schema_<%=cid%> = com.google.cloud.bigquery.Schema.of(fields_<%=cid%>);
com.google.cloud.bigquery.TableInfo tableInfo_<%=cid%> = com.google.cloud.bigquery.TableInfo.newBuilder(tableId_<%=cid%>, com.google.cloud.bigquery.StandardTableDefinition.of(schema_<%=cid%>)).build();
table_<%=cid%> = bigquery_<%=cid%>.create(tableInfo_<%=cid%>);
loadJobBuilder_<%=cid%>.setSchema(schema_<%=cid%>);
}
@@ -582,7 +583,7 @@
}
%>
loadJobBuilder_<%=cid%>.setFormatOptions(csvOptions_<%=cid%>.build());
loadJobBuilder_<%=cid%>.setNullMarker("\\N");
loadJobBuilder_<%=cid%>.setNullMarker(<%= nullMarker %>);
com.google.cloud.bigquery.Job job_<%=cid%> = bigquery_<%=cid%>.create(com.google.cloud.bigquery.JobInfo.of(loadJobBuilder_<%=cid%>.build()));
job_<%=cid%> = job_<%=cid%>.waitFor(com.google.cloud.RetryOption.initialRetryDelay(org.threeten.bp.Duration.ofSeconds(1)));
if (job_<%=cid%> != null && job_<%=cid%>.getStatus().getError() == null) {

View File

@@ -173,10 +173,16 @@
SHOW_IF="(SET_FIELD_DELIMITER == 'true')">
<DEFAULT>","</DEFAULT>
</PARAMETER>
<PARAMETER NAME="DROP" FIELD="CHECK" REQUIRED="true" NUM_ROW="25" SHOW_IF="AUTH_MODE == 'SERVICEACCOUNT'">
<PARAMETER NAME="USE_CUSTOM_NULL_MARKER" FIELD="CHECK" NUM_ROW="25">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CUSTOM_NULL_MARKER" FIELD="TEXT" NUM_ROW="25" SHOW_IF="USE_CUSTOM_NULL_MARKER=='true'">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="DROP" FIELD="CHECK" REQUIRED="true" NUM_ROW="30" SHOW_IF="AUTH_MODE == 'SERVICEACCOUNT'">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="ENCODING" FIELD="ENCODING_TYPE" NUM_ROW="30"
<PARAMETER NAME="ENCODING" FIELD="ENCODING_TYPE" NUM_ROW="35"
REQUIRED="true" REPOSITORY_VALUE="ENCODING">
<DEFAULT>"ISO-8859-15"</DEFAULT>
</PARAMETER>

View File

@@ -40,6 +40,8 @@ ENCODING.NAME=Encoding
TOKEN_NAME.NAME=token properties File Name
SET_FIELD_DELIMITER.NAME=Set the field delimiter
FIELD_DELIMITER.NAME=
USE_CUSTOM_NULL_MARKER.NAME=Use custom null marker
CUSTOM_NULL_MARKER.NAME=
DROP.NAME=Drop table if exists
DIE_ON_ERROR.NAME=Die on error

View File

@@ -55,7 +55,12 @@
<PARAMETER NAME="TEXT_ENCLOSURE" FIELD="TEXT" REQUIRED="true" NUM_ROW="30" MAX_LENGTH="1" REPOSITORY_VALUE="TEXT_ENCLOSURE" SHOW="false">
<DEFAULT>"\""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="USE_CUSTOM_NULL_MARKER" FIELD="CHECK" NUM_ROW="35">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CUSTOM_NULL_MARKER" FIELD="TEXT" NUM_ROW="35" SHOW_IF="USE_CUSTOM_NULL_MARKER=='true'">
<DEFAULT>""</DEFAULT>
</PARAMETER>
<PARAMETER NAME="CREATE" FIELD="CHECK" REQUIRED="true" NUM_ROW="40">
<DEFAULT>true</DEFAULT>
</PARAMETER>

View File

@@ -25,7 +25,8 @@
boolean checkDiskSpace = ("true").equals(ElementParameterParser.getValue(node,"__CHECK_DISK_SPACE__"));
boolean flushOnRow = ("true").equals(ElementParameterParser.getValue(node, "__FLUSHONROW__"));
String flushMod = ElementParameterParser.getValue(node, "__FLUSHONROW_NUM__");
boolean useCustomNullMarker = ElementParameterParser.getBooleanValue(node, "__USE_CUSTOM_NULL_MARKER__");
String nullMarker = useCustomNullMarker ? ElementParameterParser.getValue(node, "__CUSTOM_NULL_MARKER__") : "\"\\\\N\"";
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
List< ? extends IConnection> conns = node.getIncomingConnections();
@@ -35,8 +36,8 @@
if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)) {
List<IMetadataColumn> columns = metadata.getListColumns();
int sizeColumns = columns.size();
%>
String[] row_<%=cid%>=new String[]{<%for(int j=0;j<sizeColumns;j++){%>"\\N",<%}%>}; //empty value must be NULL('\N' in bulk file)
%>
String[] row_<%=cid%>=new String[]{<%for(int j=0;j<sizeColumns;j++){%> <%= nullMarker %>,<%}%>}; //empty value must be defined by null marker('\N' is default in bulk file)
<%
for (int i = 0; i < sizeColumns; i++) {
IMetadataColumn column = columns.get(i);
@@ -64,24 +65,10 @@
%>
row_<%=cid%>[<%=i%>] = <%=conn.getName() %>.<%=column.getLabel() %>;
<%
}else if(javaType == JavaTypesManager.DATE && pattern == null){
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd");
<%
}else if(javaType == JavaTypesManager.DATE && pattern != null){
if(pattern.length() > 12){
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd HH:mm:ss");
<%
}else if(pattern.length() == 12 || "\"\"".equals(pattern)) {
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, "yyyy-MM-dd");
<%
}else {
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, <%=pattern%>);
<%
}
}else if(javaType == JavaTypesManager.DATE){
%>
row_<%=cid%>[<%=i%>] = FormatterUtils.format_Date(<%=conn.getName() %>.<%=column.getLabel() %>, <%=(pattern != null && !("\"\"").equals(pattern)) ? pattern : "\"yyyy-MM-dd\""%>);
<%
}else if(javaType == JavaTypesManager.BYTE_ARRAY){
%>
row_<%=cid%>[<%=i%>] = java.nio.charset.Charset.forName(<%=encoding %>).decode(java.nio.ByteBuffer.wrap(<%=conn.getName() %>.<%=column.getLabel() %>)).toString();
@@ -100,7 +87,7 @@
}
%>
} else {
row_<%=cid%>[<%=i%>] = "\\N";
row_<%=cid%>[<%=i%>] = <%= nullMarker %>;
}
<%
}

View File

@@ -24,4 +24,5 @@ RECORDS_CONTAIN_NULL_VALUE.NAME=Records contain NULL value
PROPERTY.NAME=Property Type
CHECK_DISK_SPACE.NAME=Check disk space
USE_CUSTOM_NULL_MARKER.NAME=Use custom null marker
CUSTOM_NULL_MARKER.NAME=

View File

@@ -17,40 +17,40 @@ imports="
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
String useExistingConn = ElementParameterParser.getValue(node,"__USE_EXISTING_CONNECTION__");
String accessToken = ElementParameterParser.getValue(node, "__ACCESS_TOKEN__");
String refreshToken = ElementParameterParser.getValue(node, "__REFRESH_TOKEN__");
String clientId = ElementParameterParser.getValue(node, "__CLIENT_ID__");
String clientSecret = ElementParameterParser.getValue(node, "__CLIENT_SECRET__");
String connection = ElementParameterParser.getValue(node,"__CONNECTION__");
String connectionKey = "\"conn_" + connection+"\"";
String path = ElementParameterParser.getValue(node, "__PATH__");
String filelistType = ElementParameterParser.getValue(node, "__LIST_MODE__");
boolean includeSubDirectories = "true".equals(ElementParameterParser.getValue(node, "__INCLUDSUBDIR__"));
final boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
log4jFileUtil.componentStartInfo(node);
List<IMetadataTable> metadatas = node.getMetadataList();
IMetadataTable metadata = null;
if ((metadatas!=null)&&(metadatas.size()>0)) {
metadata = metadatas.get(0);
metadata = metadatas.get(0);
}
List< ? extends IConnection> outputConnections = node.getOutgoingSortedConnections();
String dataOutputConnection = null;
String dataOutputConnection = null;
for(IConnection conn : outputConnections) {
if(conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)) {
dataOutputConnection = conn.getName();
break;
} // if(conn) end
} // for(conns) end
%>
<%
if(!"true".equals(useExistingConn)){
@@ -105,115 +105,81 @@ if(!"true".equals(useExistingConn)){
<%=cid%>_client.authenticate(new com.box.boxjavalibv2.dao.BoxOAuthToken(<%=cid%>_map));
}
<%
} else {
} else {
%>
com.box.boxjavalibv2.BoxClient <%=cid%>_client = (com.box.boxjavalibv2.BoxClient)globalMap.get(<%=connectionKey%>);
<%
}
%>
String <%=cid%>_path = normalizePath(<%=path%>);
boolean <%=cid%>_hasError = false;
String <%=cid%>_name = getFolderFromPath(<%=cid%>_path);
com.box.boxjavalibv2.dao.BoxFile <%=cid%>_boxFile = null;
com.box.boxjavalibv2.dao.BoxFolder <%=cid%>_boxFolder = null;
if ((<%=cid%>_name.equals("All Files") && (<%=cid%>_path.isEmpty() || <%=cid%>_path == null)))
{
<%=cid%>_boxFolder = <%=cid%>_client.getFoldersManager().getFolder("0",null);
}
else
{
com.box.restclientv2.requestsbase.BoxDefaultRequestObject <%=cid%>_requestObject = new com.box.restclientv2.requestsbase.BoxDefaultRequestObject();
if (<%=cid%>_name.equals("All Files") || <%=cid%>_path.equals("All Files"))
{
<%=cid%>_boxFolder = <%=cid%>_client.getFoldersManager().getFolder("0", null);
}
else
{
String parentPath = normalizePath(<%=cid%>_path.replace(<%=cid%>_name, ""));
<%=cid%>_boxFolder = <%=cid%>_client.getFoldersManager().getFolder("0", <%=cid%>_requestObject);
if ("All Files".equals(parentPath)) {
for (com.box.boxjavalibv2.dao.BoxTypedObject <%=cid%>_boxTypedObject : <%=cid%>_boxFolder.getItemCollection().getEntries()) {
if (<%=cid%>_boxTypedObject.getValue("name").equals(<%=cid%>_name)) {
if ("folder".equals(<%=cid%>_boxTypedObject.getType())) {
<%=cid%>_boxFolder = <%=cid%>_client.getFoldersManager().getFolder(<%=cid%>_boxTypedObject.getId(), <%=cid%>_requestObject);
} else {
<%=cid%>_boxFile = <%=cid%>_client.getFilesManager().getFile(<%=cid%>_boxTypedObject.getId(),<%=cid%>_requestObject);
}
}
}
com.box.boxjavalibv2.dao.BoxFolder <%=cid%>_rootFolder = <%=cid%>_client.getFoldersManager().getFolder("0", null);
com.box.boxjavalibv2.dao.BoxItem <%=cid%>_boxItem;
} else {
List<String> <%=cid%>_paths = new java.util.ArrayList<String>(java.util.Arrays.asList(parentPath.split("/")));
<%=cid%>_boxFolder = getBoxFolderRecursively(<%=cid%>_paths, <%=cid%>_boxFolder, <%=cid%>_client);
}
}
}
if (<%=cid%>_boxFile == null && <%=cid%>_boxFolder == null)
{
throw new Exception("<%=cid%> - " + "No file or directory found in " + <%=path%>);
}
else if (<%=cid%>_boxFolder != null)
{
java.util.List<com.box.boxjavalibv2.dao.BoxItem> <%=cid%>_children = new java.util.ArrayList<com.box.boxjavalibv2.dao.BoxItem>();
<%if(includeSubDirectories){%>
<%if(filelistType.equalsIgnoreCase("Directories")){%>
<%=cid%>_children = getChildrenRecursively(<%=cid%>_client, <%=cid%>_boxFolder, <%=cid%>_children, true, false);
<%} else if (filelistType.equalsIgnoreCase("Files")){%>
<%=cid%>_children = getChildrenRecursively(<%=cid%>_client, <%=cid%>_boxFolder, <%=cid%>_children, false, true);
<%} else {%>
<%=cid%>_children = getChildrenRecursively(<%=cid%>_client, <%=cid%>_boxFolder, <%=cid%>_children, true, true);
<%}%>
<%}%>
<%else{%>
for (com.box.boxjavalibv2.dao.BoxTypedObject obj : <%=cid%>_boxFolder.getItemCollection().getEntries())
{
<%if(filelistType.equalsIgnoreCase("Directories")){%>
if(obj.getType().equals("folder"))
{
com.box.boxjavalibv2.dao.BoxFolder folder = <%=cid%>_client.getFoldersManager().getFolder(obj.getId(), null);
<%=cid%>_children.add(folder);
}
<%} else if (filelistType.equalsIgnoreCase("Files")){%>
if (obj.getType().equals("file"))
{
com.box.boxjavalibv2.dao.BoxFile file = <%=cid%>_client.getFilesManager().getFile(obj.getId(), null);
<%=cid%>_children.add(file);
}
<%} else {%>
if(obj.getType().equals("folder"))
{
com.box.boxjavalibv2.dao.BoxFolder folder = <%=cid%>_client.getFoldersManager().getFolder(obj.getId(), null);
<%=cid%>_children.add(folder);
}
else if (obj.getType().equals("file"))
{
com.box.boxjavalibv2.dao.BoxFile file = <%=cid%>_client.getFilesManager().getFile(obj.getId(), null);
<%=cid%>_children.add(file);
}
<%}%>
int <%=cid%>_slashPosition = <%=cid%>_path.indexOf("/");
String <%=cid%>_rootFolderName = <%=cid%>_slashPosition == -1 ? <%=cid%>_path : <%=cid%>_path.substring(0, <%=cid%>_slashPosition);
if (<%=cid%>_rootFolderName.equals("All Files")) {
List<String> <%=cid%>_pathItems = new java.util.ArrayList<String>(java.util.Arrays.asList(<%=cid%>_path.split("/")));
<%=cid%>_pathItems.remove(0);
try {
<%=cid%>_boxItem = getBoxItemRecursively(<%=cid%>_pathItems, <%=cid%>_rootFolder, <%=cid%>_client);
} catch (Exception e) {
throw new Exception("<%=cid%> - " + "The path to file or directory was not found in " + <%=cid%>_path, e);
}
<%}%>
for (com.box.boxjavalibv2.dao.BoxItem <%=cid%>_item : <%=cid%>_children)
{
String <%=cid%>_itemPath = getBoxItemPath(<%=cid%>_item);
globalMap.put("<%=cid%>_NAME", <%=cid%>_item.getName());
globalMap.put("<%=cid%>_ID", <%=cid%>_item.getId());
globalMap.put("<%=cid%>_FILE_PATH", <%=cid%>_itemPath);
globalMap.put("<%=cid%>_FILE_DIRECTORY", <%=cid%>_itemPath + <%=cid%>_item.getName());
globalMap.put("<%=cid%>_TYPE",<%=cid%>_item.getType());
globalMap.put("<%=cid%>_LAST_MODIFIED", <%=cid%>_item.getModifiedAt());
globalMap.put("<%=cid%>_SIZE", <%=cid%>_item.getSize());
} else {
throw new Exception("<%=cid%> - " + "The path to file or directory was not found in " + <%=cid%>_path);
}
java.util.List<com.box.boxjavalibv2.dao.BoxItem> <%=cid%>_items = new java.util.ArrayList<>();
if (<%=cid%>_boxItem.getType().equals("folder")) {
com.box.boxjavalibv2.dao.BoxFolder <%=cid%>_boxFolder = (com.box.boxjavalibv2.dao.BoxFolder) <%=cid%>_boxItem;
<%if(includeSubDirectories) {%>
<%if(filelistType.equalsIgnoreCase("Directories")) {%>
<%=cid%>_items = getChildrenRecursively(<%=cid%>_client, <%=cid%>_boxFolder, <%=cid%>_items, true, false);
<%} else if (filelistType.equalsIgnoreCase("Files")) {%>
<%=cid%>_items = getChildrenRecursively(<%=cid%>_client, <%=cid%>_boxFolder, <%=cid%>_items, false, true);
<%} else {%>
<%=cid%>_items = getChildrenRecursively(<%=cid%>_client, <%=cid%>_boxFolder, <%=cid%>_items, true, true);
<%}%>
<%} else {%>
for (com.box.boxjavalibv2.dao.BoxTypedObject obj : <%=cid%>_boxFolder.getItemCollection().getEntries())
{
<%if(filelistType.equalsIgnoreCase("Directories")) {%>
if(obj.getType().equals("folder"))
{
com.box.boxjavalibv2.dao.BoxFolder folder = <%=cid%>_client.getFoldersManager().getFolder(obj.getId(), null);
<%=cid%>_items.add(folder);
}
<%} else if (filelistType.equalsIgnoreCase("Files")) {%>
if (obj.getType().equals("file"))
{
com.box.boxjavalibv2.dao.BoxFile file = <%=cid%>_client.getFilesManager().getFile(obj.getId(), null);
<%=cid%>_items.add(file);
}
<%} else {%>
if(obj.getType().equals("folder"))
{
com.box.boxjavalibv2.dao.BoxFolder folder = <%=cid%>_client.getFoldersManager().getFolder(obj.getId(), null);
<%=cid%>_items.add(folder);
}
else if (obj.getType().equals("file"))
{
com.box.boxjavalibv2.dao.BoxFile file = <%=cid%>_client.getFilesManager().getFile(obj.getId(), null);
<%=cid%>_items.add(file);
}
<%}%>
}
<%}%>
} else if (<%=cid%>_boxItem.getType().equals("file")) {
com.box.boxjavalibv2.dao.BoxFile <%=cid%>_boxFile = (com.box.boxjavalibv2.dao.BoxFile) <%=cid%>_boxItem;
<%=cid%>_items.add(<%=cid%>_boxFile);
}
for (com.box.boxjavalibv2.dao.BoxItem <%=cid%>_item : <%=cid%>_items) {
String <%=cid%>_itemPath = getBoxItemPath(<%=cid%>_item);
globalMap.put("<%=cid%>_NAME", <%=cid%>_item.getName());
globalMap.put("<%=cid%>_ID", <%=cid%>_item.getId());
globalMap.put("<%=cid%>_FILE_PATH", <%=cid%>_itemPath);
globalMap.put("<%=cid%>_FILE_DIRECTORY", <%=cid%>_itemPath + <%=cid%>_item.getName());
globalMap.put("<%=cid%>_TYPE",<%=cid%>_item.getType());
globalMap.put("<%=cid%>_LAST_MODIFIED", <%=cid%>_item.getModifiedAt());
globalMap.put("<%=cid%>_SIZE", <%=cid%>_item.getSize());

View File

@@ -16,7 +16,6 @@
%>
}
}
//globalMap.put("<%=cid%>_NB_FILE", NB_FILE<%=cid%>);

View File

@@ -899,7 +899,7 @@
<IMPORT NAME="Driver-VERTICA_7.1.2" MODULE="vertica-jdbc-7.1.2-0.jar" MVN="mvn:org.talend.libraries/vertica-jdbc-7.1.2-0/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_7_1_X')" />
<IMPORT NAME="Driver-EXASolution" MODULE="exajdbc-6.0.9302.jar" MVN="mvn:org.talend.libraries/exajdbc-6.0.9302/6.3.0" REQUIRED_IF="(DBTYPE=='EXASOL') and (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-Redshift" MODULE="redshift-jdbc42-no-awssdk-1.2.32.1056.jar" MVN="mvn:com.amazon.redshift/redshift-jdbc42-no-awssdk/1.2.32.1056" REQUIRED_IF="(DBTYPE=='REDSHIFT') and (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-Snowflake" MODULE="snowflake-jdbc-3.5.3.jar" MVN="mvn:net.snowflake/snowflake-jdbc/3.5.3" REQUIRED_IF="(DBTYPE=='SNOWFLAKE') and (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-Snowflake" MODULE="snowflake-jdbc-3.11.1.jar" MVN="mvn:net.snowflake/snowflake-jdbc/3.11.1" REQUIRED_IF="(DBTYPE=='SNOWFLAKE') and (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Driver-VERTICA_9.0" MODULE="vertica-jdbc-9.0.0-0.jar" MVN="mvn:org.talend.libraries/vertica-jdbc-9.0.0-0/6.0.0" REQUIRED_IF="(DBTYPE=='VERTICA') and (USE_EXISTING_CONNECTION == 'false') and (DB_VERTICA_VERSION=='VERTICA_9_0')"/>
</IMPORTS>
</CODEGENERATION>

View File

@@ -25,7 +25,7 @@
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
org.talend.designer.dbmap.language.generation.GenericDbGenerationManager gm = new org.talend.designer.dbmap.language.generation.GenericDbGenerationManager();
org.talend.designer.dbmap.language.mssql.MssqlGenerationManager gm = new org.talend.designer.dbmap.language.mssql.MssqlGenerationManager();
String uniqueNameComponent = null;
IDbLanguage currentLanguage = gm.getLanguage();

View File

@@ -107,7 +107,7 @@ imports="
}
else{
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -149,7 +149,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talen File Enhanced"
MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar" REQUIRED="true" />
MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -17,20 +17,16 @@
if (connObj != null) {
try {
<% if (sftp == false && ftps == false) { %>
com.enterprisedt.net.ftp.FTPClient conn = (com.enterprisedt.net.ftp.FTPClient) connObj;
conn.executeCommand("quit");
<% } else if (sftp) { %>
<% if (sftp) { %>
com.jcraft.jsch.ChannelSftp channel = (com.jcraft.jsch.ChannelSftp) connObj;
com.jcraft.jsch.Session session = channel.getSession();
channel.disconnect();
session.disconnect();
<% } else if (ftps) { %>
org.apache.commons.net.ftp.FTPSClient conn = (org.apache.commons.net.ftp.FTPSClient) connObj;
conn.disconnect();
<% } else { %>
System.err.println("Invalid tFTPConnection configuration.");
<% } %>
org.apache.commons.net.ftp.FTPClient conn = (org.apache.commons.net.ftp.FTPClient) connObj;
conn.disconnect();
<% }
%>
} catch (Exception e) {
globalMap.put("<%=cid%>_ERROR_MESSAGE", e.getMessage());

View File

@@ -15,16 +15,13 @@
%>
Object connObj = globalMap.get("conn_<%=connection%>");
if (connObj != null) {
<% if (sftp == false && ftps == false) { %>
com.enterprisedt.net.ftp.FTPClient conn = (com.enterprisedt.net.ftp.FTPClient) connObj;
conn.executeCommand("quit");
<% } else if (sftp) { %>
<% if (sftp) { %>
com.jcraft.jsch.ChannelSftp channel = (com.jcraft.jsch.ChannelSftp) connObj;
com.jcraft.jsch.Session session = channel.getSession();
channel.disconnect();
session.disconnect();
<% } else if (ftps) { %>
org.apache.commons.net.ftp.FTPSClient conn = (org.apache.commons.net.ftp.FTPSClient) connObj;
<% } else { %>
org.apache.commons.net.ftp.FTPClient conn = (org.apache.commons.net.ftp.FTPClient) connObj;
conn.disconnect();
<% } %>

View File

@@ -1,88 +0,0 @@
<%@ jet
%>
boolean retry_<%=cid%> = false;
int retry_count_<%=cid%> = 0;
int retry_max_<%=cid%> = 5;
com.jcraft.jsch.Session session_<%=cid%> = null;
com.jcraft.jsch.Channel channel_<%=cid%> = null;
do {
retry_<%=cid%> = false;
com.jcraft.jsch.JSch jsch_<%=cid%> = new com.jcraft.jsch.JSch();
<%if (("PUBLICKEY").equals(authMethod)){%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - SFTP authentication using a public key.");
log.debug("<%=cid%> - Private key: '" + <%=privateKey%> + "'.");
<%}%>
jsch_<%=cid%>.addIdentity(<%=privateKey %>, defaultUserInfo_<%=cid%>.getPassphrase());
<%}%>
session_<%=cid%> = jsch_<%=cid%>.getSession(<%=user%>, <%=host%>, <%=port%>);
session_<%=cid%>.setConfig("PreferredAuthentications", "publickey,password,keyboard-interactive,gssapi-with-mic");
<%if (("PASSWORD").equals(authMethod)) {%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - SFTP authentication using a password.");
<%}%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
session_<%=cid%>.setPassword(decryptedPassword_<%=cid%>);
<%}%>
session_<%=cid%>.setUserInfo(defaultUserInfo_<%=cid%>);
<%if (!useProxy) {%>
if(("true").equals(System.getProperty("http.proxySet")) ){
com.jcraft.jsch.ProxyHTTP proxy_<%=cid%> = new com.jcraft.jsch.ProxyHTTP(System.getProperty("http.proxyHost"),Integer.parseInt(System.getProperty("http.proxyPort")));
if(!"".equals(System.getProperty("http.proxyUser"))){
proxy_<%=cid%>.setUserPasswd(System.getProperty("http.proxyUser"),System.getProperty("http.proxyPassword"));
}
session_<%=cid%>.setProxy(proxy_<%=cid%>);
}
<%}%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user%> + "'.");
<%}%>
channel_<%=cid%> = null;
try {
session_<%=cid%>.connect();
channel_<%=cid%> = session_<%=cid%>.openChannel("sftp");
channel_<%=cid%>.connect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
} catch (com.jcraft.jsch.JSchException e_<%=cid%>) {
try {
if(channel_<%=cid%>!=null) {
channel_<%=cid%>.disconnect();
}
if(session_<%=cid%>!=null) {
session_<%=cid%>.disconnect();
}
} catch(java.lang.Exception ce_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - close sftp connection failed : " + ce_<%=cid%>.getClass() + " : " + ce_<%=cid%>.getMessage());
<%}%>
}
String message_<%=cid%> = new TalendException(null, null, null).getExceptionCauseMessage(e_<%=cid%>);
if(message_<%=cid%>.contains("Signature length not correct") || message_<%=cid%>.contains("connection is closed by foreign host")) {
retry_<%=cid%> = true;
retry_count_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - connect: Signature length not correct or connection is closed by foreign host, so retry, retry time : " + retry_count_<%=cid%>);
<%}%>
} else {
throw e_<%=cid%>;
}
}
} while(retry_<%=cid%> && (retry_count_<%=cid%> < retry_max_<%=cid%>));
com.jcraft.jsch.ChannelSftp c_<%=cid%> = (com.jcraft.jsch.ChannelSftp)channel_<%=cid%>;

View File

@@ -28,7 +28,7 @@ String authMethod = ElementParameterParser.getValue(node, "__AUTH_METHOD__");
String privateKey = ElementParameterParser.getValue(node, "__PRIVATEKEY__");
boolean ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String sEncoding = ElementParameterParser.getValue(node, "__ENCODING__");
String encoding = ElementParameterParser.getValue(node, "__ENCODING__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
String protectionLevel = ElementParameterParser.getValue(node, "__FTPS_PROT__");
String protectionBufferSize = ElementParameterParser.getValue(node, "__FTPS_PROTECTION_BUFF_SIZE__");
@@ -60,42 +60,22 @@ if (ElementParameterParser.canEncrypt(node, passwordFieldName)) {
});
<%
}
%>
<%
if (!sftp && !ftps) { // *** ftp *** //
if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
%>
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
passwordFieldName = "__PASS__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=sEncoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASS__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
globalMap.put("conn_<%=cid%>",ftp_<%=cid %>);
globalMap.put("conn_<%=cid%>",ftp_<%=cid %>);
<%
} else if (!ftps) { // *** sftp *** //
%>
class MyUserInfo implements com.jcraft.jsch.UserInfo, com.jcraft.jsch.UIKeyboardInteractive {
<%
passwordFieldName = "__PASSPHRASE__";
if (ElementParameterParser.canEncrypt(node, passwordFieldName)) {
@@ -138,99 +118,20 @@ if (!sftp && !ftps) { // *** ftp *** //
passwordFieldName = "__PASS__";
%>
<%@ include file="sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=sEncoding%>);
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
<%}%>
globalMap.put("conn_<%=cid%>", c_<%=cid%>);
<%
} else { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
MyTrust_<%=cid%> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=sEncoding %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%
passwordFieldName = "__PASS__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
%>
globalMap.put("conn_<%=cid%>", ftp_<%=cid %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
globalMap.put("conn_<%=cid%>", ftp_<%=cid %>);
<%
}
%>

View File

@@ -185,7 +185,6 @@
REQUIRED="true" NOT_SHOW_IF="(SFTP == 'true') and (USE_ENCODING == 'false')">
<DEFAULT>"ISO-8859-15"</DEFAULT>
</PARAMETER>
</PARAMETERS>
<ADVANCED_PARAMETERS>
@@ -236,10 +235,10 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar" MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819" REQUIRED_IF="(FTPS == 'true')" />
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar" MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819" REQUIRED_IF="(FTPS == 'true')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -55,6 +55,8 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -129,7 +131,7 @@ int nb_file_<%=cid%> = 0;
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -166,191 +168,70 @@ int nb_file_<%=cid%> = 0;
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
} else if (ftps) { // *** ftps *** //
} else {
if (ftps) { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
if (moveToCurrentDir) {
%>
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(<%=remotedir %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
<%
}
}
%>
<% /*store real pwd*/ %>
String rootWorkDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<>();
<%
for (int i=0; i<files.size(); i++) {
Map<String, String> line = files.get(i);
} else {// *** ftp *** //
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,"");
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
} else {// *** ftp *** //
%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> =null;
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> =null;
<%if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ",Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username: '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,"");
list<%=cid %>.add(map<%=cid %><%=i %>);
<%}%>
String remotedir<%=cid %> = <%=remotedir%>;
ftp_<%=cid %>.chdir(remotedir<%=cid %>);
String root<%=cid %> = ftp_<%=cid %>.pwd();
<%} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%}%>
<%
}//common code ftp and ftps:
%>
<%
if (("true").equals(useExistingConn) && moveToCurrentDir) {
%>
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(<%=remotedir %>);
<%if(isLog4jEnabled){%>
<%if("FILE".equals(targetType)){%>
log.info("<%=cid%> - Deleting file from server.");
<%}else{%>
log.info("<%=cid%> - Deleting directory from server.");
<%}%>
<%}%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%}%>
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
<%
}
%>
<% /*store real pwd*/ %>
String rootWorkDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<>();
<%
for (int i=0; i<files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,"");
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
}
%>

View File

@@ -70,14 +70,14 @@
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
}catch(java.net.SocketException se_<%=cid%>){
}catch(java.io.IOException ioe_<%=cid%>){
//ignore failure
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
}
<%
@@ -86,18 +86,12 @@
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
<%
}
}else{
if(!moveToCurrentDir){
%>
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
<%
}
}
}
%>

View File

@@ -177,7 +177,7 @@
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="31" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="31" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -292,13 +292,13 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -95,7 +95,7 @@
<%
}else if (ftps) { // *** ftps *** //
}else { // *** ftp, ftps *** //
%>
for (String key<%=cid %> : keySet<%=cid %>) {
<% /*prepare filemask*/ %>
@@ -241,73 +241,6 @@
<% /*return to root dir for next filemask*/ %>
ftp_<%=cid %>.changeWorkingDirectory(rootWorkDir_<%=cid %>);
<%
}else {// *** ftp *** //
%>
try {
for (String key<%=cid %> : keySet<%=cid %>) {
String filemask<%=cid %> = key<%=cid %>;
String dir<%=cid %> = null;
<%if(!usePerl5) {//perl5 mode not support windows(\) path separator at the mask string%>
String mask<%=cid %> = filemask<%=cid %>.replaceAll("\\\\", "/") ;
<%}else{%>
String mask<%=cid %> = filemask<%=cid %>;
<%}%>
int i<%=cid %> = mask<%=cid %>.lastIndexOf('/');
if (i<%=cid %>!=-1) {
dir<%=cid %> = mask<%=cid %>.substring(0, i<%=cid %>);
mask<%=cid %> = mask<%=cid %>.substring(i<%=cid %>+1);
}
if (dir<%=cid %>!=null && !"".equals(dir<%=cid %>)) ftp_<%=cid %>.chdir(dir<%=cid %>);
<%if(!usePerl5) {%>
mask<%=cid %> = org.apache.oro.text.GlobCompiler.globToPerl5(mask<%=cid %>.toCharArray(), org.apache.oro.text.GlobCompiler.DEFAULT_MASK);
<%}%>
String[] fileName_<%=cid%> = null;
boolean dirFailed_<%=cid%> = false;
try{
fileName_<%=cid%> = ftp_<%=cid%>.dir(null,false);
}catch(com.enterprisedt.net.ftp.FTPException e_<%=cid%>){
dirFailed_<%=cid%> = true;
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + e_<%=cid%>.getMessage());
<%}%>
}
if((fileName_<%=cid%> == null) || dirFailed_<%=cid%> || (fileName_<%=cid%>.length <= 0)){
com.enterprisedt.net.ftp.FTPFile[] fileList_<%=cid%> = ftp_<%=cid%>.dirDetails(null);
fileName_<%=cid%> = new String[fileList_<%=cid%>.length];
for (int i_<%=cid%> = 0 ; i_<%=cid%> < fileList_<%=cid%>.length ; i_<%=cid%>++){
fileName_<%=cid%>[i_<%=cid%>] = fileList_<%=cid%>[i_<%=cid%>].getName();
}
}
for (String file_<%=cid%> : fileName_<%=cid%>) {
if (file_<%=cid%>.matches(mask<%=cid %>)) {
<%
if("FILE".equals(targetType)){
%>
ftp_<%=cid %>.delete(file_<%=cid%>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - File '" + file_<%=cid%> + "' was deleted from the remote directory " + <%=remotedir %> + ".");
<%}%>
<%
}else{
%>
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Directory '" + file_<%=cid%> + "' was deleted from the remote directory " + <%=remotedir %> + ".");
<%}%>
ftp_<%=cid %>.rmdir(file_<%=cid%>);
<%
}
%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File deleted.");
nb_file_<%=cid%>++;
}
}
}
}catch(com.enterprisedt.net.ftp.FTPException e_<%=cid %>){
globalMap.put("<%=cid %>_CURRENT_STATUS", "Deleting file action error");
throw e_<%=cid %>;
}
ftp_<%=cid %>.chdir(root<%=cid %>);
<%
}
%>

View File

@@ -57,6 +57,8 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
String passwordFieldName = "";
@@ -134,7 +136,7 @@ if (sftp) { // *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -153,134 +155,38 @@ if (sftp) { // *** sftp *** //
}
<%
}
} else if (ftps) { // *** ftps *** //
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
} else { // *** ftp+s *** //
if (ftps) {
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
passwordFieldName = "__PASSWORD__";
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
} else {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
}
} else { // *** ftp *** //
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
}
if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {
passwordFieldName = "__PASSWORD__";
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
} else { // *** ftp *** //
%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> = null;
<% if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username: '" +<%=username %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=username %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host%> + "' has succeeded.");
<%}%>
<%
}
}
}%>

View File

@@ -56,41 +56,35 @@ if (sftp) { // *** sftp *** //
<%
}
} else { // *** ftp *** //
if (!("true").equals(useExistingConn)) {
if (("true").equals(ignoreFailureAtQuit)) {
%>
try {
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
} catch(java.net.SocketException se_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
<%
} else {
%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
<%
}
} else {
if (moveToCurrentDir) {
if (!("true").equals(useExistingConn)) {
if (("true").equals(ignoreFailureAtQuit)) {
%>
ftp_<%=cid %>.chdir(<%=remoteDir %>);
try {
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
} catch(java.io.IOException ioe_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
<%
} else {
%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
<%
}
}
}
}
}
%>

View File

@@ -170,7 +170,7 @@ COMPATIBILITY="ALL"
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="61" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="61" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -280,12 +280,12 @@ COMPATIBILITY="ALL"
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -70,7 +70,7 @@ if (sftp) { // *** sftp *** //
}
globalMap.put("<%=cid %>_DIRECTORYNAME", <%=directoryname %>);
<%}
} else if (ftps) { // *** ftps *** //
} else { // *** ftp+s *** //
%>
//change working dir and save root
String remoteDir_<%=cid %> = (<%=remoteDir %>).replaceAll("\\\\","/");
@@ -155,35 +155,5 @@ if (sftp) { // *** sftp *** //
ftp_<%=cid %>.changeWorkingDirectory(remoteDir_<%=cid %>);
<%
}
} else { // *** ftp *** //
if (isFileType){%>
String remoteDir_<%=cid %> = (<%=remoteDir %>).replaceAll("\\\\","/");
if (ftp_<%=cid %>.exists<%="File" %>(remoteDir_<%=cid %> + "/" + <%=filename %>)) {
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + <%=filename%> + "' exists in the remote directory '"+ <%=remoteDir %> + "' .");
<%}%>
globalMap.put("<%=cid %>_EXISTS", true);
} else {
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + <%=filename%> + "' doesn't exist in the remote directory '"+ <%=remoteDir %> + "' .");
<%}%>
globalMap.put("<%=cid %>_EXISTS", false);
}
globalMap.put("<%=cid %>_FILENAME", <%=filename %>);
<%} else {%>
String remoteDir_<%=cid %> = (<%=remoteDir %>).replaceAll("\\\\","/");
String directoryname_<%=cid %> = (<%=directoryname %>).replaceAll("\\\\","/");
if (ftp_<%=cid %>.exists<%="Directory" %>(remoteDir_<%=cid %> + "/" + directoryname_<%=cid %>)) {
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + <%=directoryname %> + "' exists in the remote directory '"+ <%=remoteDir %> + "' .");
<%}%>
globalMap.put("<%=cid %>_EXISTS", true);
} else {
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + <%=directoryname %> + "' doesn't exist in the remote directory '"+ <%=remoteDir %> + "' .");
<%}%>
globalMap.put("<%=cid %>_EXISTS", false);
}
globalMap.put("<%=cid %>_DIRECTORYNAME", <%=directoryname %>);
<%}
}%>
}
%>

View File

@@ -56,6 +56,8 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
String passwordFieldName = "";
@@ -151,7 +153,7 @@ if (sftp) {// *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -232,260 +234,104 @@ if (sftp) {// *** sftp *** //
globalMap.put("<%=cid %>_CURRENT_FILEPATH", currentFilePath_<%=cid%>);
nb_file_<%=cid%>++;
<%
} else if (ftps) { // *** ftps *** //
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
} else {
if (ftps) { // *** ftps *** //
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if(isLog4jEnabled){
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
}
passwordFieldName = "__PASSWORD__";
} else {
%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
}
} else { // *** ftp
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
} else {
passwordFieldName = "__PASSWORD__";
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
String rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
<%
}
%>
int nb_file_<%=cid%> = 0;
org.apache.commons.net.ftp.FTPFile[] ftpFiles_<%=cid %> = null;
List<org.apache.commons.net.ftp.FTPFile> fileListTemp_<%=cid %> = new java.util.ArrayList<>();
<%/*read files*/ %>
String remotedir_<%=cid %> = (<%=remotedir%>).replaceAll("\\\\","/");
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
ftpFiles_<%=cid %> = ftp_<%=cid %>.listFiles();
<%
if ("true".equals(useExistingConn) && !moveToCurrentDir) {
%>
<%/*return to previous dir */%>
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
<%
}
%>
<%/*prepare masks*/ %>
for (String filemask_<%=cid %> : maskList_<%=cid %>) {
java.util.regex.Pattern fileNamePattern_<%=cid %> = java.util.regex.Pattern.compile(filemask_<%=cid %>.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*"));
for (org.apache.commons.net.ftp.FTPFile ftpFile_<%=cid %> : ftpFiles_<%=cid %>) {
if (fileNamePattern_<%=cid %>.matcher(ftpFile_<%=cid %>.getName()).matches()) {
fileListTemp_<%=cid %>.add(ftpFile_<%=cid %>);
}
}
}
String currentFilePath_<%=cid%> = remotedir_<%=cid %>;
if(!remotedir_<%=cid%>.endsWith("/")&&!remotedir_<%=cid%>.endsWith("\\")){
currentFilePath_<%=cid%> += "/";
}
for (int i = 0; i < fileListTemp_<%=cid %>.size(); i++) {
org.apache.commons.net.ftp.FTPFile ftpFile_<%=cid %> = fileListTemp_<%=cid %>.get(i);
String currentFileName_<%=cid%> = null;
<%
if (dirFull) {
%>
currentFileName_<%=cid%> = ftpFile_<%=cid %>.toString();
<%
} else {
%>
currentFileName_<%=cid%> = ftpFile_<%=cid %>.getName();
<%
}
if(isLog4jEnabled) {
%>
log.debug("<%=cid%> - List file : '" + currentFilePath_<%=cid%> + "' .");
<%
}
}
} //common code for ftp and ftps:
%>
<%/*set to globalMap*/ %>
globalMap.put("<%=cid %>_CURRENT_FILE", currentFileName_<%=cid%>);
globalMap.put("<%=cid %>_CURRENT_FILEPATH", currentFilePath_<%=cid%> + ftpFile_<%=cid %>.getName());
nb_file_<%=cid %>++;
int nb_file_<%=cid%> = 0;
org.apache.commons.net.ftp.FTPFile[] ftpFiles_<%=cid %> = null;
String rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
List<org.apache.commons.net.ftp.FTPFile> fileListTemp_<%=cid %> = new java.util.ArrayList<>();
<%/*read files*/ %>
String remotedir_<%=cid %> = (<%=remotedir%>).replaceAll("\\\\","/");
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
ftpFiles_<%=cid %> = ftp_<%=cid %>.listFiles();
<%
} else { // *** ftp *** //
if ("true".equals(useExistingConn) && !moveToCurrentDir) {
%>
int nb_file_<%=cid%> = 0;
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> =null;
<%if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
String remotedir_<%=cid %> = (<%=remotedir%>).replaceAll("\\\\","/");
ftp_<%=cid %>.chdir(remotedir_<%=cid %>);
com.enterprisedt.net.ftp.FTPFile[] fileList_<%=cid %> = ftp_<%=cid %>.dirDetails(null);
String[] arrayWithNames_<%=cid %> = ftp_<%=cid %>.dir(null, false);
<%/*return to previous dir */%>
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
<%
if (("true").equals(useExistingConn)) {
if(!moveToCurrentDir){
}
%>
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
<%
}
}
<%/*prepare masks*/ %>
for (String filemask_<%=cid %> : maskList_<%=cid %>) {
java.util.regex.Pattern fileNamePattern_<%=cid %> = java.util.regex.Pattern.compile(filemask_<%=cid %>.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*"));
for (org.apache.commons.net.ftp.FTPFile ftpFile_<%=cid %> : ftpFiles_<%=cid %>) {
if (fileNamePattern_<%=cid %>.matcher(ftpFile_<%=cid %>.getName()).matches()) {
fileListTemp_<%=cid %>.add(ftpFile_<%=cid %>);
}
}
}
String currentFilePath_<%=cid%> = remotedir_<%=cid %>;
if(!remotedir_<%=cid%>.endsWith("/")&&!remotedir_<%=cid%>.endsWith("\\")){
currentFilePath_<%=cid%> += "/";
}
for (int i = 0; i < fileListTemp_<%=cid %>.size(); i++) {
org.apache.commons.net.ftp.FTPFile ftpFile_<%=cid %> = fileListTemp_<%=cid %>.get(i);
String currentFileName_<%=cid%> = null;
<%
if (dirFull) {
%>
List<String> fileListTemp_<%=cid %> = new java.util.ArrayList<String>();
List<String> fileListTempWithNames_<%=cid %> = new java.util.ArrayList<String>();
currentFileName_<%=cid%> = ftpFile_<%=cid %>.toString();
<%
} else {
%>
currentFileName_<%=cid%> = ftpFile_<%=cid %>.getName();
<%
}
for (String filemask_<%=cid %> : maskList_<%=cid %>) {
java.util.regex.Pattern fileNamePattern_<%=cid %> = java.util.regex.Pattern.compile(filemask_<%=cid %>.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*"));
java.util.Set<String> setWithNames_<%=cid %> = new java.util.HashSet<>(java.util.Arrays.asList(arrayWithNames_<%=cid %>));
for (com.enterprisedt.net.ftp.FTPFile ftpFile : fileList_<%=cid %>){
String filemaskTemp_<%=cid%> = "";
String filemaskTempName_<%=cid%> = "";
if (setWithNames_<%=cid %>.contains(ftpFile.getName())){
filemaskTemp_<%=cid%> = ftpFile.getRaw();
filemaskTempName_<%=cid%> = ftpFile.getName();
if (fileNamePattern_<%=cid %>.matcher(filemaskTempName_<%=cid %>).matches()) {
fileListTemp_<%=cid %>.add(filemaskTemp_<%=cid %>);
fileListTempWithNames_<%=cid%>.add(filemaskTempName_<%=cid%>);
}
}
}
}
int i_<%=cid %> = -1;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Listing files from server.");
<%}%>
while (++i_<%=cid %> < fileListTemp_<%=cid %>.size()) {
<% if (dirFull){%>
String currentFileName_<%=cid%> = fileListTemp_<%=cid %>.get(i_<%=cid %>);
<% } else {%>
String currentFileName_<%=cid%> = fileListTempWithNames_<%=cid %>.get(i_<%=cid %>);
<% }%>
String currentFilePath_<%=cid%> = remotedir_<%=cid %>;
if(!remotedir_<%=cid%>.endsWith("/")&&!remotedir_<%=cid%>.endsWith("\\")){
currentFilePath_<%=cid%> += "/";
}
currentFilePath_<%=cid%> += fileListTempWithNames_<%=cid%>.get(i_<%=cid%>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - List file : '" + currentFilePath_<%=cid%> + "' .");
<%}%>
globalMap.put("<%=cid %>_CURRENT_FILE", currentFileName_<%=cid%>);
globalMap.put("<%=cid %>_CURRENT_FILEPATH", currentFilePath_<%=cid%>);
nb_file_<%=cid%>++;
<%}%>
if(isLog4jEnabled) {
%>
log.debug("<%=cid%> - List file : '" + currentFilePath_<%=cid%> + "' .");
<%
}
%>
<%/*set to globalMap*/ %>
globalMap.put("<%=cid %>_CURRENT_FILE", currentFileName_<%=cid%>);
globalMap.put("<%=cid %>_CURRENT_FILEPATH", currentFilePath_<%=cid%> + ftpFile_<%=cid %>.getName());
nb_file_<%=cid %>++;
<%
} //end common code part
%>

View File

@@ -65,14 +65,14 @@ if (sftp && !("true").equals(useExistingConn)) {// *** sftp *** //
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
} catch (java.net.SocketException se_<%=cid%>) {
} catch (java.io.IOException ioe_<%=cid%>) {
//ignore failure
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
}
<%
@@ -81,7 +81,7 @@ if (sftp && !("true").equals(useExistingConn)) {// *** sftp *** //
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>

View File

@@ -155,7 +155,7 @@
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="41" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="41" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -258,12 +258,12 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -62,6 +62,8 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -153,9 +155,9 @@ if (sftp) { // *** sftp *** //
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
<%}%>
@@ -202,93 +204,52 @@ if (sftp) { // *** sftp *** //
<%}%>
}
<%
} else if (ftps) { // *** ftps *** //
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
} else {
if (ftps) { // *** ftps *** //
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
} else {
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
} else { // *** ftp *** //
%>
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
if (("true").equals(useExistingConn)) {
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
}
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%
passwordFieldName = "__PASSWORD__";
if("binary".equalsIgnoreCase(mode)){
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE); //to calculate md5 for binary files correctly
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
} else {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.ASCII_FILE_TYPE);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
}
} //common code for ftp and ftps:
%>
String remoteDir_<%=cid %> = (<%=remoteDir %>).replaceAll("\\\\", "/");
org.apache.commons.net.ftp.FTPFile[] allFiles_<%=cid %> = ftp_<%=cid %>.listFiles(remoteDir_<%=cid %>);
@@ -330,84 +291,5 @@ if (sftp) { // *** sftp *** //
<%}%>
}
<%
} else {// *** ftp *** //
%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> =null;
<%if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=username %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
<%
//add feature 19709,add the "Transfer mode" option,
//the default transfer mode is ASCII,we don't change the mode when ascii is choosed.
if("binary".equalsIgnoreCase(mode)){
%>
ftp_<%=cid %>.setType(com.enterprisedt.net.ftp.FTPTransferType.BINARY);
<%
}
%>
String remoteDir_<%=cid %> = <%=remoteDir %>.replaceAll("\\\\", "/");
<%=outputConnName %> = new <%=outputConnName %>Struct();
String remoteFile_<%=cid %> = <%=remoteDir %> + "/" + <%=filename%>;
if (ftp_<%=cid %>.exists(remoteFile_<%=cid %>)) {
<%=outputConnName %>.abs_path = (remoteFile_<%=cid %>).replaceAll("//", "/");
<%=outputConnName %>.dirname = <%=remoteDir %>;
<%=outputConnName %>.basename = <%=filename%>;
<%=outputConnName %>.size = ftp_<%=cid %>.size(remoteFile_<%=cid %>);
<%=outputConnName %>.mtime = ftp_<%=cid %>.modtime(remoteFile_<%=cid %>).getTime();
<%=outputConnName %>.mtime_string =ftp_<%=cid %>.modtime(remoteFile_<%=cid %>).toString();
<%if(isLog4jEnabled){%>
log4jSb_<%=cid%>.append("<%=cid%> - File properties for remoteDir is : '" + <%=remoteDir %> + "',file name is : '" + <%=filename%> +"',file size is : '" + <%=outputConnName %>.size + "',file last modify time is : '" + <%=outputConnName %>.mtime_string + "'");
<%}%>
<%if (MD5) {%>
// Calculation of the Message Digest MD5
java.security.MessageDigest dgs_<%=cid %> = java.security.MessageDigest.getInstance("MD5");
dgs_<%=cid %>.update(ftp_<%=cid %>.get(remoteFile_<%=cid %>));
<%=outputConnName %>.md5 =String.format("%032x", new java.math.BigInteger(1, dgs_<%=cid %>.digest()));
<%if(isLog4jEnabled){%>
log4jSb_<%=cid%>.append(",md5 message is : '"+ <%=outputConnName %>.md5 + "'.");
<%}%>
<%}%>
<%if(isLog4jEnabled){%>
log.debug(log4jSb_<%=cid%>);
<%}%>
}
<%
}
%>

View File

@@ -80,13 +80,13 @@ if (sftp && !bUseExistingConn) { // *** sftp *** //
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
} catch (java.net.SocketException se_<%=cid%>) {
} catch (java.io.IOException ioe_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
@@ -94,7 +94,7 @@ if (sftp && !bUseExistingConn) { // *** sftp *** //
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>

View File

@@ -255,12 +255,12 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS />

View File

@@ -66,6 +66,8 @@ if ("true".equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
%>
int nb_file_<%=cid%> = 0;
@@ -286,7 +288,7 @@ if (sftp) { // *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -303,477 +305,247 @@ if (sftp) { // *** sftp *** //
c_<%=cid%>.cd(remotedir_<%=cid %>);
}
<%
} else if (!ftps) { // *** ftp *** //
%>
final com.enterprisedt.net.ftp.TransferCompleteStrings msg_<%=cid%> = new com.enterprisedt.net.ftp.TransferCompleteStrings();
} else {
%>
abstract class FTPGetter_<%=cid%> {
protected org.apache.commons.net.ftp.FTPClient ftpClient = null;
protected int count = 0;
class FTPGetter_<%=cid%> {
private com.enterprisedt.net.ftp.FTPClient ftpClient = null;
private int count = 0;
public void getAllFiles(String remoteDirectory, String localDirectory)
throws IllegalStateException, IOException, java.io.FileNotFoundException {
public void getAllFiles(String remoteDirectory, String localDirectory) throws java.io.IOException,
com.enterprisedt.net.ftp.FTPException, java.text.ParseException {
chdir(remoteDirectory);
String path = ftpClient.printWorkingDirectory();
org.apache.commons.net.ftp.FTPFile[] ftpFiles = null;
ftpFiles = ftpClient.listFiles();
chdir(remoteDirectory);
String path = ftpClient.pwd();
String[] ftpFileNames = null;
com.enterprisedt.net.ftp.FTPFile[] ftpFiles = null;
for (org.apache.commons.net.ftp.FTPFile ftpFile : ftpFiles) {
try{
//use dir() for Bug9562 with FTP server in AS400
ftpFileNames = ftpClient.dir(null, false);
//Bug 13272, the same as getFiles().
} catch (com.enterprisedt.net.ftp.FTPException e){
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + e.getMessage());
<%}%>
ftpFileNames = null;
ftpFiles = ftpClient.dirDetails(".");
}
if (ftpFile.isDirectory()) {
//if dirDetails(...) doesn't work, then use dir(...), distinguish file type by FTPException
if ((!(".").equals(ftpFile.getName())) && (!("..").equals(ftpFile.getName()))) {
java.io.File localFile = new java.io.File(localDirectory + "/" + ftpFile.getName());
if ((ftpFiles == null) && (ftpFileNames != null)){
//if the file is folder, catch the FTPException and recur
for (String ftpFileName : ftpFileNames){
try {
downloadFile(localDirectory + "/" + ftpFileName,ftpFileName);
} catch (com.enterprisedt.net.ftp.FTPException e) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + e.getMessage());
<%}%>
java.io.File localFile = new java.io.File(localDirectory + "/" + ftpFileName);
if (!localFile.exists()) {
localFile.mkdir();
}
getAllFiles(path + "/" + ftpFile.getName(), localDirectory + "/" + ftpFile.getName());
chdir(path);
}
} else if (!ftpFile.isSymbolicLink()) {
downloadFile(localDirectory + "/" + ftpFile.getName(),ftpFile.getName());
}
}
}
if (!localFile.exists()) {
localFile.mkdir();
}
getAllFiles(path + "/" + ftpFileName, localDirectory + "/" + ftpFileName);
chdir(path);
}
}
} else {
for (com.enterprisedt.net.ftp.FTPFile ftpFile : ftpFiles) {
public void getFiles(String remoteDirectory, String localDirectory, String maskStr)
throws IllegalStateException, IOException, java.io.FileNotFoundException {
if (ftpFile.isDir()) {
if ((!(".").equals(ftpFile.getName())) && (!("..").equals(ftpFile.getName()))) {
java.io.File localFile = new java.io.File(localDirectory + "/" + ftpFile.getName());
chdir(remoteDirectory);
org.apache.commons.net.ftp.FTPFile[] ftpFiles = null;
ftpFiles = ftpClient.listFiles(".");
if (!localFile.exists()) {
localFile.mkdir();
}
getAllFiles(path + "/" + ftpFile.getName(), localDirectory + "/" + ftpFile.getName());
chdir(path);
}
} else if (!ftpFile.isLink()) {
downloadFile(localDirectory + "/" + ftpFile.getName(),ftpFile.getName());
}
}
}
}
for(org.apache.commons.net.ftp.FTPFile ftpFile : ftpFiles) {
public void getFiles(String remoteDirectory, String localDirectory, String maskStr) throws java.io.IOException,
com.enterprisedt.net.ftp.FTPException, java.text.ParseException {
chdir(remoteDirectory);
String[] ftpFileNames = null;
com.enterprisedt.net.ftp.FTPFile[] ftpFiles = null;
if (ftpFile.isFile()) {
String fileName = ftpFile.getName();
try {
//use dir() for Bug9562 with FTP server in AS400 (the same way as getAllFiles())
ftpFileNames = ftpClient.dir(null, false);
//Bug 13272, if dir() throw exception, use dirDetails().
} catch (com.enterprisedt.net.ftp.FTPException e){
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + e.getMessage());
if (fileName.matches(maskStr)) {
downloadFile(localDirectory + "/" + fileName,fileName);
}
}
}
}
public void chdir(String path) throws IllegalStateException, IOException {
if (!".".equals(path)) {
ftpClient.changeWorkingDirectory(path);
}
}
public String pwd() throws IllegalStateException, IOException {
return ftpClient.printWorkingDirectory();
}
protected abstract void downloadFile(String localFileName, String remoteFileName)
throws IllegalStateException, java.io.FileNotFoundException, IOException;
}
<%
if (!ftps) { // *** ftp *** //
%>
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
if (("true").equals(useExistingConn)) {
String conn= "conn_" + connection;
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
<%}%>
ftpFileNames = null;
ftpFiles = ftpClient.dirDetails(".");
}
//if dirDetails(...) doesn't work, then use dir(...), but can not distinguish file type
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
}
if ((ftpFiles == null) && (ftpFileNames != null)){
for (String ftpFileName : ftpFileNames) {
if (ftpFileName.matches(maskStr)) {
downloadFile(localDirectory + "/" + ftpFileName,ftpFileName);
}
}
} else{
for (com.enterprisedt.net.ftp.FTPFile ftpFile : ftpFiles) {
if (!ftpFile.isDir() && !ftpFile.isLink()) {
String fileName = ftpFile.getName();
if (fileName.matches(maskStr)) {
downloadFile(localDirectory + "/" + fileName,fileName);
}
}
}
}
}
public void chdir(String path) throws java.io.IOException, com.enterprisedt.net.ftp.FTPException{
if (!".".equals(path)) {
ftpClient.chdir(path);
}
}
public String pwd() throws java.io.IOException, com.enterprisedt.net.ftp.FTPException{
return ftpClient.pwd();
}
private void downloadFile(String localFileName, String remoteFileName) throws java.io.IOException, com.enterprisedt.net.ftp.FTPException {
java.io.File localFile = new java.io.File(localFileName);
try {
<%if (append) {%>
java.io.FileOutputStream fout = new java.io.FileOutputStream(localFile, true);
ftpClient.get(fout, remoteFileName);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer OK.");
<%
} else if ("never".equals(overwrite)){
%>
if (!localFile.exists()) {
ftpClient.get(localFileName, remoteFileName);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer OK.");
} else {
if("binary".equalsIgnoreCase(ElementParameterParser.getValue(node, "__MODE__"))){
%>
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
} else {
%>
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.ASCII_FILE_TYPE);
<%
}
%>
final java.util.List<String> msg_<%=cid%> = new java.util.ArrayList<String>();
FTPGetter_<%=cid%> getter_<%=cid%> = new FTPGetter_<%=cid%>() {
@Override
protected void downloadFile(String localFileName, String remoteFileName)
throws IllegalStateException, java.io.FileNotFoundException, IOException {
java.io.File localFile = new java.io.File(localFileName);
<%
if (append) {
%>
try (java.io.FileOutputStream localFos = new java.io.FileOutputStream(localFileName, true)) {
ftpClient.retrieveFile(remoteFileName, localFos);
<%if(isLog4jEnabled){%>
log.info("<%= cid %> - file ["+ remoteFileName +"] exit transmission.");
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file ["+ remoteFileName +"] exit transmission.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file transfered.");
}
<%
} else if ("always".equals(overwrite)){
%>
ftpClient.get(localFileName, remoteFileName);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
<%
} else if ("size_differ".equals(overwrite)) {
%>
if (localFile.exists()) {
long ftpSize = ftpClient.size(remoteFileName);
long localSize = localFile.length();
if (ftpSize != localSize) {
ftpClient.get(localFileName, remoteFileName);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer OK.");
} else {
<%if(isLog4jEnabled){%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer OK.");
count++;
} catch (IOException e) {
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded unsuccessfully.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer fail.");
throw e;
}
<%
} else if ("never".equals(overwrite)) {
%>
if (!localFile.exists()) {
downloadFileWithOverwrite(localFileName, remoteFileName);
} else {
<%if(isLog4jEnabled){%>
log.info("<%= cid %> - file ["+ remoteFileName +"] exit transmission.");
<%}%>
msg_<%=cid%>.add("file ["+ remoteFileName +"] exit transmission.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file transfered.");
}
<%
} else if ("always".equals(overwrite)){
%>
downloadFileWithOverwrite(localFileName, remoteFileName);
<%
} else if ("size_differ".equals(overwrite)) {
%>
if (localFile.exists()) {
long ftpSize = java.util.Arrays.stream(ftpClient.listFiles(remoteFileName)).findFirst().get().getSize();
long localSize = localFile.length();
if (ftpSize != localSize) {
downloadFileWithOverwrite(localFileName, remoteFileName);
} else {
<%if(isLog4jEnabled){%>
log.info("<%= cid %> - file ["+ remoteFileName +"] exit transmission.");
<%}%>
msg_<%=cid%>.add("file ["+ remoteFileName +"] exit transmission.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file transfered.");
}
} else {
ftpClient.get(localFileName, remoteFileName);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer OK.");
}
<%
}
%>
} catch (com.enterprisedt.net.ftp.FTPException e) {
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded unsuccessfully.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer fail.");
throw e;
}
count++;
}
msg_<%=cid%>.add("file ["+ remoteFileName +"] exit transmission.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file transfered.");
}
} else {
downloadFileWithOverwrite(localFileName, remoteFileName);
}
<%
}
%>
}
private void downloadFileWithOverwrite(String localFileName, String remoteFileName)
throws IllegalStateException, java.io.FileNotFoundException, IOException {
try (java.io.FileOutputStream localFos = new java.io.FileOutputStream(localFileName)) {
ftpClient.retrieveFile(remoteFileName, localFos);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded successfully.");
<%if (bPrintMsg) {%>
System.out.println("file [" + remoteFileName + "] downloaded successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer OK.");
count++;
} catch (IOException e) {
msg_<%=cid%>.add("file [" + remoteFileName + "] downloaded unsuccessfully.");
globalMap.put("<%=cid %>_CURRENT_STATUS", "File transfer fail.");
throw e;
}
}
};
getter_<%=cid%>.ftpClient = ftp_<%=cid %>;
<%
} else { // *** ftps *** //
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
<%
if ("false".equals(useExistingConn)){
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
} else {
String conn= "conn_" + connection;
%>
<% /* get connection from globalMap*/ %>
String rootDir_<%=cid %> = null;
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient)globalMap.get("<%=conn %>");
<%
if (!moveToCurrentDir) {
%>
rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
<%
}
if(isLog4jEnabled) {
%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection.Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%
}
}
%>
FTPGetter_<%=cid%> getter_<%=cid%> = new FTPGetter_<%=cid%>() {
@Override
protected void downloadFile(String localFileName, String remoteFileName)
throws IllegalStateException, java.io.FileNotFoundException, IOException {
try (java.io.FileOutputStream localFos = new java.io.FileOutputStream(localFileName)) {
ftpClient.retrieveFile(remoteFileName, localFos);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
count++;
}
}
};
getter_<%=cid%>.ftpClient = ftp_<%=cid %>;
<%
} //common code for ftp and ftps:
%>
String remotedir_<%=cid%> = <%=remotedir%>;
if (!".".equals(remotedir_<%=cid %>)) {
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
}
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
if ("true".equals(useExistingConn)){
String conn= "conn_" + connection;
%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}
if (("ACTIVE").equals(connectMode)){
%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%
}else{
%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%
}
%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%} %>
msg_<%=cid%>.clearAll();
FTPGetter_<%=cid%> getter_<%=cid%> = new FTPGetter_<%=cid%>();
getter_<%=cid%>.ftpClient = ftp_<%=cid %>;
String remotedir_<%=cid %> = <%=remotedir%>;
if (!".".equals(remotedir_<%=cid %>)) {
ftp_<%=cid %>.chdir(remotedir_<%=cid %>);
}
<%
} else { // *** ftps *** //
%>
class FTPSGetter_<%=cid%> {
private org.apache.commons.net.ftp.FTPClient ftpClient = null;
private int count = 0;
public void getAllFiles(String remoteDirectory, String localDirectory)
throws IllegalStateException, IOException, java.io.FileNotFoundException {
chdir(remoteDirectory);
String path = ftpClient.printWorkingDirectory();
org.apache.commons.net.ftp.FTPFile[] ftpFiles = null;
ftpFiles = ftpClient.listFiles();
for (org.apache.commons.net.ftp.FTPFile ftpFile : ftpFiles) {
if (ftpFile.isDirectory()) {
if ((!(".").equals(ftpFile.getName())) && (!("..").equals(ftpFile.getName()))) {
java.io.File localFile = new java.io.File(localDirectory + "/" + ftpFile.getName());
if (!localFile.exists()) {
localFile.mkdir();
}
getAllFiles(path + "/" + ftpFile.getName(), localDirectory + "/" + ftpFile.getName());
chdir(path);
}
} else if (!ftpFile.isSymbolicLink()) {
downloadFile(localDirectory + "/" + ftpFile.getName(),ftpFile.getName());
}
}
}
public void getFiles(String remoteDirectory, String localDirectory, String maskStr)
throws IllegalStateException, IOException, java.io.FileNotFoundException {
chdir(remoteDirectory);
org.apache.commons.net.ftp.FTPFile[] ftpFiles = null;
ftpFiles = ftpClient.listFiles(".");
for(org.apache.commons.net.ftp.FTPFile ftpFile : ftpFiles) {
if (ftpFile.isFile()) {
String fileName = ftpFile.getName();
if (fileName.matches(maskStr)) {
downloadFile(localDirectory + "/" + fileName,fileName);
}
}
}
}
public void chdir(String path)
throws IllegalStateException, IOException {
if (!".".equals(path)) {
ftpClient.changeWorkingDirectory(path);
}
}
public String pwd()
throws IllegalStateException, IOException {
return ftpClient.printWorkingDirectory();
}
private void downloadFile(String localFileName, String remoteFileName)
throws IllegalStateException, java.io.FileNotFoundException, IOException {
try (java.io.FileOutputStream localFos = new java.io.FileOutputStream(localFileName)) {
ftpClient.retrieveFile(remoteFileName, localFos);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Downloaded file " + (count +1) + " : '" + remoteFileName + "' successfully.");
<%}%>
count++;
}
}
}
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
<%
if ("false".equals(useExistingConn)){
%>
<% /* create connection */ %>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid%> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid%> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid%> = null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid%>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
String conn= "conn_" + connection;
%>
<% /* get connection from globalMap*/ %>
String rootDir_<%=cid %> = null;
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient)globalMap.get("<%=conn %>");
<%
if (!moveToCurrentDir) {
%>
rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
<%
}
if(isLog4jEnabled) {
%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection.Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%
}
}
%>
FTPSGetter_<%=cid%> getter_<%=cid%> = null;
String remotedir_<%=cid%> = null;
getter_<%=cid%> = new FTPSGetter_<%=cid%>();
getter_<%=cid%>.ftpClient = ftp_<%=cid %>;
remotedir_<%=cid%> = <%=remotedir%>;
if (!".".equals(remotedir_<%=cid %>)) {
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
}
<%
}
// *** share code *** //
@@ -783,22 +555,10 @@ java.util.List<String> maskList_<%=cid %> = new java.util.ArrayList<String>();
<%
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
maskList_<%=cid %>.add(<%= line.get("FILEMASK") %>);
%>
maskList_<%=cid %>.add(<%= line.get("FILEMASK") %>);
<%
}
if (!sftp && !ftps) { // *** ftp *** //
if ("binary".equalsIgnoreCase(ElementParameterParser.getValue(node, "__MODE__"))) {
%>
ftp_<%=cid %>.setType(com.enterprisedt.net.ftp.FTPTransferType.BINARY);
<%
}else {
%>
ftp_<%=cid %>.setType(com.enterprisedt.net.ftp.FTPTransferType.ASCII);
<%
}
}
%>
String localdir_<%=cid %> = <%=localdir%>;
//create folder if local direcotry (assigned by property) not exists

View File

@@ -54,8 +54,8 @@ nb_file_<%=cid%> = getter_<%=cid %>.count;
<%}%>
<%}%>
<%} else if (!ftps) {%>
msg_<%=cid%>.add(ftp_<%=cid%>.getDownloadCount() + " files have been downloaded.");
String[] msgAll_<%=cid %> = msg_<%=cid%>.getAll();
msg_<%=cid%>.add(getter_<%=cid%>.count + " files have been downloaded.");
String[] msgAll_<%=cid %> = msg_<%=cid%>.toArray(new String[0]);
StringBuffer sb_<%=cid %> = new StringBuffer();
if (msgAll_<%=cid %> != null) {
@@ -74,13 +74,13 @@ nb_file_<%=cid%> = getter_<%=cid %>.count;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
} catch(java.net.SocketException se_<%=cid%>) {
} catch(java.io.IOException ioe_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - " + se_<%=cid%>.getMessage());
log.error("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
@@ -90,7 +90,7 @@ nb_file_<%=cid%> = getter_<%=cid %>.count;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
@@ -99,7 +99,7 @@ nb_file_<%=cid%> = getter_<%=cid %>.count;
}else{
if(!moveToCurrentDir){
%>
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
<%
}
}

View File

@@ -150,7 +150,7 @@ COMPATIBILITY="ALL"
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="61" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="61" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -298,15 +298,15 @@ COMPATIBILITY="ALL"
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
</CODEGENERATION>
<RETURNS>
<RETURN NAME="NB_FILE" TYPE="id_Integer" AVAILABILITY="AFTER"/>
<RETURN NAME="CURRENT_STATUS" TYPE="id_String" AVAILABILITY="FLOW"/>

View File

@@ -16,23 +16,10 @@ String dieOnError = ElementParameterParser.getValue(node, "__DIE_ON_ERROR__");
String remotedir = ElementParameterParser.getValue(node, "__REMOTEDIR__");
String connection = ElementParameterParser.getValue(node, "__CONNECTION__");
String useExistingConn = ElementParameterParser.getValue(node, "__USE_EXISTING_CONNECTION__");
boolean sftp = false;
boolean usePerl5 = ("true").equals(ElementParameterParser.getValue(node, "__PERL5_REGEX__"));
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
if ("true".equals(useExistingConn)) {
List<? extends INode> nodeList = node.getProcess().getGeneratingNodes();
for (INode n : nodeList) {
if (n.getUniqueName().equals(connection)) {
sftp = "true".equals(ElementParameterParser.getValue(n, "__SFTP__"));
}
}
} else {
sftp = "true".equals(ElementParameterParser.getValue(node, "__SFTP__"));
}
%>
try {

View File

@@ -62,8 +62,10 @@ if (("true").equals(useExistingConn)) {
}
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
}
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
//The following part support the socks proxy for FTP and SFTP (Socks V4 or V5, they are all OK).
//And it can not work with the FTP proxy directly, only support the socks proxy.
@@ -150,8 +152,8 @@ int nb_file_<%=cid%> = 0;
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -228,255 +230,103 @@ int nb_file_<%=cid%> = 0;
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%} else if (!ftps) {%> // *** ftp *** //
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> =null;
<%
if (("true").equals(useExistingConn)) {
String conn= "conn_" + connection;
%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
// msg_<%=cid%> likes a String[] to save the message from transfer.
com.enterprisedt.net.ftp.TransferCompleteStrings msg_<%=cid%> = ftp_<%=cid %>.getTransferCompleteMessages();
msg_<%=cid%>.clearAll();
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,<%= line.get("NEWNAME") %>);
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
%>
String remotedir<%=cid %> = (<%=remotedir%>).replaceAll("\\\\", "/");
<%} else {
if (!ftps) {%> // *** ftp *** //
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
if (createDirIfNotExist) {
if (("true").equals(useExistingConn)) {
String conn= "conn_" + connection;
%>
boolean isDirectoryExist_<%=cid %> = ftp_<%=cid %>.existsDirectory(remotedir<%=cid %>);
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
} else {
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
}
if (!isDirectoryExist_<%=cid %>) {
String rootDir = null;
String[] dirsTree_<%=cid %> = remotedir<%=cid %>.split("/");
if("binary".equalsIgnoreCase(ElementParameterParser.getValue(node, "__MODE__"))){
%>
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
} else {
%>
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.ASCII_FILE_TYPE);
<%
}
%>
// msg_<%=cid%> likes a String[] to save the message from transfer.
java.util.List<String> msg_<%=cid%> = new java.util.ArrayList<>();
<%
} else { // *** ftps *** //
String keystoreFile = ElementParameterParser.getValue(node,"__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
if(("false").equals(useExistingConn)){
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
} else {
String conn= "conn_" + connection;
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
}
} //common code for ftp and ftps:
%>
String rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
String remotedir_<%=cid%> = (<%=remotedir%>).replaceAll("\\\\", "/");
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
<%
if(createDirIfNotExist) {
%>
if(!cwdSuccess_<%=cid %>){
String[] dirsTree_<%=cid %> = remotedir_<%=cid %>.split("/");
for (String dir: dirsTree_<%=cid %>) {
if (!ftp_<%=cid %>.existsDirectory(dir)) {
ftp_<%=cid %>.mkdir(dir);
}
ftp_<%=cid %>.chdir(dir);
ftp_<%=cid %>.makeDirectory(dir);
ftp_<%=cid %>.changeWorkingDirectory(dir);
}
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
}
<%
}
%>
ftp_<%=cid %>.chdir(remotedir<%=cid %>);
<%if ("binary".equalsIgnoreCase(ElementParameterParser.getValue(node, "__MODE__"))) {%>
ftp_<%=cid %>.setType(com.enterprisedt.net.ftp.FTPTransferType.BINARY);
<%} else { %>
ftp_<%=cid %>.setType(com.enterprisedt.net.ftp.FTPTransferType.ASCII);
<%}%>
String localdir<%=cid %> = <%=localdir%>;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Uploading files to the server.");
<%}%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
} else { // *** ftps *** //
String keystoreFile = ElementParameterParser.getValue(node,"__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
String rootDir_<%=cid %> = null;
<%
if(("false").equals(useExistingConn)){
%>
<% /* create connection */ %>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid%> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid%> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid%> = null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid%>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%
passwordFieldName = "__PASSWORD__";
}
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
<%
if (dieOnError) {
%>
throw e;
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
String conn= "conn_" + connection;
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Uses an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
rootDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
<%
}
%>
String remotedir_<%=cid%> = (<%=remotedir%>).replaceAll("\\\\", "/");
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%
if(createDirIfNotExist) {
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
String[] dirsTree_<%=cid %> = remotedir_<%=cid %>.split("/");
for (String dir: dirsTree_<%=cid %>) {
ftp_<%=cid %>.makeDirectory(dir);
ftp_<%=cid %>.changeWorkingDirectory(dir);
}
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>, <%= line.get("NEWNAME") %>);
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
}
%>
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(remotedir_<%=cid %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
String localdir<%=cid %> = <%=localdir%>;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Uploading files to the server.");
<%}%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>, <%= line.get("NEWNAME") %>);
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
%>
String localdir<%=cid %> = <%=localdir%>;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Uploading files to the server.");
<%}%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
} /* close ftps */
}
%>

View File

@@ -57,9 +57,9 @@ if(sftp){ // *** sftp *** //
<%}else if(!ftps){%> // *** ftp *** //
}
msg_<%=cid%>.add(ftp_<%=cid %>.getUploadCount() + " files have been uploaded.");
msg_<%=cid%>.add(nb_file_<%=cid%> + " files have been uploaded.");
String[] msgAll_<%=cid %> = msg_<%=cid%>.getAll();
String[] msgAll_<%=cid %> = msg_<%=cid%>.toArray(new String[0]);
StringBuffer sb_<%=cid %> = new StringBuffer();
if (msgAll_<%=cid %> != null) {
for (String item_<%=cid %> : msgAll_<%=cid %>) {
@@ -76,13 +76,13 @@ if(sftp){ // *** sftp *** //
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
}catch(java.net.SocketException se_<%=cid%>){
}catch(java.io.IOException ioe_<%=cid%>){
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
@@ -92,7 +92,7 @@ if(sftp){ // *** sftp *** //
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
@@ -101,7 +101,7 @@ if(sftp){ // *** sftp *** //
}else{
if(!moveToCurrentDir){
%>
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
<%
}
}

View File

@@ -165,7 +165,7 @@ COMPATIBILITY="ALL"
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="6" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="6" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -343,12 +343,12 @@ COMPATIBILITY="ALL"
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -185,125 +185,14 @@ try{
}
}
<%
}else if (!ftps) { // *** ftp *** //
%>
String currentStatus_<%=cid %> = "No file transfered.";
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
for (String key<%=cid %> : keySet<%=cid %>){
if(key<%=cid %> == null || "".equals(key<%=cid%>)){
System.err.println("file name invalid!");
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - file name invalid!");
<%}%>
continue;
}
String tempdir<%=cid %> = localdir<%=cid %>;
String filemask<%=cid %> = key<%=cid %>;
String dir<%=cid %> = null;
<%if(!useGlob) {//perl5 mode not support windows(\) path separator at the mask string%>
String mask<%=cid %> = filemask<%=cid %>.replaceAll("\\\\", "/") ;
<%} else {%>
String mask<%=cid %> = filemask<%=cid%>;
<%}%>
int i<%=cid %> = mask<%=cid %>.lastIndexOf('/');
if (i<%=cid %>!=-1){
dir<%=cid %> = mask<%=cid %>.substring(0, i<%=cid %>);
mask<%=cid %> = mask<%=cid %>.substring(i<%=cid %>+1);
}
if (dir<%=cid %>!=null && !"".equals(dir<%=cid %>)){
tempdir<%=cid %> = tempdir<%=cid %> + "/" + dir<%=cid %>;
}
<%if(!useGlob) {%>
mask<%=cid %> = mask<%=cid %>.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*");
<%}%>
final String finalMask<%=cid %> = mask<%=cid %>;
java.io.File[] listings<%=cid %> = null;
java.io.File file<%=cid %> = new java.io.File(tempdir<%=cid %>);
if (file<%=cid %>.isDirectory()) {
listings<%=cid %> = file<%=cid %>.listFiles(new java.io.FileFilter() {
public boolean accept(java.io.File pathname) {
boolean result = false;
if (pathname != null && pathname.isFile()) {
result = java.util.regex.Pattern.compile(finalMask<%=cid %>).matcher(pathname.getName()).find();
}
return result;
}
});
}
if(listings<%=cid %> != null && listings<%=cid %>.length > 0){
for (int m<%=cid %> = 0; m<%=cid %> < listings<%=cid %>.length; m<%=cid %>++){
java.io.File file_in_localDir = listings<%=cid %>[m<%=cid %>];
String uploadingFileName_<%= cid %> = file_in_localDir.getName();
String newFileName_<%= cid %> = map<%=cid %>.get(key<%=cid %>);
String newName_<%=cid%> = ("".equals(newFileName_<%= cid %>)) ? uploadingFileName_<%= cid %> : newFileName_<%= cid %>;
boolean targetFileExistsOnRemoteServer_<%= cid %> = false;
try{
if (uploadingFileName_<%= cid %>.matches(mask<%=cid %>)){
<%
if(("never".equals(overwrite) || "size_differ".equals(overwrite)) && !append){
%>
targetFileExistsOnRemoteServer_<%= cid %> = ftp_<%= cid %>.existsFile(newName_<%= cid %>);
globalMap.put("<%=cid %>_CURRENT_FILE_EXISTS", targetFileExistsOnRemoteServer_<%= cid %>);
<%
}
%>
<%if(append){%>
ftp_<%=cid %>.put(tempdir<%=cid %> + "/" + uploadingFileName_<%= cid %>, newName_<%= cid %>, true);
<%}else if(("never").equals(overwrite)){%>
if(!targetFileExistsOnRemoteServer_<%= cid %>){
ftp_<%=cid %>.put(tempdir<%=cid %> + "/" + uploadingFileName_<%= cid %>, newName_<%=cid%>);
}
<%}else if(("always").equals(overwrite)){%>
ftp_<%=cid %>.put(tempdir<%=cid %> + "/" + uploadingFileName_<%= cid %>, newName_<%=cid%>);
<%}else if(("size_differ").equals(overwrite)){%>
if (targetFileExistsOnRemoteServer_<%= cid %>){
com.enterprisedt.net.ftp.FTPFile ftpfile<%=cid%>=ftp_<%=cid%>.fileDetails(newName_<%=cid%>);
long ftpSize<%=cid%>=ftpfile<%=cid%>.size();
long localSize<%=cid%>=file_in_localDir.length();
if(ftpSize<%=cid%> != localSize<%=cid%>){
ftp_<%=cid %>.put(tempdir<%=cid %> + "/" + uploadingFileName_<%= cid %>, newName_<%=cid%>);
}
}else{
ftp_<%=cid %>.put(tempdir<%=cid %> + "/" + uploadingFileName_<%= cid %>, newName_<%=cid%>);
}
<%}%>
currentStatus_<%=cid %> = "File rename OK.";
msg_<%=cid%>.add("file: " + file_in_localDir.getAbsolutePath() + ", size: "
+ file_in_localDir.length() + " bytes upload successfully");
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Uploaded file '" + uploadingFileName_<%= cid %> + "' successfully.");
<%}%>
nb_file_<%=cid%>++;
}
}catch(com.enterprisedt.net.ftp.FTPException e_<%=cid%>){
msg_<%=cid%>.add("file " + file_in_localDir.getAbsolutePath() + " not found?");
currentStatus_<%=cid %> = "File rename fail.";
globalMap.put("<%=cid %>_CURRENT_FILE_EXISTS", targetFileExistsOnRemoteServer_<%= cid %>);
throw e_<%=cid%>;
}
}
}else{
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - No match file exist!");
<%}%>
System.err.println("No match file exist!");
}
} else {// *** ftp+ ftps *** //
if (!ftps) {
ftpsoverwrite = overwrite; //to avoid migration, different dropdowns with same purpose
}
globalMap.put("<%=cid %>_CURRENT_STATUS", currentStatus_<%=cid %>);
<%
} else {// *** ftps *** //
%>
String currentStatus_<%=cid %> = "No file transfered.";
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file transfered.");
globalMap.put("<%=cid %>_CURRENT_STATUS", currentStatus_<%=cid %>);
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
boolean needRename_<%=cid%> = false;
for (String key<%=cid %> : keySet<%=cid %>){
@@ -368,17 +257,43 @@ try{
java.io.FileInputStream file_stream_<%=cid %> = new java.io.FileInputStream(file_in_localDir_<%=cid%>);
globalMap.put("<%=cid %>_CURRENT_FILE_EXISTS", remoteExistsFiles_<%=cid%>.contains(file_in_localDir_<%=cid%>.getName()));
String newName_<%=cid%> = ("".equals(map<%=cid %>.get(key<%=cid %>)))?file_in_localDir_<%=cid%>.getName():map<%=cid %>.get(key<%=cid %>);
<%if("never".equals(ftpsoverwrite)){%>
needRename_<%=cid%> = true;
<%
if (!ftps && append) {
%>
if ((remoteExistsFiles_<%=cid%>.contains(newName_<%=cid%>))){
ftp_<%=cid %>.appendFile(file_in_localDir_<%=cid%>.getName(), file_stream_<%=cid %>);
} else {
ftp_<%=cid %>.storeFile(file_in_localDir_<%=cid%>.getName(), file_stream_<%=cid %>);
}
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Uploaded file '" + newName_<%=cid%> + "' successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_FILE_EXISTS", remoteExistsFiles_<%=cid%>.contains(newName_<%=cid%>));
<%
} else if ("size_differ".equals(ftpsoverwrite)) {
%>
if ((remoteExistsFiles_<%=cid%>.contains(newName_<%=cid%>))){
long ftpSize = java.util.Arrays.stream(ftp_<%=cid %>.listFiles(newName_<%=cid%>)).filter(org.apache.commons.net.ftp.FTPFile::isFile).findFirst().get().getSize();
long localSize = file_in_localDir_<%=cid%>.length();
if (ftpSize != localSize) {
ftp_<%=cid %>.deleteFile(newName_<%=cid%>);
}
}
ftp_<%=cid %>.storeFile(file_in_localDir_<%=cid%>.getName(), file_stream_<%=cid %>);
<%
} else if("never".equals(ftpsoverwrite)){
%>
if (!(remoteExistsFiles_<%=cid%>.contains(newName_<%=cid%>))){
needRename_<%=cid%> = true;
ftp_<%=cid %>.storeFile(file_in_localDir_<%=cid%>.getName(), file_stream_<%=cid %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Uploaded file '" + newName_<%=cid%> + "' successfully.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_FILE_EXISTS", remoteExistsFiles_<%=cid%>.contains(newName_<%=cid%>));
} else {
needRename_<%=cid %> = false;
}
<%}else if("always".equals(ftpsoverwrite)){%>
needRename_<%=cid%> = true;
if ((remoteExistsFiles_<%=cid%>.contains(newName_<%=cid%>))){
ftp_<%=cid %>.deleteFile(newName_<%=cid%>);
}
@@ -391,6 +306,14 @@ try{
<%}%>
file_stream_<%=cid %>.close();
<%
if (!ftps) {
%>
msg_<%=cid%>.add("file: " + file_in_localDir_<%=cid%>.getAbsolutePath() + ", size: "
+ file_in_localDir_<%=cid%>.length() + " bytes upload successfully");
<%
}
%>
nb_file_<%=cid%>++;
}
}
@@ -418,6 +341,12 @@ try{
<%}%>
}catch(java.lang.Exception e_<%=cid%>){
<%
if (!ftps) {
%>
msg_<%=cid%>.add("file not found?: " + e_<%=cid%>.getMessage());
<%
}
if(("true").equals(dieOnError)){
%>
throw(e_<%=cid%>);

View File

@@ -57,6 +57,8 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -131,9 +133,9 @@ if (sftp) { // *** sftp *** //
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
<%}%>
@@ -172,102 +174,42 @@ if (sftp) { // *** sftp *** //
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
} else if (ftps) { // *** ftps *** //
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
} else {
if (ftps) { // *** ftps *** //
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
passwordFieldName = "__PASSWORD__";
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
} else {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
}
} else { // *** ftp *** //
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
}
if (("true").equals(useExistingConn)) {
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
if (dieOnError) {
} else {
passwordFieldName = "__PASSWORD__";
%>
throw e;
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
} else {
%>
e.printStackTrace();
<%
}
%>
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
}
}
} //common code for ftp and ftps:
%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
@@ -294,67 +236,5 @@ if (sftp) { // *** sftp *** //
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) { <% /* start main loop */ %>
<%
} else {// *** ftp *** //
}
%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> =null;
<%if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<%if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,<%= line.get("NEWNAME") %>);
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
%>
String remotedir<%=cid %> = <%=remotedir%>;
ftp_<%=cid %>.chdir(remotedir<%=cid %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Renaming file from server.");
<%}%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%}%>

View File

@@ -78,13 +78,13 @@
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
}catch(java.net.SocketException se_<%=cid%>){
}catch(java.io.IOException ioe_<%=cid%>){
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
@@ -94,7 +94,7 @@
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
@@ -103,7 +103,7 @@
}else{
if(!moveToCurrentDir){
%>
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
ftp_<%=cid %>.changeWorkingDirectory(rootDir_<%=cid %>);
<%
}
}

View File

@@ -143,7 +143,7 @@ COMPATIBILITY="ALL"
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="31" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="31" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -261,12 +261,12 @@ COMPATIBILITY="ALL"
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" UrlPath="platform:/plugin/org.talend.libraries.ftp/lib/edtftpj-2.5.0.jar" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -57,25 +57,74 @@ try{
}
<%
} else if (ftps) { // *** ftps *** //
} else { // *** ftp+s *** //
%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file renamed.");
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
for (String key<%=cid %> : keySet<%=cid %>) {
try{
boolean renameSuccess_<%=cid %> = ftp_<%=cid%>.rename(key<%=cid %>, map<%=cid %>.get(key<%=cid %>));
if (renameSuccess_<%=cid %>) {
nb_file_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + key<%=cid %> + "' rename ok.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
} else {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - '" + key<%=cid %> + "' rename failed. " + ftp_<%=cid %>.getReplyString());
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename fail.");
}
<%
if (!ftps && "never".equals(overwrite)) {
%>
String[] remoteFileArray_<%=cid%> = java.util.Arrays.stream(ftp_<%=cid %>.listFiles()).filter(org.apache.commons.net.ftp.FTPFile::isFile).map(org.apache.commons.net.ftp.FTPFile::getName).filter(map<%=cid %>.get(key<%=cid %>)::equals).toArray(String[]::new);
boolean isRemoteFileExists_<%=cid %> = (remoteFileArray_<%=cid%> != null) && (remoteFileArray_<%=cid%>.length > 0);
if (!isRemoteFileExists_<%=cid%>) {
boolean renameSuccess_<%=cid %> = ftp_<%=cid%>.rename(key<%=cid %>, map<%=cid %>.get(key<%=cid %>));
if (renameSuccess_<%=cid %>) {
nb_file_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + key<%=cid %> + "' rename ok.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
} else {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - '" + key<%=cid %> + "' rename failed. " + ftp_<%=cid %>.getReplyString());
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename fail.");
}
}
<%
} else if (!ftps && "size_differ".equals(overwrite)) {
%>
org.apache.commons.net.ftp.FTPFile toRenameFile_<%=cid %> = java.util.Arrays.stream(ftp_<%=cid %>.listFiles(key<%=cid %>)).filter(org.apache.commons.net.ftp.FTPFile::isFile).findFirst().orElse(null);
if (toRenameFile_<%=cid %> != null) {
org.apache.commons.net.ftp.FTPFile toOverwriteFile_<%=cid %> = java.util.Arrays.stream(ftp_<%=cid %>.listFiles(map<%=cid %>.get(key<%=cid %>))).filter(org.apache.commons.net.ftp.FTPFile::isFile).findFirst().orElse(null);
if ((toOverwriteFile_<%=cid %> != null) && toRenameFile_<%=cid %>.getSize() != toOverwriteFile_<%=cid%>.getSize()) {
ftp_<%=cid %>.deleteFile(map<%=cid %>.get(key<%=cid %>));
}
boolean renameSuccess_<%=cid %> = ftp_<%=cid%>.rename(key<%=cid %>, map<%=cid %>.get(key<%=cid %>));
if (renameSuccess_<%=cid %>) {
nb_file_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + key<%=cid %> + "' rename ok.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
} else {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - '" + key<%=cid %> + "' rename failed. " + ftp_<%=cid %>.getReplyString());
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename fail.");
}
}
<%
} else {
%>
boolean renameSuccess_<%=cid %> = ftp_<%=cid%>.rename(key<%=cid %>, map<%=cid %>.get(key<%=cid %>));
if (renameSuccess_<%=cid %>) {
nb_file_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + key<%=cid %> + "' rename ok.");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
} else {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - '" + key<%=cid %> + "' rename failed. " + ftp_<%=cid %>.getReplyString());
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename fail.");
}
<%
}
%>
}catch(IOException e){
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename fail.");
<%
@@ -94,100 +143,21 @@ try{
}
}
<%
} else {%>//normal ftp
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
String tmp_<%=cid%> = pid + System.currentTimeMillis();
try{
for (String key<%=cid %> : keySet<%=cid %>) {
<%if(("always").equals(overwrite)){%>
String fromFile_<%=cid%> = key<%=cid %>;
String toFile_<%=cid%> = map<%=cid %>.get(key<%=cid %>);
if (!fromFile_<%=cid%>.equals(toFile_<%=cid%>)){
if(ftp_<%=cid%>.exists(fromFile_<%=cid%>)){
if(ftp_<%=cid%>.exists(toFile_<%=cid%> + "." + tmp_<%=cid%>)){
ftp_<%=cid%>.delete(toFile_<%=cid%> + "." + tmp_<%=cid%>);
}
if(ftp_<%=cid%>.exists(toFile_<%=cid%>)){
ftp_<%=cid %>.rename(toFile_<%=cid%>, toFile_<%=cid%> + "." + tmp_<%=cid%>);
}
ftp_<%=cid %>.rename(fromFile_<%=cid%>, toFile_<%=cid%>);
if(ftp_<%=cid%>.exists(toFile_<%=cid%> + "." + tmp_<%=cid%>)){
ftp_<%=cid%>.delete(toFile_<%=cid%> + "." + tmp_<%=cid%>);
}
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + toFile_<%=cid%> + "' rename ok");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
nb_file_<%=cid%>++;
}
}
<%}%>
<%if(("never").equals(overwrite)){%>
if(!(ftp_<%=cid%>.exists(map<%=cid %>.get(key<%=cid %>)))) {
ftp_<%=cid %>.rename(key<%=cid %>, map<%=cid %>.get(key<%=cid %>));
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + key<%=cid %> + "' rename ok");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
nb_file_<%=cid%>++;
}
<%}%>
<%if(("size_differ").equals(overwrite)){%>
if((ftp_<%=cid%>.exists(map<%=cid %>.get(key<%=cid %>)))){
com.enterprisedt.net.ftp.FTPFile ftpnewNamefile<%=cid%>=ftp_<%=cid%>.fileDetails(map<%=cid %>.get(key<%=cid %>));
com.enterprisedt.net.ftp.FTPFile ftporigNamefile<%=cid%>=ftp_<%=cid%>.fileDetails(key<%=cid %>);
long newNameSize<%=cid%>=ftpnewNamefile<%=cid%>.size();
long origNameSize<%=cid%>=ftporigNamefile<%=cid%>.size();
if(!(newNameSize<%=cid%>==origNameSize<%=cid%>)) {
String fromFile_<%=cid%> = key<%=cid %>;
String toFile_<%=cid%> = map<%=cid %>.get(key<%=cid %>);
if(ftp_<%=cid%>.exists(fromFile_<%=cid%>)){
if(ftp_<%=cid%>.exists(toFile_<%=cid%> + "." + tmp_<%=cid%>)){
ftp_<%=cid%>.delete(toFile_<%=cid%> + "." + tmp_<%=cid%>);
}
if(ftp_<%=cid%>.exists(toFile_<%=cid%>)){
ftp_<%=cid %>.rename(toFile_<%=cid%>, toFile_<%=cid%> + "." + tmp_<%=cid%>);
}
ftp_<%=cid %>.rename(fromFile_<%=cid%>, toFile_<%=cid%>);
if(ftp_<%=cid%>.exists(toFile_<%=cid%> + "." + tmp_<%=cid%>)){
ftp_<%=cid%>.delete(toFile_<%=cid%> + "." + tmp_<%=cid%>);
}
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + toFile_<%=cid%> + "' rename ok");
<%}%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
nb_file_<%=cid%>++;
}
}
}else{
ftp_<%=cid %>.rename(key<%=cid %>, map<%=cid %>.get(key<%=cid %>));
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename OK.");
nb_file_<%=cid%>++;
}
<%}%>
}
}catch(com.enterprisedt.net.ftp.FTPException e){
globalMap.put("<%=cid %>_CURRENT_STATUS", "File rename fail.");
throw e;
}
<%}%>
}catch(java.lang.Exception e){
}
%>
} catch (Exception e) {
<%
if(("true").equals(dieOnError)){
if(("true").equals(dieOnError)){
%>
throw(e);
<%
}else{
}else{
%>
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - " + e.getMessage());
<%}%>
System.err.print(e.getMessage());
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - " + e.getMessage());
<%}%>
System.err.print(e.getMessage());
<%
}
}
%>
}

View File

@@ -55,6 +55,8 @@ if (("true").equals(useExistingConn)) {
} else {
sftp = ("true").equals(ElementParameterParser.getValue(node, "__SFTP__"));
ftps = ("true").equals(ElementParameterParser.getValue(node, "__FTPS__"));
%>
<%
}
boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.getProcess(), "__LOG4J_ACTIVATE__"));
@@ -131,7 +133,7 @@ if (sftp) {// *** sftp *** //
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="../tFTPConnection/sftp_connect.javajet"%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/sftp_connect.javajet"%>
<%if(use_encoding) {%>
c_<%=cid%>.setFilenameEncoding(<%=encoding%>);
@@ -167,179 +169,76 @@ if (sftp) {// *** sftp *** //
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%
} else if (ftps) { // *** ftps *** //
/* creating connection */
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
} else {
if (ftps) { // *** ftps *** //
%>
<%@ include file="../templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> =null;
MyTrust_<%=cid %> myTrust_<%=cid%> = null;
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = null;
<%
/* creating connection */
if (("false").equals(useExistingConn)) {
String keystoreFile = ElementParameterParser.getValue(node, "__KEYSTORE_FILE__");
String securityMode = ElementParameterParser.getValue(node, "__SECURITY_MODE__");
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftps_connect.javajet"%>
<%
} else {
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
if (moveToCurrentDir) {
%>
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(<%=remotedir %>);
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
boolean isHTTPProxyUsed = "true".equals(System.getProperty("http.proxySet"));
if (isHTTPProxyUsed) {
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
<%
}
}
} else { // *** ftp *** //
%>
org.apache.commons.net.ftp.FTPClient ftp_<%=cid %> = null;
<%
if (("true").equals(useExistingConn)) {
%>
ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPClient) globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteAddress().toString() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%
if (moveToCurrentDir) {
%>
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(<%=remotedir %>);
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=ftpsPort %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
<%
if(isLog4jEnabled){
}
} else {
passwordFieldName = "__PASSWORD__";
%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftp_connect.javajet"%>
<%
}
passwordFieldName = "__PASSWORD__";
}
} //common code for ftp and ftps:
%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
<% /*store real pwd*/ %>
String rootWorkDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,"");
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
}
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) { <% /* start main loop */ %>
<%
}
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
} else {
%>
org.apache.commons.net.ftp.FTPSClient ftp_<%=cid %> = (org.apache.commons.net.ftp.FTPSClient) globalMap.get("<%=conn %>");
<%
if (moveToCurrentDir) {
%>
boolean cwdSuccess_<%=cid %> = ftp_<%=cid %>.changeWorkingDirectory(<%=remotedir %>);
if (!cwdSuccess_<%=cid %>) {
throw new RuntimeException("Failed to change remote directory. " + ftp_<%=cid %>.getReplyString());
}
<%
}
}
%>
<% /*store real pwd*/ %>
String rootWorkDir_<%=cid %> = ftp_<%=cid %>.printWorkingDirectory();
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%
for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,"");
list<%=cid %>.add(map<%=cid %><%=i %>);
<%
}
%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) { <% /* start main loop */ %>
<%
} else {// *** ftp *** //
%>
com.enterprisedt.net.ftp.FTPClient ftp_<%=cid %> = null;
<%if (("true").equals(useExistingConn)) {%>
ftp_<%=cid %> = (com.enterprisedt.net.ftp.FTPClient)globalMap.get("<%=conn %>");
<%if(isLog4jEnabled){%>
if(ftp_<%=cid %>!=null) {
log.info("<%=cid%> - Use an existing connection. Connection hostname: " + ftp_<%=cid %>.getRemoteHost() + ", Connection port: " + ftp_<%=cid %>.getRemotePort() + ".");
}
<%}%>
<%if(!moveToCurrentDir){%>
String rootDir_<%=cid %> = ftp_<%=cid %>.pwd();
<%}%>
<%} else {%>
ftp_<%=cid %> = new com.enterprisedt.net.ftp.FTPClient();
ftp_<%=cid %>.setRemoteHost(<%=host %>);
ftp_<%=cid %>.setRemotePort(<%=port %>);
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - " + <%=(("ACTIVE").equals(connectMode))?"\"Using the active mode.\"":"\"Using the passive mode.\""%>);
<%}%>
<% if (("ACTIVE").equals(connectMode)) {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.ACTIVE);
<%} else {%>
ftp_<%=cid %>.setConnectMode(com.enterprisedt.net.ftp.FTPConnectMode.PASV);
<%}%>
ftp_<%=cid %>.setControlEncoding(<%=encoding%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" +<%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect();
<%
passwordFieldName = "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%}%>
java.util.List<java.util.Map<String,String>> list<%=cid %> = new java.util.ArrayList<java.util.Map<String,String>>();
<%for (int i = 0; i < files.size(); i++) {
Map<String, String> line = files.get(i);
%>
java.util.Map<String,String> map<%=cid %><%=i %> = new java.util.HashMap<String,String>();
map<%=cid %><%=i %>.put(<%= line.get("FILEMASK") %>,"");
list<%=cid %>.add(map<%=cid %><%=i %>);
<%}%>
String remotedir<%=cid %> = <%=remotedir%>;
ftp_<%=cid %>.chdir(remotedir<%=cid %>);
String root<%=cid %> = ftp_<%=cid %>.pwd();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Truncating file from the server.");
<%}%>
for (java.util.Map<String, String> map<%=cid %> : list<%=cid %>) {
<%}%>

View File

@@ -67,7 +67,7 @@
}
} else {// *** ftp *** //
%>
}
} <% /* close main loop */ %>
<%
if(!("true").equals(useExistingConn)){
if(("true").equals(ignoreFailureAtQuit)){
@@ -76,13 +76,13 @@
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
}catch(java.net.SocketException se_<%=cid%>){
}catch(java.io.IOException ioe_<%=cid%>){
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - " + se_<%=cid%>.getMessage());
log.warn("<%=cid%> - " + ioe_<%=cid%>.getMessage());
<%}%>
//ignore failure
}
@@ -92,18 +92,12 @@
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Closing the connection to the server.");
<%}%>
ftp_<%=cid %>.quit();
ftp_<%=cid %>.disconnect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connection to the server closed.");
<%}%>
<%
}
}else{
if(!moveToCurrentDir){
%>
ftp_<%=cid %>.chdir(rootDir_<%=cid %>);
<%
}
}
}
%>

View File

@@ -149,7 +149,7 @@ COMPATIBILITY="ALL"
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="31" SHOW_IF="USE_EXISTING_CONNECTION == 'true'">
<PARAMETER NAME="MOVE_TO_THE_CURRENT_DIRECTORY" FIELD="CHECK" NUM_ROW="31" SHOW_IF="(USE_EXISTING_CONNECTION == 'true') AND (#LINK@NODE.CONNECTION.SFTP == 'false')">
<DEFAULT>true</DEFAULT>
</PARAMETER>
@@ -249,13 +249,13 @@ COMPATIBILITY="ALL"
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java-FTP" MODULE="edtftpj-2.5.0.jar" MVN="mvn:com.enterprisedt/edtftpj/2.5.0" REQUIRED_IF="(SFTP == 'false') AND (FTPS == 'false') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="Java-SFTP" MODULE="jsch-0.1.55.jar" MVN="mvn:com.jcraft/jsch/0.1.55" REQUIRED_IF="(SFTP == 'true') AND (USE_EXISTING_CONNECTION == 'false')"/>
<IMPORT NAME="jakarta-oro" MODULE="jakarta-oro-2.0.8.jar" MVN="mvn:org.talend.libraries/jakarta-oro-2.0.8/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache/lib/jakarta-oro-2.0.8.jar" REQUIRED="true" />
<IMPORT NAME="Java-FTPS" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="Java-FTPS-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
<IMPORT NAME="Java-FTP+S" MODULE="commons-net-3.6.jar" MVN="mvn:commons-net/commons-net/3.6" REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="FTP+S-Proxy-talend" MODULE="commons-net-ftps-proxy-3.6.1-talend-20190819.jar"
MVN="mvn:org.talend.components.lib/commons-net-ftps-proxy/3.6.1-talend-20190819"
REQUIRED_IF="(FTPS == 'true') AND (USE_EXISTING_CONNECTION == 'false')" />
REQUIRED_IF="(SFTP == 'false') AND (USE_EXISTING_CONNECTION == 'false')" />
<IMPORT NAME="talend-proxy" MODULE="talend-proxy-1.0.1.jar" MVN="mvn:org.talend.components.lib/talend-proxy/1.0.1" REQUIRED_IF="USE_EXISTING_CONNECTION == 'false'" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>

View File

@@ -73,10 +73,10 @@
}
}
<%
} else if (ftps) { /* ftps */
} else { /* ftp+s */
%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file truncated.");
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
for (String key<%=cid %> : keySet<%=cid %>) {
<% /*prepare filemask*/ %>
String filemask<%=cid %> = key<%=cid %>;
@@ -188,44 +188,4 @@
<% /*return to root dir for next filemask*/ %>
ftp_<%=cid %>.changeWorkingDirectory(rootWorkDir_<%=cid %>);
<%
} else { /* ftp */
%>
globalMap.put("<%=cid %>_CURRENT_STATUS", "No file truncated.");
java.util.Set<String> keySet<%=cid %> = map<%=cid %>.keySet();
try {
for (String key<%=cid %> : keySet<%=cid %>) {
String filemask<%=cid %> = key<%=cid %>;
String dir<%=cid %> = null;
<%if(!useGlob) {//perl5 mode not support windows(\) path separator at the mask string%>
String mask<%=cid %> = filemask<%=cid %>.replaceAll("\\\\", "/") ;
<%}else{%>
String mask<%=cid %> = filemask<%=cid %>;
<%}%>
int i<%=cid %> = mask<%=cid %>.lastIndexOf('/');
if (i<%=cid %>!=-1) {
dir<%=cid %> = mask<%=cid %>.substring(0, i<%=cid %>);
mask<%=cid %> = mask<%=cid %>.substring(i<%=cid %>+1);
}
if (dir<%=cid %>!=null && !"".equals(dir<%=cid %>)) ftp_<%=cid %>.chdir(dir<%=cid %>);
<%if(!useGlob) {%>
mask<%=cid %> = org.apache.oro.text.GlobCompiler.globToPerl5(mask<%=cid %>.toCharArray(), org.apache.oro.text.GlobCompiler.DEFAULT_MASK);
<%}%>
String[] listings<%=cid %> = ftp_<%=cid %>.dir(null, false);
for (int m<%=cid %> = 0; m<%=cid %> < listings<%=cid %>.length; m<%=cid %>++) {
if (listings<%=cid %>[m<%=cid %>].matches(mask<%=cid %>)) {
ftp_<%=cid %>.put(new ByteArrayInputStream(new byte[]{}), listings<%=cid %>[m<%=cid %>]);
globalMap.put("<%=cid %>_CURRENT_STATUS", "File truncate OK.");
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - '" + listings<%=cid %>[m<%=cid %>] + "' truncate ok.");
<%}%>
nb_file_<%=cid%>++;
}
}
}
}catch(com.enterprisedt.net.ftp.FTPException e){
globalMap.put("<%=cid %>_CURRENT_STATUS", "File truncate fail.");
throw e;
}
ftp_<%=cid %>.chdir(root<%=cid %>);
<%}%>
<%} %>

View File

@@ -464,6 +464,7 @@
log.error("<%=cid%> - " +e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<% } %>
}
java.util.zip.ZipEntry entry_<%=cid %> = null;
@@ -479,6 +480,7 @@
log.error("<%=cid%> - " +e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
break;
<% } %>
}
@@ -503,6 +505,7 @@
log.error("<%=cid%> - " +e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<% } %>
}
<%
@@ -528,6 +531,7 @@
log.error("<%=cid%> - " +e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<% } %>
}
<%
@@ -606,7 +610,9 @@
} else if(rejectConnName.equals(firstConnName)){%>
<%=rejectConnName%>.errorMessage = e.getMessage() + " - Line: " + tos_count_<%=node.getUniqueName() %>;
<%
}
} %>
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<%
}
%>
}
@@ -1076,6 +1082,7 @@
log.error("<%=cid%> - " + e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<% } %>
}
java.util.zip.ZipEntry entry_<%=cid %> = null;
@@ -1091,6 +1098,7 @@
log.error("<%=cid%> - " + e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
break;
<% } %>
}
@@ -1285,6 +1293,7 @@
log.error("<%=cid%> - " +e.getMessage());
<%}%>
System.err.println(e.getMessage());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<% } %>
}//TD110 end
@@ -1388,7 +1397,9 @@
%>
<%=rejectConnName%>.errorMessage = e.getMessage() + " - Line: " + tos_count_<%=node.getUniqueName() %>;
<%
}
} %>
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<%
}
%>
}

View File

@@ -213,7 +213,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talen File Enhanced"
MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar" REQUIRED="true" />
MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar" REQUIRED="true" />
<IMPORT NAME="Talend_CSV" MODULE="talendcsv.jar" MVN="mvn:org.talend.libraries/talendcsv/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.csv/lib/talendcsv.jar"
REQUIRED="true" />
</IMPORTS>

View File

@@ -131,7 +131,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talend File Enhanced" MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar" REQUIRED="true" />
<IMPORT NAME="Talend File Enhanced" MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -118,7 +118,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talen File Enhanced"
MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar" REQUIRED="true" />
MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar" REQUIRED="true" />
<IMPORT NAME="Talend_CSV" MODULE="talendcsv.jar" MVN="mvn:org.talend.libraries/talendcsv/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.csv/lib/talendcsv.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -81,11 +81,11 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talen File Enhanced" MODULE="talend_file_enhanced_20070724.jar" REQUIRED="true" />
<IMPORT NAME="Talen File Enhanced" MODULE="talend_file_enhanced-1.1.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>
<RETURNS>
<RETURN NAME="NB_LINE" TYPE="id_Integer" AVAILABILITY="AFTER" />
</RETURNS>
</COMPONENT>
</COMPONENT>

View File

@@ -39,7 +39,7 @@
String skipEmptyRows = "true"; //("true").equals(ElementParameterParser.getValue(node, "__SKIP_EMPTY_ROWS__"));
boolean trimAll = ("true").equals(ElementParameterParser.getValue(node,"__TRIMALL__"));
String encoding = ElementParameterParser.getValue(node,"__ENCODING__");
//need to process rows longger than 100,000 characters, the property SafetySwitch(in talend_file_enhanced_20070724.jar) should be sent to false.(the default is true)
//need to process rows longger than 100,000 characters, the property SafetySwitch(in talend_file_enhanced-1.1.jar) should be sent to false.(the default is true)
//that means if check the option(true), the logic value of bSafetySwitch should be changed to false (negate the property)
boolean bSafetySwitch = !(("true").equals(ElementParameterParser.getValue(node, "__PROCESS_LONG_ROW__")));
String advancedSeparatorStr = ElementParameterParser.getValue(node, "__ADVANCED_SEPARATOR__");

View File

@@ -128,7 +128,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talen File Enhanced" MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar" REQUIRED="true" />
<IMPORT NAME="Talen File Enhanced" MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -209,8 +209,6 @@
javax.mail.Multipart mp<%=cid%> = (javax.mail.Multipart) msg_<%=cid %>.getContent();
String attachfileName<%=cid%> = "";
String path<%=cid%> = "";
java.io.BufferedOutputStream out<%=cid%> = null;
java.io.BufferedInputStream in<%=cid%> = null;
for (int i = 0; i < mp<%=cid%>.getCount(); i++) {
javax.mail.BodyPart mpart<%=cid%> = mp<%=cid%>.getBodyPart(i);
String disposition<%=cid%> = mpart<%=cid%>.getDisposition();
@@ -231,18 +229,18 @@
}
path<%=cid%> = path<%=cid%> + attachfileName<%=cid%>;
<% if(isLog4jEnabled){ %>
log.info("<%= cid %> - Extracted attachment: '"+attachfileName<%=cid%>+"'.");
log.info("<%= cid %> - Extracting attachment: '"+attachfileName<%=cid%>+"'.");
<% } %>
java.io.File attachFile = new java.io.File(path<%=cid%>);
out<%=cid%> = new java.io.BufferedOutputStream(new java.io.FileOutputStream(attachFile));
in<%=cid%> = new java.io.BufferedInputStream(mpart<%=cid%>.getInputStream());
int buffer<%=cid%> = 0;
while ((buffer<%=cid%> = in<%=cid%>.read()) != -1) {
out<%=cid%>.write(buffer<%=cid%>);
out<%=cid%>.flush();
}
out<%=cid%>.close();
in<%=cid%>.close();
try (java.io.BufferedOutputStream out = new java.io.BufferedOutputStream(new java.io.FileOutputStream(attachFile));
java.io.BufferedInputStream in = new java.io.BufferedInputStream(mpart<%=cid%>.getInputStream())){
byte[] buffer = new byte[8192];
int bytesRead = 0;
while ((bytesRead = in.read(buffer)) > 0) {
out.write(buffer, 0, bytesRead);
}
out.flush();
}
}
}
}

View File

@@ -68,7 +68,7 @@
String dieOnErrorStr = ElementParameterParser.getValue(node, "__DIE_ON_ERROR__");
boolean dieOnError = (dieOnErrorStr!=null&&!("").equals(dieOnErrorStr))?("true").equals(dieOnErrorStr):false;
//need to process rows longger than 100 000 characters, the property SafetySwitch(in talend_file_enhanced_20070724.jar) should be sent to false.(the default is true)
//need to process rows longger than 100 000 characters, the property SafetySwitch(in talend_file_enhanced-1.1.jar) should be sent to false.(the default is true)
//that means if check the option(true), the logic value of bSafetySwitch should be changed to false (XOR with 'true')
boolean bSafetySwitch = (("true").equals(ElementParameterParser.getValue(node, "__PROCESS_LONG_ROW__")) ^ true);
String advancedSeparatorStr = ElementParameterParser.getValue(node, "__ADVANCED_SEPARATOR__");

View File

@@ -258,7 +258,7 @@
<IMPORTS>
<IMPORT
NAME="Talen File Enhanced"
MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar"
MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar"
REQUIRED="true"
/>
</IMPORTS>

View File

@@ -175,7 +175,7 @@
<IMPORTS>
<IMPORT
NAME="Talen File Enhanced"
MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar"
MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar"
REQUIRED="true"
/>
</IMPORTS>

View File

@@ -54,7 +54,8 @@ List<Map<String, String>> autoSizeList = (List<Map<String,String>>)ElementParame
String advancedSeparatorStr = ElementParameterParser.getValue(node, "__ADVANCED_SEPARATOR__");
boolean advancedSeparator = (advancedSeparatorStr!=null&&!("").equals(advancedSeparatorStr))?("true").equals(advancedSeparatorStr):false;
String thousandsSeparator = ElementParameterParser.getValueWithJavaType(node, "__THOUSANDS_SEPARATOR__", JavaTypesManager.CHARACTER);
String decimalSeparator = ElementParameterParser.getValueWithJavaType(node, "__DECIMAL_SEPARATOR__", JavaTypesManager.CHARACTER);
String decimalSeparator = ElementParameterParser.getValueWithJavaType(node, "__DECIMAL_SEPARATOR__", JavaTypesManager.CHARACTER);
boolean isTruncateExceedingCharacters = ("true").equals(ElementParameterParser.getValue(node, "__TRUNCATE_EXCEEDING_CHARACTERS__" ));
Map<String,String> autColSizeMaping=new HashMap<String,String>();
List< ? extends IConnection> conns = node.getIncomingConnections();
@@ -760,6 +761,7 @@ if(!hasDynamic && isIncludeHeader){
%>
int nb_line_<%=cid%> = 0;
org.talend.ExcelTool xlsxTool_<%=cid%> = new org.talend.ExcelTool();
xlsxTool_<%=cid%>.setTruncateExceedingCharacters(<%=isTruncateExceedingCharacters%>);
<%
if(flushOnRow && (useStream || !isAppendFile)){
%>

View File

@@ -193,6 +193,10 @@
SHOW_IF="(ADVANCED_SEPARATOR == 'true')" REPOSITORY_VALUE="DECIMAL_SEPARATOR">
<DEFAULT>"."</DEFAULT>
</PARAMETER>
<PARAMETER NAME="TRUNCATE_EXCEEDING_CHARACTERS" FIELD="CHECK" REQUIRED="true" NUM_ROW="49"
SHOW_IF="(VERSION_2007=='true')">
<DEFAULT>false</DEFAULT>
</PARAMETER>
<PARAMETER
NAME="ENCODING"
@@ -220,7 +224,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Java_Excel" MODULE="jxl.jar" MVN="mvn:org.talend.libraries/jxl/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.jexcel/lib/jxl.jar" REQUIRED_IF="(VERSION_2007 == 'false')" />
<IMPORT NAME="TALEND_EXCEL" MODULE="talendExcel-1.5-20190731.jar" MVN="mvn:org.talend.libraries/talendExcel-1.5-20190731/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/talendExcel-1.5-20190731.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="TALEND_EXCEL" MODULE="talendExcel-1.6-20210127.jar" MVN="mvn:org.talend.libraries/talendExcel-1.6-20210127/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.excel/lib/talendExcel-1.6-20210127.jar" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="commons-lang3" MODULE="commons-lang3-3.8.1.jar" MVN="mvn:org.apache.commons/commons-lang3/3.8.1" REQUIRED_IF="(VERSION_2007 == 'true')" />
<IMPORT NAME="dom4j" MODULE="dom4j-1.6.1.jar" MVN="mvn:dom4j/dom4j/1.6.1" UrlPath="platform:/plugin/org.talend.libraries.dom4j-jaxen/lib/dom4j-1.6.1.jar" REQUIRED_IF="(VERSION_2007 == 'true')" BundleID="" />
<IMPORT NAME="geronimo-stax-api" MODULE="geronimo-stax-api_1.0_spec-1.0.1.jar" MVN="mvn:org.talend.libraries/geronimo-stax-api_1.0_spec-1.0.1/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.apache.axis2/lib/geronimo-stax-api_1.0_spec-1.0.1.jar" REQUIRED_IF="(VERSION_2007 == 'true')" BundleID="" />

View File

@@ -88,4 +88,5 @@ VERSION_2007.NAME=Write excel2007 file format(xlsx)
FLUSHONROW.NAME=Custom the flush buffer size
FLUSHONROW_NUM.NAME=Row number
RECALCULATE_FORMULA.NAME=Recalculate formula
RECALCULATE_FORMULA.NAME=Recalculate formula
TRUNCATE_EXCEEDING_CHARACTERS.NAME=Truncate characters exceeding max cell length

View File

@@ -79,7 +79,7 @@
<CODEGENERATION>
<IMPORTS>
<IMPORT NAME="Talen File Enhanced" MODULE="talend_file_enhanced_20070724.jar" MVN="mvn:org.talend.libraries/talend_file_enhanced_20070724/6.0.0" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced_20070724.jar" REQUIRED="true" />
<IMPORT NAME="Talen File Enhanced" MODULE="talend_file_enhanced-1.1.jar" MVN="mvn:org.talend.components.lib/talend_file_enhanced/1.1" UrlPath="platform:/plugin/org.talend.libraries.custom/lib/talend_file_enhanced-1.1.jar" REQUIRED="true" />
</IMPORTS>
</CODEGENERATION>

View File

@@ -294,6 +294,7 @@
<ITEM NAME="Object" VALUE="Object"/>
<ITEM NAME="Short" VALUE="Short"/>
<ITEM NAME="String" VALUE="String"/>
<ITEM NAME="NString" VALUE="NString"/>
<ITEM NAME="Time" VALUE="Time"/>
<ITEM NAME="Null" VALUE="Null"/>
</ITEMS>

View File

@@ -51,6 +51,7 @@ SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Long=Long
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Object=Object
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Short=Short
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.String=String
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.NString=NString
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Time=Time
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_TYPE.ITEM.Null=Null
SET_PREPAREDSTATEMENT_PARAMETERS.ITEM.PARAMETER_VALUE=Parameter Value

View File

@@ -27452,7 +27452,7 @@
<ITEM NAME="LESSTHAN" VALUE="LessThan" />
<ITEM NAME="GREATEREQUAL" VALUE="GreaterEqual" />
<ITEM NAME="LESSEQUAL" VALUE="LessEqual" />
<ITEM NAME="LIKE" VALUE="Like" SHOW_IF="((AUTH_TYPE=='ONLINE') AND (API_VERSION != 'API_2016_ODATA' OR API_VERSION != 'API_2018_ODATA')) OR ((AUTH_TYPE=='ON_PREMISE') AND (MS_CRM_VERSION != 'CRM_2016' OR MS_CRM_VERSION != 'CRM_2018'))"/>
<ITEM NAME="LIKE" VALUE="Like" SHOW_IF="((AUTH_TYPE=='ONLINE' AND (API_VERSION != 'API_2016_ODATA' AND API_VERSION != 'API_2018_ODATA')) OR (AUTH_TYPE=='ON_PREMISE' AND (MS_CRM_VERSION != 'CRM_2016' AND MS_CRM_VERSION != 'CRM_2018')))"/>
</ITEMS>
</ITEM>
<ITEM NAME="RVALUE" FIELD="String" />

View File

@@ -54,11 +54,11 @@ for(IConnection conn : outgoingConns) {
<%
}
}
log4jCodeGenerateUtil.query(node);
%>
query_<%=cid %> = <%=dbquery%>;
whetherReject_<%=cid%> = false;
<%
log4jCodeGenerateUtil.query(node, "query_" + cid);
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
@@ -131,7 +131,7 @@ try {
<%
}
}
log4jCodeGenerateUtil.logInfo(node,"info",cid+" - Execute the query: '\" + "+dbquery +" + \"' has finished.");
log4jCodeGenerateUtil.logInfo(node,"info",cid+" - Execute the query: '\" + query_" + cid + " + \"' has finished.");
%>
<% //feature 0010425
if(usePrepareStatement){

View File

@@ -185,6 +185,10 @@ String inputConnName = null;
if (i == 0){
%>
paraList_<%=cid %>.add("<%=commandLineWindows[i]%>");
String m2 = System.getProperty("talend.component.manager.m2.repository");
if (m2 != null){
paraList_<%=cid %>.add("-Dtalend.component.manager.m2.repository=" + m2);
}
<%
} else if (i > 0){
if (commandLineWindows[i].indexOf("\"") >= 0){
@@ -217,6 +221,10 @@ String inputConnName = null;
if (i == 0){
%>
paraList_<%=cid %>.add("<%=commandLineUnix[i]%>");
String m2 = System.getProperty("talend.component.manager.m2.repository");
if (m2 != null){
paraList_<%=cid %>.add("-Dtalend.component.manager.m2.repository=" + m2);
}
<%
} else if (i > 0){
String param;

View File

@@ -220,6 +220,7 @@ boolean isLog4jEnabled = ("true").equals(ElementParameterParser.getValue(node.ge
log.error("<%=cid%> - " + e.toString());
<%}%>
System.err.println(e.toString());
globalMap.put("<%=cid %>_ERROR_MESSAGE", e.getMessage());
<%}%>
}finally{
props_<%=cid %>.remove("mail.smtp.host");

View File

@@ -105,7 +105,7 @@ if(useExistingConnection) {
}
else {
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + host_<%=cid%> + ":" + port_<%=cid%> + ";DatabaseName" + db_<%=cid%> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + host_<%=cid%> + ":" + port_<%=cid%> + ";DatabaseName=" + db_<%=cid%> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -20,7 +20,7 @@ imports="
}
else{
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -36,7 +36,7 @@ if(useExistingConn) {
}
else{
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=host %> + ":" + <%=port %> + ";DatabaseName" + <%=dbName %> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=host %> + ":" + <%=port %> + ";DatabaseName=" + <%=dbName %> + ";" + <%=dbproperties%>;
<%
}
%>

View File

@@ -29,7 +29,7 @@ imports="
}
else{
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbServer %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbServer %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -188,7 +188,7 @@ if(useExistingConn){
}
else{
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbServer %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbServer %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -26,7 +26,7 @@ imports="
}
else{
%>
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbServer %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String url_<%=cid%> = "jdbc:sqlanywhere:Host=" + <%=dbServer %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -85,7 +85,7 @@ if(inputSchema != null && inputSchema.size() > 0 && outputSchema != null && outp
}
else{
%>
String connectionString_<%=cid %> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String connectionString_<%=cid %> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -60,7 +60,7 @@ if(columnList != null && columnList.size() > 0) {
}
else{
%>
String url_<%=cid %> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String url_<%=cid %> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -60,7 +60,7 @@ if(useExistingConn){
}
else{
%>
String connectionString_<%=cid %> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName" + <%=dbname %> + ";" + <%=dbproperties%>;
String connectionString_<%=cid %> = "jdbc:sqlanywhere:Host=" + <%=dbhost %> + ":" + <%=dbport %> + ";DatabaseName=" + <%=dbname %> + ";" + <%=dbproperties%>;
<%
}
}

View File

@@ -3,6 +3,7 @@ imports="
org.talend.core.model.process.INode
org.talend.core.model.process.ElementParameterParser
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.utils.NodeUtil
"
%>
@@ -17,7 +18,15 @@ String operator = ElementParameterParser.getValue(node,"__OPERATOR__");
String value = ElementParameterParser.getValue(node,"__VALUE__");
boolean isEnableWhereClause = ("true").equals(ElementParameterParser.getValue(node,"__ENABLE_WHERE_CLAUSE__"));
boolean isEnableSQLStmt = ("true").equals(ElementParameterParser.getValue(node,"__ENABLE_SQL_STMT__"));
String conn = "conn_" + connection ;
final String conn;
INode connNode = NodeUtil.getNodeByUniqueName(node.getProcess(), connection);
if((connNode!=null) && "tSnowflakeConnection".equals(connNode.getComponent().getName())){
conn = connection + "_connection";
}else{
conn = "conn_" + connection;
}
String query = "SELECT COUNT(*) FROM ";
%>

View File

@@ -77,9 +77,7 @@ imports="
<%dbInputBeginUtil.createStatement(node);%>
String dbquery_<%=cid%> = <%=dbInputBeginUtil.getQueryString(node)%>;
<%if(isLog4jEnabled){%>
log.debug("<%=cid%> - Executing the query: '"+dbquery_<%=cid%>+"'.");
<%}%>
<% log4jCodeGenerateUtil.query(node, "dbquery_" + cid); %>
<%
if(isAmazonAurora){

View File

@@ -55,11 +55,11 @@ for(IConnection conn : outgoingConns) {
<%
}
}
log4jCodeGenerateUtil.query(node);
%>
query_<%=cid %> = <%=dbquery%>;
whetherReject_<%=cid%> = false;
<%
log4jCodeGenerateUtil.query(node, "query_" + cid);
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
@@ -128,7 +128,7 @@ try {
<%
}
}
log4jCodeGenerateUtil.logInfo(node,"debug",cid+" - Execute the query: '\" + "+dbquery +" + \"' has finished.");
log4jCodeGenerateUtil.logInfo(node,"debug",cid+" - Execute the query: '\" + query_" + cid + " + \"' has finished.");
%>
<%
if(usePrepareStatement){

View File

@@ -0,0 +1,25 @@
<%@ jet
%>
ftp_<%=cid %>.setControlEncoding(<%=encoding %>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user %>+ "'.");
<%}%>
ftp_<%=cid %>.connect(<%=host %>,<%=portToConnect%>);
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
<%
passwordFieldName = cid.contains("Connection") ? "__PASS__" : "__PASSWORD__";
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
boolean isLoginSuccessful_<%=cid %> = ftp_<%=cid %>.login(<%=user %>, decryptedPassword_<%=cid%>);
if (!isLoginSuccessful_<%=cid %>) {
throw new RuntimeException("Login failed");
}
ftp_<%=cid %>.setFileType(org.apache.commons.net.ftp.FTP.BINARY_FILE_TYPE);

View File

@@ -0,0 +1,62 @@
<%@ jet
%>
<%
String portToConnect = port;
%>
try {
if(("true").equals(System.getProperty("http.proxySet")) ){
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.apache.commons.net.ftp.FTPHTTPClient(httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else if ("local".equals(System.getProperty("http.proxySet"))) {
String uriString = <%=host %> + ":" + <%=ftpsPort%>;
java.net.Proxy proxyToUse = org.talend.proxy.TalendProxySelector.getInstance().getProxyForUriString(uriString);
if (!proxyToUse.equals(java.net.Proxy.NO_PROXY)) {
java.net.InetSocketAddress proxyAddress = (java.net.InetSocketAddress) proxyToUse.address();
String httpProxyHost = proxyAddress.getAddress().getHostAddress();
int httpProxyPort = proxyAddress.getPort();
String httpProxyUser = "";
String httpProxyPass = ""; //leave it empty if proxy creds weren't specified
org.talend.proxy.ProxyCreds proxyCreds = org.talend.proxy.TalendProxyAuthenticator.getInstance().getCredsForProxyURI(httpProxyHost + ":" + httpProxyPort);
if (proxyCreds != null) {
httpProxyUser = proxyCreds.getUser();
httpProxyPass = proxyCreds.getPass();
}
ftp_<%=cid %> = new org.apache.commons.net.ftp.FTPHTTPClient(httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else { //no http proxy for ftp host defined
ftp_<%=cid %> = new org.apache.commons.net.ftp.FTPClient();
}
} else {
ftp_<%=cid %> = new org.apache.commons.net.ftp.FTPClient();
}
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/commons_net_create_connection.javajet"%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
%>

View File

@@ -0,0 +1,92 @@
<%@ jet
%>
<%
String portToConnect = ftpsPort;
%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/ftpsTrust.javajet"%>
javax.net.ssl.SSLContext sslContext_<%=cid %> = null;
javax.net.ssl.TrustManager[] trustManager_<%=cid %> = null;
javax.net.ssl.SSLSocketFactory sslSocketFactory_<%=cid %> = null;
MyTrust_<%=cid%> myTrust_<%=cid%> = null;
try {
sslContext_<%=cid %> = javax.net.ssl.SSLContext.getInstance("SSL");
myTrust_<%=cid%> = new MyTrust_<%=cid%>();
trustManager_<%=cid %> = myTrust_<%=cid%>.getTrustManagers();
sslContext_<%=cid %>.init(null, trustManager_<%=cid %>, new java.security.SecureRandom());
sslSocketFactory_<%=cid %> = sslContext_<%=cid %>.getSocketFactory();
if(("true").equals(System.getProperty("http.proxySet")) ){
String httpProxyHost = System.getProperty("http.proxyHost");
int httpProxyPort = Integer.getInteger("http.proxyPort");
String httpProxyUser = System.getProperty("http.proxyUser");
String httpProxyPass = System.getProperty("http.proxyPassword");
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else if ("local".equals(System.getProperty("http.proxySet"))) {
String uriString = <%=host %> + ":" + <%=ftpsPort%>;
java.net.Proxy proxyToUse = org.talend.proxy.TalendProxySelector.getInstance().getProxyForUriString(uriString);
if (!proxyToUse.equals(java.net.Proxy.NO_PROXY)) {
java.net.InetSocketAddress proxyAddress = (java.net.InetSocketAddress) proxyToUse.address();
String httpProxyHost = proxyAddress.getAddress().getHostAddress();
int httpProxyPort = proxyAddress.getPort();
String httpProxyUser = "";
String httpProxyPass = ""; //leave it empty if proxy creds weren't specified
org.talend.proxy.ProxyCreds proxyCreds = org.talend.proxy.TalendProxyAuthenticator.getInstance().getCredsForProxyURI(httpProxyHost + ":" + httpProxyPort);
if (proxyCreds != null) {
httpProxyUser = proxyCreds.getUser();
httpProxyPass = proxyCreds.getPass();
}
ftp_<%=cid %> = new org.talend.ftp.HTTPProxyFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>, httpProxyHost, httpProxyPort, httpProxyUser, httpProxyPass);
} else { //no http proxy for ftp host defined
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
} else {
ftp_<%=cid %> = new org.talend.ftp.SSLSessionReuseFTPSClient(<%="IMPLICIT".equals(securityMode)%>, sslContext_<%=cid %>);
}
<%if(isLog4jEnabled){%>
log.info("<%=cid%> -FTPS security Mode is <%=securityMode%>.");
<%}%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/FTP/commons_net_create_connection.javajet"%>
ftp_<%=cid %>.setRemoteVerificationEnabled(<%=useRemoteVerification %>);
<%
if (protectionBufferSize != null && !protectionBufferSize.isEmpty() ) {
%>
ftp_<%=cid %>.execPBSZ(<%=protectionBufferSize %>);
<%
}
if (protectionLevel != null && !protectionLevel.isEmpty()) {
%>
ftp_<%=cid %>.execPROT(<%=protectionLevel %>);
<%
}
%>
} catch (Exception e) {
<%if(isLog4jEnabled){%>
log.error("<%=cid%> - Can't create connection: " + e.getMessage());
<%}%>
throw e;
}
<%
if ("PASSIVE".equals(connectMode)) {
%>
ftp_<%=cid %>.enterLocalPassiveMode();
<%
if (isLog4jEnabled) {
%>
log.debug("<%=cid%> - Using the passive mode.");
<%
}
}
%>

View File

@@ -0,0 +1,106 @@
<%@ jet
%>
boolean retry_<%=cid%> = false;
int retry_count_<%=cid%> = 0;
int retry_max_<%=cid%> = 5;
com.jcraft.jsch.Session session_<%=cid%> = null;
com.jcraft.jsch.Channel channel_<%=cid%> = null;
do {
retry_<%=cid%> = false;
com.jcraft.jsch.JSch jsch_<%=cid%> = new com.jcraft.jsch.JSch();
<%if (("PUBLICKEY").equals(authMethod)){%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - SFTP authentication using a public key.");
log.debug("<%=cid%> - Private key: '" + <%=privateKey%> + "'.");
<%}%>
jsch_<%=cid%>.addIdentity(<%=privateKey %>, defaultUserInfo_<%=cid%>.getPassphrase());
<%}%>
session_<%=cid%> = jsch_<%=cid%>.getSession(<%=user%>, <%=host%>, <%=port%>);
session_<%=cid%>.setConfig("PreferredAuthentications", "publickey,password,keyboard-interactive,gssapi-with-mic");
<%if (("PASSWORD").equals(authMethod)) {%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - SFTP authentication using a password.");
<%}%>
<%@ include file="@{org.talend.designer.components.localprovider}/components/templates/password.javajet"%>
session_<%=cid%>.setPassword(decryptedPassword_<%=cid%>);
<%}%>
session_<%=cid%>.setUserInfo(defaultUserInfo_<%=cid%>);
<%if (!useProxy) {%>
if(("true").equals(System.getProperty("http.proxySet"))) {
com.jcraft.jsch.ProxyHTTP proxy_<%=cid%> = new com.jcraft.jsch.ProxyHTTP(System.getProperty("http.proxyHost"),Integer.parseInt(System.getProperty("http.proxyPort")));
if(!"".equals(System.getProperty("http.proxyUser"))){
proxy_<%=cid%>.setUserPasswd(System.getProperty("http.proxyUser"),System.getProperty("http.proxyPassword"));
}
session_<%=cid%>.setProxy(proxy_<%=cid%>);
} else if ("local".equals(System.getProperty("http.proxySet"))) {
String uriString = <%=host %> + ":" + <%=port%>;
java.net.Proxy proxyToUse = org.talend.proxy.TalendProxySelector.getInstance().getProxyForUriString(uriString);
if (!proxyToUse.equals(java.net.Proxy.NO_PROXY)) {
java.net.InetSocketAddress proxyAddress = (java.net.InetSocketAddress) proxyToUse.address();
String proxyHost = proxyAddress.getAddress().getHostAddress();
int proxyPort = proxyAddress.getPort();
com.jcraft.jsch.ProxyHTTP proxy_<%=cid%> = new com.jcraft.jsch.ProxyHTTP(proxyHost, proxyPort);
org.talend.proxy.ProxyCreds proxyCreds = org.talend.proxy.TalendProxyAuthenticator.getInstance().getCredsForProxyURI(proxyHost + ":" + proxyPort);
if (proxyCreds != null) {
proxy_<%=cid%>.setUserPasswd(proxyCreds.getUser(), proxyCreds.getPass());
}
session_<%=cid%>.setProxy(proxy_<%=cid%>);
}
}
<%}%>
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Attempt to connect to '" + <%=host %> + "' with username '" + <%=user%> + "'.");
<%}%>
channel_<%=cid%> = null;
try {
session_<%=cid%>.connect();
channel_<%=cid%> = session_<%=cid%>.openChannel("sftp");
channel_<%=cid%>.connect();
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - Connect to '" + <%=host %> + "' has succeeded.");
<%}%>
} catch (com.jcraft.jsch.JSchException e_<%=cid%>) {
try {
if(channel_<%=cid%>!=null) {
channel_<%=cid%>.disconnect();
}
if(session_<%=cid%>!=null) {
session_<%=cid%>.disconnect();
}
} catch(java.lang.Exception ce_<%=cid%>) {
<%if(isLog4jEnabled){%>
log.warn("<%=cid%> - close sftp connection failed : " + ce_<%=cid%>.getClass() + " : " + ce_<%=cid%>.getMessage());
<%}%>
}
String message_<%=cid%> = new TalendException(null, null, null).getExceptionCauseMessage(e_<%=cid%>);
if(message_<%=cid%>.contains("Signature length not correct") || message_<%=cid%>.contains("connection is closed by foreign host")) {
retry_<%=cid%> = true;
retry_count_<%=cid%>++;
<%if(isLog4jEnabled){%>
log.info("<%=cid%> - connect: Signature length not correct or connection is closed by foreign host, so retry, retry time : " + retry_count_<%=cid%>);
<%}%>
} else {
throw e_<%=cid%>;
}
}
} while(retry_<%=cid%> && (retry_count_<%=cid%> < retry_max_<%=cid%>));
com.jcraft.jsch.ChannelSftp c_<%=cid%> = (com.jcraft.jsch.ChannelSftp)channel_<%=cid%>;

View File

@@ -172,6 +172,16 @@ imports="
<%
}
public void query(INode node, String dbQueryVariableName){
beforeComponentProcess(node);
//for input
logInfo(node,"debug",cid+" - Executing the query: '\" + "+dbQueryVariableName +" + \"'.");
}
/**
* @deprecated please use another method instead: query(INode node, String dbQueryVariableName) because execution of the query expression can be not idempotent
*/
@Deprecated
public void query(INode node){
beforeComponentProcess(node);
//for input

View File

@@ -415,12 +415,13 @@ public class CLASS {
}
createSQL.append(suffix);
} else {
boolean isSpecialDBAndNotSetLength = (POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId())) && column.getColumn().getLength() == null;
if (!lengthIgnored) {
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
if (isSpecialDBAndNotSetLength) {
} else {
prefix = "(";
suffix = ") ";
createSQL.append(prefix + length);
createSQL.append(prefix + length);
}
}
if (!precisionIgnored) {
@@ -432,9 +433,11 @@ public class CLASS {
} else {
comma = ",";
}
createSQL.append(comma + precision);
if(!isSpecialDBAndNotSetLength) {
createSQL.append(comma + precision);
}
}
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
if (isSpecialDBAndNotSetLength) {
} else {
createSQL.append(suffix);
}
@@ -1588,12 +1591,9 @@ public class CLASS {
createSQL.append(suffix);
} else {
if (!lengthIgnored) {
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
prefix = "(";
suffix = ") ";
createSQL.append(prefix + length);
}
prefix = "(";
suffix = ") ";
createSQL.append(prefix + length);
}
if (!precisionIgnored) {
prefix = (prefix.equals("") ? "(" : prefix);
@@ -1606,10 +1606,9 @@ public class CLASS {
}
createSQL.append(comma + precision);
}
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
createSQL.append(suffix);
}
createSQL.append(suffix);
if(MYSQL.equalsIgnoreCase(getDBMSId()) && dataType.endsWith("UNSIGNED")) {
createSQL.append("UNSIGNED");
}
@@ -2190,12 +2189,9 @@ public class CLASS {
createSQL.append(suffix);
} else {
if (!lengthIgnored) {
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
prefix = "(";
suffix = ") ";
createSQL.append(prefix + getLengthStringRepresentation(dataType, length));
}
prefix = "(";
suffix = ") ";
createSQL.append(prefix + getLengthStringRepresentation(dataType, length));
}
if (!precisionIgnored) {
prefix = (prefix.equals("") ? "(" : prefix);
@@ -2208,10 +2204,9 @@ public class CLASS {
}
createSQL.append(comma + precision);
}
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
createSQL.append(suffix);
}
createSQL.append(suffix);
if(MYSQL.equalsIgnoreCase(getDBMSId()) && dataType.endsWith("UNSIGNED")) {
createSQL.append("UNSIGNED");
}
@@ -3044,7 +3039,7 @@ public class CLASS {
createSQL.append(suffix);
} else {
if (!lengthIgnored) {
if (column.getColumn().getLength() == null) {
if (column.getColumn().getLength() == null) {
} else {
prefix = "(";
suffix = ") ";
@@ -3060,7 +3055,10 @@ public class CLASS {
} else {
comma = ",";
}
createSQL.append(comma + precision);
if(column.getColumn().getLength() != null) {
createSQL.append(comma + precision);
}
}
if (column.getColumn().getLength() == null) {
} else {
@@ -3739,12 +3737,9 @@ public class CLASS {
createSQL.append(suffix);
} else {
if (!lengthIgnored) {
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
prefix = "(";
suffix = ") ";
createSQL.append(prefix + length);
}
prefix = "(";
suffix = ") ";
createSQL.append(prefix + length);
}
if (!precisionIgnored) {
prefix = (prefix.equals("") ? "(" : prefix);
@@ -3757,10 +3752,9 @@ public class CLASS {
}
createSQL.append(comma + precision);
}
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
createSQL.append(suffix);
}
createSQL.append(suffix);
if(MYSQL.equalsIgnoreCase(getDBMSId()) && dataType.endsWith("UNSIGNED")) {
createSQL.append("UNSIGNED");
}
@@ -4126,7 +4120,9 @@ public class CLASS {
}
createSQL.append(comma + precision);
}
createSQL.append(suffix);
createSQL.append(suffix);
if(MYSQL.equalsIgnoreCase(getDBMSId()) && dataType.endsWith("UNSIGNED")) {
createSQL.append("UNSIGNED");
}
@@ -4597,12 +4593,9 @@ public class CLASS {
createSQL.append(suffix);
} else {
if (!lengthIgnored) {
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
prefix = "(";
suffix = ") ";
createSQL.append(prefix + getLengthStringRepresentation(dataType, length));
}
prefix = "(";
suffix = ") ";
createSQL.append(prefix + getLengthStringRepresentation(dataType, length));
}
if (!precisionIgnored) {
prefix = (prefix.equals("") ? "(" : prefix);
@@ -4613,12 +4606,10 @@ public class CLASS {
} else {
comma = ",";
}
createSQL.append(comma + precision);
}
if ((POSTGRESQL.equalsIgnoreCase(getDBMSId()) || POSTGREPLUS.equalsIgnoreCase(getDBMSId()) ) && column.getColumn().getLength() == null) {
} else {
createSQL.append(suffix);
}
createSQL.append(suffix);
if(MYSQL.equalsIgnoreCase(getDBMSId()) && dataType.endsWith("UNSIGNED")) {
createSQL.append("UNSIGNED");
}

View File

@@ -24,6 +24,8 @@
<talendMQConnectionUtil.dir>${project.basedir}/components/tMomConnection</talendMQConnectionUtil.dir>
<talendMQConnectionUtil.version>1.0.1-20190215</talendMQConnectionUtil.version>
<tldapinput.dir>${project.basedir}/components/tLDAPInput</tldapinput.dir>
<tproxy.dir>${project.basedir}/components/tSetProxy</tproxy.dir>
<tproxy.version>1.0.1</tproxy.version>
</properties>
<build>
@@ -156,6 +158,14 @@
<overWrite>true</overWrite>
<outputDirectory>${tldapinput.dir}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.talend.components.lib</groupId>
<artifactId>talend-proxy</artifactId>
<version>${tproxy.version}</version>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${tproxy.dir}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>

View File

@@ -158,6 +158,21 @@ if(hasBox){
return folder;
}
private static com.box.boxjavalibv2.dao.BoxItem getBoxItemRecursively(List<String> pathItems, com.box.boxjavalibv2.dao.BoxItem item, com.box.boxjavalibv2.BoxClient client) throws Exception {
if (pathItems.isEmpty()) {
return item;
} else {
String itemName = pathItems.remove(0);
com.box.boxjavalibv2.dao.BoxFolder folder = (com.box.boxjavalibv2.dao.BoxFolder) item;
for (com.box.boxjavalibv2.dao.BoxTypedObject obj : folder.getItemCollection().getEntries()) {
if (obj.getValue("name").equals(itemName)) {
return getBoxItemRecursively(pathItems, client.getBoxItemsManager().getItem(obj.getId(), null, com.box.boxjavalibv2.dao.BoxResourceType.valueOf(obj.getType().toUpperCase())), client);
}
}
throw new Exception("The item " + itemName + " was not found in the directory " + folder.getName());
}
}
private java.util.List<com.box.boxjavalibv2.dao.BoxItem> getChildrenRecursively(com.box.boxjavalibv2.BoxClient client, com.box.boxjavalibv2.dao.BoxFolder folder,
java.util.List<com.box.boxjavalibv2.dao.BoxItem> list, boolean withFolders, boolean withFiles) throws Exception

View File

@@ -15,8 +15,10 @@ package org.talend.designer.core.generic.controller;
import java.beans.PropertyChangeEvent;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import org.apache.commons.lang.StringUtils;
@@ -41,6 +43,7 @@ import org.talend.components.api.properties.ComponentReferenceProperties;
import org.talend.core.model.process.EParameterFieldType;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.INode;
import org.talend.core.model.process.IReplaceNodeHandler;
import org.talend.core.model.utils.TalendTextUtils;
import org.talend.core.ui.properties.tab.IDynamicProperty;
import org.talend.designer.core.generic.constants.IGenericConstants;
@@ -282,12 +285,12 @@ public class ComponentRefController extends AbstractElementPropertySectionContro
INode currentNode = (INode) elem;
List<INode> refNodes = getRefNodes(param, props);
List<String> itemsLabel = new ArrayList<>();
List<String> itemsValue = new ArrayList<>();
Map<String, INode> itemsValue = new LinkedHashMap<>();
// First item is this component (see also createComboCommand)
// FIXME - I18N for this message
itemsLabel.add("Use this Component");
itemsValue.add(currentNode.getUniqueName());
itemsValue.put(currentNode.getUniqueName(), currentNode);
String selectedValue;
Object referenceType = props.referenceType.getValue();
if (referenceType != null && referenceType.equals(ComponentReferenceProperties.ReferenceType.COMPONENT_INSTANCE)) {
@@ -311,19 +314,20 @@ public class ComponentRefController extends AbstractElementPropertySectionContro
displayName = uniqueName + " - " + displayName; //$NON-NLS-1$
}
itemsLabel.add(displayName);
itemsValue.add(uniqueName);
itemsValue.put(uniqueName, node);
labelToValueMap.put(displayName, uniqueName);
}
List<String> itemValueList = new ArrayList<>(itemsValue.keySet());
param.setListItemsDisplayName(itemsLabel.toArray(new String[0]));
param.setListItemsDisplayCodeName(itemsLabel.toArray(new String[0]));
param.setListItemsValue(itemsValue.toArray(new String[0]));
param.setListItemsValue(itemValueList.toArray(new String[0]));
combo.setItems(itemsLabel.toArray(new String[0]));
String iLabel = null;
int selection = 0;
for (int i = 0; i < itemsValue.size(); i++) {
String iValue = itemsValue.get(i);
for (int i = 0; i < itemValueList.size(); i++) {
String iValue = itemValueList.get(i);
if ((selectedValue == null && (((INode) elem).getUniqueName()).equals(iValue))
|| (selectedValue != null && StringUtils.isNotEmpty(iValue) && iValue.endsWith(selectedValue))) {
|| isRefernceNode(currentNode, itemsValue.get(iValue), selectedValue, props)) {
iLabel = itemsLabel.get(i);
break;
}
@@ -336,6 +340,33 @@ public class ComponentRefController extends AbstractElementPropertySectionContro
}
private boolean isRefernceNode(INode selectedNode, INode checkingNode, String selectedUniqueName, ComponentReferenceProperties props) {
IReplaceNodeHandler selectedHandler = selectedNode.getReplaceNodeHandler();
IReplaceNodeHandler checkingHandler = checkingNode.getReplaceNodeHandler();
String selectedPrefix = Optional.ofNullable(selectedHandler).map(h -> h.getPrefix()).orElse("");
String checkingPrefix = Optional.ofNullable(checkingHandler).map(h -> h.getPrefix()).orElse("");
if (checkingPrefix.startsWith(selectedPrefix)) {
String selectedNameWithoutPrefix = selectedUniqueName;
String checkingUniqueNameWithoutPrefix = checkingNode.getUniqueName();
if (selectedNameWithoutPrefix.startsWith(selectedPrefix)) {
selectedNameWithoutPrefix = selectedNameWithoutPrefix.substring(selectedPrefix.length());
}
if (checkingUniqueNameWithoutPrefix.startsWith(selectedPrefix)) {
checkingUniqueNameWithoutPrefix = checkingUniqueNameWithoutPrefix.substring(selectedPrefix.length());
}
boolean isReferenceNode = StringUtils.equals(selectedNameWithoutPrefix, checkingUniqueNameWithoutPrefix);
if (isReferenceNode && selectedNode != checkingNode) {
props.componentInstanceId.setValue(selectedNameWithoutPrefix);
props.setReference(checkingNode.getComponentProperties());
}
return isReferenceNode;
} else {
return false;
}
}
private List<INode> getRefNodes(IElementParameter param, ComponentReferenceProperties props) {
callBeforeActive(param);
List<INode> refNodes = new ArrayList<>();

View File

@@ -399,6 +399,7 @@ GuessSchemaController.connectionFailed=Database connection is failed.
GuessSchemaController.connError=Connection error
GuessSchemaController.setParameter=Please set connection parameters
GuessSchemaController.waitOpenDatabase=Waiting for opening Database ...
GuessSchemaController.emptyQuery=Empty query
I18nPreferencePage.allTranslation=Import all translations
I18nPreferencePage.completeInfo=Translations loaded from Babili, these translations will be only displayed to test, on next restart of the application, these texts for translations will be lost.
I18nPreferencePage.importBabili=Import Translation from Babili

View File

@@ -382,6 +382,8 @@ public enum EParameterName {
ACTIVE_DATABASE_DELIMITED_IDENTIFIERS("Active database delimited identifiers"), //$NON-NLS-1$
USE_ALIAS_IN_OUTPUT_TABLE("Use alias in output table"), //$NON-NLS-1$
// for tDataprepRun
PREPARATION_ID("PREPARATION_ID");

View File

@@ -177,8 +177,7 @@ public final class Expression {
protected static boolean isThereCondition(String expression, String condition) {
expression = expression.toLowerCase();
if (!expression.contains(StringUtils.wrap(condition, StringUtils.SPACE))) {
// also exclude those like 'standard', "story"
if (!expression.contains(condition)) {
return false;
}
if (AND.equals(condition) && conditionMatcher.matches(expression, andPattern)) {

Some files were not shown because too many files have changed in this diff Show More