Compare commits

...

16 Commits

Author SHA1 Message Date
wwang-talend
dba3b1b0a1 fix readme 2020-08-31 16:08:50 +08:00
wwang-talend
e12b4545ee add readme 2020-08-31 10:43:04 +08:00
wang wei
95d7273962 fix(TDI-44643): NPE exception on tWaitForSQLData with snowflake
connection(#5006)
2020-08-31 10:13:38 +08:00
msheverda-talend
6cf2cae1b6 fix(TDI-42522): avoid 65535 issue (#3636) (#3893) (#4895)
Co-authored-by: wang wei <wwang@talend.com>
2020-07-30 18:08:10 +03:00
Chao MENG
604fb23a71 patch(TPS-4193): remove localprovider, make it compatible with javajet modifications (#4984)
* patch(TPS-4193): remove localprovider, make it compatible with javajet
modifications

* patch(TPS-4193): remove localprovider, make it compatible with javajet
modifications
2020-07-23 18:56:58 +08:00
apoltavtsev
e3f6f81cde TESB-29647 Bundle version for Jobs is corrected 2020-07-20 15:23:29 +02:00
Andreas Mattes
26ff46ddeb TESB-27583 TESB-29690 Ensure WSDL is included in WS consumer build. (#4966)
This fix corrects TESB-27081 and TESB-27583 ensuring SOAP Action is set from binding definition in WSDL, and TESB-2960 ensuring the WSDL is present for properly configuring the SOAP 1.2 binding. This fix is identical with the temporarily reverted fix from pull request #4926.
2020-07-20 11:25:28 +02:00
Andreas Mattes
cd955b7ad4 Revert "TESB-29690 Ensure WSDL is included in WS consumer build". 2020-07-14 11:36:46 +02:00
Andreas Mattes
dc11694fd1 TESB-29690 Ensure WSDL is included in WS consumer build. (#4926)
This fix includes fixes for TESB-27081 and TESB-27583 handling missing SOAP Action due to missing WSDL and TESB-2960 ensuring the WSDL is present for properly configuring the SOAP 1.2 binding.
2020-07-13 18:25:54 +02:00
hzhao-talendbj
b1598aca9f tup-27447 backport to patch721 (#4822) 2020-07-02 14:28:41 +08:00
hzhao-talendbj
938c1dadb2 TUP-27070 (#4855)
on patch721, eltmap don't have 'update' option  so remove this junit
from 721
2020-06-24 15:36:07 +08:00
hzhao-talendbj
ba7622b528 tup-27070 (#4848)
add this junit by mistake,it should not in branch patch721, so remove it
2020-06-23 18:18:04 +08:00
hzhao-talendbj
2cdfb08e18 tup-27070 fix junit failed on patch 7.2.1 (#4846) 2020-06-23 09:18:11 +08:00
bhe-talendbj
33fb855c55 fix(TUP-27053): backport (#4767) (#4801) 2020-06-19 15:13:02 +08:00
hzhao-talendbj
e6458bc956 Hengzhao/backport patch721/tup 27070 (#4774)
* Tup-27070 (#4734)
backport to 7.2.1 and fix conflicts

Conflicts:
	main/plugins/org.talend.designer.dbmap/src/main/java/org/talend/designer/dbmap/language/generation/DbGenerationManager.java

* TUP-27070  fix cross join cases

* Fix(TDI-43264):Fix issues. (#4182)

Co-authored-by: vyu-talend <49019399+vyu-talend@users.noreply.github.com>
2020-06-19 15:01:05 +08:00
hcyi
699ac6f1d4 fix(TUP-26939:Tmap shows null values when using Expression Builder. (#4802)
(#4632)

Conflicts:
	main/plugins/org.talend.expressionbuilder/src/main/java/org/talend/expressionbuilder/test/shadow/VirtualRowGeneratorNode.java
2020-06-19 14:55:49 +08:00
20 changed files with 1950 additions and 288 deletions

58
PATCH_RELEASE_NOTE.md Normal file
View File

@@ -0,0 +1,58 @@
---
version: 7.2.1
module: https://talend.poolparty.biz/coretaxonomy/42
product:
- https://talend.poolparty.biz/coretaxonomy/23
---
# TPS-4346
| Info | Value |
| ---------------- | ---------------- |
| Patch Name | Patch\_20200901\_TPS-4346\_v1-7.2.1 |
| Release Date | 2020-09-01 |
| Target Version | 20190620_1446-V7.2.1 |
| Product affected | Talend Studio |
## Introduction
This is a self-contained patch.
**NOTE**: For information on how to obtain this patch, reach out to your Support contact at Talend.
## Fixed issues
This patch contains the following fixes:
- TPS-4346 [7.2.1] NPE exception on tWaitForSQLData with snowflake connection (TDI-44643)
## Prerequisites
Consider the following requirements for your system:
- Talend Studio 7.2.1 must be installed.
- If already installed one of the 4 patches(TPS-4193,TPS-3855,TPS-4183,TPS-3858), please install TPS-4186 or later cumulative patch before installing current patch.
## Installation
### Installing the patch using Software update
1) Logon TAC and switch to Configuration->Software Update, then enter the correct values and save referring to the documentation: https://help.talend.com/reader/f7Em9WV_cPm2RRywucSN0Q/j9x5iXV~vyxMlUafnDejaQ
2) Switch to Software update page, where the new patch will be listed. The patch can be downloaded from here into the nexus repository.
3) On Studio Side: Logon Studio with remote mode, on the logon page the Update button is displayed: click this button to install the patch.
### Installing the patch using Talend Studio
1) Create a folder named "patches" under your studio installer directory and copy the patch .zip file to this folder.
2) Restart your studio: a window pops up, then click OK to install the patch, or restart the commandline and the patch will be installed automatically.
### Installing the patch using Commandline
Execute the following commands:
1. Talend-Studio-win-x86_64.exe -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002 --talendDebug
2. initRemote {tac_url} -ul {TAC login username} -up {TAC login password}
3. checkAndUpdate -tu {TAC login username} -tup {TAC login password}

View File

@@ -59,88 +59,76 @@ if((codePart.equals(ECodePart.END))&&(stat || logstashCurrent)){
List<String> needToEndConnNames = new ArrayList<String>();
INode nextNode = node.getOutgoingConnections(EConnectionType.ITERATE).get(0).getTarget();
NodeUtil.fillConnectionsForStat(needToEndConnNames, nextNode);
%>
<%if(stat) {%>
if(execStat){
<%
for(String connName : needToEndConnNames){
if(!needToEndConnNames.isEmpty()) {
if(stat && logstashCurrent) {
%>
runStat.updateStatOnConnection("<%=connName%>"+iterateId,2, 0);
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,2,0<%for(String connName : needToEndConnNames){%>,"<%=connName%>"<%}%>);
<%
} else {
if(stat) {%>
if(execStat){
runStat.updateStatOnConnection(iterateId,2,0<%for(String connName : needToEndConnNames){%>,"<%=connName%>"<%}%>);
}
<%}%>
<%if(logstashCurrent) {//now only finish the log, not send, TODO%>
if(enableLogStash){
runStat.log(iterateId,2,0<%for(String connName : needToEndConnNames){%>,"<%=connName%>"<%}%>);
}
<%
}
}
%>
}
<%}%>
<%if(logstashCurrent) {//now only finish the log, not send, TODO%>
if(enableLogStash){
<%
for(String connName : needToEndConnNames){
%>
runStat.logStatOnConnection("<%=connName%>"+iterateId,2, 0);
<%
}
%>
}
<%
}
%>
<%
}
if(connSet.size()>0){
for(IConnection con:connSet){
INode source = con.getSource();
String sourceNodeId = source.getUniqueName();
String sourceNodeComponent = source.getComponent().getName();
%>
<%if(stat) {%>
if(execStat){
if(resourceMap.get("inIterateVComp") == null || !((Boolean)resourceMap.get("inIterateVComp"))){
runStat.updateStatOnConnection("<%=con.getUniqueName()%>"+iterateId,2, 0);
}
if(connSet.size()>0) {
if(stat && logstashCurrent && (connSet.size()==1)) {//the most common case, write this ugly logic for 65535 issue
for(IConnection con:connSet){
INode source = con.getSource();
String sourceNodeId = source.getUniqueName();
String sourceNodeComponent = source.getComponent().getName();
for (INode jobStructureCatcher : jobCatcherNodes) {
%>
if(runStat.updateStatAndLog(execStat,enableLogStash,resourceMap,iterateId,"<%=con.getUniqueName()%>",2,0,
<%=jobStructureCatcher.getUniqueName()%>,"<%=sourceNodeId%>","<%=sourceNodeComponent%>","<%=node.getUniqueName()%>","<%=node.getComponent().getName()%>","<%="REJECT".equals(con.getConnectorName()) ? "reject" : "output"%>")) {
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
}
<%
break;
}
}
} else {
if(stat){
%>
if(execStat){
runStat.updateStat(resourceMap,iterateId,2,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash){
if(resourceMap.get("inIterateVComp") == null || !((Boolean)resourceMap.get("inIterateVComp"))){
<%
if(logstashCurrent){
%>
if(enableLogStash) {
<%
for(IConnection con:connSet){
INode source = con.getSource();
String sourceNodeId = source.getUniqueName();
String sourceNodeComponent = source.getComponent().getName();
for (INode jobStructureCatcher : jobCatcherNodes) {
%>
RunStat.StatBean talend_statebean = runStat.logStatOnConnection("<%=con.getUniqueName()%>"+iterateId,2, 0);
<%=jobStructureCatcher.getUniqueName() %>.addConnectionMessage(
"<%=sourceNodeId%>",
"<%=sourceNodeComponent%>",
false,
"<%="REJECT".equals(con.getConnectorName()) ? "reject" : "output"%>",
"<%=con.getUniqueName()%>",
talend_statebean.getNbLine(),
talend_statebean.getStartTime(),
talend_statebean.getEndTime()
);
<%=jobStructureCatcher.getUniqueName() %>.addConnectionMessage(
"<%=node.getUniqueName()%>",
"<%=node.getComponent().getName()%>",
true,
"input",
"<%=con.getUniqueName()%>",
talend_statebean.getNbLine(),
talend_statebean.getStartTime(),
talend_statebean.getEndTime()
);
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
if(runStat.log(resourceMap,iterateId,"<%=con.getUniqueName()%>",2,0,
<%=jobStructureCatcher.getUniqueName()%>,"<%=sourceNodeId%>","<%=sourceNodeComponent%>","<%=node.getUniqueName()%>","<%=node.getComponent().getName()%>","<%="REJECT".equals(con.getConnectorName()) ? "reject" : "output"%>")) {
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
}
<%
break;
}
%>
}
}
%>
}
<%
}
<%}%>
<%
}
}
}
}
%>

View File

@@ -146,83 +146,82 @@
boolean logstashCurrent = !cid.startsWith("tJobStructureCatcher") && !cid.startsWith("talend") && enableLogStash;
if ((codePart.equals(ECodePart.BEGIN))&&(stat || logstashCurrent)&&connSet.size()>0) {
for(IConnection con:connSet){
%>
if (<%if(stat){%>execStat<%}%><%if(stat && logstashCurrent){%> || <%}%><%if(logstashCurrent){%>enableLogStash<%}%>) {
if(resourceMap.get("inIterateVComp") == null){
<%if(containsTPartitioner){%>
java.util.concurrent.ConcurrentHashMap<Object, Object> concurrentHashMap_<%=con.getUniqueName() %> = (java.util.concurrent.ConcurrentHashMap) globalMap.get("concurrentHashMap");
concurrentHashMap_<%=con.getUniqueName() %>.putIfAbsent("<%=con.getUniqueName() %>" + iterateLoop,new java.util.concurrent.atomic.AtomicInteger(0));
java.util.concurrent.atomic.AtomicInteger stats_<%=con.getUniqueName() %> = (java.util.concurrent.atomic.AtomicInteger) concurrentHashMap_<%=con.getUniqueName() %>.get("<%=con.getUniqueName() %>" + iterateLoop);
int step_<%=con.getUniqueName() %> = stats_<%=con.getUniqueName() %>.incrementAndGet()<=1?0:1;
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId, step_<%=con.getUniqueName()%>, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName()%>"+iterateId, step_<%=con.getUniqueName()%>, 0);
}
<%}%>
<%}else{%>
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId, 0, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName()%>"+iterateId, 0, 0);
}
<%}%>
<%}%>
if(containsTPartitioner) {
%>
if(<%if(stat){%>execStat<%}%><%if(stat && logstashCurrent){%> || <%}%><%if(logstashCurrent){%>enableLogStash<%}%>) {
runStat.updateStatOnConnectionAndLog(resourceMap,globalMap,iterateLoop,iterateId,<%if(stat){%>execStat<%} else {%>false<%}%>,enableLogStash,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
} else {
if(stat && logstashCurrent) {
%>
runStat.updateStatAndLog(execStat,enableLogStash,resourceMap,iterateId,0,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
<%
} else {
if(stat) {
%>
if(execStat) {
runStat.updateStatOnConnection(resourceMap,iterateId,0,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
}
<%
if(logstashCurrent) {
%>
if(enableLogStash) {
runStat.log(resourceMap,iterateId,0,0<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
}
}
}
if((codePart.equals(ECodePart.MAIN))&&(stat || logstashCurrent)&&connSet.size()>0){
for(IConnection con:connSet){
%>
//<%=con.getUniqueName()%>
//<%=(String)codeGenArgument.getIncomingName()%>
<%if (!node.getComponent().useMerge()) {%>
<%if(stat) {%>
if(execStat){
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
}
<%}%>
<%
} else if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){
if(!node.getComponent().useMerge()) {
if(stat && logstashCurrent) {
%>
<%if(stat) {%>
if(execStat){
runStat.updateStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,1,1<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
<%
} else {
if(stat) {
%>
if(execStat){
runStat.updateStatOnConnection(iterateId,1,1<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=con.getUniqueName() %>"+iterateId,1, 1);
if(logstashCurrent) {
%>
if(enableLogStash) {
runStat.log(iterateId,1,1<%for(IConnection con : connSet){%>,"<%=con.getUniqueName()%>"<%}%>);
}
<%
}
<%}%>
<%}%>
<%
}
} else {
for(IConnection connection:connSet){
if(connection.getUniqueName().equals((String)codeGenArgument.getIncomingName())){
if(stat && logstashCurrent) {
%>
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,1,1<%for(IConnection con : connSet){if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){%>,"<%=con.getUniqueName()%>"<%}}%>);
<%
} else {
if(stat) {%>
if(execStat){
runStat.updateStatOnConnection(iterateId,1,1<%for(IConnection con : connSet){if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){%>,"<%=con.getUniqueName()%>"<%}}%>);
}
<%}
if(logstashCurrent) {%>
if(enableLogStash) {
runStat.log(iterateId,1,1<%for(IConnection con : connSet){if(con.getUniqueName().equals((String)codeGenArgument.getIncomingName())){%>,"<%=con.getUniqueName()%>"<%}}%>);
}
<%}
}
}
}
}
}
@@ -251,7 +250,7 @@
for (INode jobStructureCatcher : jobCatcherNodes) {
%>
if(enableLogStash) {
<%=jobStructureCatcher.getUniqueName() %>.addComponentMessage("<%=node.getUniqueName()%>", "<%=node.getComponent().getName()%>");
<%=jobStructureCatcher.getUniqueName() %>.addCM("<%=node.getUniqueName()%>", "<%=node.getComponent().getName()%>");
<%=jobStructureCatcher.getDesignSubjobStartNode().getUniqueName() %>Process(globalMap);
}
<%
@@ -266,46 +265,37 @@
List<String> needToStartConnNames = new ArrayList<String>();
INode nextNode = node.getOutgoingConnections(EConnectionType.ITERATE).get(0).getTarget();
NodeUtil.fillConnectionsForStat(needToStartConnNames, nextNode);
if(needToStartConnNames.isEmpty()) {
//do nothing
} else if(containsTPartitioner){
%>
if(<%if(stat){%>execStat<%}%><%if(stat && logstashCurrent){%> || <%}%><%if(logstashCurrent){%>enableLogStash<%}%>){
<%
for(String connName : needToStartConnNames){
%>
<%if(containsTPartitioner){%>
java.util.concurrent.ConcurrentHashMap<Object, Object> concurrentHashMap_<%=connName%> = (java.util.concurrent.ConcurrentHashMap) globalMap.get("concurrentHashMap");
concurrentHashMap_<%=connName%>.putIfAbsent("<%=connName%>" + iterateLoop,new java.util.concurrent.atomic.AtomicInteger(0));
java.util.concurrent.atomic.AtomicInteger stats_<%=connName%> = (java.util.concurrent.atomic.AtomicInteger) concurrentHashMap_<%=connName%>.get("<%=connName%>" + iterateLoop);
int step_<%=connName %> = stats_<%=connName%>.incrementAndGet()<=1?0:1;
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=connName%>"+iterateId, step_<%=connName%>, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=connName%>"+iterateId, step_<%=connName%>, 0);
}
<%}%>
<%}else{%>
<%if(stat) {%>
if(execStat) {
runStat.updateStatOnConnection("<%=connName%>"+iterateId, 0, 0);
}
<%}%>
<%if(logstashCurrent) {%>
if(enableLogStash) {
runStat.logStatOnConnection("<%=connName%>"+iterateId, 0, 0);
}
<%}%>
<%}%>
<%
}
%>
runStat.updateStatOnConnectionAndLog(globalMap,iterateLoop,iterateId,<%if(stat){%>execStat<%} else {%>false<%}%>,enableLogStash,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
}
<%
} else {
if(stat && logstashCurrent) {
%>
runStat.updateStatAndLog(execStat,enableLogStash,iterateId,0,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
<%
} else {
if(stat){
%>
if(execStat){
runStat.updateStatOnConnection(iterateId,0,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
}
<%
}
%>
<%if(logstashCurrent) {%>
if(enableLogStash){
runStat.log(iterateId,0,0<%for(String connName : needToStartConnNames){%>,"<%=connName%>"<%}%>);
}
<%
}
}
}
<%
}else if(codePart.equals(ECodePart.MAIN)){
%>
resourceMap.put("inIterateVComp", true);

View File

@@ -25,7 +25,7 @@
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
org.talend.designer.dbmap.language.generation.GenericDbGenerationManager gm = new org.talend.designer.dbmap.language.generation.GenericDbGenerationManager();
org.talend.designer.dbmap.language.mssql.MssqlGenerationManager gm = new org.talend.designer.dbmap.language.mssql.MssqlGenerationManager();
String uniqueNameComponent = null;
IDbLanguage currentLanguage = gm.getLanguage();

View File

@@ -305,48 +305,54 @@ for(Column colStmt:stmtStructure){
if(("INSERT").equals(dataAction)){
%>
String insertQuery_<%=cid %> = "INSERT INTO "+tableName_<%=cid%>+"(<%=insertColName.toString()%>) ("+select_query_<%=cid %>+")";
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(insertQuery_<%=cid %>);
String insertQuery_<%=cid %> = "INSERT INTO "+tableName_<%=cid%>+"(<%=insertColName.toString()%>) ("+select_query_<%=cid %>+")";
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(insertQuery_<%=cid %>);
<%
}else if (("UPDATE").equals(dataAction)){
if(useUpdateStatement){
if(useUpdateStatement){
%>
String updateQuery_<%=cid %> = select_query_<%=cid %>
String updateQuery_<%=cid %> = select_query_<%=cid %>;
<%
}else{
}else{
%>
String updateQuery_<%=cid %> = "UPDATE "+tableName_<%=cid%>+" SET <%=updateSetStmt.toString()%> "
String updateQuery_<%=cid %> = "UPDATE "+tableName_<%=cid%>+" SET <%=updateSetStmt.toString()%> ";
<%
}
}
if(CodeGenerationUtils.hasAlphaNumericCharacter(whereClause)) {
if(useUpdateStatement){
%>
if(select_query_<%=cid %>.indexOf("WHERE")==-1){
updateQuery_<%=cid %> +=" WHERE ";
}else{
updateQuery_<%=cid %> +=" AND ";
}
updateQuery_<%=cid %>+= <%=CodeGenerationUtils.replaceAllCrBySpace(whereClause)%>;
<%
}else{
%>
updateQuery_<%=cid %> += " WHERE " + <%=CodeGenerationUtils.replaceAllCrBySpace(whereClause)%>;
<%
}
}
%>
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(updateQuery_<%=cid %>);
<%
}else if (("DELETE").equals(dataAction)){
%>
String deleteQuery_<%=cid %> = "DELETE FROM "+ tableName_<%=cid%>+" WHERE EXISTS ("+select_query_<%=cid %>+") "
<%
if(CodeGenerationUtils.hasAlphaNumericCharacter(whereClause)) {
%>
+" WHERE " + <%=CodeGenerationUtils.replaceAllCrBySpace(whereClause)%>
+" AND " + <%=CodeGenerationUtils.replaceAllCrBySpace(whereClause)%>
<%
}
%>
;
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(deleteQuery_<%=cid %>);
<%
}
%>
;
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(updateQuery_<%=cid %>);
<%
}else if (("DELETE").equals(dataAction)){
%>
String deleteQuery_<%=cid %> = "DELETE FROM "+ tableName_<%=cid%>+" WHERE EXISTS ("+select_query_<%=cid %>+") "
<%
if(CodeGenerationUtils.hasAlphaNumericCharacter(whereClause)) {
%>
+" AND " + <%=CodeGenerationUtils.replaceAllCrBySpace(whereClause)%>
<%
}
%>
;
pstmt_<%=cid %> = conn_<%=cid%>.prepareStatement(deleteQuery_<%=cid %>);
<%
}
%>
int nb_line_<%=cid%> = 0;
int nb_line_updated_<%=cid%> = 0;
int nb_line_inserted_<%=cid%> = 0;

View File

@@ -33,4 +33,4 @@ USE_DIFFERENT_TABLE.NAME=Use different table name
DIFFERENT_TABLE_NAME.NAME=Table name
IS_TABLE_NAME_VARIABLE.NAME=Table name from connection name is variable
USE_UPDATE_STATEMENT.NAME=Use update statement
USE_UPDATE_STATEMENT.NAME=Use update statement without subqueries

View File

@@ -3,6 +3,7 @@ imports="
org.talend.core.model.process.INode
org.talend.core.model.process.ElementParameterParser
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.utils.NodeUtil
"
%>
@@ -17,7 +18,15 @@ String operator = ElementParameterParser.getValue(node,"__OPERATOR__");
String value = ElementParameterParser.getValue(node,"__VALUE__");
boolean isEnableWhereClause = ("true").equals(ElementParameterParser.getValue(node,"__ENABLE_WHERE_CLAUSE__"));
boolean isEnableSQLStmt = ("true").equals(ElementParameterParser.getValue(node,"__ENABLE_SQL_STMT__"));
String conn = "conn_" + connection ;
final String conn;
INode connNode = NodeUtil.getNodeByUniqueName(node.getProcess(), connection);
if((connNode!=null) && "tSnowflakeConnection".equals(connNode.getComponent().getName())){
conn = connection + "_Connection";
}else{
conn = "conn_" + connection;
}
String query = "SELECT COUNT(*) FROM ";
%>

View File

@@ -54,6 +54,7 @@ import org.talend.designer.dbmap.i18n.Messages;
import org.talend.designer.dbmap.language.generation.DbGenerationManager;
import org.talend.designer.dbmap.language.generation.GenericDbGenerationManager;
import org.talend.designer.dbmap.language.hive.HiveGenerationManager;
import org.talend.designer.dbmap.language.mssql.MssqlGenerationManager;
import org.talend.designer.dbmap.language.mysql.MysqlGenerationManager;
import org.talend.designer.dbmap.language.oracle.OracleGenerationManager;
import org.talend.designer.dbmap.language.postgres.PostgresGenerationManager;
@@ -602,6 +603,8 @@ public class DbMapComponent extends AbstractMapComponent {
generationManager = new PostgresGenerationManager();
} else if (value.contains("tELTHiveMap")) { //$NON-NLS-1$
generationManager = new HiveGenerationManager();
} else if (value.contains("tELTMSSqlMap")) {
generationManager = new MssqlGenerationManager();
} else if (value.startsWith("tELT") && value.endsWith("Map")) //$NON-NLS-1$ //$NON-NLS-2$
{
generationManager = new GenericDbGenerationManager();

View File

@@ -12,7 +12,11 @@
// ============================================================================
package org.talend.designer.dbmap.language.generation;
import static java.util.Optional.ofNullable;
import static java.util.stream.Collectors.toList;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
@@ -272,6 +276,9 @@ public abstract class DbGenerationManager {
return tableName;
}
protected String addQuotes(String name) {
return name;
}
/**
*
* ggu Comment method "buildSqlSelect".
@@ -751,6 +758,8 @@ public abstract class DbGenerationManager {
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, inputTable.getAlias());
}
boolean commaCouldBeAdded = i > 0;
buildTableDeclaration(component, sb, inputTable, commaCouldBeAdded, false, false);
}
// On
@@ -767,7 +776,7 @@ public abstract class DbGenerationManager {
boolean isFirstClause = true;
for (int i = 0; i < lstSizeInputTables; i++) {
ExternalDbMapTable inputTable = inputTables.get(i);
if (buildConditions(component, sbWhere, inputTable, false, isFirstClause, false)) {
if (buildConditions4WhereClause(isFirstClause, component, sbWhere, inputTable, false)) {
isFirstClause = false;
}
}
@@ -895,6 +904,20 @@ public abstract class DbGenerationManager {
return false;
}
/**
* This piece regulates which columns are going to be skipped in UPDATE QUERY generation in SET section if
* <b>SET_COLUMN</b> property exists. This enhancement is needed for tELTOutput component in case of
* <b>USE_UPDATE_STATEMENT</b>
*
* @return List of columns that going to be skipped, if no <b>SET_COLUMN</b> specified then return empty list.
*/
protected List<Boolean> getSetColumnsForUpdateQuery() {
return ofNullable(source.getElementParameter("SET_COLUMN")).filter(iep -> iep.isShow(source.getElementParameters()))
.flatMap(iep -> ofNullable((List<Map<String, ? extends Object>>) iep.getValue()))
.orElseGet(Collections::emptyList).stream().map(m -> !Boolean.valueOf(m.get("UPDATE_COLUMN").toString()))
.collect(toList());
}
protected void checkUseDelimitedIdentifiers(DbMapComponent component) {
/**
* in elt related component javajets(like tELTMSSqlMap_main.javajet), they don't get DbGenerationManager by
@@ -1088,6 +1111,35 @@ public abstract class DbGenerationManager {
return atLeastOneConditionWritten;
}
/**
* build conditions for update case only DOC hzhao Comment method "buildConditions".
*
* @param isFirstClause
* @param component
* @param sb
* @param inputTable
* @param isSqlQuert
* @return
*/
protected boolean buildConditions4WhereClause(boolean isFirstClause, DbMapComponent component, StringBuilder sb,
ExternalDbMapTable inputTable,
boolean isSqlQuert) {
List<ExternalDbMapEntry> inputEntries = inputTable.getMetadataTableEntries();
int lstSizeEntries = inputEntries.size();
boolean atLeastOneConditionWritten = false;
for (int j = 0; j < lstSizeEntries; j++) {
ExternalDbMapEntry dbMapEntry = inputEntries.get(j);
boolean conditionWritten = buildCondition(component, sb, inputTable, isFirstClause, dbMapEntry, true, isSqlQuert);
if (conditionWritten) {
atLeastOneConditionWritten = true;
}
if (isFirstClause && conditionWritten) {
isFirstClause = false;
}
}
return atLeastOneConditionWritten;
}
/**
* DOC amaumont Comment method "buildCondition".
*
@@ -1711,5 +1763,329 @@ public abstract class DbGenerationManager {
public void setUseDelimitedIdentifiers(boolean useDelimitedIdentifiers) {
this.useDelimitedIdentifiers = useDelimitedIdentifiers;
}
public boolean isConditionChecked(DbMapComponent component, ExternalDbMapTable inputTable) {
List<ExternalDbMapEntry> inputEntries = inputTable.getMetadataTableEntries();
boolean atLeastOneConditionIsChecked = false;
for (ExternalDbMapEntry dbMapEntry : inputEntries) {
if (dbMapEntry.isJoin()) {
if (org.apache.commons.lang.StringUtils.isNotEmpty(dbMapEntry.getExpression())
&& org.apache.commons.lang.StringUtils.isNotEmpty(dbMapEntry.getOperator())) {
atLeastOneConditionIsChecked = true;
break;
}
}
}
return atLeastOneConditionIsChecked;
}
public String buildSqlUpdate(DbMapComponent dbMapComponent, String outputTableName, String tabString) {
queryColumnsName = "\""; //$NON-NLS-1$
aliasAlreadyDeclared.clear();
queryColumnsSegments.clear();
querySegments.clear();
subQueryTable.clear();
inputSchemaContextSet.clear();
this.tabSpaceString = tabString;
DbMapComponent component = getDbMapComponent(dbMapComponent);
List<IConnection> outputConnections = (List<IConnection>) component.getOutgoingConnections();
Map<String, IConnection> nameToOutputConnection = new HashMap<String, IConnection>();
for (IConnection connection : outputConnections) {
nameToOutputConnection.put(connection.getUniqueName(), connection);
}
ExternalDbMapData data = component.getExternalData();
StringBuilder sb = new StringBuilder();
List<ExternalDbMapTable> outputTables = data.getOutputTables();
int lstOutputTablesSize = outputTables.size();
ExternalDbMapTable outputTable = null;
for (int i = 0; i < lstOutputTablesSize; i++) {
ExternalDbMapTable temp = outputTables.get(i);
if (outputTableName.equals(temp.getName())) {
outputTable = temp;
break;
}
}
if (outputTable != null) {
String outTableName = outputTable.getTableName();
IConnection connection = nameToOutputConnection.get(outputTable.getName());
List<IMetadataColumn> columns = new ArrayList<IMetadataColumn>();
if (connection != null) {
IMetadataTable metadataTable = connection.getMetadataTable();
if (metadataTable != null) {
columns.addAll(metadataTable.getListColumns());
}
}
// Update
String targetSchemaTable = outTableName;
IElementParameter eltSchemaNameParam = source.getElementParameter("ELT_SCHEMA_NAME"); //$NON-NLS-1$
if (eltSchemaNameParam != null && eltSchemaNameParam.getValue() != null) {
String schema = TalendQuoteUtils.removeQuotesIfExist(String.valueOf(eltSchemaNameParam.getValue()));
if (org.apache.commons.lang.StringUtils.isNotEmpty(schema)) {
targetSchemaTable = addQuotes(schema) + DbMapSqlConstants.DOT + addQuotes(outTableName);
}
}
appendSqlQuery(sb, "\"", false); //$NON-NLS-1$
appendSqlQuery(sb, DbMapSqlConstants.UPDATE);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, targetSchemaTable);
appendSqlQuery(sb, tabSpaceString);
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
// Set
String keyColumn = DbMapSqlConstants.EMPTY;
List<ExternalDbMapEntry> metadataTableEntries = outputTable.getMetadataTableEntries();
if (metadataTableEntries != null) {
appendSqlQuery(sb, "SET"); //$NON-NLS-1$
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
boolean isKey = false;
int lstSizeOutTableEntries = metadataTableEntries.size();
List<Boolean> setColumns = getSetColumnsForUpdateQuery();
final boolean hasDeactivatedColumns = !setColumns.isEmpty();
boolean isFirstColumn = true;
for (int i = 0; i < lstSizeOutTableEntries; i++) {
if (hasDeactivatedColumns && setColumns.get(i)) {
continue;
}
ExternalDbMapEntry dbMapEntry = metadataTableEntries.get(i);
String columnEntry = dbMapEntry.getName();
String expression = dbMapEntry.getExpression();
expression = initExpression(component, dbMapEntry);
expression = addQuoteForSpecialChar(expression, component);
//
if (!DEFAULT_TAB_SPACE_STRING.equals(this.tabSpaceString)) {
expression += DbMapSqlConstants.SPACE + DbMapSqlConstants.AS + DbMapSqlConstants.SPACE
+ getAliasOf(dbMapEntry.getName());
}
String exp = replaceVariablesForExpression(component, expression);
String columnSegment = exp;
// Added isFirstColumn to conform old behaior if first column is skipped
if (i > 0 && !isFirstColumn) {
queryColumnsName += DbMapSqlConstants.COMMA + DbMapSqlConstants.SPACE;
columnSegment = DbMapSqlConstants.COMMA + DbMapSqlConstants.SPACE + columnSegment;
}
if (expression != null && expression.trim().length() > 0) {
queryColumnsName += exp;
queryColumnsSegments.add(columnSegment);
}
//
if (!isKey) {
for (IMetadataColumn column : columns) {
String columnName = column.getLabel();
if (columnName.equals(dbMapEntry.getName()) && column.isKey()) {
isKey = column.isKey();
keyColumn = addQuotes(columnEntry) + " = " + expression;//$NON-NLS-1$
break;
}
}
if (isKey) {
continue;
}
}
if (expression != null && expression.trim().length() > 0) {
// Append COMMA and NEW_LINE for all columns except FIRST.
if (!isFirstColumn) {
appendSqlQuery(sb, DbMapSqlConstants.COMMA);
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
} else {
isFirstColumn = false;
}
appendSqlQuery(sb, addQuotes(columnEntry) + " = " + expression); //$NON-NLS-1$
}
}
}
if ("\"".equals(queryColumnsName)) {
throw new IllegalArgumentException("Specify at least 1 column for UPDATE QUERY in SET section");
}
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
// From
appendSqlQuery(sb, tabSpaceString);
appendSqlQuery(sb, DbMapSqlConstants.FROM);
// load input table in hash
List<ExternalDbMapTable> inputTables = data.getInputTables();
// load input table in hash
boolean explicitJoin = false;
int lstSizeInputTables = inputTables.size();
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, tabSpaceString);
IJoinType previousJoinType = null;
for (int i = 0; i < lstSizeInputTables; i++) {
ExternalDbMapTable inputTable = inputTables.get(i);
IJoinType joinType = language.getJoin(inputTable.getJoinType());
if (!language.unuseWithExplicitJoin().contains(joinType) && i > 0) {
explicitJoin = true;
} else {
explicitJoin = false;
}
if (i == 0) {
joinType = AbstractDbLanguage.JOIN.NO_JOIN;
previousJoinType = joinType;
} else {
joinType = language.getJoin(inputTable.getJoinType());
}
boolean commaCouldBeAdded = !explicitJoin && i > 0;
boolean crCouldBeAdded = false;
if (language.unuseWithExplicitJoin().contains(joinType) && !explicitJoin) {
buildTableDeclaration(component, sb, inputTable, commaCouldBeAdded, crCouldBeAdded, false);
} else if (!language.unuseWithExplicitJoin().contains(joinType) && explicitJoin) {
if (i > 0) {
if (previousJoinType == null) {
buildTableDeclaration(component, sb, inputTables.get(i - 1), commaCouldBeAdded, crCouldBeAdded, true);
previousJoinType = joinType;
} else {
// appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, tabSpaceString);
}
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
}
String labelJoinType = joinType.getLabel();
if (joinType == AbstractDbLanguage.JOIN.CROSS_JOIN) {
ExternalDbMapTable nextTable = null;
if (i < lstSizeInputTables) {
nextTable = inputTables.get(i);
appendSqlQuery(sb, labelJoinType);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
buildTableDeclaration(component, sb, nextTable, false, true, true);
}
} else {
if (isConditionChecked(component, inputTable)) {
appendSqlQuery(sb, labelJoinType);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
buildTableDeclaration(component, sb, inputTable, false, false, true);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, DbMapSqlConstants.ON);
appendSqlQuery(sb, DbMapSqlConstants.LEFT_BRACKET);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
buildConditions(component, sb, inputTable, true, true, true);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, DbMapSqlConstants.RIGHT_BRACKET);
} else {
commaCouldBeAdded = true;
buildTableDeclaration(component, sb, inputTable, commaCouldBeAdded, crCouldBeAdded, false);
}
}
}
}
// where
StringBuilder sbWhere = new StringBuilder();
this.tabSpaceString = DEFAULT_TAB_SPACE_STRING;
boolean isFirstClause = true;
for (int i = 0; i < lstSizeInputTables; i++) {
ExternalDbMapTable inputTable = inputTables.get(i);
IJoinType joinType = language.getJoin(inputTable.getJoinType());
if (joinType == AbstractDbLanguage.JOIN.CROSS_JOIN) {
// if join type is CROSS JOIN the condition will only in where clause no matter explicit join
// checked or not
if (buildConditions4WhereClause(isFirstClause, component, sbWhere, inputTable, false)) {
isFirstClause = false;
}
} else {
if (buildConditions(component, sbWhere, inputTable, false, isFirstClause, false)) {
isFirstClause = false;
}
}
}
/*
* for addition conditions
*/
// like as input.newcolumn1>100
List<String> whereAddition = new ArrayList<String>();
// olny pure start with group or order, like as order/group by input.newcolumn1
// List<String> byAddition = new ArrayList<String>();
// like as input.newcolumn1>100 group/oder by input.newcolumn1
// List<String> containWhereAddition = new ArrayList<String>();
// like as "OR/AND input.newcolumn1", will keep original
List<String> originalWhereAddition = new ArrayList<String>();
List<String> otherAddition = new ArrayList<String>();
if (outputTable != null) {
List<ExternalDbMapEntry> customWhereConditionsEntries = outputTable.getCustomWhereConditionsEntries();
if (customWhereConditionsEntries != null) {
for (ExternalDbMapEntry entry : customWhereConditionsEntries) {
String exp = initExpression(component, entry);
if (exp != null && !DbMapSqlConstants.EMPTY.equals(exp.trim())) {
if (containWith(exp, DbMapSqlConstants.OR, true) || containWith(exp, DbMapSqlConstants.AND, true)) {
exp = replaceVariablesForExpression(component, exp);
originalWhereAddition.add(exp);
} else {
exp = replaceVariablesForExpression(component, exp);
whereAddition.add(exp);
}
}
}
}
List<ExternalDbMapEntry> customOtherConditionsEntries = outputTable.getCustomOtherConditionsEntries();
if (customOtherConditionsEntries != null) {
for (ExternalDbMapEntry entry : customOtherConditionsEntries) {
String exp = initExpression(component, entry);
if (exp != null && !DbMapSqlConstants.EMPTY.equals(exp.trim())) {
exp = replaceVariablesForExpression(component, exp);
otherAddition.add(exp);
}
}
}
}
this.tabSpaceString = tabString;
String whereClauses = sbWhere.toString();
boolean whereFlag = whereClauses.trim().length() > 0;
boolean whereAddFlag = !whereAddition.isEmpty();
boolean whereOriginalFlag = !originalWhereAddition.isEmpty();
if (whereFlag || whereAddFlag || whereOriginalFlag) {
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, tabSpaceString);
appendSqlQuery(sb, DbMapSqlConstants.WHERE);
}
if (whereFlag) {
appendSqlQuery(sb, whereClauses);
}
if (whereAddFlag) {
for (int i = 0; i < whereAddition.size(); i++) {
if (i == 0 && whereFlag || i > 0) {
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, tabSpaceString);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, DbMapSqlConstants.AND);
}
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, whereAddition.get(i));
}
}
if (whereOriginalFlag) {
for (String s : originalWhereAddition) {
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, DbMapSqlConstants.SPACE);
appendSqlQuery(sb, s);
}
}
if (!otherAddition.isEmpty()) {
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, tabSpaceString);
for (String s : otherAddition) {
appendSqlQuery(sb, s);
appendSqlQuery(sb, DbMapSqlConstants.NEW_LINE);
appendSqlQuery(sb, tabSpaceString);
}
}
}
String sqlQuery = sb.toString();
sqlQuery = handleQuery(sqlQuery);
queryColumnsName = handleQuery(queryColumnsName);
return sqlQuery;
}
}

View File

@@ -12,8 +12,12 @@
// ============================================================================
package org.talend.designer.dbmap.language.generation;
import java.util.Optional;
import org.talend.commons.exception.ExceptionHandler;
import org.talend.designer.dbmap.DbMapComponent;
import org.talend.designer.dbmap.language.GenericDbLanguage;
import org.talend.designer.dbmap.language.mssql.MssqlGenerationManager;
/**
* wzhang class global comment. Detailled comment
@@ -26,6 +30,17 @@ public class GenericDbGenerationManager extends DbGenerationManager {
@Override
public String buildSqlSelect(DbMapComponent component, String outputTableName) {
try {
if (Optional.ofNullable(component).map(c -> c.getComponent()).map(c -> "tELTMSSqlMap".equals(c.getName()))
.orElse(false)) {
MssqlGenerationManager genManager = new MssqlGenerationManager();
String result = genManager.buildSqlSelect(component, outputTableName);
this.queryColumnsName = genManager.getQueryColumnsName();
return result;
}
} catch (Throwable e) {
ExceptionHandler.process(e);
}
return super.buildSqlSelect(component, outputTableName);
}
}

View File

@@ -0,0 +1,34 @@
// ============================================================================
//
// Copyright (C) 2006-2019 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.designer.dbmap.language.mssql;
import org.talend.designer.dbmap.DbMapComponent;
import org.talend.designer.dbmap.language.GenericDbLanguage;
import org.talend.designer.dbmap.language.generation.DbGenerationManager;
public class MssqlGenerationManager extends DbGenerationManager {
public MssqlGenerationManager() {
super(new GenericDbLanguage());
}
@Override
public String buildSqlSelect(DbMapComponent component, String outputTableName) {
boolean checkUseUpdateStatement = checkUseUpdateStatement(component, outputTableName);
if (checkUseUpdateStatement) {
return buildSqlUpdate(component, outputTableName, DEFAULT_TAB_SPACE_STRING);
} else {
return super.buildSqlSelect(component, outputTableName, DEFAULT_TAB_SPACE_STRING);
}
}
}

View File

@@ -14,6 +14,7 @@
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**/*.class</include>
<include>${talend.job.path}/**/*.wsdl</include>
<include>__tdm/**</include>
</includes>
</fileSet>
@@ -22,8 +23,8 @@
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>OSGI-INF/**</include>
<include>TALEND-INF/**</include>
<include>MAVEN-INF/**</include>
<include>TALEND-INF/**</include>
<include>MAVEN-INF/**</include>
</includes>
</fileSet>
<fileSet> <!-- add resources -->
@@ -46,8 +47,8 @@
<includes>
<include>**</include>
</includes>
</fileSet>
</fileSet>
<fileSet> <!-- add metadata -->
<directory>${basedir}/src/main/resources</directory>
<outputDirectory>${file.separator}</outputDirectory>
@@ -55,7 +56,7 @@
<include>metadata/**</include>
</includes>
</fileSet>
<fileSet>
<directory>${current.resources.dir}</directory>
<outputDirectory>${file.separator}</outputDirectory>

View File

@@ -1,67 +1,68 @@
<?xml version="1.0" encoding="UTF-8"?>
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>jar-with-dependencies</id>
<includeBaseDirectory>false</includeBaseDirectory>
<formats>
<format>jar</format>
</formats>
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>jar-with-dependencies</id>
<includeBaseDirectory>false</includeBaseDirectory>
<formats>
<format>jar</format>
</formats>
<fileSets>
<fileSets>
<fileSet> <!-- add java classes -->
<directory>${project.build.outputDirectory}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**/*.class</include>
<include>__tdm/**</include>
</includes>
</fileSet>
<fileSet> <!-- add osgi resources -->
<directory>${current.bundle.resources.dir}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>OSGI-INF/**</include>
<include>TALEND-INF/**</include>
<include>MAVEN-INF/**</include>
</includes>
</fileSet>
<fileSet> <!-- add java classes -->
<directory>${project.build.outputDirectory}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**/*.class</include>
<include>${talend.job.path}/**/*.wsdl</include>
<include>__tdm/**</include>
</includes>
</fileSet>
<fileSet> <!-- add route resources -->
<directory>${current.bundle.resources.dir}/resources</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>**</include>
</includes>
</fileSet>
<fileSet> <!-- add osgi resources -->
<directory>${current.bundle.resources.dir}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>OSGI-INF/**</include>
<include>TALEND-INF/**</include>
<include>MAVEN-INF/**</include>
</includes>
</fileSet>
<fileSet> <!-- add route lib -->
<directory>${current.bundle.resources.dir}/lib</directory>
<outputDirectory>lib</outputDirectory>
<includes>
<include>**</include>
</includes>
</fileSet>
<fileSet> <!-- add route resources -->
<directory>${current.bundle.resources.dir}/resources</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>**</include>
</includes>
</fileSet>
<fileSet> <!-- add route lib -->
<directory>${current.bundle.resources.dir}/lib</directory>
<outputDirectory>lib</outputDirectory>
<includes>
<include>**</include>
</includes>
</fileSet>
<fileSet> <!-- add context resources -->
<directory>${current.resources.dir}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**</include>
</includes>
</fileSet>
<fileSet> <!-- add context resources -->
<directory>${resources.path}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**</include>
</includes>
</fileSet>
<fileSet><!-- add context resources -->
<directory>${current.resources.dir}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**</include>
</includes>
</fileSet>
<fileSet><!-- add context resources -->
<directory>${resources.path}</directory>
<outputDirectory>${file.separator}</outputDirectory>
<includes>
<include>${talend.job.path}/**</include>
</includes>
</fileSet>
<fileSet> <!-- add provided-lib -->
<directory>${current.bundle.resources.dir}/provided-lib</directory>
<outputDirectory>lib</outputDirectory>
@@ -69,17 +70,17 @@
<include>**</include>
</includes>
</fileSet>
</fileSets>
<!-- <dependencySets>
<dependencySet>
<outputDirectory>lib</outputDirectory>
<includes>
<include>${routines.binaries.includes.set}</include>
</includes>
without version for talend libraries
<outputFileNameMapping>${artifact.artifactId}.${artifact.extension}</outputFileNameMapping>
<useProjectArtifact>false</useProjectArtifact>
</dependencySet>
</dependencySets> -->
</fileSets>
<!-- <dependencySets>
<dependencySet>
<outputDirectory>lib</outputDirectory>
<includes>
<include>${routines.binaries.includes.set}</include>
</includes>
without version for talend libraries
<outputFileNameMapping>${artifact.artifactId}.${artifact.extension}</outputFileNameMapping>
<useProjectArtifact>false</useProjectArtifact>
</dependencySet>
</dependencySets> -->
</assembly>

View File

@@ -44,6 +44,7 @@ import org.eclipse.jface.preference.IPreferenceStore;
import org.eclipse.jface.viewers.IStructuredSelection;
import org.osgi.framework.Bundle;
import org.talend.commons.exception.ExceptionHandler;
import org.talend.commons.utils.workbench.resources.ResourceUtils;
import org.talend.core.CorePlugin;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.IESBService;
@@ -931,7 +932,7 @@ public class DefaultRunProcessService implements IRunProcessService {
IFolder targetFolder = mainProject.getFolder(refPath.removeLastSegments(1));
if (!targetFolder.exists()) {
targetFolder.create(true, false, progressMonitor);
ResourceUtils.createFolder(targetFolder);
}
if (codeFile.getLocation().removeLastSegments(1).equals(targetFolder.getLocation())) {
continue;

View File

@@ -1213,12 +1213,28 @@ public class JavaProcessor extends AbstractJavaProcessor implements IJavaBreakpo
} else {
localM2Path = localM2Path + PomUtil.getLocalRepositoryPath();
}
asList.add(3, localM2Path);
insertArgument(asList, localM2Path);
}
return asList.toArray(new String[0]);
}
}
private static void insertArgument(List<String> asList, String arg) throws ProcessorException {
// https://jira.talendforge.org/browse/TUP-27053
int idx = -1;
for (int i = 0; i < asList.size(); i++) {
if (asList.get(i).equals("-cp")) {
idx = i;
break;
}
}
if (idx > -1) {
asList.add(idx, arg);
} else {
throw new ProcessorException("Can not insert " + arg);
}
}
@Override
public List<String> extractAheadCommandSegments() {
List<String> aheadSegments = new ArrayList<>();

View File

@@ -389,8 +389,9 @@ public class MavenJavaProcessor extends JavaProcessor {
if (!isMainJob && isGoalInstall) {
if (!buildCacheManager.isJobBuild(getProperty())) {
deleteExistedJobJarFile(talendJavaProject);
if ("ROUTE".equalsIgnoreCase(getBuildType(getProperty())) && project != null &&
ERepositoryObjectType.PROCESS.equals(ERepositoryObjectType.getType(getProperty()))) {
String buildType = getBuildType(getProperty());
if (("ROUTE".equalsIgnoreCase(buildType) || "OSGI".equalsIgnoreCase(buildType)) && project != null &&
ERepositoryObjectType.PROCESS.equals(ERepositoryObjectType.getType(getProperty()))) {
// TESB-23870
// child routes job project must be compiled explicitly for
// correct child job manifest generation during OSGi packaging

View File

@@ -36,6 +36,7 @@ import org.talend.core.model.metadata.types.JavaTypesManager;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.runprocess.shadow.ObjectElementParameter;
import org.talend.core.model.runprocess.shadow.TextElementParameter;
import org.talend.core.utils.TalendQuoteUtils;
import org.talend.designer.rowgenerator.PluginUtils;
import org.talend.designer.rowgenerator.RowGeneratorComponent;
import org.talend.designer.rowgenerator.data.Function;
@@ -123,7 +124,7 @@ public class VirtualRowGeneratorNode extends RowGeneratorComponent {
}
} catch (NumberFormatException e1) {
for (Variable var : variables) {
expression = renameValues(expression, var.getName(), "\"" + var.getValue() + "\"");//$NON-NLS-1$ //$NON-NLS-2$
expression = renameValues(expression, var.getName(), TalendQuoteUtils.addQuotesIfNotExist(var.getValue()));
}
}
value.put(RowGeneratorComponent.ARRAY, "\"\"+(" + expression + ")+\"\""); //$NON-NLS-1$ //$NON-NLS-2$
@@ -182,6 +183,15 @@ public class VirtualRowGeneratorNode extends RowGeneratorComponent {
}
private String renameVaribleValue(String newValue, String type) {
if (newValue == null) {
return newValue;
}
if (JavaTypesManager.STRING.getLabel().equals(type)) {
return TalendQuoteUtils.addQuotesIfNotExist(newValue);
}
if (newValue.equals("null")) { //$NON-NLS-1$
return newValue;
}
if (JavaTypesManager.BIGDECIMAL.getLabel().equals(type)) {
newValue = " new " + JavaTypesManager.BIGDECIMAL.getNullableClass().getName() + "(" + newValue + ")";//$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
} else if (JavaTypesManager.CHARACTER.getLabel().contains(type)) {

View File

@@ -38,6 +38,7 @@ import org.talend.core.model.properties.ProcessItem;
import org.talend.designer.maven.launch.MavenPomCommandLauncher;
import org.talend.designer.maven.model.MavenSystemFolders;
import org.talend.designer.maven.model.TalendMavenConstants;
import org.talend.designer.maven.utils.PomIdsHelper;
import org.talend.designer.runprocess.IProcessor;
import org.talend.repository.documentation.ExportFileResource;
import org.talend.repository.ui.wizards.exportjob.scriptsmanager.JobScriptsManager;
@@ -61,7 +62,17 @@ public class BuildOSGiBundleHandler extends BuildJobHandler {
IProcessor.NO_STATISTICS, IProcessor.NO_TRACES);
osgiMavenManager.setJobVersion(version);
osgiMavenManager.setBundleVersion(version);
osgiMavenManager.setBundleVersion(getArtifactVersion(processItem));
}
protected String getArtifactVersion(ProcessItem processItem) {
if (processItem != null) {
String v = PomIdsHelper.getJobVersion(processItem.getProperty());
if (v != null) {
return v.replace("-", ".");
}
}
return null;
}
/*

View File

@@ -792,4 +792,5 @@ public class DbGenerationManagerTest extends DbGenerationManagerTestHelper {
+ " FROM\n" + " table1\n" + " WHERE t.column2 = '\" +context.param2+ \"'\n" + " ) table2\"";
assertEquals(exceptQuery.replaceAll("\n", "").trim(), query.trim());
}
}