Compare commits

...

116 Commits

Author SHA1 Message Date
apoltavtsev
2ffcce909a Update AbstractMavenProcessorPom.java 2020-11-18 13:24:51 +01:00
apoltavtsev
aeab2f8da9 Update AbstractMavenProcessorPom.java 2020-11-18 13:22:25 +01:00
apoltavtsev
1e4010ce99 fix(TESB-29553, TESB-31088) Could not generate ctalenjob microservice jar 2020-11-18 12:40:57 +01:00
kjwang
49aab27be9 Kjwang/fix tup 29290 detect and update all jobs r73 (#3845)
* fix(TUP-28952):TOS: NPE when delete items to recycle bin (#3738)

https://jira.talendforge.org/browse/TUP-28952

Signed-off-by: jding-tlnd <jding@talend.com>

* Fix TUP-29290 "Detect and Update all jobs " is not identifying the changes in imported projects. (#3833)

* Fix TUP-29290 "Detect and Update all jobs " is not identifying the
changes in imported projects.
https://jira.talendforge.org/browse/TUP-29290

Co-authored-by: Jane Ding <jding@talend.com>
2020-11-16 17:58:44 +08:00
apoltavtsev
c7965cf552 fix(TESB-29553) Publishing a route with cTalendJob from Studio and commandline gives different results 2020-11-12 11:06:44 +01:00
Chao MENG
61b55c0d2a fix(TUP-29285): Improve reloading logic of Custom Uri Mapping (#3830)
https://jira.talendforge.org/browse/TUP-29285
2020-11-12 17:09:53 +08:00
bhe-talendbj
be41bcbef3 feat(TESB-28330): remove workaround for cConfig (#3754)
* feat(TESB-28330): remove workaround for cConfig

* TESB-28330:cConfig/Beans - Support customer groupid and artifact name
(additional to custom version)

* TESB-28330:cConfig/Beans - Support customer groupid and artifact name
(additional to custom version)

Co-authored-by: SunChaoqun <csun@talend.com>
2020-11-12 16:31:56 +08:00
Zhiwei Xue
2827919dae fix(TUP-29265):"<routine_name> cannot be resolved" error after (#3825)
installing latest R2020-10 patch
2020-11-12 09:55:16 +08:00
jiezhang-tlnd
33848db788 Jzhang/bufix/73/tup 28893 support more typesfor share libs1 (#3816)
* fix(TUP-28893)Support more types of artifact for share libs
https://jira.talendforge.org/browse/TUP-28893

* ignore snapshot

* Maintenance/7.3 (#3774)

* fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758)

* fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)

Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>

* Maintenance/7.3 (#3775)

* fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758)

* fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)

Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>

* Maintenance/7.3 (#3776)

* fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758)

* fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)

Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* modify for nexus2

* support classifier for artifact repository

* add getClassifier and junits

* remove snapshot judgment

* fix(TUP-28893)Support more types of artifact for share libs

* fix(TUP-28893)Support more types of artifact for share libs

* fix(TUP-28893)Support more types of artifact for share libs

* fix pom deploy fail on artifactory

* revert the fix of redeploy pom file

* revert the fix of redeploy pom file

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>
2020-11-11 13:00:41 +08:00
apoltavtsev
ac359e295a fix(TESB-30691) Incorrect Route version in Runtime after sequential publish from Studio 2020-11-10 16:45:36 +01:00
apoltavtsev
6965eb79fa bugfix(TESB-30661) Wrong artifact version of the job called by a route 2020-11-10 15:59:21 +01:00
apoltavtsev
80f02ef1e7 feat(TESB-29949) Pass the data source to a job using a context variable 2020-11-10 15:58:06 +01:00
hzhao-talendbj
7c50efdf21 bugfix(TUP-26868):xmlMapping file are not correctly add to class path in (#3229) (#3810)
bugfix(TUP-26868):xmlMapping file are not correctly add to class path in debug mode

Co-authored-by: hwang-talend <hwang@talend.com>
2020-11-10 15:38:16 +08:00
chmyga
7f40393753 chore(TDI-45157): bump TCK components version (#3821)
Co-authored-by: Dmytro Chmyga <dmytro.chmyga@synapse.com>
2020-11-10 08:44:41 +02:00
Jane Ding
356dba9940 Fix junit failure (#3820)
Signed-off-by: jding-tlnd <jding@talend.com>
2020-11-09 19:24:48 +08:00
Chao MENG
83f7a577d9 feat(TUP-28790): Enhance detection of localhost (#3729)
* feat(TUP-28790): Enhance detection of localhost

https://jira.talendforge.org/browse/TUP-28790

* feat(TUP-28790): Enhance detection of localhost

https://jira.talendforge.org/browse/TUP-28790

* feat(TUP-28790): Enhance detection of localhost

https://jira.talendforge.org/browse/TUP-28790

* feat(TUP-28790): Enhance detection of localhost

https://jira.talendforge.org/browse/TUP-28790

* feat(TUP-28790): Enhance detection of localhost

https://jira.talendforge.org/browse/TUP-28790
2020-11-09 17:32:13 +08:00
Chao MENG
e87f557a49 feat(TUP-29126): Github Renaming the default branch from master (#3793)
* feat(TUP-29126): Github Renaming the default branch from master
https://jira.talendforge.org/browse/TUP-29126

* feat(TUP-29126): Github Renaming the default branch from master
https://jira.talendforge.org/browse/TUP-29126
2020-11-09 10:10:52 +08:00
Zhiwei Xue
a51b3316d3 fix(TUP-28852):Keytab fields missing for impala metadata (#3743) 2020-11-07 15:44:29 +08:00
bkatiukhov
46a22c4ad0 TESB-30649 Fix NPE for bean relation (#3805)
Co-authored-by: bohdan.katiukhov <bohdan.katiukhov@KBP1-LHP-A00322.synapse.com>
2020-11-06 16:52:05 +02:00
sbliu
f8cc600d06 TUP-28904 CI/CD build for DI jobs that contain 2 different versions of httpcore and httpclient are inconsistent with Studio build。
Add dependency jars from parent job, child job to ci job dependencies, the dependency jars is the latest version in parent and the latest version in child job, others is optional.
2020-11-06 17:22:16 +08:00
bhe-talendbj
3146ed307f fix(TUP-29208): add warning message for checking remote artifactory (#3803)
* fix(TUP-29208): add warning message for checking remote artifactory

* fix(TUP-29208): share module based on gavtc and sha1

* fix(TUP-29208): resolve comments
2020-11-06 15:40:57 +08:00
kjwang
f4f91a5323 TUP-28833 Multi-User: support custom javajet component in shared studio
TUP-28833 Multi-User: support custom javajet component in shared studio
https://jira.talendforge.org/browse/TUP-28833
2020-11-06 15:02:50 +08:00
Jane Ding
2aee38595f feat(TUP-29076):support memsql in studio metadata and components (#3779)
https://jira.talendforge.org/browse/TUP-29076

Signed-off-by: jding-tlnd <jding@talend.com>
2020-11-06 10:52:43 +08:00
wang wei
dde32c8b49 fix(TDI-44910): add memsql mapping (#3757)
* fix(TDI-44910): add memsql mapping

* rename it
2020-11-06 10:51:56 +08:00
jiezhang-tlnd
563b69231f Jzhang/bufix/73/tup 28893 support more typesfor share libs1 (#3777)
* fix(TUP-28893)Support more types of artifact for share libs
https://jira.talendforge.org/browse/TUP-28893

* ignore snapshot

* Maintenance/7.3 (#3774)

* fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758)

* fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)

Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>

* Maintenance/7.3 (#3775)

* fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758)

* fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)

Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>

* Maintenance/7.3 (#3776)

* fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758)

* fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)

Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route

* modify for nexus2

* support classifier for artifact repository

* add getClassifier and junits

* remove snapshot judgment

Co-authored-by: hcyi <hcyi@talend.com>
Co-authored-by: Zhiwei Xue <zwxue@talend.com>
Co-authored-by: apoltavtsev <apoltavtsev@gmail.com>
2020-11-05 14:19:31 +08:00
bhe-talendbj
1ea6227f59 bugfix(TUP-26278): Validate table name before persisting (#3108) (#3789)
* fix(TUP-26278): Validate table name before persisting

* fix(TUP-26278): Remove validating table name
2020-11-03 17:28:20 +08:00
kjwang
148e9e9253 Fix a NPE issue (#3795)
Fix TUP-28050 Multi-User: Avoid possible actions that are not supported on shared studio (Fix a NPE issue)
https://jira.talendforge.org/browse/TUP-28050
2020-11-03 15:25:25 +08:00
kjwang
1e6a43650f Kjwang/fix tup 28050 avoid possible actions (#3683)
* TUP-28274 Shared Studio: Can't find EMR 5.29 option in Hadoop Cluster
wizard after enable shared studio mode
https://jira.talendforge.org/browse/TUP-28274
2020-11-03 11:21:43 +08:00
jiezhang-tlnd
67616537b2 fix(TUP-28839)NPE when retrieve schema for JDBC connection of postgresql (#3700) (#3745)
https://jira.talendforge.org/browse/TUP-28839
2020-11-03 09:08:31 +08:00
bhe-talendbj
1df4e819e0 fix(TUP-29170): Fix jarpath change (#3786) 2020-11-02 17:30:12 +08:00
sbliu
b3d6c45f2d TUP-29059 Add logon project name to performance log. 2020-11-02 16:49:09 +08:00
wang wei
f1341cd69f fix(TDI-45038): add "timestamp" info for component start event for studio audit log(#3742) 2020-11-02 09:36:06 +08:00
bhe-talendbj
a64bcecb0c bugfix(TUP-28978): Add migration and make module name consistent (#3753)
* fix(TUP-28978): search local support other type and strip snapshot version

* fix(TUP-28978): Fix strip timestamp

* fix(TUP-2897): search remote dll, strip timestamp

* fix(TUP-28978): search local filtering timestamp

* fix(TUP-28978): Revert unnecessary changed files
2020-10-29 15:55:58 +08:00
apoltavtsev
5707d4db94 Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route 2020-10-27 14:00:05 +01:00
apoltavtsev
5e20a78752 Revert bugfix(TESB-30661) Wrong artifact version of the job called by a route 2020-10-27 13:59:10 +01:00
apoltavtsev
85e44cd5c6 bugfix(TESB-30661) Wrong artifact version of the job called by a route 2020-10-27 13:53:20 +01:00
apoltavtsev
8582c20f1a bugfix(TESB-30661) Wrong artifact version of the job called by a route 2020-10-27 13:52:46 +01:00
Zhiwei Xue
6216fd76ea fix(TUP-28846): Export Type Dropdown is empty when publishing Big Data (#3703)
Batch jobs job into Artifact Repository for Talend Studio 7.2 and 7.3
2020-10-27 18:18:36 +08:00
hcyi
71d339fc0a fix(TUP-28983):Can't switch to another version on Impala connection (#3741) (#3758) 2020-10-27 15:54:23 +08:00
Jane Ding
d286276e6d fix(TUP-28952):TOS: NPE when delete items to recycle bin (#3737)
https://jira.talendforge.org/browse/TUP-28952

Signed-off-by: jding-tlnd <jding@talend.com>
2020-10-20 10:15:38 +08:00
bhe-talendbj
3996d24893 feat(TUP-25246): Automatically detect custom maven URI for external jars (#3636)
* feat(TUP-25246): Automatically detect custom maven URI for external dependencies

* feat(TUP-25246): add Jar detector

* feat(TUP-25246): fix NPE

* feat(TUP-25246): fix NPE

* feat(TUP-25246): refactor

* feat(TUP-25246): fix error selections

* feat(TUP-25246): set default value for custom uri

* feat(TUP-25246): call LibManagerUiPlugin.getDefault().getLibrariesService() to deploy jar

* feat(TUP-25246): fix customuri for tLibraryload

* feat(TUP-25246): autofill custom uri if detected

* feat(TUP-25246): autofill default uri if detected

* feat(TUP-25246): Update metadata driver jar

* feat(TUP-25246): hide remote search if no remote repositoris

* feat(TUP-28342):Save maven url for components with parameter MODULE_LIST

* feat(TUP-25246): Fix dialog and moduleNeed

* feat(TUP-25246): set error message if module was already installed

* feat(TUP-25246): remove error message if module is already installed

* feat(TUP-25246): Fix metadata driver path

* feat(TUP-25246): add check for copy

* feat(TUP-25246): Fix cConfig and UI

* feat(TUP-25246): TUP-28910

* feat(TUP-25246): fix retrieve jar

* feat(TUP-25246): add migration

* feat(TUP-25246): fix diaglog behavior

* feat(TUP-25246): Fix jar path inside context

* feat(TUP-25246): remove quotes for jar

* feat(): Fix config dialog and revert TalendLibsServerManager

* feat(TUP-25246): fix configdialog

* feat(TUP-25246): remove nl plugin

* feat(TUP-25246): fix resolve local jar

* feat(TUP-25246): fix resolve local jar

Co-authored-by: Zhiwei Xue <zwxue@talend.com>
2020-10-16 16:22:56 +08:00
sbliu
f37b5b6073 TUP-28944 Issue with performance tracking with french locale (#3723)
* TUP-28944 fix the number format problem introduced by different locale.

* TUP-28944 make format method suitable to test.

* TUP-28944 add unit test for format a number.

* TUP-28944 fix the unit test.
2020-10-16 10:42:14 +08:00
jiezhang-tlnd
060a9c7f9e feat(TUP-28758)Add a warning when we login to a project and if there is (#3675)
* feat(TUP-28758)Add a warning when we login to a project and if there is
migration to do

* feat(TUP-28758)Add a warning when we login to a project and if there is
migration to do

* update warning info

* format code

* Add a warning when we login to a project and if there is migration to do

* collect warnings in one dialog

* remove nexus warning dialog

* update login info

* adjust warning content

* adjust warning content

* adjust warning content

* remove useless cases

* add junits

* update method and junits

* modify the text

* get right monthly patch name

* modify project patch version name

* remove Talend word in warning message
2020-10-15 15:18:29 +08:00
hzhao-talendbj
789b4c02c0 backport 27849 to 73 (#3707) 2020-10-14 16:52:29 +08:00
Chao MENG
4c3e3dfb2c fix(TUP-28903): NPE exception when clicking on context (#3711)
https://jira.talendforge.org/browse/TUP-28903
2020-10-14 11:40:24 +08:00
bhe-talendbj
7b3a20a3b3 fix(TUP-28659): Remove snapshot version of jars (#3664) 2020-10-14 09:44:21 +08:00
Jane Ding
e3880f3904 feat(TUP-28640):Improve JDBC database support framework to load (#3686)
* feat(TUP-28640):Improve JDBC database support framework to load
supported DBs and components automatically
https://jira.talendforge.org/browse/TUP-28640

* feat(TUP-28640):Improve JDBC database support framework to load
supported DBs and components automatically
https://jira.talendforge.org/browse/TUP-28640
2020-10-13 16:22:28 +08:00
Jane Ding
d54ed2a62c fix(TUP-28618):[Bug] db type dont show Delta in impact page. (#3676)
https://jira.talendforge.org/browse/TUP-28618
2020-10-13 16:20:23 +08:00
Mike Yan
cde1129ce4 Yyan/feat/tesb 29271 route debug 731 (#3709)
* feat(TESB-29271): Add camel breakpoint category

* fix(TESB-30624)[Debugging for Routes] NameAlreadyBoundException when
debug the ctalendjob
2020-10-13 15:15:33 +08:00
kjwang
1081ae3680 Kjwang/feat tup 27762 new version of ci (#3656)
* TUP-28274 Shared Studio: Can't find EMR 5.29 option in Hadoop Cluster
wizard after enable shared studio mode
https://jira.talendforge.org/browse/TUP-28274
2020-10-13 10:41:54 +08:00
chmyga
3e02adb157 fix(TDI-44962): fix stream closed exception (#3692)
* Recreate FileWriter with append after writing header

* Add csv writer to shared writer map

Co-authored-by: Dmytro Chmyga <dmytro.chmyga@synapse.com>
2020-10-12 11:24:55 +03:00
hzhao-talendbj
c212142789 change to filter CI case (#3697) 2020-10-12 12:46:38 +08:00
wang wei
ab67a4cc91 fix(TDI-44937): TalendDate.formatDateLocale()gives random value for FR country code in java 11.(#3679) 2020-10-12 10:20:41 +08:00
Chao MENG
675da3fc7d fix(TUP-27076): Unable to connect to codecommit GIT after the first (#3673)
login.
https://jira.talendforge.org/browse/TUP-27076
2020-10-10 10:40:59 +08:00
hzhao-talendbj
6f9ad53b0d remove exception log (#3688) 2020-10-09 11:10:19 +08:00
Mike Yan
6cfffe1775 feat(TESB-29271): Add camel breakpoint category (#3680) 2020-10-07 21:50:29 +08:00
chmyga
3c2d416677 Revert "fix(TDI-44826): Missing line in Resume log (#3640)" (#3691)
This reverts commit cf52e1e004.

Co-authored-by: Dmytro Chmyga <dmytro.chmyga@synapse.com>
2020-10-05 14:17:44 +03:00
vyu-talend
f7f5627679 fix(TDI-44726):add some necessary annotation. (#3666) 2020-09-27 15:14:19 +08:00
kjwang
939c328d1b Fix TUP-28603 'Export item' missing signature (#3643)
* Fix TUP-28603 'Export item' missing signature
https://jira.talendforge.org/browse/TUP-28603
2020-09-27 10:36:44 +08:00
Jane Ding
cb31b968bf feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE (#3670)
https://jira.talendforge.org/browse/TUP-27654
2020-09-23 14:20:02 +08:00
hcyi
c0636a6822 fix(TUP-25171):Issue when using components inside Joblet. (#3578) 2020-09-18 15:55:05 +08:00
Chao MENG
1e31b9ed97 fix(TUP-26413): Be able to compare metadata connection conflicts (#3657)
https://jira.talendforge.org/browse/TUP-26413
2020-09-18 12:25:15 +08:00
Jane Ding
d07e5beaf9 feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE (#3592)
* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
https://jira.talendforge.org/browse/TUP-27654

* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
https://jira.talendforge.org/browse/TUP-27654

Signed-off-by: jding-tlnd <jding@talend.com>

* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
https://jira.talendforge.org/browse/TUP-27654

Signed-off-by: jding-tlnd <jding@talend.com>

* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
TUP-28615:[Bug] Save the property to metadata will change Delta Lake to
JDBC.
https://jira.talendforge.org/browse/TUP-27654

* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
https://jira.talendforge.org/browse/TUP-27654

* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
https://jira.talendforge.org/browse/TUP-27654

* feat(TUP-27654):Databricks Delta Lake - Support ELT & MERGE
https://jira.talendforge.org/browse/TUP-27654
2020-09-18 12:01:17 +08:00
Chao MENG
281c39428c fix(TUP-28682): Connection info missing when create dataset for tck (#3655)
components

https://jira.talendforge.org/browse/TUP-28682
2020-09-17 14:29:02 +08:00
sbliu
3eaef89b5d feature(TUP-28405) Collect hardware info.Add functio to measure IO speed, startup time.
Collect hardware info.
Add functio to measure IO speed, startup time, remove unit.
2020-09-17 14:23:29 +08:00
wang wei
de34e9bcaa fix(TDI-44745): FileInputDelimited fails to properly consume CipherInputStream(#3590) 2020-09-17 14:16:06 +08:00
Chao MENG
5d1956966c Cmeng/patch/7.3/tup 28063 improve start (#3564)
* workitem(TUP-28063): Improve studio bundles reloading logic
https://jira.talendforge.org/browse/TUP-28063

* workitem(TUP-28063): Improve studio bundles reloading logic
https://jira.talendforge.org/browse/TUP-28063

* workitem(TUP-28063): Improve studio bundles reloading logic
https://jira.talendforge.org/browse/TUP-28063

Conflicts:
	main/plugins/org.talend.rcp/src/main/java/org/talend/rcp/intro/Application.java

* workitem(TUP-28063): Improve studio bundles reloading logic
https://jira.talendforge.org/browse/TUP-28063

Conflicts:
	main/plugins/org.talend.rcp/src/main/java/org/talend/rcp/intro/Application.java
2020-09-16 16:40:41 +08:00
Zhiwei Xue
1b1966058f feat(TUP-28323):Improve the maven build time spent during login project (#3606) 2020-09-16 16:35:28 +08:00
vdrokov
ca65a35c8e fix(TESB-29853): TESB-29819 Studio doesn't select model beans during exportingTESB-29853 (#3512)
Alternative fix
2020-09-16 08:19:44 +02:00
wang wei
c0a2a27815 fix(TDI-44623): [OBSERVABILITY] : add PID and (Thread-ID OR UUID) information (#3614)
* fix(TDI-44623): [OBSERVABILITY] : add PID and (Thread-ID OR UUID)
information

* decrease the expensive call number for getting pid

* fix(TDI-44807): Observability: initialize moment only when necessary

* fix(TDI-44806): Observability: Add customization for event frequency
2020-09-15 17:29:56 +08:00
hcyi
832b460e4b fix(TUP-28635):[bug] output path always be changed when path contains \ (#3645) 2020-09-15 14:51:51 +08:00
wang wei
3e8c7bb5d6 fix(TDI-44601): add databricks delta lake mapping file (#3511) 2020-09-15 10:10:26 +08:00
hcyi
2f5f944d0b feat(TUP-23827):Runtime Lineage for studio. (#3513)
* feat(TUP-23827):Runtime Lineage for studio.

* feat(TUP-23827):improve and add junits for Runtime Lineage.

* feat(TUP-23827):TUP-28161[bug]runtimelineage.prefs will generate
reproduced jobid+version。

* feat(TUP-23827):TUP-28186[bug]runtimelineage.prefs better remove job
version

* fix(TDI-44670): Runtime Lineage - Finalize component part

* fix the compiler issue for null value

* Revert "fix the compiler issue for null value"

This reverts commit 227fbb99bb.

* fix the compiler and NPE issue

* feat(TUP-23827):only support for DI jobs if selected "Use runtime
lineage for all jobs".

* feat(TUP-28539):Enhance GUI of Runtime Lineage.

* fix the compiler issue

* make private

* feat(TUP-28539):improve for Enhance GUI of Runtime Lineage.

* fix the compiler issue

* feat(TUP-28539):improve to filter dataviewer/guess schema for Enhance
GUI of Runtime Lineage.

* feat(TUP-28539):fix a path bug for Enhance
 GUI of Runtime Lineage.

Co-authored-by: wwang-talend <wwang@talend.com>
2020-09-14 18:42:45 +08:00
pyzhou
cf52e1e004 fix(TDI-44826): Missing line in Resume log (#3640) 2020-09-14 17:10:48 +08:00
Emmanuel GALLOIS
a05b2538a1 fix(TCOMP-1770): bump component-runtime to 1.1.25 2020-09-14 11:10:08 +02:00
Oleksandr Zhelezniak
b456595669 chore(TDI-44671): bump tck components (#3633)
* bump version of TCK components [1.9.0] -> [1.13.0]
2020-09-11 12:22:56 +03:00
bhe-talendbj
bf2699c7c9 fix(TUP-28617): add commons-codec (#3629) 2020-09-11 11:37:46 +08:00
kjwang
35047b473d Fix: TUP-28372 Modules Tab is not updated when switching branches. (Fix (#3623)
Fix: TUP-28372 Modules Tab is not updated when switching branches. (Fix a problem when switching branch may cause wrong save operation )
https://jira.talendforge.org/browse/TUP-28372
2020-09-11 10:45:29 +08:00
hzhao-talendbj
50233f93f3 tup-28322 remove networkValid and call method only once. (#3594)
* tup-28322 remove networkValid  and call method only once.

* add to createNewsPage

* add accessable

* change spelling mistake accessible

* use updateHistory to avoid call two times
2020-09-10 14:03:25 +08:00
nrousseau
bf3bbf6430 fix(TUP-28487): fix performance issue (#3598)
* fix(TUP-28487): fix performance issue
2020-09-10 09:52:02 +08:00
sbliu
735bbef8f3 feature(TUP-28334) Enhance logging of performance. 2020-09-09 10:34:17 +08:00
Chao MENG
28782a40d4 fix(TUP-28316): Left git related issues checked with support (#3566)
* fix(TUP-28316): Left git related issues checked with support
https://jira.talendforge.org/browse/TUP-28316

Conflicts:
	main/plugins/org.talend.core.repository/src/main/java/org/talend/core/repository/recyclebin/RecycleBinManager.java

* fix(TUP-28316): Left git related issues checked with support
https://jira.talendforge.org/browse/TUP-28316
2020-09-08 16:58:16 +08:00
Emmanuel GALLOIS
da57b8a80a feat(TCOMP-1651): bump component-runtime to 1.1.24 (#3620) 2020-09-08 10:20:07 +02:00
zshen-talend
9d0049bede Zshen/bugfix/fix switch context issue (#3619) 2020-09-08 15:21:41 +08:00
jiezhang-tlnd
1bd0f7939c feat(TUP-28432)Upgrade AWS SDK's version (#3609)
https://jira.talendforge.org/browse/TUP-28432
2020-09-07 16:41:56 +08:00
kjwang
b8e4c3bc7a Fix: TUP-28372 Modules Tab is not updated when switching branches. (#3587)
Fix: TUP-28372 Modules Tab is not updated when switching branches.
https://jira.talendforge.org/browse/TUP-28372
2020-09-02 17:02:10 +08:00
bhe-talendbj
931178184c fix(TUP-27268): Add isRequiredAlways for ILoginTask (#3593) 2020-09-02 16:02:34 +08:00
Chao MENG
3f8fc4ef4f fix(TUP-26341): Talend Token Collector doesn't work with http/https (#3575)
proxy
https://jira.talendforge.org/browse/TUP-26341

Conflicts:
	main/plugins/org.talend.core.ui/src/main/java/org/talend/core/ui/token/TokenCollectorFactory.java
2020-09-01 16:01:24 +08:00
hcyi
ad3490606d fix(TUP-27820):context group with same key causing the issue. (#3581) 2020-08-26 17:41:53 +08:00
Zhiwei Xue
085406b8da fix(TUP-28324): Avoid pom resolution for studio artifacts (#3570) 2020-08-26 09:54:52 +08:00
Emmanuel GALLOIS
e9ef85f8e5 patch(TPS-4101): bump component-runtime to 1.1.15.2 (#3583) 2020-08-25 12:41:47 +02:00
Zhiwei Xue
7543617b31 fix(TUP-28419):Sync patch m2 jars should not be done after TCK server (#3584)
startup
2020-08-25 11:15:06 +08:00
bhe-talendbj
e21bb53627 fix(TUP-28308): build error and joblet maven xml file has incorrect values and Couldn't parse data error message (#3576)
* fix(TUP-28308): Check multiple version of joblets

* fix(TUP-28308): Revert unnecessary changes
2020-08-25 10:27:14 +08:00
hcyi
a8f36c79f5 fix(TUP-28134):TCK Components shown as hidden by default in the palette (#3530) 2020-08-25 09:57:04 +08:00
jiezhang-tlnd
58171338ac fix(TUP-28078)metadata folder generated under .../poms/jobs when create (#3549)
new connection in remote project
https://jira.talendforge.org/browse/TUP-28078
2020-08-24 10:28:50 +08:00
Jane Ding
cebd4646d9 fix(TUP-25566):All the recycle bin should not be build or executed (#3557)
https://jira.talendforge.org/browse/TUP-25566
2020-08-17 16:03:06 +08:00
jiezhang-tlnd
9f2997692a Jzhang/feat/731/tup 27862 re (#3542)
* feat(TUP-27862)Add possibility to customize pattern for dates when
2020-08-13 18:02:25 +08:00
jiezhang-tlnd
1387f66674 fix(TUP-26329)impala metadata not in sync with the impala component (#3514)
* fix(TUP-26329)impala metadata not in sync with the impala component
(#3118)

* fix(TUP-26329)impala metadata not in sync with the impala component

* fix(TUP-26329)impala metadata not in sync with the impala component

* fix(TUP-26329)impala metadata not in sync with the impala component

Conflicts:
	main/plugins/org.talend.core.runtime/src/main/java/org/talend/core/model/metadata/designerproperties/RepositoryToComponentProperty.java

* fix when "revert context", the "Additional JDBD Settings" is still (#3174)

context value like
context.cdh513judy_impala1_hiveAdditionalJDBCParameters, which should be
the detailed value.

Co-authored-by: hzhao-talendbj <49395568+hzhao-talendbj@users.noreply.github.com>
2020-08-13 14:42:55 +08:00
hzhao-talendbj
7ee7eddb9d Hengzhao/backport73/tup 25103 (#3541)
* TUP-25103

* remove service

* TUP-25103
2020-08-13 13:56:34 +08:00
Jane Ding
e674ec78f7 fix(TUP-28180):[bug]Only latest version deleted from root pom when check (#3529)
* fix(TUP-28180):[bug]Only latest version deleted from root pom when check
"Exclude deleted items" and delete to recycle bin

Signed-off-by: jding-tlnd <jding@talend.com>

* fix(TUP-28205):[bug] items not removed from pom file by default if the
imported items have some in recycle bin before
https://jira.talendforge.org/browse/TUP-28205

Signed-off-by: jding-tlnd <jding@talend.com>

* fix(TUP-28205):[bug] items not removed from pom file by default if the
imported items have some in recycle bin before
https://jira.talendforge.org/browse/TUP-28205

Signed-off-by: jding-tlnd <jding@talend.com>
2020-08-13 10:42:48 +08:00
Nicolas Rousseau
08c2082e8a chore(TUP-26710):update version 2020-08-12 15:20:28 +08:00
bhe-talendbj
8740f3b1da fix(TUP-27506): Update minor version (#3546) 2020-08-12 14:56:30 +08:00
jiezhang-tlnd
e08c225198 revert fix (#3547) 2020-08-12 14:54:57 +08:00
bhe-talendbj
fef6605f83 fix(TUP-27374): fix test case (#3545)
* fix(TUP-27374): fix test case

* fix(TUP-27374): fix test case
2020-08-11 15:10:32 +08:00
jiezhang-tlnd
af83d9d393 fix(TUP-28078)metadata folder generated under .../poms/jobs when create (#3520)
new connection in remote project
https://jira.talendforge.org/browse/TUP-28078
2020-08-11 14:23:22 +08:00
wang wei
45a92717f9 fix(TDI-44192): Update dom4j to 2.1.3 (#3453) 2020-08-11 10:19:08 +08:00
Dmytro Grygorenko
7191b77c5c Enable precision check for Timestamp datatype. (#3495)
* fix(TDI-44457): added precision check for Datetime and Timestamp datatypes.

* fix(TDI-44457): Remove default precision value

* fix(TDI-44457): add default precision 3 (to be tested)

* fix(TDI-44457): enable precision for Timestamp only (to be tested)
2020-08-10 15:47:42 +03:00
Jürgen Kindler
18c2e53223 TPSVC-15154 - allow debug logs for deletions to see which files are purged (#3531) 2020-08-06 21:13:32 +08:00
Zhiwei Xue
16fbcf34f3 feat(TUP-28142):Add talendDebug for CI (#3525) 2020-08-06 17:50:53 +08:00
Chao MENG
495ebda155 fix(TUP-27989): Git related issues checked with support (#3494)
https://jira.talendforge.org/browse/TUP-27989
2020-08-06 15:02:39 +08:00
wang wei
8f36da5ee1 fix(TDI-44481): upgrade commons-beanutils to 1.9.4 (#3448) 2020-08-06 12:45:58 +08:00
bhe-talendbj
0828f2951a fix(TUP-28002): Search release repository before uploading jars (#3503)
* fix(TUP-28002): Search release repository before uploading jars

* fix(TUP-28002): Fix NPE

* fix(TUP-28002): Check snapshot as well
2020-08-04 18:11:02 +08:00
Jane Ding
5da999c4bd feat(TUP-25566):All the recycle bin should not be build or executed (#3527)
https://jira.talendforge.org/browse/TUP-25566

Signed-off-by: jding-tlnd <jding@talend.com>
2020-08-04 11:28:47 +08:00
jiezhang-tlnd
39eee08210 Jzhang/feat/73/tup 27862 (#3480)
* feat(TUP-27862)Add possibility to customize pattern for dates when
retrieve table

* feat(TUP-27862)Add possibility to customize pattern for dates when
retrieve table

* feat(TUP-27862)Add possibility to customize pattern for dates when
retrieve table

* feat(TUP-27862)Add possibility to customize pattern for dates when
retrieve table

* feat(TUP-27862)Add possibility to customize pattern for dates when
retrieve table

* add junit

* add junits

* add junits
2020-08-03 09:40:57 +08:00
Chao MENG
1ab6486d9a fix(TUP-27953): tSalesforceInput component displays in-job (#3505)
tSalesforceConnection incorrectly
https://jira.talendforge.org/browse/TUP-27953
2020-07-31 11:11:20 +08:00
Jane Ding
aa7da99e0d feat(TUP-25566)All the recycle bin should not be build or executed (#3484)
* feat(TUP-25566)All the recycle bin should not be build or executed
https://jira.talendforge.org/browse/TUP-25566

* feat(TUP-25566)All the recycle bin should not be build or executed
https://jira.talendforge.org/browse/TUP-25566

* feat(TUP-25566)All the recycle bin should not be build or executed
https://jira.talendforge.org/browse/TUP-25566
2020-07-31 02:35:32 +08:00
209 changed files with 7640 additions and 1269 deletions

View File

@@ -13,7 +13,6 @@
<plugin id="org.talend.libraries.jdbc.ingres" download-size="0" install-size="0" version="0.0.0"/>
<plugin id="org.talend.libraries.jdbc.mysql" download-size="0" install-size="0" version="0.0.0"/>
<plugin id="org.talend.libraries.jdbc.paraccel" download-size="0" install-size="0" version="0.0.0"/>
<plugin id="org.talend.libraries.jdbc.postgresql" download-size="0" install-size="0" version="0.0.0"/>
<plugin id="org.talend.libraries.jdbc.sqlite3" download-size="0" install-size="0" version="0.0.0"/>
<plugin id="org.talend.libraries.jdbc.teradata" download-size="0" install-size="0" version="0.0.0"/>
</feature>

View File

@@ -16,7 +16,8 @@ Require-Bundle: org.apache.log4j;visibility:=reexport,
org.talend.utils,
org.eclipse.core.net,
org.eclipse.m2e.core,
org.eclipse.m2e.maven.runtime
org.eclipse.m2e.maven.runtime,
org.eclipse.core.resources
Export-Package: org.talend.commons,
org.talend.commons.exception,
org.talend.commons.i18n,

View File

@@ -111,7 +111,8 @@ public class CommonsPlugin implements BundleActivator {
}
public static boolean isDebugMode() {
return ArrayUtils.contains(Platform.getApplicationArgs(), TalendDebugHandler.TALEND_DEBUG);
return Boolean.getBoolean("talendDebug") //$NON-NLS-1$
|| ArrayUtils.contains(Platform.getApplicationArgs(), TalendDebugHandler.TALEND_DEBUG);
}
public static boolean isJUnitTest() {

View File

@@ -232,10 +232,50 @@ public class VersionUtils {
* Check if studio version < other studio version record in remote project.
*/
public static boolean isInvalidProductVersion(String remoteFullProductVersion) {
String localProductVersion = getInternalVersion();
return isInvalidProductVersion(localProductVersion, remoteFullProductVersion);
}
protected static boolean isInvalidProductVersion(String localProductVersion, String remoteFullProductVersion) {
if (remoteFullProductVersion == null) {
return false;
}
return getInternalVersion().compareTo(getProductVersionWithoutBranding(remoteFullProductVersion)) < 0;
if (skipCheckingNightlyBuilds(localProductVersion, remoteFullProductVersion)) {
return false;
}
return localProductVersion.compareTo(getProductVersionWithoutBranding(remoteFullProductVersion)) < 0;
}
public static boolean productVersionIsNewer(String remoteFullProductVersion) {
String localProductVersion = getInternalVersion();
return productVersionIsNewer(localProductVersion, remoteFullProductVersion);
}
protected static boolean productVersionIsNewer(String localProductVersion, String remoteFullProductVersion) {
if (remoteFullProductVersion == null) {
return false;
}
if (skipCheckingNightlyBuilds(localProductVersion, remoteFullProductVersion)) {
return false;
}
return localProductVersion.compareTo(getProductVersionWithoutBranding(remoteFullProductVersion)) > 0;
}
private static boolean skipCheckingNightlyBuilds(String localProductVersion, String remoteFullProductVersion) {
String separator = "-"; //$NON-NLS-1$
String localSuffix = StringUtils.substringAfterLast(localProductVersion, separator);
String remoteProductVersion = getProductVersionWithoutBranding(remoteFullProductVersion);
String remoteSuffix = StringUtils.substringAfterLast(remoteProductVersion, separator);
String nightly = "SNAPSHOT"; //$NON-NLS-1$
String milestone = "M"; //$NON-NLS-1$
if ((localSuffix.equals(nightly) || localSuffix.startsWith(milestone))
&& (remoteSuffix.equals(nightly) || remoteSuffix.startsWith(milestone))) {
// skip checking between nightly/milestone build.
return true;
}
return false;
}
public static String getTalendVersion(String productVersion) {
@@ -310,4 +350,24 @@ public class VersionUtils {
}
}
public static String getSimplifiedPatchName(String projectPatchName) {
if (projectPatchName != null) {
String result = null;
if (projectPatchName.contains("_") && projectPatchName.split("_").length >= 3) {
result = projectPatchName.split("_")[2];
if (!result.startsWith("R")) {
return null;
}
}
if (projectPatchName.contains("-")) {
String[] split = projectPatchName.split("-");
if (split != null && split.length > 0) {
return result + "-" + split[split.length - 1];
}
}
}
return null;
}
}

View File

@@ -15,6 +15,7 @@ package org.talend.commons.utils.network;
import java.lang.reflect.Field;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.Inet6Address;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.PasswordAuthentication;
@@ -22,7 +23,11 @@ import java.net.SocketException;
import java.net.URI;
import java.net.URL;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import java.util.regex.Pattern;
import org.apache.commons.lang.StringUtils;
@@ -150,6 +155,28 @@ public class NetworkUtil {
@Override
protected PasswordAuthentication getPasswordAuthentication() {
String httpProxyHost = System.getProperty("http.proxyHost"); //$NON-NLS-1$
String httpProxyPort = System.getProperty("http.proxyPort"); //$NON-NLS-1$
String httpsProxyHost = System.getProperty("https.proxyHost"); //$NON-NLS-1$
String httpsProxyPort = System.getProperty("https.proxyPort"); //$NON-NLS-1$
String requestingHost = getRequestingHost();
int requestingPort = getRequestingPort();
String proxyHost = null;
String proxyPort = null;
boolean isHttp = false;
if ("http".equalsIgnoreCase(getRequestingScheme())) {
isHttp = true;
}
if (isHttp && StringUtils.isNotBlank(httpProxyHost)) {
proxyHost = httpProxyHost;
proxyPort = httpProxyPort;
} else {
proxyHost = httpsProxyHost;
proxyPort = httpsProxyPort;
}
if (!StringUtils.equals(proxyHost, requestingHost) || !StringUtils.equals(proxyPort, "" + requestingPort)) {
return null;
}
String httpProxyUser = System.getProperty("http.proxyUser"); //$NON-NLS-1$
String httpProxyPassword = System.getProperty("http.proxyPassword"); //$NON-NLS-1$
String httpsProxyUser = System.getProperty("https.proxyUser"); //$NON-NLS-1$
@@ -167,7 +194,11 @@ public class NetworkUtil {
proxyPassword = httpsProxyPassword.toCharArray();
}
}
return new PasswordAuthentication(proxyUser, proxyPassword);
if (StringUtils.isBlank(proxyUser)) {
return null;
} else {
return new PasswordAuthentication(proxyUser, proxyPassword);
}
}
});
@@ -206,6 +237,54 @@ public class NetworkUtil {
}
}
public static List<String> getLocalLoopbackAddresses(boolean wrapIpV6) {
Set<String> addresses = new LinkedHashSet<>();
try {
addresses.add(getIp(InetAddress.getLoopbackAddress(), wrapIpV6));
} catch (Exception e) {
ExceptionHandler.process(e);
}
try {
Enumeration<NetworkInterface> networkInterfaces = NetworkInterface.getNetworkInterfaces();
while (networkInterfaces.hasMoreElements()) {
NetworkInterface networkInterface = networkInterfaces.nextElement();
Enumeration<InetAddress> inetAddresses = networkInterface.getInetAddresses();
while (inetAddresses.hasMoreElements()) {
InetAddress inetAddress = inetAddresses.nextElement();
if (inetAddress != null && inetAddress.isLoopbackAddress()) {
addresses.add(getIp(inetAddress, wrapIpV6));
}
}
}
} catch (Exception e) {
ExceptionHandler.process(e);
}
if (addresses.isEmpty()) {
addresses.add("127.0.0.1");
String ipv6Loopback = "::1";
if (wrapIpV6) {
ipv6Loopback = "[" + ipv6Loopback + "]";
}
addresses.add(ipv6Loopback);
}
return new ArrayList<>(addresses);
}
private static String getIp(InetAddress inetAddress, boolean wrapIpV6) {
if (wrapIpV6 && Inet6Address.class.isInstance(inetAddress)) {
String addr = inetAddress.getHostAddress();
if (!addr.startsWith("[") || !addr.endsWith("]")) {
addr = "[" + addr + "]";
}
return addr;
} else {
return inetAddress.getHostAddress();
}
}
public static boolean isSelfAddress(String addr) {
if (addr == null || addr.isEmpty()) {
return false; // ?

View File

@@ -25,6 +25,7 @@ import java.net.URISyntaxException;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
@@ -38,6 +39,8 @@ import java.util.Set;
import org.apache.commons.lang.StringUtils;
import org.apache.log4j.Priority;
import org.eclipse.core.internal.net.ProxyManager;
import org.eclipse.core.internal.net.ProxyType;
import org.eclipse.core.net.proxy.IProxyChangeEvent;
import org.eclipse.core.net.proxy.IProxyService;
import org.talend.commons.CommonsPlugin;
import org.talend.commons.exception.ExceptionHandler;
@@ -77,6 +80,14 @@ public class TalendProxySelector extends ProxySelector {
private static final String PROP_DISABLE_DEFAULT_SELECTOR_PROVIDER = "talend.studio.proxy.disableDefaultSelectorProvider";
private static final String PROP_PROXY_EXCLUDE_LOOPBACK_ADDRESS_AUTOMATICALLY = "talend.studio.proxy.excludeLoopbackAutomatically";
private static final String PROP_PROXY_EXCLUDE_LOOPBACK_ADDRESS_AUTOMATICALLY_DEFAULT = "true";
private static final String PROP_PROXY_HTTP_NON_PROXYHOSTS = "http.nonProxyHosts";
private static final String PROP_PROXY_HTTPS_NON_PROXYHOSTS = "https.nonProxyHosts";
/**
* Example: update.talend.com,socket:http,https:http;nexus.talend.com,socket,http;,socket:http
*/
@@ -104,6 +115,8 @@ public class TalendProxySelector extends ProxySelector {
private EProxySelector eProxySelector;
private IProxyService proxyManager;
final private Map<Object, Collection<IProxySelectorProvider>> selectorProviders;
private Map<String, Map<String, String>> hostMap;
@@ -114,6 +127,10 @@ public class TalendProxySelector extends ProxySelector {
private static Object instanceLock = new Object();
private List<String> localLoopbackAddresses;
private Object localLoopbackAddressesLock = new Object();
private boolean printProxyLog = false;
private boolean allowProxyRedirect = false;
@@ -126,6 +143,8 @@ public class TalendProxySelector extends ProxySelector {
private boolean updateSystemPropertiesForJre = true;
private boolean excludeLoopbackAddressAutomatically = false;
private TalendProxySelector(final ProxySelector eclipseDefaultSelector) {
this.eclipseDefaultSelector = eclipseDefaultSelector;
this.jreDefaultSelector = new DefaultProxySelector();
@@ -139,6 +158,8 @@ public class TalendProxySelector extends ProxySelector {
executeConnectionFailed = Boolean.valueOf(System.getProperty(PROP_EXECUTE_CONNECTION_FAILED, Boolean.TRUE.toString()));
updateSystemPropertiesForJre = Boolean
.valueOf(System.getProperty(PROP_UPDATE_SYSTEM_PROPERTIES_FOR_JRE, Boolean.TRUE.toString()));
excludeLoopbackAddressAutomatically = Boolean.valueOf(System.getProperty(
PROP_PROXY_EXCLUDE_LOOPBACK_ADDRESS_AUTOMATICALLY, PROP_PROXY_EXCLUDE_LOOPBACK_ADDRESS_AUTOMATICALLY_DEFAULT));
switch (System.getProperty(PROP_PROXY_SELECTOR, PROP_PROXY_SELECTOR_DEFAULT).toLowerCase()) {
case PROP_PROXY_SELECTOR_JRE:
@@ -148,11 +169,83 @@ public class TalendProxySelector extends ProxySelector {
this.eProxySelector = EProxySelector.eclipse_default;
break;
}
proxyManager = ProxyManager.getProxyManager();
checkProxyManager(IProxyChangeEvent.PROXY_DATA_CHANGED);
proxyManager.addProxyChangeListener(event -> checkProxyManager(event.getChangeType()));
initHostMap();
initRedirectList();
}
private void checkProxyManager(int changeEvent) {
try {
if (IProxyChangeEvent.PROXY_DATA_CHANGED == changeEvent
|| IProxyChangeEvent.NONPROXIED_HOSTS_CHANGED == changeEvent) {
if (this.excludeLoopbackAddressAutomatically && proxyManager.isProxiesEnabled()) {
List<String> addresses = getLocalLoopbackAddresses();
if (addresses != null && !addresses.isEmpty()) {
if (org.eclipse.core.internal.net.ProxySelector
.canSetBypassHosts(org.eclipse.core.internal.net.ProxySelector.getDefaultProvider())) {
List<String> configuredProxies = Arrays.asList(proxyManager.getNonProxiedHosts());
if (!configuredProxies.containsAll(addresses)) {
Set<String> nonProxyHosts = new HashSet<>(addresses);
nonProxyHosts.addAll(configuredProxies);
ExceptionHandler.log(
this.getClass().getName() + ":" + "-D" + PROP_PROXY_EXCLUDE_LOOPBACK_ADDRESS_AUTOMATICALLY
+ "=true, adding missing loopback addresses into eclipse nonProxyHosts: "
+ nonProxyHosts);
proxyManager.setNonProxiedHosts(nonProxyHosts.toArray(new String[0]));
}
} else {
updateNonProxyHosts(addresses, PROP_PROXY_HTTP_NON_PROXYHOSTS);
updateNonProxyHosts(addresses, PROP_PROXY_HTTPS_NON_PROXYHOSTS);
}
}
}
}
} catch (Exception e) {
ExceptionHandler.process(e);
}
}
private void updateNonProxyHosts(List<String> localLoopbackAddresses, final String nonProxyProperty) {
if (localLoopbackAddresses != null && !localLoopbackAddresses.isEmpty()) {
Set<String> nonProxyHosts = new HashSet<>(localLoopbackAddresses);
String property = System.getProperty(nonProxyProperty);
boolean update = true;
if (StringUtils.isNotBlank(property)) {
List<String> configuredProxies = Arrays.asList(ProxyType.convertPropertyStringToHosts(property));
if (configuredProxies.containsAll(localLoopbackAddresses)) {
update = false;
} else {
nonProxyHosts.addAll(configuredProxies);
}
}
if (update) {
ExceptionHandler.log(this.getClass().getName() + ":" + "-D" + PROP_PROXY_EXCLUDE_LOOPBACK_ADDRESS_AUTOMATICALLY
+ "=true, adding missing loopback addresses into " + nonProxyProperty + ": " + nonProxyHosts);
System.setProperty(nonProxyProperty,
ProxyType.convertHostsToPropertyString(nonProxyHosts.toArray(new String[0])));
}
}
}
private List<String> getLocalLoopbackAddresses() {
if (this.localLoopbackAddresses == null) {
synchronized (localLoopbackAddressesLock) {
if (this.localLoopbackAddresses == null) {
List<String> addresses = NetworkUtil.getLocalLoopbackAddresses(false);
final String localhost = "localhost";
if (!addresses.contains(localhost)) {
addresses.add(localhost);
}
this.localLoopbackAddresses = addresses;
}
}
}
return this.localLoopbackAddresses;
}
private void initHostMap() {
try {
hostMap = new HashMap<>();
@@ -282,7 +375,27 @@ public class TalendProxySelector extends ProxySelector {
ExceptionHandler.log("TalendProxySelector.select " + uri);
}
if (uri == null) {
return Collections.EMPTY_LIST;
List<Proxy> result = new ArrayList<>();
result.add(Proxy.NO_PROXY);
return result;
}
try {
if (this.excludeLoopbackAddressAutomatically) {
List<String> addresses = getLocalLoopbackAddresses();
if (addresses != null) {
String host = uri.getHost();
if (addresses.contains(host)) {
if (printProxyLog) {
ExceptionHandler.log(uri + " is excluded from proxy");
}
List<Proxy> result = new ArrayList<>();
result.add(Proxy.NO_PROXY);
return result;
}
}
}
} catch (Exception e) {
ExceptionHandler.process(e);
}
URI validatedUri = validateUri(uri);
Set<Proxy> results = new LinkedHashSet<>();
@@ -332,6 +445,9 @@ public class TalendProxySelector extends ProxySelector {
ExceptionHandler.log("Selected proxys for " + uri + ", " + proxys);
ExceptionHandler.process(new Exception("Proxy call stacks"), Priority.INFO);
}
if (results.isEmpty()) {
results.add(Proxy.NO_PROXY);
}
return new LinkedList<Proxy>(results);
}

View File

@@ -49,11 +49,17 @@ public class EclipseCommandLine {
*/
static public final String TALEND_PROJECT_TYPE_COMMAND = "-talendProjectType"; //$NON-NLS-1$
static public final String TALEND_LICENCE_PATH = "talend.licence.path"; //$NON-NLS-1$
static public final String ARG_TALEND_LICENCE_PATH = "-" + TALEND_LICENCE_PATH; //$NON-NLS-1$
/**
* for relaunch of the plugins when relaunching the Studio
*/
static public final String TALEND_RELOAD_COMMAND = "-talendReload"; //$NON-NLS-1$
static public final String LOGIN_ONLINE_UPDATE = "--loginOnlineUpdate";
static public final String ARG_TALEND_BUNDLES_CLEANED = "-talend.studio.bundles.cleaned"; //$NON-NLS-1$
static public final String PROP_TALEND_BUNDLES_DO_CLEAN = "-talend.studio.bundles.doclean"; //$NON-NLS-1$

View File

@@ -0,0 +1,127 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.commons.utils.time;
import java.util.Enumeration;
import java.util.Properties;
import org.apache.log4j.Hierarchy;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;
import org.apache.log4j.RollingFileAppender;
import org.apache.log4j.spi.LoggerFactory;
import org.apache.log4j.spi.RootLogger;
import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.Platform;
public class PerformanceLogManager {
private Hierarchy hierarchy;
public PerformanceLogManager() {
Properties properties = new Properties();
properties.put("log4j.rootCategory", ", A1");
properties.put("log4j.appender.A1", RollingFileAppender.class.getName());
IPath performanceLogPath = Platform.getLogFileLocation().removeLastSegments(1).append("performance.log");
properties.put("log4j.appender.A1.File", performanceLogPath.toOSString());
properties.put("log4j.appender.A1.MaxBackupIndex", "10");// same as .log's max backup log file count
properties.put("log4j.appender.A1.MaxFileSize", "1000000");//1000*1000 byte, same as .log's max file size
properties.put("log4j.appender.A1.layout", "org.apache.log4j.PatternLayout");
properties.put("log4j.appender.A1.layout.ConversionPattern", "%d %-5p %c %x - %m%n");
this.hierarchy = new Hierarchy(new RootLogger(Level.INFO));
new PropertyConfigurator().doConfigure(properties,hierarchy);
}
/**
* Checks if this PluginLogManager is disabled for this level.
* @param level level value
* @return boolean true if it is disabled
*/
public boolean isDisabled(int level) {
return this.hierarchy.isDisabled(level);
}
/**
* Enable logging for logging requests with level l or higher.
* By default all levels are enabled.
* @param level level object
*/
public void setThreshold(Level level) {
this.hierarchy.setThreshold(level);
}
/**
* The string version of setThreshold(Level level)
* @param level level string
*/
public void setThreshold(String level) {
this.hierarchy.setThreshold(level);
}
/**
* Get the repository-wide threshold.
* @return Level
*/
public Level getThreshold() {
return this.hierarchy.getThreshold();
}
/**
* Returns a new logger instance named as the first parameter
* using the default factory. If a logger of that name already exists,
* then it will be returned. Otherwise, a new logger will be instantiated
* and then linked with its existing ancestors as well as children.
* @param name logger name
* @return Logger
*/
public Logger getLogger(String name) {
return this.hierarchy.getLogger(name);
}
/**
* The same as getLogger(String name) but using a factory instance instead of
* a default factory.
* @param name logger name
* @param factory factory instance
* @return Logger
*/
public Logger getLogger(String name, LoggerFactory factory) {
return this.hierarchy.getLogger(name,factory);
}
public Logger getRootLogger() {
return this.hierarchy.getRootLogger();
}
public Logger exists(String name) {
return this.hierarchy.exists(name);
}
public void shutdown() {
this.hierarchy.shutdown();
}
/**
* Returns all the loggers in this manager.
* @return Enumeration logger enumeration
*/
public Enumeration getCurrentLoggers() {
return this.hierarchy.getCurrentLoggers();
}
public void resetConfiguration() {
this.hierarchy.resetConfiguration();
}
}

View File

@@ -0,0 +1,371 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.commons.utils.time;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Properties;
import java.util.Random;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import org.eclipse.core.resources.IWorkspaceRoot;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.Platform;
import org.talend.commons.exception.CommonExceptionHandler;
/**
* DOC sbliu class global comment. Detailled comment
*/
public class PerformanceStatisticUtil {
private static final int MEGABYTE = 1024 * 1024;// MB = 1024*1024 byte
private static final int KILOBYTE = 1024;// kb=1024 byte
private static final int numOfBlocks = 256;
private static final int blockSizeKb = 512;
private static final String dataFile = "testio.data";
private static String recordingFileName = "performance_record";
private static File recordingFile = null;
private static enum BlockSequence {
SEQUENTIAL,
RANDOM;
}
public static enum StatisticKeys {
IO_COUNT("I/O.count"), // io count
IO_W_MB_SEC("I/O.write"), // write speed MB
IO_R_MB_SEC("I/O.read"), // read speed MB
IO_W_AVERAGE_MB_SEC("I/O.write.average"), // average speed of write MB
IO_R_AVERAGE_MB_SEC("I/O.read.average"), // average speed of read
STARTUP_AVERAGE("startup.average"),
STARTUP_MAX("startup.max"),
STARTUP_COUNT("startup.count");
private String key;
StatisticKeys(String _key) {
key = _key;
}
public String get() {
return key;
}
}
public static void recordStartupEpapsedTime(double elapsedTimeInSeconds) {
File file = getRecordingFile();
Properties props = read(file, true);
String propCount = props.getProperty(StatisticKeys.STARTUP_COUNT.get(), "0");
String propMax = props.getProperty(StatisticKeys.STARTUP_MAX.get(), "0");
String propAverage = props.getProperty(StatisticKeys.STARTUP_AVERAGE.get(), "0");
int iPropCount = Integer.parseInt(propCount);
double iPropMax = Double.parseDouble(propMax);
double iPropAverage = Double.parseDouble(propAverage);
iPropMax = iPropMax > elapsedTimeInSeconds ? iPropMax : elapsedTimeInSeconds;
iPropAverage = (iPropAverage * iPropCount + elapsedTimeInSeconds) / (iPropCount + 1);
iPropCount++;
props.setProperty(StatisticKeys.STARTUP_COUNT.get(), "" + iPropCount);
props.setProperty(StatisticKeys.STARTUP_MAX.get(), "" + iPropMax);
props.setProperty(StatisticKeys.STARTUP_AVERAGE.get(), "" + iPropAverage);
store(file, props);
}
public static File getRecordingFile() {
if (recordingFile != null) {
return recordingFile;
}
String configurationLocation = Platform.getConfigurationLocation().getURL().getPath();
File file = new File(configurationLocation + "/" + recordingFileName);
return file;
}
public static void setRecordingFile(File _recordingFile) {
recordingFile = _recordingFile;
}
public static synchronized Properties read(File recordFile, boolean createIfNotExist) {
Properties props = new Properties();
if (recordFile != null && exist(recordFile, createIfNotExist)) {
FileInputStream inStream = null;
try {
inStream = new FileInputStream(recordFile);
props.load(inStream);
} catch (Exception e) {
CommonExceptionHandler.log(e.getMessage());
} finally {
if (inStream != null) {
try {
inStream.close();
} catch (IOException e) {//
}
}
}
}
return props;
}
public static synchronized void store(File recordFile, Properties props) {
if (props == null) {
return;
}
if (recordFile != null && exist(recordFile, true)) {
FileOutputStream outputStream = null;
try {
outputStream = new FileOutputStream(recordFile);
props.store(outputStream, "");
} catch (IOException e) {
CommonExceptionHandler.log(e.getMessage());
} finally {
if (outputStream != null) {
try {
outputStream.close();
} catch (IOException e) {
//
}
}
}
}
}
private static boolean exist(File recordFile, boolean createIfNotExist) {
boolean exists = recordFile.exists();
if (!exists && createIfNotExist) {
try {
exists = recordFile.createNewFile();
if (!exists) {
throw new FileNotFoundException(recordFile.getName());
}
} catch (Exception e) {
CommonExceptionHandler.log(e.getMessage());
return false;
}
}
return exists;
}
private static Lock lock = new ReentrantLock();
private static Condition condition = lock.newCondition();
private static boolean measureIOFinished = true;
public static void waitUntilFinish() throws InterruptedException {
lock.lock();
try {
if(!measureIOFinished) {
condition.await(20, TimeUnit.SECONDS);
}
} finally {
lock.unlock();
}
}
public static void measureIO() {
new Thread() {
public void run() {
measureIOFinished = false;
try {
_measureIO();
} finally {
measureIOFinished = true;
}
}
}.start();
}
private static void _measureIO() {
File file = getRecordingFile();
Properties props = read(file, true);
IWorkspaceRoot root = ResourcesPlugin.getWorkspace().getRoot();
File workspace = root.getLocation().makeAbsolute().toFile();
File locationDir = new File(workspace, "temp"); // here is workspace/temp dir
File testFile = detectTestDataFile(locationDir);
if (testFile != null) {
measureWrite(props, testFile);
measureRead(props, testFile);
store(file, props);
}
}
private static void measureWrite(Properties props, File testFile) {
int blockSize = blockSizeKb * KILOBYTE;
long startTime = System.nanoTime();
long totalBytesWrittenInMark = writeIO(numOfBlocks, BlockSequence.RANDOM, blockSize, testFile);
totalBytesWrittenInMark = totalBytesWrittenInMark + writeIO(numOfBlocks, BlockSequence.SEQUENTIAL, blockSize, testFile);
long endTime = System.nanoTime();
long elapsedTimeNs = endTime - startTime;
double sec = (double) elapsedTimeNs / (double) 1000000000;
double mbWritten = (double) totalBytesWrittenInMark / (double) MEGABYTE;
double bwMbSec = mbWritten / sec;
String ioCount = props.getProperty(StatisticKeys.IO_COUNT.get(), "0");
String ioWAverageMbSec = props.getProperty(StatisticKeys.IO_W_AVERAGE_MB_SEC.get(), "0");
String ioWMbSec = props.getProperty(StatisticKeys.IO_W_MB_SEC.get(), "0");
int digital_ioCount = Integer.parseInt(ioCount);
double digital_ioWAverageMbSec = Double.parseDouble(ioWAverageMbSec);
double digital_ioWMbSec = Double.parseDouble(ioWMbSec);
digital_ioWAverageMbSec = (digital_ioWAverageMbSec * digital_ioCount + bwMbSec) / (digital_ioCount + 1);
digital_ioWMbSec = bwMbSec;
props.setProperty(StatisticKeys.IO_W_AVERAGE_MB_SEC.get(), format(digital_ioWAverageMbSec));
props.setProperty(StatisticKeys.IO_W_MB_SEC.get(), format(digital_ioWMbSec));
}
private static long writeIO(int numOfBlocks, BlockSequence blockSequence, int blockSize, File testFile) {
byte[] blockArr = new byte[blockSize];
for (int b = 0; b < blockArr.length; b++) {
if (b % 2 == 0) {
blockArr[b] = (byte) 0xFF;
}
}
String mode = "rwd";// "rwd"
long totalBytesWrittenInMark = 0;
try {
try (RandomAccessFile rAccFile = new RandomAccessFile(testFile, mode)) {
for (int b = 0; b < numOfBlocks; b++) {
if (blockSequence == BlockSequence.RANDOM) {
int rLoc = randInt(0, numOfBlocks - 1);
rAccFile.seek(rLoc * blockSize);
} else {
rAccFile.seek(b * blockSize);
}
rAccFile.write(blockArr, 0, blockSize);
totalBytesWrittenInMark += blockSize;
}
}
} catch (IOException e) {
CommonExceptionHandler.log(e.getMessage());
}
return totalBytesWrittenInMark;
}
private static File detectTestDataFile(File location) {
if (!location.exists()) {
location.mkdirs();
}
File testFile = null;
try {
testFile = new File(location.getAbsolutePath() + File.separator + dataFile);
testFile.deleteOnExit();
testFile.createNewFile();
} catch (IOException e) {
CommonExceptionHandler.log(e.getMessage());
}
return testFile;
}
public static void measureRead(Properties props, File testFile) {
int blockSize = blockSizeKb * KILOBYTE;
long startTime = System.nanoTime();
long totalBytesReadInMark = readIO(numOfBlocks, BlockSequence.RANDOM, blockSize, testFile);
totalBytesReadInMark = totalBytesReadInMark + readIO(numOfBlocks, BlockSequence.SEQUENTIAL, blockSize, testFile);
long endTime = System.nanoTime();
long elapsedTimeNs = endTime - startTime;
double sec = (double) elapsedTimeNs / (double) 1000000000;
double mbRead = (double) totalBytesReadInMark / (double) MEGABYTE;
double bwMbSec = mbRead / sec;
String ioCount = props.getProperty(StatisticKeys.IO_COUNT.get(), "0");
String ioRAverageMbSec = props.getProperty(StatisticKeys.IO_R_AVERAGE_MB_SEC.get(), "0");
String ioRMbSec = props.getProperty(StatisticKeys.IO_R_MB_SEC.get(), "0");
int digital_ioCount = Integer.parseInt(ioCount);
double digital_ioRAverageMbSec = Double.parseDouble(ioRAverageMbSec);
double digital_ioRMbSec = Double.parseDouble(ioRMbSec);
digital_ioRAverageMbSec = (digital_ioRAverageMbSec * digital_ioCount + bwMbSec) / (digital_ioCount + 1);
digital_ioRMbSec = bwMbSec;
digital_ioCount++;
props.setProperty(StatisticKeys.IO_R_AVERAGE_MB_SEC.get(), format(digital_ioRAverageMbSec));
props.setProperty(StatisticKeys.IO_R_MB_SEC.get(), format(digital_ioRMbSec));
props.setProperty(StatisticKeys.IO_COUNT.get(), "" + digital_ioCount);
}
public static String format(double dvalue) {
return BigDecimal.valueOf(dvalue).setScale(2, RoundingMode.HALF_UP).toString();
}
private static long readIO(int numOfBlocks, BlockSequence blockSequence, int blockSize, File testFile) {
long totalBytesReadInMark = 0;
byte[] blockArr = new byte[blockSize];
for (int b = 0; b < blockArr.length; b++) {
if (b % 2 == 0) {
blockArr[b] = (byte) 0xFF;
}
}
try {
try (RandomAccessFile rAccFile = new RandomAccessFile(testFile, "r")) {
for (int b = 0; b < numOfBlocks; b++) {
if (blockSequence == BlockSequence.RANDOM) {
int rLoc = randInt(0, numOfBlocks - 1);
rAccFile.seek(rLoc * blockSize);
} else {
rAccFile.seek(b * blockSize);
}
rAccFile.readFully(blockArr, 0, blockSize);
totalBytesReadInMark += blockSize;
}
}
} catch (IOException e) {
CommonExceptionHandler.log(e.getMessage());
}
return totalBytesReadInMark;
}
private static int randInt(int min, int max) {
// nextInt is normally exclusive of the top value,
// so add 1 to make it inclusive
int randomNum = new Random().nextInt((max - min) + 1) + min;
return randomNum;
}
}

View File

@@ -0,0 +1,121 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.commons.utils.time;
import java.util.HashMap;
import org.apache.log4j.Logger;
/**
* DOC sbliu class global comment. Detailled comment
*/
public class TimeMeasurePerformance extends TimeMeasure{
static private Logger logger;
private static HashMap<String, TimeStack> timers;
private static long startTime = -1L;
private static int indent = 0;
public static void begin(String idTimer, String description) {
startTime = System.nanoTime();
init();
if (timers.containsKey(idTimer)) {
log(indent(indent) + "Warning (start): timer " + idTimer + " already exists"); //$NON-NLS-1$ //$NON-NLS-2$
} else {
indent++;
TimeStack times = new TimeStack();
timers.put(idTimer, times);
String message = "Start '" + idTimer + "' ...";
if (description != null) {
message = "Start '" + idTimer + "', " + description + " ...";
}
log(indent(indent) + message); //$NON-NLS-1$ //$NON-NLS-2$
}
}
private static void init() {
if (timers == null) {
timers = new HashMap<String, TimeStack>();
}
if(logger == null) {
configureLogger();
}
}
private static void log (String message) {
logger.info(message);
}
public static long end(String idTimer) {
init();
if (!timers.containsKey(idTimer)) {
log(indent(indent) + "Warning (end): timer " + idTimer + " doesn't exist"); //$NON-NLS-1$ //$NON-NLS-2$
return -1;
} else {
TimeStack timeStack = timers.get(idTimer);
timers.remove(idTimer);
long elapsedTimeSinceLastRequest = timeStack.getLastStepElapsedTime();
log(indent(indent) + "End '" + idTimer + "', elapsed time since last request: " //$NON-NLS-1$ //$NON-NLS-2$
+ elapsedTimeSinceLastRequest + " ms "); //$NON-NLS-1$
long totalElapsedTime = timeStack.getTotalElapsedTime();
log(indent(indent) + "End '" + idTimer + "', total elapsed time: " + totalElapsedTime + " ms "); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
indent--;
return totalElapsedTime;
}
}
public static long step(String idTimer, String stepName) {
init();
if (!timers.containsKey(idTimer)) {
log(indent(indent) + "Warning (end): timer " + idTimer + " does'nt exist"); //$NON-NLS-1$ //$NON-NLS-2$
return -1;
} else {
TimeStack timeStack = timers.get(idTimer);
timeStack.addStep();
/*
* trace the timeline of every step,problem is that the code below " Calendar ca = Calendar.getInstance();
* Date now = ca.getTime();" will cost almost 13ms~15ms
*/
long time = timeStack.getLastStepElapsedTime();
String timerStepName = idTimer + "', step name '" + stepName; //$NON-NLS-1$
log(indent(indent)
+ "-> '" + timerStepName + "', elapsed time since previous step: " + time + " ms"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
return time;
}
}
private static void configureLogger() {
try {
PerformanceLogManager logManager = new PerformanceLogManager();
logger = logManager.getLogger(TimeMeasurePerformance.class.getName());
} catch (Exception e) {
throw new RuntimeException("Error while initializing log properties.", e);
}
}
public static void afterStartup() {
double elapsedTimeInSeconds = (double)(System.nanoTime() - startTime)/1000000000;
PerformanceStatisticUtil.recordStartupEpapsedTime(elapsedTimeInSeconds);
PerformanceStatisticUtil.measureIO();
}
}

View File

@@ -212,4 +212,8 @@ public class CommonTextCellEditorWithProposal {
return this.contentProposalAdapter;
}
public int getPreviousActivatedIndex() {
return previousActivatedIndex;
}
}

View File

@@ -97,4 +97,7 @@ public class ExtendedTextCellEditorWithProposal extends ExtendedTextCellEditor i
super.fireCancelEditor();
}
public CommonTextCellEditorWithProposal getCommonTextEditor() {
return commonTextEditor;
}
}

View File

@@ -67,4 +67,9 @@ public class SyncLibrariesLoginTask extends AbstractLoginTask implements IRunnab
});
}
@Override
public boolean isRequiredAlways() {
return true;
}
}

View File

@@ -837,8 +837,8 @@ public abstract class AbstractEMFRepositoryFactory extends AbstractRepositoryFac
Object fullFolder = getFullFolder(project, type, relativeFolder);
serializableAllVersion = getSerializableFromFolder(project, fullFolder, id, type, false, false, true, true);
if (serializableAllVersion.isEmpty()) {
// look in all folders
serializableAllVersion = getSerializable(project, id, false, false);
// look in all folders for this item type
serializableAllVersion = getSerializableFromFolder(project, fullFolder, id, type, false, true, true, true, true);
}
int size = serializableAllVersion.size();

View File

@@ -270,6 +270,8 @@ public interface IRepositoryFactory {
public void create(Project project, Item item, IPath path, boolean... isImportItem) throws PersistenceException;
public void save(Project project, Item item, boolean isMigrationTask) throws PersistenceException;
public void save(Project project, Item item) throws PersistenceException;
public void save(Project project, Property property) throws PersistenceException;

View File

@@ -1064,6 +1064,9 @@ public class ProjectRepositoryNode extends RepositoryNode implements IProjectRep
}
if (newProject != null && newProject.getEmfProject() != null) {
List<FolderItem> folderItems = ProjectManager.getInstance().getFolders(newProject.getEmfProject());
if (folderItems != null) {
folderItems = new ArrayList<>(folderItems);
}
for (FolderItem folder : folderItems) {
String folderName = folder.getProperty().getLabel();
if (("process".equals(folderName) || "joblets".equals(folderName)) && folder.getChildren() != null

View File

@@ -15,6 +15,7 @@ package org.talend.core.repository.model;
import java.beans.PropertyChangeListener;
import java.beans.PropertyChangeSupport;
import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.net.URL;
import java.util.ArrayList;
import java.util.Date;
@@ -71,7 +72,7 @@ import org.talend.commons.ui.gmf.util.DisplayUtils;
import org.talend.commons.ui.runtime.exception.MessageBoxExceptionHandler;
import org.talend.commons.utils.data.container.RootContainer;
import org.talend.commons.utils.network.TalendProxySelector;
import org.talend.commons.utils.time.TimeMeasure;
import org.talend.commons.utils.time.TimeMeasurePerformance;
import org.talend.commons.utils.workbench.resources.ResourceUtils;
import org.talend.core.AbstractDQModelService;
import org.talend.core.GlobalServiceRegister;
@@ -84,6 +85,7 @@ import org.talend.core.context.Context;
import org.talend.core.context.RepositoryContext;
import org.talend.core.exception.TalendInternalPersistenceException;
import org.talend.core.hadoop.BigDataBasicUtil;
import org.talend.core.model.general.ILibrariesService;
import org.talend.core.model.general.ModuleNeeded;
import org.talend.core.model.general.Project;
import org.talend.core.model.metadata.MetadataTalendType;
@@ -124,11 +126,13 @@ import org.talend.core.repository.constants.Constant;
import org.talend.core.repository.constants.FileConstants;
import org.talend.core.repository.i18n.Messages;
import org.talend.core.repository.recyclebin.RecycleBinManager;
import org.talend.core.repository.utils.LoginTaskRegistryReader;
import org.talend.core.repository.utils.ProjectDataJsonProvider;
import org.talend.core.repository.utils.RepositoryPathProvider;
import org.talend.core.repository.utils.XmiResourceManager;
import org.talend.core.runtime.CoreRuntimePlugin;
import org.talend.core.runtime.repository.item.ItemProductKeys;
import org.talend.core.runtime.services.IGenericWizardService;
import org.talend.core.runtime.services.IMavenUIService;
import org.talend.core.runtime.util.ItemDateParser;
import org.talend.core.service.ICoreUIService;
@@ -136,6 +140,7 @@ import org.talend.core.service.IUpdateService;
import org.talend.cwm.helper.SubItemHelper;
import org.talend.cwm.helper.TableHelper;
import org.talend.designer.runprocess.IRunProcessService;
import org.talend.login.ILoginTask;
import org.talend.repository.ProjectManager;
import org.talend.repository.ReferenceProjectProblemManager;
import org.talend.repository.ReferenceProjectProvider;
@@ -177,6 +182,8 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
private boolean isCancelled;
private static final LoginTaskRegistryReader LOGIN_TASK_REGISTRY_READER = new LoginTaskRegistryReader();
@Override
public synchronized void addPropertyChangeListener(PropertyChangeListener l) {
if (l == null) {
@@ -226,6 +233,13 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
return null;
}
private ILibrariesService getLibrariesService() {
if (GlobalServiceRegister.getDefault().isServiceRegistered(ILibrariesService.class)) {
return GlobalServiceRegister.getDefault().getService(ILibrariesService.class);
}
return null;
}
/*
* (non-Javadoc)
*
@@ -1213,6 +1227,28 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
return this.repositoryFactoryFromProvider.getLastVersion(project, ProcessUtils.getPureItemId(id), folderPath, type);
}
@Override
public IRepositoryViewObject getLastVersion(String id, ERepositoryObjectType type)
throws PersistenceException {
return getLastVersion(id , "", type);
}
@Override
public IRepositoryViewObject getLastVersion(String id, List<ERepositoryObjectType> types) throws PersistenceException {
if (types != null) {
IRepositoryViewObject object = null;
for (ERepositoryObjectType type : types) {
object = getLastVersion(id, type);
if (object != null) {
return object;
}
}
}
return null;
}
@Override
public IRepositoryViewObject getLastVersion(String id, String folderPath, ERepositoryObjectType type)
throws PersistenceException {
String objId = id;
@@ -1224,7 +1260,25 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
return this.repositoryFactoryFromProvider.getLastVersion(project, objId, folderPath, type);
}
}
return this.repositoryFactoryFromProvider.getLastVersion(projectManager.getCurrentProject(), objId , folderPath, type);
return getLastRefVersion(projectManager.getCurrentProject(), objId , folderPath, type);
}
@Override
public IRepositoryViewObject getLastRefVersion(Project project, String id, String folderPath, ERepositoryObjectType type) throws PersistenceException {
String projectLabel = ProcessUtils.getProjectLabelFromItemId(id);
IRepositoryViewObject lastVersion = getLastVersion(project, ProcessUtils.getPureItemId(id), folderPath, type);
if (lastVersion == null) {
for (Project p : projectManager.getReferencedProjects(project)) {
if (projectLabel != null && !projectLabel.equals(p.getTechnicalLabel())) {
continue;
}
lastVersion = getLastRefVersion(p, id);
if (lastVersion != null) {
break;
}
}
}
return lastVersion;
}
@Override
@@ -1546,14 +1600,15 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
@Override
public void save(Project project, Item item, boolean... isMigrationTask) throws PersistenceException {
this.repositoryFactoryFromProvider.save(project, item);
if (isMigrationTask == null || isMigrationTask.length == 0 || !isMigrationTask[0]) {
this.repositoryFactoryFromProvider.save(project, item);
boolean avoidGenerateProm = false;
if (isMigrationTask != null && isMigrationTask.length == 2) {
avoidGenerateProm = isMigrationTask[1];
}
fireRepositoryPropertyChange(ERepositoryActionName.SAVE.getName(), avoidGenerateProm, item);
} else {
this.repositoryFactoryFromProvider.save(project, item, true);
}
}
@@ -2035,11 +2090,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
*/
public void logOnProject(Project project, IProgressMonitor monitor) throws LoginException, PersistenceException {
try {
TimeMeasure.display = CommonsPlugin.isDebugMode();
TimeMeasure.displaySteps = CommonsPlugin.isDebugMode();
TimeMeasure.measureActive = CommonsPlugin.isDebugMode();
TimeMeasure.begin("logOnProject"); //$NON-NLS-1$
TimeMeasurePerformance.begin("logOnProject", "logon project name '" + project.getLabel()+"'"); //$NON-NLS-1$ //$NON-NLS-2$
try {
/**
* init/check proxy selector, in case default proxy selector is not registed yet
@@ -2083,12 +2134,35 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
this.repositoryFactoryFromProvider.beforeLogon(project);
ProjectManager.getInstance().getBeforeLogonRecords().clear();
ProjectManager.getInstance().getUpdatedRemoteHandlerRecords().clear();
ILibrariesService librariesService = getLibrariesService();
if (librariesService != null) {
librariesService.setForceReloadCustomUri();
}
ProjectDataJsonProvider.checkAndRectifyRelationShipSetting(project.getEmfProject());
try {
// load additional jdbc
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericWizardService.class)) {
IGenericWizardService service = GlobalServiceRegister.getDefault()
.getService(IGenericWizardService.class);
if (service != null) {
service.loadAdditionalJDBC();
}
}
} catch (Exception e) {
// in case, to avoid block logon
ExceptionHandler.process(e);
}
// init dynamic distirbution after `beforeLogon`, before loading libraries.
initDynamicDistribution(monitor);
if (GlobalServiceRegister.getDefault().isServiceRegistered(IUpdateService.class)) {
IUpdateService updateService = GlobalServiceRegister.getDefault().getService(IUpdateService.class);
updateService.syncComponentM2Jars(currentMonitor);
}
// init sdk component
try {
currentMonitor = subMonitor.newChild(1, SubMonitor.SUPPRESS_NONE);
@@ -2106,7 +2180,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
throw new OperationCanceledException(""); //$NON-NLS-1$
}
// monitorWrap.worked(1);
TimeMeasure.step("logOnProject", "beforeLogon"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "beforeLogon"); //$NON-NLS-1$ //$NON-NLS-2$
// Check project compatibility
checkProjectCompatibility(project);
@@ -2124,7 +2198,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
currentMonitor = subMonitor.newChild(1, SubMonitor.SUPPRESS_NONE);
currentMonitor.beginTask(Messages.getString("ProxyRepositoryFactory.synchronizeLibraries"), 1); //$NON-NLS-1$
coreService.syncLibraries(currentMonitor);
TimeMeasure.step("logOnProject", "Sync components libraries"); //$NON-NLS-1$
TimeMeasurePerformance.step("logOnProject", "Sync components libraries"); //$NON-NLS-1$
}
currentMonitor = subMonitor.newChild(1, SubMonitor.SUPPRESS_NONE);
@@ -2133,7 +2207,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
executeMigrations(project, true, currentMonitor);
ProjectManager.getInstance().getMigrationRecords().clear();
// monitorWrap.worked(1);
TimeMeasure.step("logOnProject", "executeMigrations(beforeLogonTasks)"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "executeMigrations(beforeLogonTasks)"); //$NON-NLS-1$ //$NON-NLS-2$
currentMonitor = subMonitor.newChild(1, SubMonitor.SUPPRESS_NONE);
currentMonitor.beginTask(Messages.getString("ProxyRepositoryFactory.logonInProgress"), 1); //$NON-NLS-1$
@@ -2141,7 +2215,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
this.repositoryFactoryFromProvider.logOnProject(project);
ProjectManager.getInstance().getLogonRecords().clear();
// monitorWrap.worked(1);
TimeMeasure.step("logOnProject", "logOnProject"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "logOnProject"); //$NON-NLS-1$ //$NON-NLS-2$
emptyTempFolder(project);
@@ -2157,11 +2231,6 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
fireRepositoryPropertyChange(ERepositoryActionName.PROJECT_PREFERENCES_RELOAD.getName(), null, null);
if (GlobalServiceRegister.getDefault().isServiceRegistered(IUpdateService.class)) {
IUpdateService updateService = GlobalServiceRegister.getDefault().getService(IUpdateService.class);
updateService.syncComponentM2Jars(currentMonitor);
}
IRunProcessService runProcessService = getRunProcessService();
if (runProcessService != null) {
runProcessService.initMavenJavaProject(monitor, project);
@@ -2172,7 +2241,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
ProjectManager.getInstance().getMigrationRecords().clear();
executeMigrations(project, false, currentMonitor);
ProjectManager.getInstance().getMigrationRecords().clear();
TimeMeasure.step("logOnProject", "executeMigrations(afterLogonTasks)"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "executeMigrations(afterLogonTasks)"); //$NON-NLS-1$ //$NON-NLS-2$
if (monitor != null && monitor.isCanceled()) {
throw new OperationCanceledException(""); //$NON-NLS-1$
}
@@ -2188,12 +2257,12 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
// clean workspace
currentMonitor.beginTask(Messages.getString("ProxyRepositoryFactory.cleanWorkspace"), 1); //$NON-NLS-1$
TimeMeasure.step("logOnProject", "clean Java project"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "clean Java project"); //$NON-NLS-1$ //$NON-NLS-2$
if (workspace instanceof Workspace) {
((Workspace) workspace).getFileSystemManager().getHistoryStore().clean(currentMonitor);
}
TimeMeasure.step("logOnProject", "clean workspace history"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "clean workspace history"); //$NON-NLS-1$ //$NON-NLS-2$
currentMonitor = subMonitor.newChild(1, SubMonitor.SUPPRESS_NONE);
currentMonitor.beginTask(Messages.getString("ProxyRepositoryFactory.synch.repo.items"), 1); //$NON-NLS-1$
@@ -2219,12 +2288,12 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
if (monitor != null && monitor.isCanceled()) {
throw new OperationCanceledException(""); //$NON-NLS-1$
}
TimeMeasure.step("logOnProject", "sync repository (routines/rules/beans)"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "sync repository (routines/rules/beans)"); //$NON-NLS-1$ //$NON-NLS-2$
// log4j prefs
if (coreUiService != null && coreService != null) {
coreService.syncLog4jSettings(null);
TimeMeasure.step("logOnProject", "sync log4j"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "sync log4j"); //$NON-NLS-1$ //$NON-NLS-2$
}
try {
@@ -2233,6 +2302,8 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
// set the project mappings url
System.setProperty("talend.mappings.url", url.toString()); // $NON-NLS-1$
}
// for new added mapping file, sync to project mapping folder
MetadataTalendType.syncNewMappingFileToProject();
} catch (SystemException e) {
// ignore
ExceptionHandler.process(e);
@@ -2241,7 +2312,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
if (runProcessService != null && !isCommandLineLocalRefProject) {
runProcessService.initializeRootPoms(monitor);
TimeMeasure.step("logOnProject", "install / setup root poms"); //$NON-NLS-1$ //$NON-NLS-2$
TimeMeasurePerformance.step("logOnProject", "install / setup root poms"); //$NON-NLS-1$ //$NON-NLS-2$
}
if (GlobalServiceRegister.getDefault().isServiceRegistered(ITDQRepositoryService.class)) {
ITDQRepositoryService tdqRepositoryService = GlobalServiceRegister.getDefault()
@@ -2258,10 +2329,7 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
fullLogonFinished = true;
this.repositoryFactoryFromProvider.afterLogon(monitor);
} finally {
TimeMeasure.end("logOnProject"); //$NON-NLS-1$
TimeMeasure.display = false;
TimeMeasure.displaySteps = false;
TimeMeasure.measureActive = false;
TimeMeasurePerformance.end("logOnProject"); //$NON-NLS-1$
}
String str[] = new String[] { getRepositoryContext().getUser() + "", projectManager.getCurrentProject() + "" }; //$NON-NLS-1$ //$NON-NLS-2$
log.info(Messages.getString("ProxyRepositoryFactory.log.loggedOn", str)); //$NON-NLS-1$
@@ -2662,4 +2730,13 @@ public final class ProxyRepositoryFactory implements IProxyRepositoryFactory {
public RepositoryWorkUnit getWorkUnitInProgress() {
return repositoryFactoryFromProvider.getWorkUnitInProgress();
}
public void executeRequiredLoginTasks(IProgressMonitor monitor) throws InvocationTargetException, InterruptedException {
ILoginTask[] allLoginTasks = LOGIN_TASK_REGISTRY_READER.getAllTaskListInstance();
for (ILoginTask task : allLoginTasks) {
if (task.isRequiredAlways()) {
task.run(monitor);
}
}
}
}

View File

@@ -14,6 +14,7 @@ package org.talend.core.repository.recyclebin;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.Date;
import java.util.HashMap;
@@ -24,6 +25,7 @@ import java.util.List;
import java.util.Map;
import java.util.Set;
import org.apache.commons.lang3.StringUtils;
import org.eclipse.core.resources.IProject;
import org.eclipse.core.runtime.IPath;
import org.eclipse.emf.common.util.EList;
@@ -106,7 +108,8 @@ public class RecycleBinManager {
List<IRepositoryViewObject> deletedObjects = new ArrayList<IRepositoryViewObject>();
final EList<TalendItem> deletedItems = projectRecyclebins.get(project.getTechnicalLabel()).getDeletedItems();
List<TalendItem> notDeletedItems = new ArrayList<TalendItem>();
for (TalendItem deletedItem : deletedItems) {
List<TalendItem> dup_deletedItems = new ArrayList<>(deletedItems);
for (TalendItem deletedItem : dup_deletedItems) {
try {
final ERepositoryObjectType type = ERepositoryObjectType.getType(deletedItem.getType());
// ignore the generated doc in recycle bin
@@ -274,7 +277,40 @@ public class RecycleBinManager {
resource = createRecycleBinResource(project);
}
resource.getContents().clear();
recycleBin.setLastUpdate(new Date());
EList<String> deletedFolders = recycleBin.getDeletedFolders();
if (deletedFolders != null) {
List<String> folders = new LinkedList<>(deletedFolders);
Collections.sort(folders);
deletedFolders.clear();
deletedFolders.addAll(folders);
}
EList<TalendItem> deletedItems = recycleBin.getDeletedItems();
if (deletedItems != null) {
List<TalendItem> items = new LinkedList<>(deletedItems);
items.sort((l, r) -> {
if (l == null && r == null) {
return 0;
} else if (l == null) {
return -1;
} else if (r == null) {
return 1;
}
int result = StringUtils.compare(l.getType(), r.getType());
if (result != 0) {
return result;
}
result = StringUtils.compare(l.getPath(), r.getPath());
if (result != 0) {
return result;
}
return StringUtils.compare(l.getId(), r.getId());
});
deletedItems.clear();
deletedItems.addAll(items);
}
// set date to null to avoid timezone conflict
recycleBin.setLastUpdate(null);
resource.getContents().add(recycleBin);
EmfHelper.saveResource(resource);
lastSavedRecycleBinMap.put(recycleBin, EcoreUtil.copy(recycleBin));

View File

@@ -26,6 +26,7 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import org.apache.log4j.Priority;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IProject;
@@ -37,6 +38,7 @@ import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.core.runtime.Path;
import org.eclipse.core.runtime.Platform;
import org.eclipse.emf.common.util.EList;
import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.EObject;
import org.eclipse.emf.ecore.resource.Resource;
@@ -69,6 +71,7 @@ import org.talend.core.model.properties.ValidationRulesConnectionItem;
import org.talend.core.model.properties.helper.ByteArrayResource;
import org.talend.core.model.repository.ERepositoryObjectType;
import org.talend.core.repository.constants.FileConstants;
import org.talend.core.repository.model.ProxyRepositoryFactory;
import org.talend.core.repository.utils.ResourceFilenameHelper.FileName;
import org.talend.core.ui.ITestContainerProviderService;
import org.talend.designer.core.model.utils.emf.talendfile.ProcessType;
@@ -647,6 +650,23 @@ public class XmiResourceManager {
}
public void saveResource(Resource resource) throws PersistenceException {
try {
if (resource != null) {
Object objectByType = EcoreUtil.getObjectByType(resource.getContents(), PropertiesPackage.eINSTANCE.getProject());
if (objectByType != null) {
Project project = (Project) objectByType;
EList migrationTasks = project.getMigrationTask();
if (migrationTasks != null && 1 < migrationTasks.size()) {
org.talend.commons.exception.ExceptionHandler.process(new Exception("Bad saving logic for Project"),
Priority.WARN);
ProxyRepositoryFactory.getInstance().saveProject(new org.talend.core.model.general.Project(project));
return;
}
}
}
} catch (Throwable e) {
org.talend.commons.exception.ExceptionHandler.process(e);
}
EmfHelper.saveResource(resource);
}

View File

@@ -0,0 +1,128 @@
<?xml version="1.0"?>
<mapping>
<dbms product="DATABRICKS_DELTA_LAKE" id="databricks_delta_lake_id" label="Mapping Delta Lake" default="true">
<dbTypes>
<dbType type="SMALLINT" ignoreLen="true" ignorePre="true"/>
<dbType type="FLOAT" ignoreLen="true" ignorePre="true"/>
<dbType type="DOUBLE" ignoreLen="true" ignorePre="true"/>
<dbType type="BIGINT" ignoreLen="true" ignorePre="true"/>
<dbType type="INT" ignoreLen="true" ignorePre="true"/>
<dbType type="TINYINT" ignoreLen="true" ignorePre="true"/>
<dbType type="STRING" ignoreLen="true" ignorePre="true"/>
<dbType type="DECIMAL" ignoreLen="false" ignorePre="false"/>
<dbType type="BOOLEAN" ignoreLen="true" ignorePre="true"/>
<dbType type="STRUCT" ignoreLen="true" ignorePre="true"/>
<dbType type="MAP" ignoreLen="true" ignorePre="true"/>
<dbType type="ARRAY" ignoreLen="true" ignorePre="true"/>
<dbType type="TIMESTAMP" ignoreLen="true" ignorePre="true"/>
<dbType type="DATE" ignoreLen="true" ignorePre="true"/>
</dbTypes>
<language name="java">
<talendToDbTypes><!-- Adviced mappings -->
<talendType type="id_List"/>
<talendType type="id_Boolean">
<dbType type="BOOLEAN" default="true"/>
</talendType>
<talendType type="id_Byte">
<dbType type="TINYINT" default="true"/>
<dbType type="BIGINT"/>
<dbType type="INT"/>
<dbType type="SMALLINT"/>
</talendType>
<talendType type="id_byte[]"/>
<talendType type="id_Character">
<dbType type="STRING" default="true"/>
</talendType>
<talendType type="id_Date">
<dbType type="TIMESTAMP" default="true"/>
<dbType type="DATE"/>
</talendType>
<talendType type="id_BigDecimal">
<dbType type="DECIMAL" default="true"/>
<dbType type="BIGINT"/>
<dbType type="FLOAT"/>
<dbType type="DOUBLE"/>
</talendType>
<talendType type="id_Double">
<dbType type="DOUBLE" default="true" />
<dbType type="FLOAT"/>
</talendType>
<talendType type="id_Float">
<dbType type="FLOAT" default="true" />
<dbType type="DOUBLE"/>
</talendType>
<talendType type="id_Integer">
<dbType type="INT" default="true" />
<dbType type="BIGINT" />
</talendType>
<talendType type="id_Long">
<dbType type="BIGINT" default="true" />
</talendType>
<talendType type="id_Object">
<dbType type="STRUCT" default="true" />
<dbType type="MAP" />
<dbType type="ARRAY" />
</talendType>
<talendType type="id_Short">
<dbType type="SMALLINT" default="true" />
<dbType type="INT" />
</talendType>
<talendType type="id_String">
<dbType type="STRING" default="true"/>
</talendType>
</talendToDbTypes>
<dbToTalendTypes><!-- Adviced mappings -->
<dbType type="STRING">
<talendType type="id_String" default="true"/>
</dbType>
<dbType type="BOOLEAN">
<talendType type="id_Boolean" default="true"/>
</dbType>
<dbType type="DOUBLE">
<talendType type="id_Double" default="true"/>
<talendType type="id_BigDecimal"/>
<talendType type="id_Float"/>
</dbType>
<dbType type="DECIMAL">
<talendType type="id_BigDecimal" default="true"/>
</dbType>
<dbType type="BIGINT">
<talendType type="id_Long" default="true"/>
<talendType type="id_Integer"/>
</dbType>
<dbType type="INT">
<talendType type="id_Integer" default="true"/>
<talendType type="id_Short"/>
</dbType>
<dbType type="SMALLINT">
<talendType type="id_Short" default="true"/>
<talendType type="id_Byte"/>
</dbType>
<dbType type="TINYINT">
<talendType type="id_Byte" default="true"/>
</dbType>
<dbType type="FLOAT">
<talendType type="id_Float" default="true"/>
<talendType type="id_Double"/>
<talendType type="id_BigDecimal"/>
</dbType>
<dbType type="STRUCT">
<talendType type="id_Object" default="true"/>
</dbType>
<dbType type="MAP">
<talendType type="id_Object" default="true"/>
</dbType>
<dbType type="ARRAY">
<talendType type="id_Object" default="true"/>
</dbType>
<dbType type="TIMESTAMP">
<talendType type="id_Date" default="true"/>
</dbType>
<dbType type="DATE">
<talendType type="id_Date" default="true"/>
</dbType>
</dbToTalendTypes>
</language>
</dbms>
</mapping>

View File

@@ -40,7 +40,7 @@
<dbType type="SET" ignoreLen="true" ignorePre="true" />
<dbType type="TEXT" ignoreLen="true" ignorePre="true" />
<dbType type="TIME" ignoreLen="true" ignorePre="true" />
<dbType type="TIMESTAMP" ignoreLen="true" ignorePre="true" />
<dbType type="TIMESTAMP" ignoreLen="true" ignorePre="false" />
<dbType type="TINYBLOB" ignoreLen="true" ignorePre="true" />
<dbType type="TINYINT" ignorePre="true" />
<dbType type="TINYINT UNSIGNED" ignoreLen="true" ignorePre="true" />

View File

@@ -0,0 +1,288 @@
<?xml version="1.0"?>
<mapping>
<dbms product="SINGLESTORE" id="singlestore_id" label="Mapping SingleStore"
default="true">
<dbTypes>
<dbType type="BOOL" ignoreLen="true" ignorePre="true" />
<dbType type="BOOLEAN" ignoreLen="true" ignorePre="true" />
<dbType type="BIGINT" ignorePre="true"/>
<dbType type="BIGINT UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="BINARY" ignorePre="true"/>
<dbType type="BIT" ignorePre="true" />
<dbType type="BLOB" ignoreLen="true" ignorePre="true" />
<dbType type="CHAR" defaultLength="200" ignorePre="true"/>
<dbType type="DATE" ignoreLen="true" ignorePre="true" />
<dbType type="DATETIME" ignoreLen="true" ignorePre="false" />
<dbType type="DECIMAL" defaultLength="20" defaultPrecision="10" preBeforelen="false"/>
<dbType type="DOUBLE" defaultLength="20" defaultPrecision="10"/>
<dbType type="DOUBLE UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="ENUM" ignoreLen="true" ignorePre="true" />
<dbType type="FLOAT" defaultPrecision="2"/>
<dbType type="FLOAT UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="GEOGRAPHY" ignoreLen="true" ignorePre="true" />
<dbType type="GEOGRAPHYPOINT" ignoreLen="true" ignorePre="true" />
<dbType type="INT" ignorePre="true" />
<dbType type="INT UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="INTEGER" ignorePre="true" />
<dbType type="INTEGER UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="LONGTEXT" ignoreLen="true" ignorePre="true" />
<dbType type="LONGBLOB" ignoreLen="true" ignorePre="true"/>
<dbType type="MEDIUMBLOB" ignoreLen="true" ignorePre="true" />
<dbType type="MEDIUMINT" ignorePre="true" />
<dbType type="MEDIUMINT UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="MEDIUMTEXT" ignoreLen="true" ignorePre="true" />
<dbType type="SMALLINT" ignorePre="true" />
<dbType type="SMALLINT UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="SET" ignoreLen="true" ignorePre="true" />
<dbType type="TEXT" ignoreLen="true" ignorePre="true" />
<dbType type="TIME" ignoreLen="true" ignorePre="true" />
<dbType type="TIMESTAMP" ignoreLen="true" ignorePre="false" />
<dbType type="TINYBLOB" ignoreLen="true" ignorePre="true" />
<dbType type="TINYINT" ignorePre="true" />
<dbType type="TINYINT UNSIGNED" ignoreLen="true" ignorePre="true" />
<dbType type="TINYTEXT" ignoreLen="true" ignorePre="true" />
<dbType type="VARBINARY" ignorePre="true" />
<dbType type="VARCHAR" default="true" defaultLength="100" ignorePre="true"/>
<dbType type="YEAR" ignorePre="true"/>
<dbType type="JSON" ignoreLen="true" ignorePre="true"/>
</dbTypes>
<language name="java">
<talendToDbTypes><!-- Adviced mappings -->
<talendType type="id_List"/>
<talendType type="id_Boolean">
<dbType type="BOOL" default="true" />
</talendType>
<talendType type="id_Byte">
<dbType type="TINYINT" default="true" />
<dbType type="BIGINT" />
<dbType type="INT" />
<dbType type="MEDIUMINT" />
<dbType type="SMALLINT" />
</talendType>
<talendType type="id_byte[]">
<dbType type="VARBINARY" default="true" />
<dbType type="BINARY" />
<dbType type="BIT" />
<dbType type="BLOB" />
<dbType type="LONGBLOB" />
<dbType type="MEDIUMBLOB" />
<dbType type="TINYBLOB" />
</talendType>
<talendType type="id_Character">
<dbType type="CHAR" default="true" />
<dbType type="VARCHAR"/>
</talendType>
<talendType type="id_Date">
<dbType type="DATE" />
<dbType type="DATETIME" default="true" />
<dbType type="TIME" />
<dbType type="YEAR" />
<dbType type="TIMESTAMP" />
</talendType>
<talendType type="id_BigDecimal">
<dbType type="DECIMAL" default="true" />
<dbType type="FLOAT"/>
<dbType type="DOUBLE" />
</talendType>
<talendType type="id_Double">
<dbType type="DOUBLE" default="true" />
<dbType type="FLOAT"/>
<dbType type="DECIMAL" />
</talendType>
<talendType type="id_Float">
<dbType type="FLOAT" default="true" />
<dbType type="DOUBLE"/>
<dbType type="DECIMAL" />
</talendType>
<talendType type="id_Integer">
<dbType type="INT" default="true" />
<dbType type="BIGINT" />
</talendType>
<talendType type="id_Long">
<dbType type="BIGINT" default="true" />
</talendType>
<talendType type="id_Object">
<dbType type="BLOB" default="true"/>
<dbType type="ENUM" />
<dbType type="GEOGRAPHY" />
<dbType type="GEOGRAPHYPOINT" />
<dbType type="MEDIUMINT" />
<dbType type="LONGBLOB" />
<dbType type="MEDIUMBLOB" />
<dbType type="SET" />
<dbType type="TINYBLOB" />
</talendType>
<talendType type="id_Short">
<dbType type="SMALLINT" default="true" />
<dbType type="INT" />
<dbType type="BIGINT"/>
<dbType type="MEDIUMINT" />
</talendType>
<talendType type="id_String">
<dbType type="VARCHAR" default="true" />
<dbType type="LONGTEXT"/>
<dbType type="MEDIUMTEXT" />
<dbType type="TEXT" />
<dbType type="TINYTEXT" />
<dbType type="CHAR" />
<dbType type="JSON" />
</talendType>
</talendToDbTypes>
<dbToTalendTypes>
<dbType type="BOOL">
<talendType type="id_Boolean" default="true" />
</dbType>
<dbType type="BOOLEAN">
<talendType type="id_Boolean" default="true" />
</dbType>
<dbType type="BIGINT">
<talendType type="id_Long" default="true" />
</dbType>
<dbType type="BINARY">
<talendType type="id_byte[]" />
<talendType type="id_Boolean" default="true" />
</dbType>
<dbType type="BIT">
<talendType type="id_byte[]" />
<talendType type="id_Boolean" default="true" />
</dbType>
<dbType type="BLOB">
<talendType type="id_byte[]" default="true" />
<talendType type="id_Object" />
</dbType>
<dbType type="CHAR">
<talendType type="id_String" default="true" />
</dbType>
<dbType type="DATE">
<talendType type="id_Date" default="true" />
</dbType>
<dbType type="DATETIME">
<talendType type="id_Date" default="true" />
</dbType>
<dbType type="DECIMAL">
<talendType type="id_Float"/>
<talendType type="id_Double"/>
<talendType type="id_BigDecimal" default="true"/>
</dbType>
<dbType type="DOUBLE">
<talendType type="id_Double" default="true" />
<talendType type="id_BigDecimal"/>
</dbType>
<dbType type="ENUM">
<talendType type="id_Object" default="true" />
</dbType>
<dbType type="FLOAT">
<talendType type="id_Float" default="true" />
<talendType type="id_Double"/>
<talendType type="id_BigDecimal"/>
</dbType>
<dbType type="GEOGRAPHY">
<talendType type="id_Object" default="true" />
</dbType>
<dbType type="GEOGRAPHYPOINT">
<talendType type="id_Object" default="true" />
</dbType>
<dbType type="JSON">
<talendType type="id_String" default="true" />
<talendType type="id_Object" />
</dbType>
<dbType type="INT">
<talendType type="id_Integer" default="true" />
<talendType type="id_Long"/>
</dbType>
<dbType type="INTEGER">
<talendType type="id_Integer" default="true" />
<talendType type="id_Long"/>
</dbType>
<dbType type="LONGTEXT">
<talendType type="id_String" default="true" />
</dbType>
<dbType type="LONGBLOB">
<talendType type="id_byte[]" default="true" />
<talendType type="id_Object" />
</dbType>
<dbType type="MEDIUMBLOB">
<talendType type="id_byte[]" default="true" />
<talendType type="id_Object" />
</dbType>
<dbType type="MEDIUMINT">
<talendType type="id_Integer" default="true" />
<talendType type="id_Long"/>
</dbType>
<dbType type="MEDIUMTEXT">
<talendType type="id_String" default="true" />
</dbType>
<dbType type="SMALLINT">
<talendType type="id_Short" default="true" />
<talendType type="id_Long"/>
<talendType type="id_Integer"/>
</dbType>
<dbType type="SET">
<talendType type="id_Object" default="true" />
</dbType>
<dbType type="TEXT">
<talendType type="id_String" default="true" />
</dbType>
<dbType type="TIME">
<talendType type="id_Date" default="true" />
</dbType>
<dbType type="TIMESTAMP">
<talendType type="id_Date" default="true" />
</dbType>
<dbType type="TINYBLOB">
<talendType type="id_byte[]" default="true" />
<talendType type="id_Object" />
</dbType>
<dbType type="TINYINT">
<talendType type="id_Byte" default="true" />
<talendType type="id_Integer"/>
<talendType type="id_Long"/>
<talendType type="id_Short"/>
</dbType>
<dbType type="TINYTEXT">
<talendType type="id_String" default="true" />
</dbType>
<dbType type="VARBINARY">
<talendType type="id_byte[]" default="true" />
</dbType>
<dbType type="VARCHAR">
<talendType type="id_String" default="true" />
</dbType>
<dbType type="YEAR">
<talendType type="id_Date" default="true" />
</dbType>
<dbType type="BIGINT UNSIGNED" >
</dbType>
<dbType type="DOUBLE UNSIGNED" >
<talendType type="id_Double" default="true" />
<talendType type="id_BigDecimal"/>
</dbType>
<dbType type="FLOAT UNSIGNED" >
<talendType type="id_Double" default="true" />
<talendType type="id_BigDecimal"/>
</dbType>
<dbType type="INT UNSIGNED" >
<talendType type="id_Long" default="true" />
</dbType>
<dbType type="INTEGER UNSIGNED" >
<talendType type="id_Long" default="true" />
</dbType>
<dbType type="MEDIUMINT UNSIGNED" >
<talendType type="id_Integer" default="true" />
<talendType type="id_Long" />
</dbType>
<dbType type="SMALLINT UNSIGNED" >
<talendType type="id_Integer" default="true" />
<talendType type="id_Long" />
</dbType>
<dbType type="TINYINT UNSIGNED" >
<talendType type="id_Short" default="true" />
<talendType type="id_Integer" />
<talendType type="id_Long" />
</dbType>
</dbToTalendTypes>
</language>
</dbms>
</mapping>

View File

@@ -9,7 +9,7 @@
<dbType type="CHAR" defaultLength="50" ignorePre="true"/>
<dbType type="CHARACTER" ignoreLen="true" ignorePre="true" />
<dbType type="CHARACTER VARYING" ignoreLen="true" ignorePre="true" />
<dbType type="DATE" ignoreLen="true" ignorePre="true" />
<dbType type="DATE" ignoreLen="true" ignorePre="true" defaultPattern="dd-MM-yyyy" />
<dbType type="DATETIME" ignoreLen="true" ignorePre="true" />
<dbType type="DECIMAL" defaultLength="20" defaultPrecision="10" />
<dbType type="DOUBLE PRECISION" ignoreLen="true" ignorePre="true" />

View File

@@ -100,6 +100,11 @@
<xs:restriction base="xs:short"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="defaultPattern">
<xs:simpleType>
<xs:restriction base="xs:string"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="default" type="xs:boolean"/>
</xs:complexType>
</xs:element>

View File

@@ -42,6 +42,7 @@ import java.util.Enumeration;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
@@ -50,11 +51,16 @@ import org.eclipse.core.resources.IContainer;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IResource;
import org.eclipse.core.resources.IWorkspace;
import org.eclipse.core.resources.IWorkspaceRunnable;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.FileLocator;
import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.OperationCanceledException;
import org.eclipse.core.runtime.jobs.ISchedulingRule;
import org.eclipse.core.runtime.jobs.Job;
import org.osgi.framework.Bundle;
import org.talend.commons.exception.CommonExceptionHandler;
import org.talend.commons.exception.ExceptionHandler;
@@ -1173,4 +1179,33 @@ public class FilesUtils {
}
}
public static void executeFolderAction(IProgressMonitor monitor, IResource parentFolder, IWorkspaceRunnable run)
throws CoreException {
if (Job.getJobManager().currentRule() == null) {
IWorkspace workspace = ResourcesPlugin.getWorkspace();
ISchedulingRule defaultRule = workspace.getRuleFactory().modifyRule(parentFolder);
ISchedulingRule noBlockRule = new ISchedulingRule() {
@Override
public boolean isConflicting(ISchedulingRule rule) {
return this.contains(rule);
}
@Override
public boolean contains(ISchedulingRule rule) {
if (this.equals(rule)) {
return true;
}
if (defaultRule.contains(rule)) {
return true;
}
return false;
}
};
workspace.run(run, noBlockRule, IWorkspace.AVOID_UPDATE, monitor);
} else {
run.run(monitor);
}
}
}

View File

@@ -0,0 +1,84 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.database;
import java.util.ArrayList;
import java.util.List;
import org.talend.core.runtime.hd.hive.HiveMetadataHelper;
/**
* DOC hzhao class global comment. Detailled comment
*/
public enum EImpalaDriver {
HIVE2("HIVE2", "HIVE2", "org.apache.hive.jdbc.HiveDriver"),
IMPALA40("IMPALA40", "IMPALA40", "com.cloudera.impala.jdbc4.Driver"),
IMPALA41("IMPALA41", "IMPALA41", "com.cloudera.impala.jdbc41.Driver");
EImpalaDriver(String displayName, String name, String driver) {
this.displayName = displayName;
this.name = name;
this.driver = driver;
}
private String displayName;
private String name;
private String driver;
public String getDisplayName() {
return displayName;
}
public String getName() {
return name;
}
public String getDriver() {
return driver;
}
public static boolean isSupport(String distribution, String version, boolean byDisplay, String supportMethodName) {
return HiveMetadataHelper.doSupportMethod(distribution, version, byDisplay, supportMethodName);
}
public static String[] getImpalaDriverDisplay(String distribution, String version, boolean byDisplay) {
List<String> list = new ArrayList<>(0);
for (EImpalaDriver driver : EImpalaDriver.values()) {
if (isSupport(distribution, version, byDisplay, "doSupportImpalaConnector")) {
list.add(driver.getDisplayName());
}
}
return list.toArray(new String[0]);
}
public static EImpalaDriver getByDisplay(String display) {
for (EImpalaDriver driver : EImpalaDriver.values()) {
if (driver.getDisplayName().equals(display)) {
return driver;
}
}
return null;
}
public static EImpalaDriver getByName(String name) {
for (EImpalaDriver driver : EImpalaDriver.values()) {
if (driver.getName().equals(name)) {
return driver;
}
}
return null;
}
}

View File

@@ -267,6 +267,8 @@ public class ConnParameterKeys {
public static final String IMPALA_AUTHENTICATION_PRINCIPLA = "IMPALA_AUTHENTICATION_PRINCIPLA";//$NON-NLS-1$
public static final String IMPALA_DRIVER = "IMPALA_DRIVER";
/**
* Google Dataproc keys.
*/

View File

@@ -224,6 +224,7 @@ public class DatabaseConnStrUtil {
if (template.startsWith(DbConnStrForHive.URL_HIVE_2_TEMPLATE)) {
url = getImpalaURLString(false, server, port, sidOrDatabase, impalaPrincipal);
}
url = attachAdditionalHiveParameters(url, dbConn, false);
return url;
}

View File

@@ -23,6 +23,7 @@ import org.talend.core.database.EDatabaseTypeName;
import org.talend.core.database.conn.version.EDatabaseVersion4Drivers;
import org.talend.core.model.repository.ERepositoryObjectType;
import org.talend.core.runtime.services.IGenericDBService;
import org.talend.core.runtime.services.IGenericWizardService;
/**
* cli class global comment. Detailled comment
@@ -326,6 +327,16 @@ public enum EDatabaseConnTemplate {
databaseType.add(typeName);
}
}
// add additional jdbc (actually JDBC RepositoryObjectType)
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericWizardService.class)) {
IGenericWizardService service = GlobalServiceRegister.getDefault().getService(IGenericWizardService.class);
if (service != null) {
List<String> allAdditionalJDBCTypes = service.getAllAdditionalJDBCTypes();
if (!allAdditionalJDBCTypes.isEmpty()) {
databaseType.addAll(allAdditionalJDBCTypes);
}
}
}
if (sort) {
String[] sortedArray = databaseType.toArray(new String[0]);
Arrays.sort(sortedArray, new Comparator<String>() {

View File

@@ -102,12 +102,12 @@ public enum EDatabaseVersion4Drivers {
GREENPLUM(new DbVersion4Drivers(EDatabaseTypeName.GREENPLUM, "postgresql-8.4-703.jdbc4.jar")), //$NON-NLS-1$
// PSQL_V10(new DbVersion4Drivers(EDatabaseTypeName.PSQL, "v10", "V10", "postgresql-42.2.5.jar")),
PSQL_V9_X(new DbVersion4Drivers(EDatabaseTypeName.PSQL, "v9 and later", "V9_X", "postgresql-42.2.9.jar")), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
PSQL_V9_X(new DbVersion4Drivers(EDatabaseTypeName.PSQL, "v9 and later", "V9_X", "postgresql-42.2.14.jar")), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
PSQL_PRIOR_TO_V9(new DbVersion4Drivers(EDatabaseTypeName.PSQL, "Prior to v9", "PRIOR_TO_V9", "postgresql-8.4-703.jdbc4.jar")), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
PLUSPSQL_PRIOR_TO_V9(new DbVersion4Drivers(EDatabaseTypeName.PLUSPSQL,
"Prior to v9", "PRIOR_TO_V9", "postgresql-8.4-703.jdbc4.jar")), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
PLUSPSQL_V9_X(new DbVersion4Drivers(EDatabaseTypeName.PLUSPSQL, "v9 and later", "V9_X", "postgresql-9.4-1201.jdbc41.jar")), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
PLUSPSQL_V9_X(new DbVersion4Drivers(EDatabaseTypeName.PLUSPSQL, "v9 and later", "V9_X", "postgresql-42.2.14.jar")), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
IBMDB2(new DbVersion4Drivers(EDatabaseTypeName.IBMDB2, new String[] { "db2jcc4.jar", "db2jcc_license_cu.jar", //$NON-NLS-1$ //$NON-NLS-2$
"db2jcc_license_cisuz.jar" })), //$NON-NLS-1$
IBMDB2ZOS(new DbVersion4Drivers(EDatabaseTypeName.IBMDB2ZOS, new String[] { "db2jcc4.jar", "db2jcc_license_cu.jar", //$NON-NLS-1$ //$NON-NLS-2$
@@ -166,7 +166,7 @@ public enum EDatabaseVersion4Drivers {
REDSHIFT(new DbVersion4Drivers(EDatabaseTypeName.REDSHIFT, "redshift", "REDSHIFT", //$NON-NLS-1$ //$NON-NLS-2$
"redshift-jdbc42-no-awssdk-1.2.37.1061.jar")), //$NON-NLS-1$
REDSHIFT_SSO(new DbVersion4Drivers(EDatabaseTypeName.REDSHIFT_SSO, "redshift sso", "REDSHIFT_SSO", //$NON-NLS-1$ //$NON-NLS-2$
new String[] { "redshift-jdbc42-no-awssdk-1.2.37.1061.jar", "aws-java-sdk-1.11.729.jar", "jackson-core-2.10.1.jar", //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
new String[] { "redshift-jdbc42-no-awssdk-1.2.37.1061.jar", "aws-java-sdk-1.11.848.jar", "jackson-core-2.10.1.jar", //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
"jackson-databind-2.10.1.jar", "jackson-annotations-2.10.1.jar", "httpcore-4.4.11.jar", "httpclient-4.5.9.jar", //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$//$NON-NLS-4$
"joda-time-2.8.1.jar", "commons-logging-1.2.jar", "commons-codec-1.11.jar" })), //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$

View File

@@ -111,4 +111,6 @@ public interface IComponentsFactory {
public Map<String, File> getComponentsProvidersFolder();
public List<ComponentProviderInfo> getComponentsProvidersInfo();
public String getCustomComponentBundlePath();
}

View File

@@ -12,6 +12,7 @@
// ============================================================================
package org.talend.core.model.context;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
@@ -47,12 +48,14 @@ import org.talend.core.model.properties.ContextItem;
import org.talend.core.model.properties.Item;
import org.talend.core.model.properties.JobletProcessItem;
import org.talend.core.model.properties.ProcessItem;
import org.talend.core.model.repository.ERepositoryObjectType;
import org.talend.core.model.repository.IRepositoryViewObject;
import org.talend.core.runtime.CoreRuntimePlugin;
import org.talend.cwm.helper.ResourceHelper;
import org.talend.designer.core.model.utils.emf.talendfile.ContextParameterType;
import org.talend.designer.core.model.utils.emf.talendfile.ContextType;
import org.talend.designer.core.model.utils.emf.talendfile.ProcessType;
import org.talend.migration.IMigrationTask.ExecutionResult;
import org.talend.repository.model.IProxyRepositoryFactory;
/**
@@ -71,6 +74,7 @@ public class ContextUtils {
private static final Set<String> SECURE_SENSITIVE_CONTEXT_NAMES_EXP = new HashSet<String>(Arrays.asList("resource_flow_temp_folder", "resource_webhook_payload", "resource_file_[\\w]+",
"resource_directory_[\\w]+", "connection_[a-zA-Z0-9]+_[\\w]"));
/**
*
* ggu Comment method "isJavaKeyWords".
@@ -88,7 +92,7 @@ public class ContextUtils {
}
return false;
}
/**
*
* ggu Comment method "isSecureSensitiveParam".
@@ -241,7 +245,6 @@ public class ContextUtils {
return parameterType;
}
@SuppressWarnings("unchecked")
private static boolean checkObject(Object obj) {
if (obj == null) {
return true;
@@ -288,7 +291,7 @@ public class ContextUtils {
IProxyRepositoryFactory factory = CoreRuntimePlugin.getInstance().getProxyRepositoryFactory();
try {
final IRepositoryViewObject lastVersion = factory.getLastVersion(contextId);
final IRepositoryViewObject lastVersion = factory.getLastVersion(contextId, ERepositoryObjectType.CONTEXT);
if (lastVersion != null) {
final Item item = lastVersion.getProperty().getItem();
if (item != null && item instanceof ContextItem) {
@@ -505,9 +508,13 @@ public class ContextUtils {
return null;
}
IProxyRepositoryFactory factory = CoreRuntimePlugin.getInstance().getProxyRepositoryFactory();
List<ERepositoryObjectType> possibleTypes = new ArrayList<ERepositoryObjectType>();
possibleTypes.add(ERepositoryObjectType.CONTEXT);
possibleTypes.addAll(ERepositoryObjectType.getAllTypesOfJoblet());
possibleTypes.addAll(ERepositoryObjectType.getAllTypesOfProcess());
try {
final IRepositoryViewObject lastVersion = factory.getLastVersion(contextId);
IRepositoryViewObject lastVersion = CoreRuntimePlugin.getInstance().getProxyRepositoryFactory()
.getLastVersion(contextId, possibleTypes);
if (lastVersion != null) {
final Item item = lastVersion.getProperty().getItem();
if (item != null) {
@@ -587,7 +594,7 @@ public class ContextUtils {
return false;
}
// need check the raw value, because in sourceParam, it's raw
if (!sourceParam.getValue().equals(targetParamType.getRawValue())) {
if (!StringUtils.equals(sourceParam.getValue(), targetParamType.getRawValue())) {
return false;
}
@@ -601,7 +608,8 @@ public class ContextUtils {
return false;
}
// preference name must match TalendDesignerPrefConstants.PROPAGATE_CONTEXT_VARIABLE
// preference name must match
// TalendDesignerPrefConstants.PROPAGATE_CONTEXT_VARIABLE
return Boolean.parseBoolean(
CoreRuntimePlugin.getInstance().getDesignerCoreService().getPreferenceStore("propagateContextVariable")); //$NON-NLS-1$
}
@@ -1056,4 +1064,70 @@ public class ContextUtils {
}
return false;
}
public static ExecutionResult doCreateContextLinkMigration(Item item) {
IProxyRepositoryFactory proxyRepositoryFactory = CoreRuntimePlugin.getInstance().getProxyRepositoryFactory();
boolean modified = false, hasLinkFile = false;
try {
List<ContextType> contextTypeList = ContextUtils.getAllContextType(item);
if (contextTypeList != null && contextTypeList.size() > 0) {
for (ContextType contextType : contextTypeList) {
for (Object obj : contextType.getContextParameter()) {
if (obj instanceof ContextParameterType) {
ContextParameterType paramType = (ContextParameterType) obj;
if (ContextUtils.isBuildInParameter(paramType)) {
if (StringUtils.isEmpty(paramType.getInternalId())) {
paramType.setInternalId(proxyRepositoryFactory.getNextId());
modified = true;
}
}
}
}
}
}
hasLinkFile = ContextLinkService.getInstance().saveContextLink(item);
} catch (Exception ex) {
ExceptionHandler.process(ex);
return ExecutionResult.FAILURE;
}
if (modified || hasLinkFile) {
try {
if (modified) {
proxyRepositoryFactory.save(item, true);
}
return ExecutionResult.SUCCESS_NO_ALERT;
} catch (Exception ex) {
ExceptionHandler.process(ex);
return ExecutionResult.FAILURE;
}
}
return ExecutionResult.NOTHING_TO_DO;
}
public static ExecutionResult doCreateContextLinkMigration(ERepositoryObjectType repositoryType, Item item) {
if (item != null && getAllSupportContextLinkTypes().contains(repositoryType)) {
return doCreateContextLinkMigration(item);
}
return ExecutionResult.NOTHING_TO_DO;
}
public static List<ERepositoryObjectType> getAllSupportContextLinkTypes() {
List<ERepositoryObjectType> toReturn = new ArrayList<ERepositoryObjectType>();
toReturn.addAll(ERepositoryObjectType.getAllTypesOfProcess());
toReturn.addAll(ERepositoryObjectType.getAllTypesOfProcess2());
toReturn.addAll(ERepositoryObjectType.getAllTypesOfTestContainer());
toReturn.addAll(getAllMetaDataType());
return toReturn;
}
private static List<ERepositoryObjectType> getAllMetaDataType() {
List<ERepositoryObjectType> list = new ArrayList<ERepositoryObjectType>();
ERepositoryObjectType[] allTypes = (ERepositoryObjectType[]) ERepositoryObjectType.values();
for (ERepositoryObjectType object : allTypes) {
if (object.isChildTypeOf(ERepositoryObjectType.METADATA)) {
list.add(object);
}
}
return list;
}
}

View File

@@ -18,15 +18,18 @@ import java.util.List;
import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.eclipse.emf.common.util.EList;
import org.talend.commons.exception.ExceptionHandler;
import org.talend.commons.exception.PersistenceException;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.model.context.ContextUtils;
import org.talend.core.model.properties.ContextItem;
import org.talend.core.model.properties.Item;
import org.talend.core.runtime.CoreRuntimePlugin;
import org.talend.cwm.helper.ResourceHelper;
import org.talend.designer.core.model.utils.emf.talendfile.ContextParameterType;
import org.talend.designer.core.model.utils.emf.talendfile.ContextType;
import org.talend.repository.ProjectManager;
import org.talend.repository.model.IRepositoryService;
@@ -106,16 +109,26 @@ public abstract class AbstractItemContextLinkService implements IItemContextLink
}
Item contextItem = tempCache.get(repositoryContextId);
IRepositoryService repositoryService = GlobalServiceRegister.getDefault().getService(IRepositoryService.class);
if (contextItem == null) {
contextItem = ContextUtils.getRepositoryContextItemById(repositoryContextId);
tempCache.put(repositoryContextId, contextItem);
if (contextItem != null && !(contextItem instanceof ContextItem)
&& ProjectManager.getInstance().isInCurrentMainProject(contextItem)
&& checkRepoItemContextParamInternalId(contextItem)) {
try {
repositoryService.getProxyRepositoryFactory().save(contextItem, true); // This should only using for migration phase // case the internal id is null
} catch (Exception ex) {
ExceptionHandler.process(ex);
}
}
}
if (contextItem != null) {
ContextType contextType = ContextUtils.getContextTypeByName(contextItem, contextName);
ContextParameterType repoContextParameterType = ContextUtils.getContextParameterTypeByName(contextType, paramName);
if (repoContextParameterType != null && repoContextParameterType.eResource() == null) {
// processItem save before than contextItem, caused eResource null
IRepositoryService repositoryService = GlobalServiceRegister.getDefault().getService(IRepositoryService.class);
try {
repositoryService.getProxyRepositoryFactory().save(contextItem, false);
} catch (PersistenceException e) {
@@ -170,6 +183,30 @@ public abstract class AbstractItemContextLinkService implements IItemContextLink
}
return null;
}
protected boolean checkRepoItemContextParamInternalId(Item item) {
boolean isModified = false;
EList<?> contextTypeList = ContextUtils.getAllContextType(item);
if (contextTypeList != null) {
for (Object typeObj : contextTypeList) {
if (typeObj instanceof ContextType) {
ContextType type = (ContextType) typeObj;
for (Object obj : type.getContextParameter()) {
if (obj instanceof ContextParameterType) {
ContextParameterType contextParam = (ContextParameterType) obj;
if (ContextUtils.isBuildInParameter(contextParam)
&& StringUtils.isEmpty(contextParam.getInternalId())) {
contextParam
.setInternalId(CoreRuntimePlugin.getInstance().getProxyRepositoryFactory().getNextId());
isModified = true;
}
}
}
}
}
}
return isModified;
}
}

View File

@@ -20,6 +20,7 @@ import java.util.Set;
import org.eclipse.core.runtime.IProgressMonitor;
import org.talend.commons.exception.BusinessException;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.IService;
import org.talend.core.model.general.ModuleNeeded.ELibraryInstallStatus;
import org.talend.core.model.process.IElement;
@@ -119,6 +120,8 @@ public interface ILibrariesService extends IService {
List<ModuleNeeded> getModuleNeeded(String id, boolean isGroup);
public void deployProjectLibrary(File source) throws IOException;
public void setForceReloadCustomUri();
/**
* Listener used to fire that libraries status has been changed (new lib or new check install).
@@ -131,4 +134,14 @@ public interface ILibrariesService extends IService {
public void afterChangingLibraries();
}
public static ILibrariesService get() {
GlobalServiceRegister gsr = GlobalServiceRegister.getDefault();
if (gsr != null) {
if (gsr.isServiceRegistered(ILibrariesService.class)) {
return gsr.getService(ILibrariesService.class);
}
}
return null;
}
}

View File

@@ -28,6 +28,7 @@ import org.talend.core.runtime.CoreRuntimePlugin;
import org.talend.core.runtime.maven.MavenArtifact;
import org.talend.core.runtime.maven.MavenConstants;
import org.talend.core.runtime.maven.MavenUrlHelper;
import org.talend.core.utils.TalendQuoteUtils;
/**
* This bean is use to manage needed moduless (perl) and libraries (java).<br/>
@@ -110,6 +111,15 @@ public class ModuleNeeded {
}
public static ModuleNeeded newInstance(String context, String value, String informationMsg, boolean required) {
String val = TalendQuoteUtils.removeQuotesIfExist(value);
if (val.startsWith(MavenUrlHelper.MVN_PROTOCOL)) {
return new ModuleNeeded(context, informationMsg, required, val);
}
// won't do migration for old MODULE_LIST but still make it compatible
return new ModuleNeeded(context, val, informationMsg, required);
}
/**
* DOC smallet ModuleNeeded constructor comment.
*

View File

@@ -0,0 +1,26 @@
package org.talend.core.model.metadata;
public class DbDefaultDatePattern {
private String defaultPattern;
private String dbTypeName;
public DbDefaultDatePattern() {
}
public String getDefaultPattern() {
return defaultPattern;
}
public void setDefaultPattern(String defaultPattern) {
this.defaultPattern = defaultPattern;
}
public String getDbTypeName() {
return dbTypeName;
}
public void setDbTypeName(String dbTypeName) {
this.dbTypeName = dbTypeName;
}
}

View File

@@ -41,6 +41,8 @@ public class Dbms {
private Set<MappingType> talendToDbTypes;
private List<DbDefaultLengthAndPrecision> defaultLengthPrecision;
private List<DbDefaultDatePattern> defaultPattern;
private List<DbIgnoreLengthAndPrecision> ignoreLengthPrecision;
@@ -62,6 +64,14 @@ public class Dbms {
List<DbDefaultLengthAndPrecision> defaultLengthPrecision) {
this.defaultLengthPrecision = defaultLengthPrecision;
}
public List<DbDefaultDatePattern> getDefaultPattern() {
return defaultPattern;
}
public void setDefaultPattern(
List<DbDefaultDatePattern> defaultLengthPrecision) {
this.defaultPattern = defaultLengthPrecision;
}
/**
* DOC amaumont Dbms constructor comment.

View File

@@ -125,10 +125,12 @@ public class MappingFileLoader {
// search and load preBeforelen nodes
ArrayList<String> dbTypes = new ArrayList<String>();
ArrayList<DbDefaultLengthAndPrecision> dbDefault = new ArrayList<DbDefaultLengthAndPrecision>();
ArrayList<DbDefaultDatePattern> dbDefaultPattern = new ArrayList<DbDefaultDatePattern>();
ArrayList<DbIgnoreLengthAndPrecision> dbIgnore = new ArrayList<DbIgnoreLengthAndPrecision>();
ArrayList<DbPreBeforeLength> dbPbeforeLList = new ArrayList<DbPreBeforeLength>();
dbms.setDbmsTypes(dbTypes);
dbms.setDefaultLengthPrecision(dbDefault);
dbms.setDefaultPattern(dbDefaultPattern);
dbms.setIgnoreLengthPrecision(dbIgnore);
dbms.setPrebeforelength(dbPbeforeLList);
List<Node> typeNodes = getChildElementNodes(dbTypesNode);
@@ -149,19 +151,25 @@ public class MappingFileLoader {
dbTypes.add(typeValue);
hAllDbTypes.add(typeValue);
DbDefaultLengthAndPrecision dbDefaultLP = new DbDefaultLengthAndPrecision();
DbDefaultDatePattern dbDefaultPt = new DbDefaultDatePattern();
DbIgnoreLengthAndPrecision dbIgnoreLP = new DbIgnoreLengthAndPrecision();
DbPreBeforeLength dbPBeforeL = new DbPreBeforeLength();
// default length and precision
Node defaultLengthItem = typeNodeAtttributes.getNamedItem("defaultLength"); //$NON-NLS-1$
Node defaultPrecision = typeNodeAtttributes.getNamedItem("defaultPrecision"); //$NON-NLS-1$
Node defaultPattern = typeNodeAtttributes.getNamedItem("defaultPattern"); //$NON-NLS-1$
if (defaultLengthItem != null)
dbDefaultLP.setDefaultLength(Integer.parseInt(defaultLengthItem.getNodeValue()));
if (defaultPrecision != null)
dbDefaultLP.setDefaultPrecision(Integer.parseInt(defaultPrecision.getNodeValue()));
if (defaultPattern != null)
dbDefaultPt.setDefaultPattern(defaultPattern.getNodeValue());
dbDefaultLP.setDbTypeName(typeValue);
dbDefaultPt.setDbTypeName(typeValue);
dbDefault.add(dbDefaultLP);
dbDefaultPattern.add(dbDefaultPt);
// ignore Length and Precision
Node ignoreLength = typeNodeAtttributes.getNamedItem("ignoreLen"); //$NON-NLS-1$

View File

@@ -191,6 +191,24 @@ public class MappingTypeRetriever {
}// end for
return false;
}
public String getDefaultPattern(String dbmsId, String dbType) {
Dbms dbms = MetadataTalendType.getDbms(dbmsId);
List<DbDefaultDatePattern> defaultPattern = dbms.getDefaultPattern();
String pattern = new String(""); //$NON-NLS-1$
for (int i = 0; i < defaultPattern.size(); i++) {
DbDefaultDatePattern dbPattern = (DbDefaultDatePattern) defaultPattern.get(i);
if (dbPattern.getDbTypeName().equalsIgnoreCase(dbType)) {
pattern = dbPattern.getDefaultPattern();
if (pattern == null) {
return "";
} else {
return pattern;
}
}// end if
}// end for
return pattern;
}
private boolean isExtensionPrecisionIgnored(String dbmsId, String dbType) {
Map<String, Map<String, List<DBTypeUtil>>> javaTypeMappingFromExtension = JavaTypesManager

View File

@@ -13,6 +13,7 @@
package org.talend.core.model.metadata;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.URL;
import java.util.ArrayList;
@@ -31,8 +32,11 @@ import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IProject;
import org.eclipse.core.resources.ProjectScope;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.FileLocator;
import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.Path;
@@ -488,6 +492,42 @@ public final class MetadataTalendType {
}
}
public static void syncNewMappingFileToProject() throws SystemException {
try {
File sysMappingFiles = new File(MetadataTalendType.getSystemForderURLOfMappingsFile().getPath());
IFolder projectMappingFolder = ResourceUtils.getProject(ProjectManager.getInstance().getCurrentProject())
.getFolder(MetadataTalendType.PROJECT_MAPPING_FOLDER);
File projectMappingFiles = projectMappingFolder.getFullPath().toFile();
if (sysMappingFiles.list().length == new File(projectMappingFolder.getLocationURI()).list().length) {
return;
}
for (File sysMapping : sysMappingFiles.listFiles()) {
IFile projectMapping = projectMappingFolder.getFile(sysMapping.getName());
if (!projectMapping.exists()) {
FileInputStream fis = null;
try {
fis = new FileInputStream(sysMapping);
projectMapping.create(fis, true, null);
} catch (CoreException coreExc) {
throw new SystemException(coreExc);
} finally {
if (fis != null) {
try {
fis.close();
} catch (IOException ioExc) {
throw new SystemException(ioExc);
}
}
}
}
}
} catch (Exception e) {
throw new SystemException(e);
}
}
/**
*
* Load db types and mapping with the current activated language (Java, Perl, ...).

View File

@@ -18,6 +18,7 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import org.apache.commons.lang.StringUtils;
import org.eclipse.emf.common.util.EList;
import org.eclipse.jface.dialogs.MessageDialog;
import org.eclipse.swt.widgets.Display;
@@ -26,9 +27,11 @@ import org.talend.components.api.properties.ComponentProperties;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.database.EDatabase4DriverClassName;
import org.talend.core.database.EDatabaseTypeName;
import org.talend.core.database.conn.ConnParameterKeys;
import org.talend.core.database.conn.DatabaseConnStrUtil;
import org.talend.core.database.conn.version.EDatabaseVersion4Drivers;
import org.talend.core.model.components.EComponentType;
import org.talend.core.model.metadata.Dbms;
import org.talend.core.model.metadata.IMetadataTable;
import org.talend.core.model.metadata.MetadataTalendType;
import org.talend.core.model.metadata.builder.ConvertionHelper;
@@ -70,6 +73,7 @@ import org.talend.core.model.utils.ContextParameterUtils;
import org.talend.core.model.utils.IDragAndDropServiceHandler;
import org.talend.core.runtime.i18n.Messages;
import org.talend.core.runtime.services.IGenericDBService;
import org.talend.core.runtime.services.IGenericWizardService;
import org.talend.core.utils.TalendQuoteUtils;
import org.talend.cwm.helper.ConnectionHelper;
import org.talend.cwm.helper.PackageHelper;
@@ -152,6 +156,17 @@ public class ComponentToRepositoryProperty {
conn.setDbmsId(mapping);
}
}
// set default mapping for additional jdbc
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericWizardService.class)) {
IGenericWizardService service = GlobalServiceRegister.getDefault().getService(IGenericWizardService.class);
if (service != null) {
Dbms dbms4AdditionalJDBC = service.getDbms4AdditionalJDBC(conn.getProductId());
if (dbms4AdditionalJDBC != null) {
conn.setDbmsId(dbms4AdditionalJDBC.getId());
}
}
}
}
for (IElementParameter param : node.getElementParameters()) {
String repositoryValue = param.getRepositoryValue();
@@ -365,6 +380,20 @@ public class ComponentToRepositoryProperty {
if (para.getRepositoryValue().endsWith(EDatabaseTypeName.GENERAL_JDBC.getProduct())) {
connection.setDatabaseType(EDatabaseTypeName.GENERAL_JDBC.getProduct());
connection.setProductId(EDatabaseTypeName.GENERAL_JDBC.getProduct());
// additional JDBC e.g. Delta Lake
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericWizardService.class)) {
IGenericWizardService service = GlobalServiceRegister.getDefault()
.getService(IGenericWizardService.class);
if (service != null) {
String database = service.getDatabseNameByNode(node);
if (StringUtils.isNotBlank(database) && service.getIfAdditionalJDBCDBType(database)) {
connection.setProductId(database);
}
}
}
}
// vertica output component have no TYPE ElementParameter .
if (para.getRepositoryValue().endsWith(EDatabaseTypeName.VERTICA.getProduct())) {
@@ -667,6 +696,18 @@ public class ComponentToRepositoryProperty {
connection.setServerName(value);
}
}
if ("IMPALA_DRIVER".equals(param.getRepositoryValue())) {
String value = getParameterValue(connection, node, param);
if (value != null) {
connection.getParameters().put(ConnParameterKeys.IMPALA_DRIVER, value);
}
}
if ("IMPALA_ADDITIONAL_JDBC".equals(param.getRepositoryValue())) {
String value = getParameterValue(connection, node, param);
if (value != null) {
connection.getParameters().put(ConnParameterKeys.CONN_PARA_KEY_HIVE_ADDITIONAL_JDBC_SETTINGS, value);
}
}
if (connection.getDatabaseType().equals(EDatabaseTypeName.ORACLEFORSID.getDisplayName())) {
setDatabaseValueForOracleSid(connection, node, param);

View File

@@ -1490,21 +1490,24 @@ public class RepositoryToComponentProperty {
}
if (value.equals("USE_SSL") && (EDatabaseTypeName.HIVE.getDisplayName().equals(databaseType)
|| EDatabaseTypeName.ORACLE_CUSTOM.getDisplayName().equals(databaseType))) {
|| EDatabaseTypeName.ORACLE_CUSTOM.getDisplayName().equals(databaseType)
|| EDatabaseTypeName.IMPALA.getDisplayName().equals(databaseType))) {
String message = connection.getParameters().get(ConnParameterKeys.CONN_PARA_KEY_USE_SSL);
return Boolean.parseBoolean(message);
}
if ((value.equals("SSL_TRUST_STORE") || value.equals("SSL_TRUSTSERVER_TRUSTSTORE"))
&& (EDatabaseTypeName.HIVE.getDisplayName().equals(databaseType)
|| EDatabaseTypeName.ORACLE_CUSTOM.getDisplayName().equals(databaseType))) {
|| EDatabaseTypeName.ORACLE_CUSTOM.getDisplayName().equals(databaseType)
|| EDatabaseTypeName.IMPALA.getDisplayName().equals(databaseType))) {
return getAppropriateValue(connection,
connection.getParameters().get(ConnParameterKeys.CONN_PARA_KEY_SSL_TRUST_STORE_PATH));
}
if ((value.equals("SSL_TRUST_STORE_PASSWORD") || value.equals("SSL_TRUSTSERVER_PASSWORD"))
&& (EDatabaseTypeName.HIVE.getDisplayName().equals(databaseType)
|| EDatabaseTypeName.ORACLE_CUSTOM.getDisplayName().equals(databaseType))) {
|| EDatabaseTypeName.ORACLE_CUSTOM.getDisplayName().equals(databaseType)
|| EDatabaseTypeName.IMPALA.getDisplayName().equals(databaseType))) {
return getAppropriateValue(connection, connection
.getValue(connection.getParameters().get(ConnParameterKeys.CONN_PARA_KEY_SSL_TRUST_STORE_PASSWORD), false));
}
@@ -1755,8 +1758,18 @@ public class RepositoryToComponentProperty {
if (StringUtils.equals("MAPPING", value)) {//$NON-NLS-1$
return connection.getDbmsId();
}
if ("IMPALA_ADDITIONAL_JDBC".equals(value)) { //$NON-NLS-1$
String additionJdbc = connection.getParameters().get(ConnParameterKeys.CONN_PARA_KEY_HIVE_ADDITIONAL_JDBC_SETTINGS);
if (isContextMode(connection, additionJdbc)) {
return additionJdbc;
} else {
return TalendQuoteUtils.addQuotes(additionJdbc);
}
}
if ("IMPALA_DRIVER".equals(value)) {
return connection.getParameters().get(ConnParameterKeys.IMPALA_DRIVER);
}
return null;
}
private static String getAppropriateValue(Connection connection, String rawValue) {

View File

@@ -0,0 +1,25 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.model.process;
/**
* DOC cmeng class global comment. Detailled comment
*/
public abstract class AbsJobletReplaceNodeHandler extends AbsReplaceNodeHandler implements IJobletReplaceNodeHandler {
public AbsJobletReplaceNodeHandler(String prefix) {
super();
this.setPrefix(prefix);
}
}

View File

@@ -0,0 +1,31 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.model.process;
/**
* DOC cmeng class global comment. Detailled comment
*/
public abstract class AbsReplaceNodeHandler implements IReplaceNodeHandler {
private String prefix;
@Override
public String getPrefix() {
return prefix;
}
public void setPrefix(String prefix) {
this.prefix = prefix;
}
}

View File

@@ -122,6 +122,8 @@ public abstract class AbstractNode implements INode {
// for MR, tag this component is the ref(lookup) start node
private boolean isRefNode = false;
private IReplaceNodeHandler replaceNodeHandler;
public String getComponentName() {
return componentName;
}
@@ -1322,6 +1324,15 @@ public abstract class AbstractNode implements INode {
return componentProperties;
}
@Override
public IReplaceNodeHandler getReplaceNodeHandler() {
return replaceNodeHandler;
}
public void setReplaceNodeHandler(IReplaceNodeHandler replaceNodeHandler) {
this.replaceNodeHandler = replaceNodeHandler;
}
public INode getRealGraphicalNode() {
return realGraphicalNode;
}

View File

@@ -44,6 +44,7 @@ public enum EComponentCategory {
DYNAMICS_SETTINGS(Messages.getString("EComponentCategory_dynamicSetting"), 13), //$NON-NLS-1$
SQL_PATTERN(Messages.getString("EComponentCategory_sqlTemplate"), 14), //$NON-NLS-1$
BREAKPOINT(Messages.getString("EComponentCategory.breakpoint"), 15), //$NON-NLS-1$
BREAKPOINT_CAMEL(Messages.getString("EComponentCategory.breakpoint"), 16), //$NON-NLS-1$
BASICRUN(Messages.getString("EComponentCategory.basicRun"), 1), //$NON-NLS-1$
DEBUGRUN(Messages.getString("EComponentCategory.debugRun"), 2), //$NON-NLS-1$
ADVANCESETTING(Messages.getString("EComponentCategory.advancedSettings"), 3), //$NON-NLS-1$

View File

@@ -113,8 +113,6 @@ public enum EParameterFieldType {
GROUPING_CAMPAIGN_CHOOSER, // htyin added for DQ Matching components
MULTI_PATTERN, // yyin added TDQ-13437
SYNC_NEXUS_BUTTON,
CHECK_NEXUS_BUTTON,
PATTERN_PROPERTY, // yyin, added TDQ-13437
UNIFIED_COMPONENTS,

View File

@@ -0,0 +1,20 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.model.process;
/**
* DOC cmeng class global comment. Detailled comment
*/
public interface IJobletReplaceNodeHandler extends IReplaceNodeHandler {
}

View File

@@ -312,6 +312,10 @@ public interface INode extends IElement {
public boolean isSubtreeStart();
default public IReplaceNodeHandler getReplaceNodeHandler() {
return null;
}
public void setComponentProperties(ComponentProperties props);
public ComponentProperties getComponentProperties();

View File

@@ -0,0 +1,22 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.model.process;
/**
* DOC cmeng class global comment. Detailled comment
*/
public interface IReplaceNodeHandler {
String getPrefix();
}

View File

@@ -0,0 +1,25 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.model.process;
/**
* DOC cmeng class global comment. Detailled comment
*/
public class JobletReplaceNodeHandler extends AbsJobletReplaceNodeHandler {
public JobletReplaceNodeHandler(String prefix) {
super(prefix);
}
}

View File

@@ -1003,4 +1003,12 @@ public final class ProcessUtils {
}
return false;
}
public static boolean isChildRouteProcess(IProcess process) {
List n = process.getNodesOfType("tRouteInput");
if (n!=null && n.size()!=0) {
return true;
}
return false;
}
}

View File

@@ -51,6 +51,7 @@ import org.talend.designer.core.model.utils.emf.talendfile.NodeType;
import org.talend.designer.core.model.utils.emf.talendfile.ProcessType;
import org.talend.repository.ProjectManager;
import org.talend.repository.model.IProxyRepositoryFactory;
import org.talend.repository.model.IRepositoryNode;
/**
* This class store all relationships between jobs/joblets and other items from the repository. Be sure to update the
@@ -493,7 +494,7 @@ public class RelationshipItemBuilder {
}
private Set<Relation> getItemsRelatedTo(Map<Relation, Set<Relation>> itemsRelations, String itemId, String version,
String relationType) {
String relationType) {
Relation itemToTest = new Relation();
itemId = ProcessUtils.getPureItemId(itemId);
@@ -929,7 +930,10 @@ public class RelationshipItemBuilder {
}
}
private void addRelationShip(Item baseItem, String relatedId, String relatedVersion, String type) {
public void addRelationShip(Item baseItem, String relatedId, String relatedVersion, String type) {
if (!loaded) {
loadRelations();
}
Relation relation = new Relation();
relation.setId(baseItem.getProperty().getId());
relation.setType(getTypeFromItem(baseItem));
@@ -945,8 +949,32 @@ public class RelationshipItemBuilder {
itemRelations.put(relation, new HashSet<Relation>());
}
itemRelations.get(relation).add(addedRelation);
autoSaveRelations();
}
public Set<Relation> getBeanRelations(Collection<Item> items) {
Set<Relation> relationships = new HashSet<Relation>();
for (Item item : items) {
Relation relation = new Relation();
relation.setId(item.getProperty().getId());
relation.setType(getTypeFromItem(item));
relation.setVersion(item.getProperty().getVersion());
Map<Relation, Set<Relation>> itemRelations = getRelatedRelations(item);
Set<Relation> repositoryNode = (Set<Relation>) itemRelations.getOrDefault(relation, new HashSet<Relation>());
for (Relation rel : repositoryNode) {
if (rel.getType().equals("Beans")) {
relationships.add(rel);
}
}
}
return relationships;
}
private Map<Relation, Set<Relation>> getRelatedRelations(Item baseItem) {
Map<Relation, Set<Relation>> itemRelations = currentProjectItemsRelations;
if (!ProjectManager.getInstance().isInMainProject(getAimProject(), baseItem)) {
@@ -1022,7 +1050,7 @@ public class RelationshipItemBuilder {
ExceptionHandler.process(e);
}
}
private List<ERepositoryObjectType> allSupportedTypes() {
List<ERepositoryObjectType> toReturn = new ArrayList<ERepositoryObjectType>();
toReturn.addAll(ERepositoryObjectType.getAllTypesOfProcess());
@@ -1494,4 +1522,5 @@ public class RelationshipItemBuilder {
return 0;
}
}
}

View File

@@ -1414,12 +1414,8 @@ public class ERepositoryObjectType extends DynaEnum<ERepositoryObjectType> {
return null;
}
public static List<ERepositoryObjectType> getAllTypesOfProcess() {
public static List<ERepositoryObjectType> getAllTypesOfBigDataProcess() {
List<ERepositoryObjectType> allTypes = new ArrayList<ERepositoryObjectType>();
if (ERepositoryObjectType.PROCESS != null) {
allTypes.add(ERepositoryObjectType.PROCESS);
}
if (ERepositoryObjectType.PROCESS_MR != null) {
allTypes.add(ERepositoryObjectType.PROCESS_MR);
}
@@ -1432,6 +1428,17 @@ public class ERepositoryObjectType extends DynaEnum<ERepositoryObjectType> {
if (ERepositoryObjectType.PROCESS_SPARKSTREAMING != null) {
allTypes.add(ERepositoryObjectType.PROCESS_SPARKSTREAMING);
}
return allTypes;
}
public static List<ERepositoryObjectType> getAllTypesOfProcess() {
List<ERepositoryObjectType> allTypes = new ArrayList<ERepositoryObjectType>();
allTypes.addAll(getAllTypesOfBigDataProcess());
if (ERepositoryObjectType.PROCESS != null) {
allTypes.add(ERepositoryObjectType.PROCESS);
}
if (ERepositoryObjectType.PROCESS_ROUTE != null) {
allTypes.add(ERepositoryObjectType.PROCESS_ROUTE);
}

View File

@@ -27,6 +27,8 @@ public interface SVNConstant {
public static final String NAME_MASTER = "master"; //$NON-NLS-1$
public static final String NAME_MAIN = "main"; //$NON-NLS-1$
public static final String NAME_ORIGIN = "origin"; //$NON-NLS-1$
/*

View File

@@ -320,24 +320,28 @@ public class RepositoryUpdateManagerHelper {
// all the jobs
for (IRepositoryViewObject process : processRep) {
Item item = process.getProperty().getItem();
boolean found = false;
for (IProcess2 open : openedProcessList) {
if (open.getId().equals(item.getProperty().getId())) {
found = true;
try {
Item item = process.getProperty().getItem();
boolean found = false;
for (IProcess2 open : openedProcessList) {
if (open.getId().equals(item.getProperty().getId())) {
found = true;
}
}
}
if (found) {
continue;
}
checkMonitorCanceled(parentMonitor);
parentMonitor.subTask(RepositoryUpdateManager.getUpdateJobInfor(item.getProperty()));
if (found) {
continue;
}
checkMonitorCanceled(parentMonitor);
parentMonitor.subTask(RepositoryUpdateManager.getUpdateJobInfor(item.getProperty()));
// List<UpdateResult> resultFromProcess = getResultFromProcess(process, types, onlySimpleShow);
// List<UpdateResult> resultFromProcess = getResultFromProcess(process, types, onlySimpleShow);
List<UpdateResult> resultFromProcess = getUpdatesNeededFromItems(parentMonitor, item, types);
if (resultFromProcess != null) {
resultList.addAll(resultFromProcess);
List<UpdateResult> resultFromProcess = getUpdatesNeededFromItems(parentMonitor, item, types);
if (resultFromProcess != null) {
resultList.addAll(resultFromProcess);
}
} catch (Exception ex) {
ExceptionHandler.process(ex);
}
parentMonitor.worked(1);
}

View File

@@ -46,7 +46,11 @@ import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.INode;
import org.talend.core.model.process.INodeConnector;
import org.talend.core.model.process.IProcess;
import org.talend.core.model.process.IProcess2;
import org.talend.core.model.process.ProcessUtils;
import org.talend.core.model.properties.Property;
import org.talend.core.runtime.IAdditionalInfo;
import org.talend.core.runtime.projectsetting.RuntimeLineageManager;
import org.talend.designer.core.ICamelDesignerCoreService;
/**
@@ -866,6 +870,170 @@ public class NodeUtil {
}
throw new IllegalArgumentException();
}
public static String getRuntimeParameterValue(INode node, IElementParameter ep) {
if (EParameterFieldType.TABLE.equals(ep.getFieldType())) {
Map<String, IElementParameter> types = new HashMap<String, IElementParameter>();
Object[] itemsValue = ep.getListItemsValue();
if (itemsValue != null) {
for (Object o : itemsValue) {
IElementParameter cep = (IElementParameter) o;
if (cep.isShow(node.getElementParameters())) {
types.put(cep.getName(), cep);
}
}
}
List<Map<String, String>> lines = (List<Map<String, String>>) ElementParameterParser.getObjectValue(node,
"__" + ep.getName() + "__");
StringBuilder value = new StringBuilder();
// implement List & Map toString(), different is the value of Map
Iterator<Map<String, String>> linesIter = lines.iterator();
if (!linesIter.hasNext()) {
return "\"[]\"";
}
value.append("new StringBuilder().append(\"[");
for (;;) {
Map<String, String> columns = linesIter.next();
Iterator<Entry<String, String>> columnsIter = columns.entrySet().iterator();
value.append("{");
Entry<String, String> column = null;
boolean printedColumnExist = false;
while (columnsIter.hasNext()) {
column = columnsIter.next();
if (types.get(column.getKey()) == null) {
continue;
}
printedColumnExist = true;
value.append(column.getKey());
value.append("=\").append(");
value.append(getRuntimeParameterValue(column.getValue(), types.get(column.getKey()), true));
value.append(").append(\"");
if (columnsIter.hasNext()) {
value.append(", ");
}
}
if (printedColumnExist && column != null && (types.get(column.getKey()) == null)) {
value.setLength(value.length() - 2);
}
value.append("}");
if (!linesIter.hasNext()) {
return value.append("]\").toString()").toString();
}
value.append(",").append(" ");
}
} else {
String value = ElementParameterParser.getValue(node, "__" + ep.getName() + "__");
if (EParameterFieldType.TABLE_BY_ROW.equals(ep.getFieldType())) {
value = ep.getValue().toString();
}
return getRuntimeParameterValue(value, ep, false);
}
}
private static String getRuntimeParameterValue(String value, IElementParameter ep, boolean itemFromTable) {
if (value == null) {
value = "";
}
value = value.trim();
boolean isMemo = false;
List<EParameterFieldType> needRemoveCRLFList = Arrays.asList(EParameterFieldType.MEMO, EParameterFieldType.MEMO_JAVA,
EParameterFieldType.MEMO_SQL, EParameterFieldType.MEMO_IMPORT, EParameterFieldType.MEMO_MESSAGE);
if (needRemoveCRLFList.contains(ep.getFieldType())) {
isMemo = true;
value = value.replaceAll("[\r\n]", " ");
}
List<EParameterFieldType> needQuoteList = Arrays.asList(EParameterFieldType.CLOSED_LIST,
EParameterFieldType.COMPONENT_LIST, EParameterFieldType.COLUMN_LIST, EParameterFieldType.PREV_COLUMN_LIST,
EParameterFieldType.CONNECTION_LIST, EParameterFieldType.LOOKUP_COLUMN_LIST,
EParameterFieldType.CONTEXT_PARAM_NAME_LIST, EParameterFieldType.PROCESS_TYPE, EParameterFieldType.COLOR,
EParameterFieldType.TABLE_BY_ROW, EParameterFieldType.HADOOP_JARS_DIALOG, EParameterFieldType.UNIFIED_COMPONENTS);
List<EParameterFieldType> needQuoteListForItem = itemFromTable ? Arrays.asList(EParameterFieldType.SCHEMA_TYPE,
EParameterFieldType.SAP_SCHEMA_TYPE, EParameterFieldType.MODULE_LIST) : new ArrayList<EParameterFieldType>();
// TODO: add RAW attribute when SCHEMA_COLUMN generated by BASED_ON_SCHEMA
List<String> needQuoteListByName = Arrays.asList("SCHEMA_COLUMN");// SCHEMA_COLUMN for BASED_ON_SCHEMA="true"
if (needQuoteList.contains(ep.getFieldType()) || needQuoteListForItem.contains(ep.getFieldType())
|| needQuoteListByName.contains(ep.getName()) || ep.isRaw()) {
value = value.replaceAll("\\\\", "\\\\\\\\");
value = value.replaceAll("\\\"", "\\\\\\\"");
return "\"" + value + "\"";
}
if (itemFromTable) {
if ("*".equals(value)) {
return "\"" + value + "\"";
}
if (value.endsWith(";")) {
value = value.substring(0, value.length() - 1);
}
}
if("".equals(value) || "\"\"".equals(value)) {
return "\"\"";
} else if("null".equals(value)) {
return "(Object)null";
}
// copied it from Log4jFileUtil.javajet but need more comment for this script
if ("\"\\n\"".equals(value) || "\"\\r\"".equals(value) || "\"\\r\\n\"".equals(value)) {
// for the value is "\n" "\r" "\r\n"
return value.replaceAll("\\\\", "\\\\\\\\");
} else if ("\"\"\"".equals(value)) {
return "\"" + "\\" + "\"" + "\"";
} else if ("\"\"\\r\\n\"\"".equals(value)) {
return "\"\\\\r\\\\n\"";
} else if ("\"\"\\r\"\"".equals(value)) {
return "\"\\\\r\"";
} else if ("\"\"\\n\"\"".equals(value)) {
return "\"\\\\n\"";
}
// ftom 20141008 - patch to fix javajet compilation errors due to hard-coded studio TableEditor mechanism
// linked to BUILDIN properties checks, this item is a boolean set to TRUE or FALSE
// fix is just transforming into true or false to make logging OK
else if ("BUILDIN".equals(ep.getName())) {
return value.toLowerCase();
}
//suppose all memo fields are processed well already, no need to go though this with dangerous
if (!isMemo && !org.talend.core.model.utils.ContextParameterUtils.isDynamic(value)) {
if(value.length() > 1 && value.startsWith("\"") && value.endsWith("\"")) {
if(itemFromTable && "ARGS".equals(ep.getName())) {
value = value.substring(1, value.length());
value = value.substring(0, value.length() - 1);
return "\"" + checkStringQuotationMarks(value) + "\"";
} else {
//do nothing
return value;
}
} else {
return "\"" + checkStringQuotationMarks(value) + "\"";
}
}
//TODO remove it
if (value.endsWith("*")) {
return value.substring(0, value.length() - 1) + "\"*\"";
}
return value;
}
private static String checkStringQuotationMarks(String str) {
String result = str;
if (result.contains("\"")) {
result = result.replace("\"", "\\\"");
}
return result;
}
public static String getNormalizeParameterValue(INode node, IElementParameter ep) {
if (EParameterFieldType.TABLE.equals(ep.getFieldType())) {
@@ -1189,7 +1357,7 @@ public class NodeUtil {
} else if (ComponentCategory.CATEGORY_4_CAMEL.getName().equals(node.getComponent().getType())) {
INodeConnector tmp = null;
if (GlobalServiceRegister.getDefault().isServiceRegistered(ICamelDesignerCoreService.class)) {
ICamelDesignerCoreService camelService = (ICamelDesignerCoreService) GlobalServiceRegister.getDefault()
ICamelDesignerCoreService camelService = GlobalServiceRegister.getDefault()
.getService(ICamelDesignerCoreService.class);
tmp = node.getConnectorFromType(camelService.getTargetConnectionType(node));
} else {
@@ -1232,4 +1400,37 @@ public class NodeUtil {
return true;
}
public static boolean isJobUsingRuntimeLineage(IProcess process) {
// Just support DI jobs now
boolean isSupport = isStandardJob(process) && !ProcessUtils.isTestContainer(process) && !isGuessSchemaJob(process);
if (!isSupport) {
return false;
}
RuntimeLineageManager runtimeLineageManager = new RuntimeLineageManager();
if (runtimeLineageManager.isUseRuntimeLineageAll()) {
return true;
}
if (runtimeLineageManager.getSelectedJobIds().isEmpty()) {
runtimeLineageManager.load();
}
return runtimeLineageManager.isRuntimeLineageSetting(process.getId());
}
public static boolean isStandardJob(IProcess process) {
if (process != null && process instanceof IProcess2) {
Property property = ((IProcess2) process).getProperty();
return property != null && property.getItem() != null
&& ComponentCategory.CATEGORY_4_DI.getName().equals(process.getComponentsType());
}
return false;
}
public static boolean isGuessSchemaJob(IProcess process) {
if (process != null && process instanceof IProcess2) {
Property property = ((IProcess2) process).getProperty();
return property != null && "ID".equals(property.getId()) && "Mock_job_for_Guess_schema".equals(property.getLabel()); //$NON-NLS-1$ //$NON-NLS-2$
}
return false;
}
}

View File

@@ -15,6 +15,7 @@ package org.talend.core.model.utils;
import java.util.ArrayList;
import java.util.List;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.model.metadata.builder.connection.Connection;
import org.talend.core.model.metadata.builder.connection.DatabaseConnection;
import org.talend.core.model.metadata.designerproperties.RepositoryToComponentProperty;
@@ -23,6 +24,7 @@ import org.talend.core.model.properties.ContextItem;
import org.talend.core.model.properties.Item;
import org.talend.core.model.repository.ERepositoryObjectType;
import org.talend.core.model.update.UpdatesConstants;
import org.talend.core.runtime.services.IGenericWizardService;
/**
* ggu class global comment. Detailled comment
@@ -47,6 +49,14 @@ public final class UpdateRepositoryHelper {
if (connection instanceof DatabaseConnection) {
String currentDbType = (String) RepositoryToComponentProperty.getValue(connection, UpdatesConstants.TYPE,
null);
String productId = ((DatabaseConnection) connection).getProductId();
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericWizardService.class)) {
IGenericWizardService service = GlobalServiceRegister.getDefault()
.getService(IGenericWizardService.class);
if (service != null && service.getIfAdditionalJDBCDBType(productId)) {
currentDbType = productId;
}
}
aliasName += " (" + currentDbType + ")"; //$NON-NLS-1$ //$NON-NLS-2$
}
if (repositoryObjectType.getType().equals("SERVICES")) {

View File

@@ -73,4 +73,6 @@ public interface IRepositoryArtifactHandler {
public String resolveRemoteSha1(MavenArtifact artifact, boolean fromRelease) throws Exception;
public List<MavenArtifact> search(String name, boolean fromSnapshot) throws Exception;
}

View File

@@ -19,7 +19,9 @@ import java.io.InputStream;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import org.apache.commons.lang.StringUtils;
import org.apache.http.HttpEntity;
@@ -53,6 +55,14 @@ public class NexusServerUtils {
// the max search result is 200 by defult from nexus
private static final int MAX_SEARCH_COUNT = 200;
public static final Set<String> IGNORED_TYPES = new HashSet<String>();
static {
IGNORED_TYPES.add("pom");
IGNORED_TYPES.add("sha1");
IGNORED_TYPES.add("md5");
}
/**
*
* DOC check if the repository exist or not
@@ -236,36 +246,36 @@ public class NexusServerUtils {
private static int readDocument(Document document, List<MavenArtifact> artifacts) throws Exception {
List<Node> list = document.selectNodes("/searchNGResponse/data/artifact");//$NON-NLS-1$
for (Node arNode : list) {
MavenArtifact artifact = new MavenArtifact();
artifacts.add(artifact);
artifact.setGroupId(arNode.selectSingleNode("groupId").getText());//$NON-NLS-1$
artifact.setArtifactId(arNode.selectSingleNode("artifactId").getText());//$NON-NLS-1$
artifact.setVersion(arNode.selectSingleNode("version").getText());//$NON-NLS-1$
Node descNode = arNode.selectSingleNode("description");//$NON-NLS-1$
if (descNode != null) {
artifact.setDescription(descNode.getText());
}
Node urlNode = arNode.selectSingleNode("url");//$NON-NLS-1$
if (urlNode != null) {
artifact.setUrl(urlNode.getText());
}
Node licenseNode = arNode.selectSingleNode("license");//$NON-NLS-1$
if (licenseNode != null) {
artifact.setLicense(licenseNode.getText());
}
Node licenseUrlNode = arNode.selectSingleNode("licenseUrl");//$NON-NLS-1$
if (licenseUrlNode != null) {
artifact.setLicenseUrl(licenseUrlNode.getText());
}
List<Node> artLinks = arNode.selectNodes("artifactHits/artifactHit/artifactLinks/artifactLink");//$NON-NLS-1$
for (Node link : artLinks) {
MavenArtifact artifact = new MavenArtifact();
artifact.setGroupId(arNode.selectSingleNode("groupId").getText());//$NON-NLS-1$
artifact.setArtifactId(arNode.selectSingleNode("artifactId").getText());//$NON-NLS-1$
artifact.setVersion(arNode.selectSingleNode("version").getText());//$NON-NLS-1$
Node descNode = arNode.selectSingleNode("description");//$NON-NLS-1$
if (descNode != null) {
artifact.setDescription(descNode.getText());
}
Node urlNode = arNode.selectSingleNode("url");//$NON-NLS-1$
if (urlNode != null) {
artifact.setUrl(urlNode.getText());
}
Node licenseNode = arNode.selectSingleNode("license");//$NON-NLS-1$
if (licenseNode != null) {
artifact.setLicense(licenseNode.getText());
}
Node licenseUrlNode = arNode.selectSingleNode("licenseUrl");//$NON-NLS-1$
if (licenseUrlNode != null) {
artifact.setLicenseUrl(licenseUrlNode.getText());
}
Node extensionElement = link.selectSingleNode("extension");//$NON-NLS-1$
String extension = null;
String classifier = null;
if (extensionElement != null) {
if ("pom".equals(extensionElement.getText())) {//$NON-NLS-1$
if (IGNORED_TYPES.contains(extensionElement.getText())) {// $NON-NLS-1$
continue;
}
extension = extensionElement.getText();
@@ -276,6 +286,7 @@ public class NexusServerUtils {
}
artifact.setType(extension);
artifact.setClassifier(classifier);
artifacts.add(artifact);
}
}
return list.size();
@@ -370,4 +381,37 @@ public class NexusServerUtils {
}
public static List<MavenArtifact> search(String nexusUrl, String userName, String password, String repositoryId, String name)
throws Exception {
List<MavenArtifact> artifacts = new ArrayList<MavenArtifact>();
int totalCount = 0;
String service = NexusConstants.SERVICES_SEARCH + getSearchQuery(repositoryId, null, null, null, 0, MAX_SEARCH_COUNT)
+ "&q=" + name;
URI requestURI = getSearchURI(nexusUrl, service);
Document document = downloadDocument(requestURI, userName, password);
if (document != null) {
Node countNode = document.selectSingleNode("/searchNGResponse/totalCount");
if (countNode != null) {
try {
totalCount = Integer.parseInt(countNode.getText());
} catch (NumberFormatException e) {
totalCount = 0;
}
}
int searchDone = readDocument(document, artifacts);
while (searchDone < totalCount) {
service = NexusConstants.SERVICES_SEARCH
+ getSearchQuery(repositoryId, null, null, null, searchDone, MAX_SEARCH_COUNT) + "&q=" + name;
requestURI = getSearchURI(nexusUrl, service);
document = downloadDocument(requestURI, userName, password);
searchDone = searchDone + readDocument(document, artifacts);
}
}
return artifacts;
}
}

View File

@@ -15,6 +15,7 @@ package org.talend.core.runtime.evaluator;
import java.util.List;
import org.apache.commons.lang3.StringEscapeUtils;
import org.talend.core.model.utils.ContextParameterUtils;
import org.talend.core.runtime.util.GenericTypeUtils;
import org.talend.core.utils.TalendQuoteUtils;
import org.talend.daikon.properties.property.Property;
@@ -28,6 +29,10 @@ import org.talend.daikon.properties.property.StringProperty;
public abstract class AbstractPropertyValueEvaluator implements PropertyValueEvaluator {
public Object getTypedValue(Property property, Object rawValue) {
return getTypedValue(property, null, rawValue);
}
public Object getTypedValue(Property property, Object storedValue, Object rawValue) {
if (GenericTypeUtils.isSchemaType(property)) {
return rawValue;
}
@@ -105,6 +110,12 @@ public abstract class AbstractPropertyValueEvaluator implements PropertyValueEva
String stringStoredValue = TalendQuoteUtils.removeQuotes(stringValue);
for (Object possibleValue : possibleValues) {
if (possibleValue.toString().equals(stringStoredValue)) {
// Update since enum type set as context for tcompv0 .
String currentStoredValue = String.valueOf(storedValue);
if (storedValue != null && ContextParameterUtils.isContainContextParam(currentStoredValue)) {
property.setTaggedValue("IS_CONTEXT_MODE", false);
property.setValue(possibleValue);
}
return possibleValue;
}
}

View File

@@ -44,6 +44,8 @@ public interface MavenConstants {
static final String USE_PROFILE_MODULE = "USE_PROFILE_MODULE";
static final String EXCLUDE_DELETED_ITEMS = "EXCLUDE_DELETED_ITEMS";
/*
* for lib
*/

View File

@@ -174,7 +174,7 @@ public class MavenUrlHelper {
if (jarName != null && jarName.length() > 0) {
String artifactId = jarName;
String type = null;
if (jarName.endsWith(MavenConstants.TYPE_JAR)) { // remove the extension .jar
if (jarName.endsWith("." + MavenConstants.TYPE_JAR)) { // remove the extension .jar
artifactId = jarName.substring(0, jarName.lastIndexOf(MavenConstants.TYPE_JAR) - 1);
if (withPackage) {
type = MavenConstants.TYPE_JAR;
@@ -380,4 +380,14 @@ public class MavenUrlHelper {
}
return buffer.toString();
}
public static String getSNAPSHOTVersion(String rVersion) {
if (rVersion == null) {
return rVersion;
}
if (rVersion.contains("-")) {
return rVersion.substring(0, rVersion.indexOf("-") + 1) + MavenUrlHelper.VERSION_SNAPSHOT;
}
return rVersion;
}
}

View File

@@ -0,0 +1,156 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.runtime.projectsetting;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import org.apache.commons.lang3.StringUtils;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IProject;
import org.talend.commons.exception.PersistenceException;
import org.talend.commons.ui.runtime.exception.ExceptionHandler;
import org.talend.commons.utils.workbench.resources.ResourceUtils;
import org.talend.repository.ProjectManager;
import org.talend.repository.model.RepositoryConstants;
import org.talend.repository.model.RepositoryNode;
import org.talend.utils.json.JSONObject;
import us.monoid.json.JSONArray;
/**
* created by hcyi on Jul 27, 2020
* Detailled comment
*
*/
public class RuntimeLineageManager {
public static final String RUNTIMELINEAGE_RESOURCES = "org.talend.runtimelineage"; //$NON-NLS-1$
public static final String RUNTIMELINEAGE_ALL = "runtimelineage.all"; //$NON-NLS-1$
public static final String RUNTIMELINEAGE_SELECTED = "runtimelineage.selected"; //$NON-NLS-1$
public static final String JOB_ID = "id"; //$NON-NLS-1$
public static final String RUNTIMELINEAGE_OUTPUT_PATH = "-Druntime.lineage.outputpath="; //$NON-NLS-1$
public static final String OUTPUT_PATH = "output.path"; //$NON-NLS-1$
private List<String> selectedJobIds = new ArrayList<String>();
private ProjectPreferenceManager prefManager = null;
private boolean useRuntimeLineageAll = false;
private String outputPath = null;
public RuntimeLineageManager() {
if (prefManager == null) {
prefManager = new ProjectPreferenceManager(RUNTIMELINEAGE_RESOURCES, true);
}
useRuntimeLineageAll = prefManager.getBoolean(RUNTIMELINEAGE_ALL);
outputPath = prefManager.getValue(OUTPUT_PATH);
}
public void load() {
try {
String jobsJsonStr = prefManager.getValue(RUNTIMELINEAGE_SELECTED);
if (StringUtils.isNotBlank(jobsJsonStr)) {
JSONArray jobsJsonArray = new JSONArray(jobsJsonStr);
for (int i = 0; i < jobsJsonArray.length(); i++) {
Object jobJsonObj = jobsJsonArray.get(i);
JSONObject jobJson = new JSONObject(String.valueOf(jobJsonObj));
Iterator sortedKeys = jobJson.sortedKeys();
String jobId = null;
while (sortedKeys.hasNext()) {
String key = (String) sortedKeys.next();
if (JOB_ID.equals(key)) {
jobId = jobJson.getString(key);
}
}
if (jobId != null) {
selectedJobIds.add(jobId);
}
}
}
} catch (Exception e) {
ExceptionHandler.process(e);
}
}
public void save(List<RepositoryNode> checkedObjects, boolean all) {
try {
JSONArray jobsJson = new JSONArray();
if (!all) {
for (RepositoryNode node : checkedObjects) {
JSONObject jobJson = new JSONObject();
if (!jobsJson.toString().contains(node.getId())) {
jobJson.put(JOB_ID, node.getId());
jobsJson.put(jobJson);
}
}
}
prefManager.setValue(RUNTIMELINEAGE_ALL, all);
prefManager.setValue(RUNTIMELINEAGE_SELECTED, jobsJson.toString());
prefManager.setValue(OUTPUT_PATH, outputPath);
prefManager.save();
} catch (Exception e) {
ExceptionHandler.process(e);
}
}
public boolean isRuntimeLineageSetting(String id) {
return selectedJobIds.contains(id);
}
public boolean isRuntimeLineagePrefsExist() {
try {
IProject project = ResourceUtils.getProject(ProjectManager.getInstance().getCurrentProject());
IFolder prefSettingFolder = ResourceUtils.getFolder(project, RepositoryConstants.SETTING_DIRECTORY, false);
IFile presRuntimeLineageFile = prefSettingFolder.getFile(RUNTIMELINEAGE_RESOURCES + ".prefs"); //$NON-NLS-1$
if (presRuntimeLineageFile.exists()) {
return true;
}
} catch (PersistenceException e) {
e.printStackTrace();
}
return false;
}
public boolean isUseRuntimeLineageAll() {
return this.useRuntimeLineageAll;
}
public ProjectPreferenceManager getPrefManager() {
return this.prefManager;
}
public List<String> getSelectedJobIds() {
return this.selectedJobIds;
}
public void setSelectedJobIds(List<String> selectedJobIds) {
this.selectedJobIds = selectedJobIds;
}
public String getOutputPath() {
return this.outputPath;
}
public void setOutputPath(String outputPath) {
this.outputPath = outputPath;
}
}

View File

@@ -18,6 +18,7 @@ import java.util.Map;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.IService;
import org.talend.core.model.components.IComponent;
import org.talend.core.model.metadata.builder.connection.Connection;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.INode;
@@ -46,6 +47,8 @@ public interface IGenericService extends IService {
public boolean isTcompv0(IComponent component);
public void validateGenericConnection(Connection conn) throws Exception;
public static IGenericService getService() {
if (GlobalServiceRegister.getDefault().isServiceRegistered(IGenericService.class)) {
return GlobalServiceRegister.getDefault().getService(IGenericService.class);

View File

@@ -20,11 +20,14 @@ import org.eclipse.swt.widgets.Composite;
import org.talend.commons.ui.swt.actions.ITreeContextualAction;
import org.talend.components.api.properties.ComponentProperties;
import org.talend.core.IService;
import org.talend.core.model.metadata.Dbms;
import org.talend.core.model.metadata.IMetadataTable;
import org.talend.core.model.metadata.builder.connection.Connection;
import org.talend.core.model.metadata.builder.connection.DatabaseConnection;
import org.talend.core.model.metadata.builder.connection.MetadataTable;
import org.talend.core.model.process.EComponentCategory;
import org.talend.core.model.process.Element;
import org.talend.core.model.process.IElement;
import org.talend.core.model.process.INode;
import org.talend.core.model.properties.Item;
import org.talend.core.model.repository.ERepositoryObjectType;
@@ -146,4 +149,19 @@ public interface IGenericWizardService extends IService {
* @return the default action which will be invoked when double click the node.
*/
public ITreeContextualAction getDefaultAction(RepositoryNode node);
public void loadAdditionalJDBC();
public List<String> getAllAdditionalJDBCTypes();
public boolean getIfAdditionalJDBCDBType(String dbType);
public void initAdditonalJDBCConnectionValue(DatabaseConnection connection, Composite dynamicForm, String dbType,
String propertyId);
public String getDefinitionName4AdditionalJDBC(IElement element);
public String getDatabseNameByNode(IElement node);
public Dbms getDbms4AdditionalJDBC(String typeName);
}

View File

@@ -0,0 +1,16 @@
package org.talend.core.runtime.util;
//============================================================================
//
//Copyright (C) 2006-2019 Talend Inc. - www.talend.com
//
//This source code is available under agreement available at
//%InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
//You should have received a copy of the agreement
//along with this program; if not, write to Talend SA
//9 rue Pages 92150 Suresnes, France
//
//============================================================================
public interface ComponentsLocationProvider {
}

View File

@@ -0,0 +1,17 @@
package org.talend.core.runtime.util;
//============================================================================
//
//Copyright (C) 2006-2019 Talend Inc. - www.talend.com
//
//This source code is available under agreement available at
//%InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
//You should have received a copy of the agreement
//along with this program; if not, write to Talend SA
//9 rue Pages 92150 Suresnes, France
//
//============================================================================
public interface SharedStudioInfoProvider {
public boolean isSupportCurrentMode();
}

View File

@@ -0,0 +1,111 @@
// ============================================================================
//
// Copyright (C) 2006-2019 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.runtime.util;
import java.io.File;
import java.io.IOException;
import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.core.runtime.Path;
import org.eclipse.core.runtime.Platform;
import org.talend.commons.exception.ExceptionHandler;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.model.components.IComponentsFactory;
import org.talend.core.runtime.CoreRuntimePlugin;
import org.talend.core.service.IUpdateService;
import org.talend.utils.io.FilesUtils;
public class SharedStudioUtils {
public static final String FILE_EXTRA_FEATURE_INDEX = "extra_feature.index"; //$NON-NLS-1$
public static final String SIGNATURE_FILE_NAME_SUFFIX = ".sig"; //$NON-NLS-1$
public static final String PROP_DEVMODE = "osgi.dev"; //$NON-NLS-1$
public static boolean updateExtraFeatureFile() {
File userConfigFolder = new File(Platform.getConfigurationLocation().getURL().getPath());
File studioConfigFolder = new File(Platform.getInstallLocation().getURL().getPath(), "configuration");//$NON-NLS-1$
if (!userConfigFolder.getAbsolutePath().equals(studioConfigFolder.getAbsolutePath())) {
File studioExtraFile = new File(studioConfigFolder, FILE_EXTRA_FEATURE_INDEX);
File studioExtraSignFile = new File(studioConfigFolder, FILE_EXTRA_FEATURE_INDEX + SIGNATURE_FILE_NAME_SUFFIX);
File userExtraFile = new File(userConfigFolder, FILE_EXTRA_FEATURE_INDEX);
File userExtraSignFile = new File(userConfigFolder, FILE_EXTRA_FEATURE_INDEX + SIGNATURE_FILE_NAME_SUFFIX);
boolean isNeedUpdate = false;
if (!studioExtraSignFile.exists() && userExtraSignFile.exists()) {
userExtraSignFile.delete();
if (userExtraFile.exists()) {
userExtraFile.delete();
}
return true;
} else if (studioExtraSignFile.exists()) {
isNeedUpdate = true;
}
if (isNeedUpdate) {
try {
FilesUtils.copyFile(studioExtraFile, userExtraFile);
FilesUtils.copyFile(studioExtraSignFile, userExtraSignFile);
} catch (IOException ex) {
ExceptionHandler.process(ex);
}
return true;
}
}
return false;
}
public static boolean isSharedStudioMode() {
if (isDevEnvironment()) {
return false;
}
File configFolder = new File (Platform.getConfigurationLocation().getURL().getFile());
File studioFolder = new File (Platform.getInstallLocation().getURL().getFile());
if (configFolder != null && studioFolder != null && configFolder.getParentFile() != null
&& configFolder.getParentFile().getAbsolutePath().equals(studioFolder.getAbsolutePath())) {
return false;
}
return true;
}
private static boolean isDevEnvironment() {
if (CoreRuntimePlugin.getInstance().getBundle().getBundleContext().getProperty(PROP_DEVMODE) != null) {
return true;
}
return false;
}
public static boolean installedPatch() {
if (GlobalServiceRegister.getDefault().isServiceRegistered(IUpdateService.class)) {
IUpdateService updateService = GlobalServiceRegister.getDefault().getService(IUpdateService.class);
try {
return updateService.syncSharedStudioLibraryInPatch(new NullProgressMonitor());
} catch (Exception e) {
ExceptionHandler.process(e);
}
}
return false;
}
public static File getSharedStudioComponentsParentFolder() {
File configFolder = new File(Platform.getConfigurationLocation().getURL().getFile());
return configFolder;
}
public static File getSharedStudioComponentsExtFolder() {
File componentFolder = SharedStudioUtils.getSharedStudioComponentsParentFolder();
IPath path = new Path(IComponentsFactory.COMPONENTS_INNER_FOLDER);
path = path.append(IComponentsFactory.EXTERNAL_COMPONENTS_INNER_FOLDER);
File extchangeFolder = new File (componentFolder, path.toOSString());
return extchangeFolder;
}
}

View File

@@ -20,5 +20,9 @@ public interface IUpdateService extends IService {
boolean checkComponentNexusUpdate();
void syncComponentM2Jars(IProgressMonitor monitor);
public boolean syncSharedStudioLibraryInPatch(IProgressMonitor monitor) throws Exception;
public String getSharedStudioMissingPatchVersion();
}

View File

@@ -13,6 +13,7 @@
package org.talend.core.ui;
import org.talend.core.IService;
import org.talend.core.runtime.maven.MavenArtifact;
/**
* @author hwang
@@ -21,5 +22,7 @@ import org.talend.core.IService;
public interface IInstalledPatchService extends IService {
public String getLatestInstalledVersion(boolean isBar);
public MavenArtifact getLastIntalledP2Patch();
}

View File

@@ -0,0 +1,60 @@
package org.talend.core.utils;
import java.util.Arrays;
import java.util.List;
import org.eclipse.core.runtime.OperationCanceledException;
import org.eclipse.jface.dialogs.IDialogConstants;
import org.eclipse.jface.dialogs.MessageDialog;
import org.eclipse.swt.SWT;
import org.eclipse.swt.widgets.Display;
import org.talend.commons.CommonsPlugin;
public class DialogUtils {
private static ELoginInfoCase finalCase;
public static void setWarningInfo(ELoginInfoCase warnningInfo) {
finalCase = warnningInfo;
}
public static void syncOpenWarningDialog(String title) {
if (CommonsPlugin.isHeadless() || DialogUtils.finalCase == null) {
return;
}
int dialogType = DialogUtils.finalCase.getDialogType();
String[] contents = DialogUtils.finalCase.getContents();
List<String> asList = Arrays.asList(contents);
StringBuffer sb = new StringBuffer();
asList.forEach(w -> {
sb.append(w);
sb.append("\n");// $NON-NLS-1$
});
int[] selectIndex = new int[1];
Display.getDefault().syncExec(new Runnable() {
@Override
public void run() {
String[] dialogButtonLabels = new String[] { IDialogConstants.OK_LABEL, IDialogConstants.CANCEL_LABEL, };
if (dialogType == MessageDialog.ERROR) {
dialogButtonLabels = new String[] { IDialogConstants.CANCEL_LABEL };
}
int open = MessageDialog.open(dialogType, Display.getDefault().getActiveShell(), title, sb.toString(), SWT.NONE,
dialogButtonLabels);
selectIndex[0] = open;
}
});
DialogUtils.finalCase = null;
if (dialogType == MessageDialog.ERROR) {
throw new OperationCanceledException(""); //$NON-NLS-1$
}
if (1 == selectIndex[0]) {
throw new OperationCanceledException(""); //$NON-NLS-1$
}
}
}

View File

@@ -0,0 +1,40 @@
package org.talend.core.utils;
import org.eclipse.jface.dialogs.MessageDialog;
public enum ELoginInfoCase {
STUDIO_LOWER_THAN_PROJECT(MessageDialog.ERROR),
STUDIO_HIGHER_THAN_PROJECT(MessageDialog.WARNING);
private int dialogType;
private String[] contents;
ELoginInfoCase(int dialogType) {
this.dialogType = dialogType;
}
ELoginInfoCase(int dialogType, String[] contents) {
this.dialogType = dialogType;
this.contents = contents;
}
public int getDialogType() {
return dialogType;
}
public void setDialogType(int dialogType) {
this.dialogType = dialogType;
}
public String[] getContents() {
return contents;
}
public void setContents(String[] contents) {
this.contents = contents;
}
}

View File

@@ -246,6 +246,8 @@ public interface IRunProcessService extends IService {
public boolean isCIMode();
public boolean isExcludeDeletedItems(Property property);
public static IRunProcessService get() {
if (GlobalServiceRegister.getDefault().isServiceRegistered(IRunProcessService.class)) {
return GlobalServiceRegister.getDefault().getService(IRunProcessService.class);

View File

@@ -42,4 +42,9 @@ public abstract class AbstractLoginTask implements ILoginTask {
return gc.getTime();
}
@Override
public boolean isRequiredAlways() {
return false;
}
}

View File

@@ -27,4 +27,10 @@ public interface ILoginTask {
public boolean isCommandlineTask();
public void run(IProgressMonitor monitor) throws InvocationTargetException, InterruptedException;
/**
* Which indicates the task will be executed for each logon of a project, by default return false(execute only once
* at the time of logon studio).
*/
boolean isRequiredAlways();
}

View File

@@ -606,7 +606,8 @@ public final class ProjectManager {
*/
if (!branchSelection.contains(NAME_TAGS) && !branchSelection.contains(NAME_BRANCHES)
&& !branchSelection.contains(NAME_TRUNK) && !branchSelection.contains("master")) { //$NON-NLS-1$
&& !branchSelection.contains(NAME_TRUNK) && !branchSelection.contains(SVNConstant.NAME_MASTER)
&& !branchSelection.contains(SVNConstant.NAME_MAIN)) {
branchSelection = NAME_BRANCHES + branchSelection;
}
return branchSelection;
@@ -649,7 +650,8 @@ public final class ProjectManager {
if (!branchName.startsWith(SVNConstant.NAME_TAGS + SVNConstant.SEP_CHAR)
&& !branchName.startsWith(SVNConstant.NAME_BRANCHES + SVNConstant.SEP_CHAR)
&& !branchName.startsWith(SVNConstant.NAME_ORIGIN + SVNConstant.SEP_CHAR)
&& !branchName.equals(SVNConstant.NAME_TRUNK) && !branchName.equals(SVNConstant.NAME_MASTER)) {
&& !branchName.equals(SVNConstant.NAME_TRUNK) && !branchName.equals(SVNConstant.NAME_MASTER)
&& !branchName.equals(SVNConstant.NAME_MAIN)) {
formatedBranchName = SVNConstant.NAME_BRANCHES + SVNConstant.SEP_CHAR + branchName;
}
return formatedBranchName;

View File

@@ -268,12 +268,20 @@ public interface IProxyRepositoryFactory {
public abstract IRepositoryViewObject getLastVersion(Project project, String id, String relativeFolder,
ERepositoryObjectType type) throws PersistenceException;
public abstract IRepositoryViewObject getLastVersion(String id, String folderPath, ERepositoryObjectType type) throws PersistenceException;
public abstract IRepositoryViewObject getLastVersion(String id, ERepositoryObjectType type) throws PersistenceException;
public abstract IRepositoryViewObject getLastVersion(String id, List<ERepositoryObjectType> types) throws PersistenceException;
public abstract IRepositoryViewObject getLastVersion(Project project, String id) throws PersistenceException;
public abstract IRepositoryViewObject getLastVersion(String id) throws PersistenceException;
public IRepositoryViewObject getLastRefVersion(Project project, String id) throws PersistenceException;
public IRepositoryViewObject getLastRefVersion(Project project, String id, String folderPath, ERepositoryObjectType type) throws PersistenceException;
public abstract IRepositoryViewObject getSpecificVersion(Project project, String id, String version, boolean avoidSaveProject)
throws PersistenceException;

View File

@@ -125,7 +125,16 @@ public interface IRepositoryService extends IService {
public void openProjectSettingDialog(final String pageId);
public List<String> getProjectBranch(Project project) throws JSONException;
/**
*
* get branches of project
*
* @param project
* @param onlyLocalIfPossible try to only get branches from local repository to improve performance
* @return
* @throws JSONException
*/
public List<String> getProjectBranch(Project project, boolean onlyLocalIfPossible) throws JSONException;
public boolean askRetryForNetworkIssue(Throwable ex);

View File

@@ -47,6 +47,8 @@ public class RepositoryConstants {
public static final String JOBLET_DOCUMENTATION_PATH = "documentations/generated/joblets"; //$NON-NLS-1$
public static final String PROJECT_SETTINGS_CUSTOM_URI_MAP = "custom_uri_mapping.json"; //$NON-NLS-1$
public static final String SYSTEM_DIRECTORY = "system"; //$NON-NLS-1$
public static final String USER_DEFINED = "UserDefined"; //$NON-NLS-1$

View File

@@ -1,5 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry exported="true" kind="lib" path="lib/jna-platform.jar"/>
<classpathentry exported="true" kind="lib" path="lib/jna.jar"/>
<classpathentry exported="true" kind="lib" path="lib/oshi-core.jar"/>
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
<classpathentry kind="con" path="org.eclipse.pde.core.requiredPlugins"/>
<classpathentry kind="src" path="src/main/java"/>

View File

@@ -31,7 +31,10 @@ Require-Bundle: org.apache.commons.lang,
org.talend.themes.core,
ca.odell.glazedlists,
org.talend.core,
org.apache.commons.io
org.apache.commons.io,
org.apache.httpcomponents.httpcore,
org.apache.httpcomponents.httpclient,
org.slf4j.api
Import-Package: org.eclipse.jdt.internal.ui.workingsets
Export-Package: org.talend.core.ui,
org.talend.core.ui.actions,
@@ -79,3 +82,7 @@ Bundle-Vendor: .Talend SA.
Bundle-ActivationPolicy: lazy
Bundle-Activator: org.talend.core.ui.CoreUIPlugin
Bundle-Localization: plugin
Bundle-ClassPath: lib/jna-platform.jar,
lib/jna.jar,
lib/oshi-core.jar,
.

View File

@@ -5,4 +5,7 @@ bin.includes = .,\
icons/,\
plugin.properties,\
schema/,\
META-INF/
META-INF/,\
lib/jna-platform.jar,\
lib/jna.jar,\
lib/oshi-core.jar

View File

@@ -24,6 +24,12 @@
id="org.talend.core.runtime.defaultProvider"
name="default">
</provider>
<provider
collector="org.talend.core.ui.token.PerformanceTokenCollector"
description="collect such as hardware info, I/O info, startup time"
id="org.talend.core.ui.token.PerformanceProvider"
name="performance">
</provider>
</extension>
<extension

View File

@@ -9,4 +9,42 @@
</parent>
<artifactId>org.talend.core.ui</artifactId>
<packaging>eclipse-plugin</packaging>
<dependencies>
<dependency>
<groupId>com.github.oshi</groupId>
<artifactId>oshi-core</artifactId>
<version>5.2.5</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.0.2</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
<outputDirectory>${project.basedir}/lib</outputDirectory>
<stripVersion>true</stripVersion>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

View File

@@ -75,9 +75,9 @@ public class ComponentsSettingsHelper {
hiddenComponents.put(component.getPaletteType(), new HashMap<String, Set<String>>());
}
if (!component.isVisibleInComponentDefinition()) {
hiddenComponents.get(component.getPaletteType()).put(component.getName(), new HashSet<String>());
hiddenComponents.get(component.getPaletteType()).put(component.getDisplayName(), new HashSet<String>());
for (String family : component.getOriginalFamilyName().split(ComponentsFactoryProvider.FAMILY_SEPARATOR_REGEX)) {
hiddenComponents.get(component.getPaletteType()).get(component.getName()).add(family);
hiddenComponents.get(component.getPaletteType()).get(component.getDisplayName()).add(family);
}
}
}
@@ -168,8 +168,8 @@ public class ComponentsSettingsHelper {
}
if (hiddenComponents.containsKey(component.getPaletteType())) {
if (hiddenComponents.get(component.getPaletteType()).containsKey(component.getName())) {
if (hiddenComponents.get(component.getPaletteType()).get(component.getName()).contains(family)) {
if (hiddenComponents.get(component.getPaletteType()).containsKey(component.getDisplayName())) {
if (hiddenComponents.get(component.getPaletteType()).get(component.getDisplayName()).contains(family)) {
// not visible if in the hidden component list;
return false;
}
@@ -225,7 +225,7 @@ public class ComponentsSettingsHelper {
EList list = emfProject.getComponentsSettings();
if (!list.isEmpty()) {
list.clear();
IRepositoryService service = (IRepositoryService) GlobalServiceRegister.getDefault().getService(
IRepositoryService service = GlobalServiceRegister.getDefault().getService(
IRepositoryService.class);
IProxyRepositoryFactory prf = service.getProxyRepositoryFactory();

View File

@@ -497,7 +497,11 @@ public class ContextTreeTable {
int max = 0;
String text = "";
for (int i = 0; i < dataLayer.getPreferredRowCount(); i++) {
text = dataLayer.getDataValueByPosition(colPos, i).toString();
Object dataValueByPosition = dataLayer.getDataValueByPosition(colPos, i);
if (dataValueByPosition == null) {
continue;
}
text = dataValueByPosition.toString();
Point size = gc.textExtent(text, SWT.DRAW_MNEMONIC);
int temp = size.x;
if (temp > max) {

View File

@@ -29,6 +29,8 @@ public class ContextAutoResizeTextPainter extends TextPainter {
private boolean changeBackgroundColor = false;
private boolean containsRowName = false;
public ContextAutoResizeTextPainter(boolean wrapText, boolean paintBg, boolean calculate) {
super(wrapText, paintBg, calculate);
}
@@ -43,6 +45,8 @@ public class ContextAutoResizeTextPainter extends TextPainter {
super.setupGCFromConfig(gc, cellStyle);
if (cellStyle.getAttributeValue(CellStyleAttributes.FOREGROUND_COLOR).equals(GUIHelper.COLOR_RED)) {
gc.setForeground(GUIHelper.COLOR_BLACK);
} else if (containsRowName) {
gc.setForeground(GUIHelper.COLOR_RED);
} else if (changeBackgroundColor) {
gc.setForeground(GUIHelper.COLOR_WIDGET_DARK_SHADOW);
}
@@ -52,6 +56,9 @@ public class ContextAutoResizeTextPainter extends TextPainter {
changeBackgroundColor = isChange;
}
public void setContainsRowName(boolean containsRowName) {
this.containsRowName = containsRowName;
}
@Override
protected void setNewMinLength(ILayerCell cell, int contentWidth) {

View File

@@ -12,6 +12,11 @@
// ============================================================================
package org.talend.core.ui.context.nattableTree;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import org.apache.commons.lang.StringUtils;
import org.eclipse.nebula.widgets.nattable.config.IConfigRegistry;
import org.eclipse.nebula.widgets.nattable.data.IDataProvider;
import org.eclipse.nebula.widgets.nattable.extension.glazedlists.GlazedListsDataProvider;
@@ -23,6 +28,7 @@ import org.eclipse.swt.graphics.GC;
import org.eclipse.swt.graphics.Rectangle;
import org.talend.core.ui.context.ContextTreeTable.ContextTreeNode;
import org.talend.core.ui.context.model.ContextTabChildModel;
import org.talend.core.ui.context.model.ContextTabParentModel;
import org.talend.core.ui.context.model.table.ContextTableTabParentModel;
/**
@@ -33,6 +39,8 @@ public class ContextNatTableBackGroudPainter extends BackgroundPainter {
private IDataProvider dataProvider;
private Map<String, String> rowNames = new HashMap<String, String>();
public ContextNatTableBackGroudPainter(ICellPainter painter, IDataProvider dataProvider) {
super(painter);
this.dataProvider = dataProvider;
@@ -53,9 +61,11 @@ public class ContextNatTableBackGroudPainter extends BackgroundPainter {
ContextTabChildModel rowChildModel = (ContextTabChildModel) rowNode.getTreeData();
if (rowChildModel != null) {
((ContextAutoResizeTextPainter) getWrappedPainter()).setChangeBackgroundColor(true);
checkContainsRowName(rowNode, rowChildModel);
}
}
super.paintCell(cell, gc, bounds, configRegistry);
((ContextAutoResizeTextPainter) getWrappedPainter()).setContainsRowName(false);
}
@Override
@@ -63,4 +73,33 @@ public class ContextNatTableBackGroudPainter extends BackgroundPainter {
return super.getBackgroundColour(cell, configRegistry);
}
private void checkContainsRowName(ContextTreeNode rowNode, ContextTabChildModel rowChildModel) {
// Check any duplicate / similar variable from different context to show in red .
String rowName = rowNode.getName();
String parentSourceName = null;
ContextTabParentModel rowTabParentModel = rowChildModel.getParent();
if (rowTabParentModel != null && rowTabParentModel instanceof ContextTableTabParentModel) {
ContextTableTabParentModel rowTableTabParentModel = (ContextTableTabParentModel) rowTabParentModel;
parentSourceName = rowTableTabParentModel.getSourceName();
}
if (StringUtils.isNotBlank(rowName) && StringUtils.isNotBlank(parentSourceName)) {
rowName = rowName.toUpperCase();
parentSourceName = parentSourceName.toUpperCase();
if (rowNames.containsValue(rowName)) {
Iterator<String> iterator = rowNames.keySet().iterator();
while (iterator.hasNext()) {
String key = iterator.next();
String value = rowNames.get(key);
if (rowName.equalsIgnoreCase(value)) {
if (!parentSourceName.equalsIgnoreCase(key)) {
((ContextAutoResizeTextPainter) getWrappedPainter()).setContainsRowName(true);
}
break;
}
}
} else {
rowNames.put(parentSourceName, rowName);
}
}
}
}

View File

@@ -20,7 +20,6 @@ import org.eclipse.gef.commands.Command;
import org.eclipse.jface.dialogs.IDialogConstants;
import org.eclipse.jface.viewers.DialogCellEditor;
import org.eclipse.jface.viewers.TableViewer;
import org.eclipse.jface.window.Window;
import org.eclipse.swt.SWT;
import org.eclipse.swt.events.FocusAdapter;
import org.eclipse.swt.events.FocusEvent;
@@ -37,7 +36,6 @@ import org.talend.core.model.process.IElement;
import org.talend.core.model.process.IElementParameter;
import org.talend.core.model.process.INode;
import org.talend.core.model.process.IProcess2;
import org.talend.core.runtime.services.IGenericDBService;
import org.talend.core.ui.CoreUIPlugin;
import org.talend.core.ui.process.IGEFProcess;
import org.talend.core.ui.services.IDesignerCoreUIService;
@@ -171,7 +169,7 @@ public class ModuleListCellEditor extends DialogCellEditor {
ILibraryManagerUIService.class);
IConfigModuleDialog dialog = libUiService.getConfigModuleDialog(cellEditorWindow.getShell(), "\"newLine\"".equals(value) ? "" : value);
if (dialog.open() == IDialogConstants.OK_ID) {
String selecteModule = dialog.getModuleName();
String selecteModule = dialog.getMavenURI();
if (selecteModule != null && (value == null || !value.equals(selecteModule))) {
setModuleValue(selecteModule, null, null);
return selecteModule;
@@ -192,24 +190,16 @@ public class ModuleListCellEditor extends DialogCellEditor {
}
// enable to refresh component setting after change modules.
IElement element = this.tableParam.getElement();
boolean isNotCConfig = element.getElementParameter("COMPONENT_NAME") == null ?
true : !"cConfig".equals(element.getElementParameter("COMPONENT_NAME").getValue());
if (element != null && isNotCConfig) {
if (element != null) {
IElementParameter updateComponentsParam = element.getElementParameter("UPDATE_COMPONENTS"); //$NON-NLS-1$
if (updateComponentsParam != null) {
updateComponentsParam.setValue(Boolean.TRUE);
}
}
//
executeCommand(new ModelChangeCommand(tableParam, param.getName(), newValue, index));
if (newVal != null) {
executeCommand(new ModelChangeCommand(tableParam, "JAR_PATH", newVal, index));
}
if (nexusVersion != null) {
executeCommand(new ModelChangeCommand(tableParam, "JAR_NEXUS_VERSION", nexusVersion, index));
}
oldValue = newValue;
if (getTableViewer() != null) {
getTableViewer().refresh(true);

View File

@@ -81,7 +81,7 @@ public class DefaultTokenCollector extends AbstractTokenCollector {
jsonObject.put("os.arch", System.getProperty("os.arch"));
jsonObject.put("os.version", System.getProperty("os.version"));
tokenStudioObject.put(OS.getKey(), jsonObject);
final IPreferenceStore preferenceStore = CoreUIPlugin.getDefault().getPreferenceStore();
long syncNb = preferenceStore.getLong(COLLECTOR_SYNC_NB);
tokenStudioObject.put(SYNC_NB.getKey(), syncNb);
@@ -91,6 +91,7 @@ public class DefaultTokenCollector extends AbstractTokenCollector {
} else {
tokenStudioObject.put(STOP_COLLECTOR.getKey(), "0"); //$NON-NLS-1$
}
return tokenStudioObject;
}
}

View File

@@ -0,0 +1,83 @@
// ============================================================================
//
// Copyright (C) 2006-2020 Talend Inc. - www.talend.com
//
// This source code is available under agreement available at
// %InstallDIR%\features\org.talend.rcp.branding.%PRODUCTNAME%\%PRODUCTNAME%license.txt
//
// You should have received a copy of the agreement
// along with this program; if not, write to Talend SA
// 9 rue Pages 92150 Suresnes, France
//
// ============================================================================
package org.talend.core.ui.token;
import java.util.Properties;
import org.talend.commons.exception.CommonExceptionHandler;
import org.talend.commons.utils.time.PerformanceStatisticUtil;
import org.talend.commons.utils.time.PerformanceStatisticUtil.StatisticKeys;
import oshi.SystemInfo;
import oshi.hardware.Baseboard;
import oshi.hardware.CentralProcessor;
import oshi.hardware.CentralProcessor.ProcessorIdentifier;
import oshi.hardware.ComputerSystem;
import oshi.hardware.GlobalMemory;
import oshi.hardware.HardwareAbstractionLayer;
import us.monoid.json.JSONObject;
/**
* DOC sbliu class global comment. Detailled comment
*/
public class PerformanceTokenCollector extends AbstractTokenCollector {
/* (non-Javadoc)
* @see org.talend.core.ui.token.AbstractTokenCollector#collect()
*/
@Override
public JSONObject collect() throws Exception {
checkAndWait();
JSONObject tokenStudioObject = new JSONObject();
//
JSONObject jsonObjectHDInfo = new JSONObject();
SystemInfo si = new SystemInfo();
HardwareAbstractionLayer hal = si.getHardware();
CentralProcessor processor = hal.getProcessor();
ProcessorIdentifier processorIdentifier = processor.getProcessorIdentifier();
ComputerSystem cs = hal.getComputerSystem();//computer system
Baseboard baseboard = cs.getBaseboard();//motherboard
GlobalMemory memory = hal.getMemory();
jsonObjectHDInfo.put("computer vendor", cs.getManufacturer());
jsonObjectHDInfo.put("board vendor", baseboard.getManufacturer());
jsonObjectHDInfo.put("board version", baseboard.getVersion());
jsonObjectHDInfo.put("processor", processorIdentifier.getName());
jsonObjectHDInfo.put("physical memory", Math.ceil((memory.getTotal() /(1024d*1024*1024))) + "GB");
tokenStudioObject.put("hardware", jsonObjectHDInfo);
//
JSONObject jsonObjectIOInfo = new JSONObject();
Properties props = PerformanceStatisticUtil.read(PerformanceStatisticUtil.getRecordingFile(),false);
jsonObjectIOInfo.put(StatisticKeys.STARTUP_AVERAGE.get(), props.getProperty(StatisticKeys.STARTUP_AVERAGE.get()));
jsonObjectIOInfo.put(StatisticKeys.STARTUP_MAX.get(), props.getProperty(StatisticKeys.STARTUP_MAX.get()));
jsonObjectIOInfo.put(StatisticKeys.IO_R_MB_SEC.get(), props.getProperty(StatisticKeys.IO_R_MB_SEC.get()));
jsonObjectIOInfo.put(StatisticKeys.IO_R_AVERAGE_MB_SEC.get(), props.getProperty(StatisticKeys.IO_R_AVERAGE_MB_SEC.get()));
jsonObjectIOInfo.put(StatisticKeys.IO_W_MB_SEC.get(), props.getProperty(StatisticKeys.IO_W_MB_SEC.get()));
jsonObjectIOInfo.put(StatisticKeys.IO_W_AVERAGE_MB_SEC.get(), props.getProperty(StatisticKeys.IO_W_AVERAGE_MB_SEC.get()));
tokenStudioObject.put("performance", jsonObjectIOInfo);
return tokenStudioObject;
}
private void checkAndWait() {
try {
PerformanceStatisticUtil.waitUntilFinish();
} catch (InterruptedException e) {
CommonExceptionHandler.log(e.getMessage());
}
}
}

View File

@@ -14,15 +14,39 @@ package org.talend.core.ui.token;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.lang.reflect.Field;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.PasswordAuthentication;
import java.net.Proxy;
import java.net.Proxy.Type;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.charset.StandardCharsets;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.zip.GZIPOutputStream;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.StatusLine;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.ByteArrayBody;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.apache.log4j.Logger;
import org.apache.log4j.Priority;
import org.eclipse.core.runtime.CoreException;
@@ -36,17 +60,13 @@ import org.eclipse.jface.preference.IPreferenceStore;
import org.eclipse.ui.preferences.ScopedPreferenceStore;
import org.talend.commons.exception.ExceptionHandler;
import org.talend.commons.utils.network.NetworkUtil;
import org.talend.commons.utils.network.TalendProxySelector;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.prefs.ITalendCorePrefConstants;
import org.talend.core.ui.CoreUIPlugin;
import org.talend.core.ui.branding.IBrandingService;
import us.monoid.json.JSONObject;
import us.monoid.web.AbstractContent;
import us.monoid.web.FormData;
import us.monoid.web.Resty;
import us.monoid.web.TextResource;
import us.monoid.web.mime.MultipartContent;
/**
* ggu class global comment. Detailled comment
@@ -214,26 +234,18 @@ public final class TokenCollectorFactory {
@Override
protected IStatus run(IProgressMonitor monitor) {
if (NetworkUtil.isNetworkValid()) {
Authenticator defaultAuth = NetworkUtil.getDefaultAuthenticator();
try {
JSONObject tokenInfors = collectTokenInfors();
Resty r = new Resty();
// set back the rath for Resty.
Field rathField = Resty.class.getDeclaredField("rath"); //$NON-NLS-1$
rathField.setAccessible(true);
Authenticator auth = (Authenticator) rathField.get(null);
Authenticator.setDefault(auth);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gzos = new GZIPOutputStream(baos);
gzos.write(tokenInfors.toString().getBytes());
gzos.close();
AbstractContent ac = Resty.content(baos.toByteArray());
byte[] data = baos.toByteArray();
baos.close();
MultipartContent mpc = Resty.form(new FormData("data", ac)); //$NON-NLS-1$
TextResource result = r.text("https://www.talend.com/TalendRegisterWS/tokenstudio_v2.php", mpc); //$NON-NLS-1$
String resultStr = new JSONObject(result.toString()).getString("result"); //$NON-NLS-1$
String responseString = sendData(data);
String resultStr = new JSONObject(responseString).getString("result"); //$NON-NLS-1$
boolean okReturned = (resultStr != null && resultStr.endsWith("OK")); //$NON-NLS-1$
if (okReturned) {
// set new days
@@ -253,7 +265,6 @@ public final class TokenCollectorFactory {
} catch (Exception e) {
ExceptionHandler.process(e);
} finally {
Authenticator.setDefault(defaultAuth);
}
}
return org.eclipse.core.runtime.Status.OK_STATUS;
@@ -274,4 +285,84 @@ public final class TokenCollectorFactory {
}
}
}
private void addProxy(String url, HttpClientBuilder clientBuilder) throws URISyntaxException {
TalendProxySelector proxySelector = TalendProxySelector.getInstance();
final List<Proxy> proxyList = proxySelector.getDefaultProxySelector().select(new URI(url));
Proxy usedProxy = null;
if (proxyList != null && !proxyList.isEmpty()) {
usedProxy = proxyList.get(0);
}
if (usedProxy != null) {
if (!Type.DIRECT.equals(usedProxy.type())) {
final Proxy finalProxy = usedProxy;
InetSocketAddress address = (InetSocketAddress) finalProxy.address();
String proxyServer = address.getHostString();
int proxyPort = address.getPort();
PasswordAuthentication proxyAuthentication = proxySelector.getHttpPasswordAuthentication();
if (proxyAuthentication != null) {
String proxyUser = proxyAuthentication.getUserName();
if (StringUtils.isNotBlank(proxyUser)) {
String proxyPassword = "";
char[] passwordChars = proxyAuthentication.getPassword();
if (passwordChars != null) {
proxyPassword = new String(passwordChars);
}
BasicCredentialsProvider credProvider = new BasicCredentialsProvider();
credProvider.setCredentials(new AuthScope(proxyServer, proxyPort),
new UsernamePasswordCredentials(proxyUser, proxyPassword));
clientBuilder.setDefaultCredentialsProvider(credProvider);
}
}
HttpHost proxyHost = new HttpHost(proxyServer, proxyPort);
clientBuilder.setProxy(proxyHost);
}
}
}
private String sendData(byte[] data) throws Exception {
CloseableHttpClient client = null;
CloseableHttpResponse response = null;
try {
final String url = "https://www.talend.com/TalendRegisterWS/tokenstudio_v2.php";
HttpClientBuilder clientBuilder = HttpClients.custom();
clientBuilder.disableCookieManagement();
addProxy(url, clientBuilder);
client = clientBuilder.build();
HttpPost httpPost = new HttpPost(url);
httpPost.setConfig(RequestConfig.DEFAULT);
MultipartEntityBuilder dataBuilder = MultipartEntityBuilder.create();
dataBuilder.addPart("data", new ByteArrayBody(data, null));
HttpEntity reqEntity = dataBuilder.build();
httpPost.setEntity(reqEntity);
response = client.execute(httpPost, HttpClientContext.create());
StatusLine statusLine = response.getStatusLine();
String responseStr = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
if (HttpURLConnection.HTTP_OK != statusLine.getStatusCode()) {
throw new Exception(statusLine.toString() + ", server message: [" + responseStr + "]");
}
return responseStr;
} finally {
if (response != null) {
try {
response.close();
} catch (Throwable e) {
ExceptionHandler.process(e);
}
}
if (client != null) {
try {
client.close();
} catch (Throwable e) {
ExceptionHandler.process(e);
}
}
}
}
}

View File

@@ -91,6 +91,10 @@ public class PluginUtil {
return "org.talend.camel.testcontainer.ui.editor.CamelTestContainerMultiPageEditor".equals(getActiveEditorId()); //$NON-NLS-1$
}
public static boolean isRouteletEditor() {
return "org.talend.repository.routelets.editor.RouteletMultiPageTalendEditor".equals(getActiveEditorId()); //$NON-NLS-1$
}
/**
* DOC yyan Get active editor ID.
*

View File

@@ -23,9 +23,11 @@ import org.eclipse.emf.common.util.URI;
import org.osgi.service.prefs.BackingStoreException;
import org.talend.commons.exception.PersistenceException;
import org.talend.commons.exception.SystemException;
import org.talend.core.GlobalServiceRegister;
import org.talend.core.IService;
import org.talend.core.model.general.Project;
import org.talend.core.model.process.INode;
import org.talend.core.model.properties.Property;
import org.talend.core.runtime.maven.MavenArtifact;
/**
@@ -78,4 +80,18 @@ public interface ICoreTisService extends IService {
Set<String> getComponentBlackList();
public void afterImport (Property property) throws PersistenceException;
boolean hasNewPatchInPatchesFolder();
boolean isDefaultLicenseAndProjectType();
void refreshPatchesFolderCache();
static ICoreTisService get() {
if (GlobalServiceRegister.getDefault().isServiceRegistered(ICoreTisService.class)) {
return GlobalServiceRegister.getDefault().getService(ICoreTisService.class);
}
return null;
}
}

View File

@@ -34,7 +34,14 @@ public interface IGITProviderService extends IService {
public String getCurrentGITRevision(Object process);
public String[] getBranchList(Project project);
/**
* get project branches
*
* @param project
* @param onlyLocalIfPossible try to only get branches from local repository to improve performance
* @return
*/
public String[] getBranchList(Project project, boolean onlyLocalIfPossible);
public boolean isGITProject(Project p) throws PersistenceException;
@@ -48,6 +55,8 @@ public interface IGITProviderService extends IService {
void createOrUpdateGitIgnoreFile(IProject eclipseProject) throws CoreException;
String getDefaultBranch(Project project);
public static IGITProviderService get() {
GlobalServiceRegister register = GlobalServiceRegister.getDefault();
if (!register.isServiceRegistered(IGITProviderService.class)) {

View File

@@ -1605,6 +1605,10 @@ public class ProcessorUtilities {
List<ProcessItem> testsItems =
testContainerService.getTestContainersByVersion(jobInfo.getProcessItem());
for (ProcessItem testItem : testsItems) {
if (testItem.getProperty().getItem().getState().isDeleted()
&& IRunProcessService.get().isExcludeDeletedItems(testItem.getProperty())) {
continue;
}
JobInfo subJobInfo = new JobInfo(testItem, testItem.getProcess().getDefaultContext());
subJobInfo.setTestContainer(true);
subJobInfo.setFatherJobInfo(jobInfo);

View File

@@ -56,8 +56,8 @@
<version>17</version>
<type>pom</type>
</dependency>
<!-- It fixes provided dependency scope to compile, because maven-dependency-plugin is setup to include
only compile and runtime scopes, but provided is also required. It's not possible to setup plugin to
<!-- It fixes provided dependency scope to compile, because maven-dependency-plugin is setup to include
only compile and runtime scopes, but provided is also required. It's not possible to setup plugin to
include all: compile, runtime and provided dependencies -->
<dependency>
<groupId>biz.aQute.bnd</groupId>
@@ -161,7 +161,7 @@
<artifactId>components-api</artifactId>
<version>${components.version}</version>
</dependency>
<!-- Dependencies in provided scope should be explicitly added as dependency in this module,
<!-- Dependencies in provided scope should be explicitly added as dependency in this module,
because provided scope is not transitive -->
<dependency>
<groupId>org.osgi</groupId>
@@ -199,6 +199,11 @@
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.14-TALEND</version>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.4</version>
</dependency>
</dependencies>
<build>
<plugins>

View File

@@ -10,7 +10,7 @@
<artifactId>studio-tacokit-dependencies</artifactId>
<packaging>pom</packaging>
<properties>
<tacokit.components.version>1.9.0</tacokit.components.version>
<tacokit.components.version>1.15.0</tacokit.components.version>
</properties>
<repositories>
<repository>

View File

@@ -11,7 +11,7 @@
<packaging>pom</packaging>
<properties>
<tcomp.version>1.1.15</tcomp.version>
<tcomp.version>1.1.25</tcomp.version>
<slf4j.version>1.7.25</slf4j.version>
</properties>
@@ -106,6 +106,11 @@
<artifactId>slf4j-simple</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.14</version>
</dependency>
</dependencies>
<build>

Some files were not shown because too many files have changed in this diff Show More