Compare commits

..

100 Commits

Author SHA1 Message Date
Sanat Nayar
fcbf22bcd5 test 2020-02-28 11:07:51 -05:00
Sanat Nayar
b4daf52ef5 readme config change 2020-02-20 14:46:49 -05:00
Sanat Nayar
a7e757e15f added readme for config 2020-02-20 14:30:55 -05:00
Sanat Nayar
c47aabc066 added readme for config 2020-02-20 14:28:20 -05:00
Sanat Nayar
78422af050 added readme for config 2020-02-20 14:23:05 -05:00
Boris Kuschel
cb515f216d Merge pull request #121 from qlik-oss/windows_ansi
Add ansi translator for windows
2020-02-20 08:46:21 -05:00
Boris Kuschel
f6eacefd82 Add ansi translator for windows
Signed-off-by: Boris Kuschel <boris.kuschel@qlik.com>
2020-02-20 07:40:27 -05:00
Sanat Nayar
2dd37ab985 Merge pull request #118 from qlik-oss/context_list
QlikSense config list-contexts
2020-02-19 14:58:50 -05:00
Sanat Nayar
8142bb5fa9 added formatting changes 2020-02-19 11:48:50 -05:00
Sanat Nayar
22ea225b8c added configs sub-command 2020-02-19 09:56:50 -05:00
Andriy Bulynko
2f854fd6e4 Fixing about command output 2020-02-19 09:38:07 -05:00
Andriy Bulynko
47bcc016fc About from current context (#117) 2020-02-19 09:28:21 -05:00
Ashwathi Shiva
4e2083309e Encrypt secret in the case: secret=false (#114)
qliksense config set-secrets <key>=<value> --secrets=false
2020-02-18 16:16:48 -05:00
Sanat Nayar
017aa63726 Merge pull request #98 from qlik-oss/upgrade_commmand
QlikSense Upgrade Command
2020-02-18 11:14:05 -05:00
Sanat Nayar
f4275a47ad added upgrade func 2020-02-14 15:39:06 -05:00
Sanat Nayar
75e4e43f9b added upgrade func 2020-02-14 15:14:48 -05:00
Sanat Nayar
21cbc0b44d added upgrade func 2020-02-14 14:22:54 -05:00
Sanat Nayar
003f7f31fc added upgrade func 2020-02-14 14:22:54 -05:00
Sanat Nayar
c3b8837402 added upgrade func 2020-02-14 14:22:54 -05:00
Sanat Nayar
838ed3069c init 2020-02-14 14:22:54 -05:00
Andriy Bulynko
5668da13a7 Creating uniquely named branches (#94) 2020-02-14 14:22:54 -05:00
Andriy Bulynko
2ed9bcb7bf Placing ejson ENV var mangling behind a check (#93) 2020-02-14 14:22:54 -05:00
Ashwathi Shiva
505bb51f95 Added tests (#90)
* Tests added
2020-02-14 14:22:54 -05:00
Andriy Bulynko
60feff3292 Faster-running kuz test (#89) 2020-02-14 14:22:54 -05:00
Ashwathi Shiva
af1afbef8f Support configuration through multiple args (#81)
* support configuration with multiple args
2020-02-14 14:22:54 -05:00
Boris Kuschel
3c4ada848a Make file paths portable
Signed-off-by: Boris Kuschel <boris.kuschel@qlik.com>
2020-02-14 14:22:54 -05:00
Boris Kuschel
6da6415c44 Make makefile portable:
Signed-off-by: Boris Kuschel <boris.kuschel@qlik.com>
2020-02-14 14:22:54 -05:00
Andriy Bulynko
a3287fc1a9 Storing/referencing ejson keys at path: ${qliksenseHome}/ejson/keys (#77) 2020-02-14 14:22:54 -05:00
Foysal Iqbal
39607652a8 Fix default install (#79) 2020-02-14 14:22:54 -05:00
Ashwathi Shiva
6768f74d40 Add support for rotateKeys and included releaseName as part of CRSpec (#78)
* added support for rotateKeys, and included releaseName as part of CRSpec
2020-02-14 14:22:54 -05:00
Ashwathi Shiva
e159e8bd90 Imperative config through cli (#75)
* fixed a regexp bug and another one where qliksense-context was set as default context every time
* modified file creation permissions
2020-02-14 14:22:54 -05:00
Ashwathi Shiva
65bf3fb185 Imperative config through cli (#70)
* minor fix
2020-02-14 14:22:54 -05:00
Ashwathi Shiva
1f245546cd Imperative config through cli (#69)
* removed unnecessary check
2020-02-14 14:22:54 -05:00
Ashwathi Shiva
e38b66f039 Imperative config through cli (#66)
qliksense config commands implemented
2020-02-14 14:22:54 -05:00
Foysal Iqbal
4fd7f2ecbf Install flags (#68) 2020-02-14 14:22:54 -05:00
Andriy Bulynko
b07995dfb1 Setting kubeConfigPath based on homedir.Dir() for consistency (#65) 2020-02-14 14:22:54 -05:00
Foysal Iqbal
caf318410d Fetch install command (#62) 2020-02-14 14:22:54 -05:00
Andriy Bulynko
78fde72c92 Setting kubeConfigPath based on os.UserHomeDir() (#64) 2020-02-14 14:22:54 -05:00
Andriy Bulynko
4b0543b7b0 upgrading k-apis to v0.0.2 2020-02-14 14:22:54 -05:00
Andriy Bulynko
3ff45e47d7 Using k-apis and underlying go-git for the about command (#56) 2020-02-14 14:22:54 -05:00
Foysal Iqbal
72f7a450cf Config view apply (#55) 2020-02-14 14:22:54 -05:00
Foysal Iqbal
0371fa0d9b Clean porter (#53) 2020-02-14 14:22:54 -05:00
Andriy Bulynko
baf394160f About command (#50) 2020-02-14 14:22:54 -05:00
Foysal Iqbal
e411219da8 bring static files into cli (#49) 2020-02-14 14:22:54 -05:00
Andriy Bulynko
a1cb7eda9f kustomize API in-process (#48) 2020-02-14 14:22:54 -05:00
Boris Kuschel
240b9242fa Pull defaults for no version and no directory 2020-02-14 14:22:54 -05:00
Foysal Iqbal
314ff5a14d About doc (#46) 2020-02-14 14:22:54 -05:00
Boris Kuschel
766a2babc7 Make pull/push Docker engineless (#42) 2020-02-14 14:22:54 -05:00
Sanat Nayar
3c1709dcb5 added upgrade func 2020-02-14 14:22:37 -05:00
Sanat Nayar
48e8c997e4 added upgrade func 2020-02-14 11:11:21 -05:00
Jacob Martin
a9b5599d35 update release to include tars (#92)
* update release to include tars
2020-02-14 06:59:32 -05:00
Sanat Nayar
b092356fba added upgrade func 2020-02-13 14:49:24 -05:00
Foysal Iqbal
488f162dff uninstall command (#97)
Signed-off-by: Foysal Iqbal <mqb@qlik.com>
2020-02-13 14:06:06 -05:00
Sanat Nayar
a29e7acf70 init 2020-02-12 17:03:34 -05:00
Andriy Bulynko
4a6e49f393 Creating uniquely named branches (#94) 2020-02-11 16:32:56 -05:00
Andriy Bulynko
50b2712456 Placing ejson ENV var mangling behind a check (#93) 2020-02-11 15:09:49 -05:00
Ashwathi Shiva
477f049c3e Added tests (#90)
* Tests added
2020-02-11 14:46:26 -05:00
Andriy Bulynko
b2ce12bd62 Faster-running kuz test (#89) 2020-02-10 14:07:07 -05:00
Ashwathi Shiva
ee4352e9d6 Support configuration through multiple args (#81)
* support configuration with multiple args
2020-02-10 09:27:44 -05:00
Boris Kuschel
11822db2cb Merge pull request #82 from qlik-oss/fix_Makefile
Make makefile portable
2020-02-08 08:16:30 -05:00
Boris Kuschel
58b027f361 Merge pull request #83 from qlik-oss/portable_paths
Make file paths portable
2020-02-08 08:16:03 -05:00
Boris Kuschel
25fb2c2407 Make file paths portable
Signed-off-by: Boris Kuschel <boris.kuschel@qlik.com>
2020-02-08 08:13:35 -05:00
Boris Kuschel
05b90314e4 Make makefile portable:
Signed-off-by: Boris Kuschel <boris.kuschel@qlik.com>
2020-02-08 07:40:29 -05:00
Andriy Bulynko
5825ba127b Storing/referencing ejson keys at path: ${qliksenseHome}/ejson/keys (#77) 2020-02-07 21:19:28 -05:00
Foysal Iqbal
156f21fab2 Fix default install (#79) 2020-02-07 16:05:11 -05:00
Ashwathi Shiva
835235a109 Add support for rotateKeys and included releaseName as part of CRSpec (#78)
* added support for rotateKeys, and included releaseName as part of CRSpec
2020-02-07 15:05:11 -05:00
Ashwathi Shiva
53127d00d8 Imperative config through cli (#75)
* fixed a regexp bug and another one where qliksense-context was set as default context every time
* modified file creation permissions
2020-02-07 11:17:23 -05:00
Ashwathi Shiva
4415c8e02b Imperative config through cli (#70)
* minor fix
2020-02-06 14:56:20 -05:00
Ashwathi Shiva
c70e123878 Imperative config through cli (#69)
* removed unnecessary check
2020-02-06 13:41:27 -05:00
Ashwathi Shiva
3595d70b7c Imperative config through cli (#66)
qliksense config commands implemented
2020-02-06 00:39:19 -05:00
Foysal Iqbal
cb2001996c Install flags (#68) 2020-02-05 11:59:48 -05:00
Andriy Bulynko
867106afd3 Setting kubeConfigPath based on homedir.Dir() for consistency (#65) 2020-02-04 17:13:16 -05:00
Foysal Iqbal
6c345c9164 Fetch install command (#62) 2020-02-04 16:50:13 -05:00
Andriy Bulynko
ee0a670018 Setting kubeConfigPath based on os.UserHomeDir() (#64) 2020-02-04 15:26:53 -05:00
Andriy Bulynko
644498ddb8 upgrading k-apis to v0.0.2 2020-02-04 13:39:08 -05:00
Andriy Bulynko
bcf2b1ab4b Using k-apis and underlying go-git for the about command (#56) 2020-02-04 12:16:58 -05:00
Foysal Iqbal
0545fd7d16 Config view apply (#55) 2020-02-04 10:03:09 -05:00
Foysal Iqbal
ea240ce3f1 Clean porter (#53) 2020-02-03 12:01:38 -05:00
Andriy Bulynko
d6a16cea8b About command (#50) 2020-02-03 10:07:25 -05:00
Foysal Iqbal
ee557c2068 bring static files into cli (#49) 2020-01-30 15:37:17 -05:00
Andriy Bulynko
fb14d30328 kustomize API in-process (#48) 2020-01-30 13:05:45 -05:00
Boris Kuschel
ca83942fbe Pull defaults for no version and no directory 2020-01-30 06:48:16 -05:00
Foysal Iqbal
7f68dad586 About doc (#46) 2020-01-29 13:37:42 -05:00
Boris Kuschel
fdc2877174 Make pull/push Docker engineless (#42) 2020-01-28 18:05:49 -05:00
Ashwathi Shiva
b9b7068689 Auto fetch invoc image deps (#36)
changed minimum qliksense mixin version and removed old porter config
2020-01-22 13:59:59 -05:00
Ashwathi Shiva
cada3690e1 Auto fetch invocation image deps (#23)
Check minimum versions of CLI, porter, and qliksense mixin. fixes #7.
2020-01-22 11:43:02 -05:00
Boris Kuschel
947486d347 Update TOC 2020-01-21 12:47:27 -05:00
Boris Kuschel
793f6e9f36 REmove multiline markers from script 2020-01-21 12:45:49 -05:00
Boris Kuschel
a569bc1ddd Fix scripts 2020-01-21 12:42:48 -05:00
Boris Kuschel
f5165fbeea Add Pull instructions 2020-01-21 12:31:29 -05:00
Boris Kuschel
4437b31592 Re-arrange porter instructions 2020-01-21 12:28:12 -05:00
Boris Kuschel
a943efe5df Bad Anchor (2nd try) 2020-01-21 12:26:59 -05:00
Boris Kuschel
70aea58d3a Bad anchor 2020-01-21 12:25:05 -05:00
Boris Kuschel
f5dadd522a Make porter CLI optional 2020-01-21 12:23:32 -05:00
Boris Kuschel
deb103c592 Fix credential name in readme 2020-01-21 12:16:58 -05:00
Boris Kuschel
dd25d07fcb Update README.md 2020-01-21 12:14:28 -05:00
Boris Kuschel
0328607a77 Fix readme indentation 2020-01-21 12:13:49 -05:00
Boris Kuschel
550aea24d6 Add PowerShell command to cred alternative 2020-01-21 12:09:20 -05:00
Foysal Iqbal
edca43ca4f fix doc (#27) 2020-01-20 10:53:34 -05:00
Boris Kuschel
513125a9ca Complete README instructions (#26)
* Fixup README

Signed-off-by: Boris Kuschel <boris.kuschel@qlik.com>
2020-01-20 08:38:14 -05:00
49 changed files with 4768 additions and 2452 deletions

View File

@@ -9,6 +9,7 @@ jobs:
working_directory: /go/src/github.com/qlik-oss/sense-installer
steps:
- checkout
- run: make test
- run: make build
build_release:
docker:
@@ -16,45 +17,27 @@ jobs:
working_directory: /go/src/github.com/qlik-oss/sense-installer
steps:
- checkout
- run: make test
- run: make xbuild-all
- run:
name: "Build latest master from porter repo"
command: |
export GO111MODULE=off
go get -u get.porter.sh/porter || true
cd /go/src/get.porter.sh/porter
# store porter master commit
git rev-parse HEAD > /go/porter-master-commit.txt
make xbuild-all VERSION=latest
cp -r bin/latest/* /go/src/github.com/${CIRCLE_PROJECT_USERNAME}/${CIRCLE_PROJECT_REPONAME}/bin/${CIRCLE_TAG}/
- run:
name: "Publish Release on GitHub"
command: |
go get github.com/tcnksm/ghr
# VERSION=v$(./artifacts/qliksense-linux-amd64 version | sed -nre 's/^[^0-9]*(([0-9]+\.)*[0-9]+).*/\1/p')
PORTER_REPO_COMMIT=$(cat /go/porter-master-commit.txt)
ghr -t ${GITHUB_TOKEN} -u ${CIRCLE_PROJECT_USERNAME} -r ${CIRCLE_PROJECT_REPONAME} -c ${CIRCLE_SHA1} -b "porter build based on commit: https://github.com/deislabs/porter/commit/${PORTER_REPO_COMMIT}" -delete ${CIRCLE_TAG} /go/src/github.com/${CIRCLE_PROJECT_USERNAME}/${CIRCLE_PROJECT_REPONAME}/bin/${CIRCLE_TAG}/
ghr -t ${GITHUB_TOKEN} -u ${CIRCLE_PROJECT_USERNAME} -r ${CIRCLE_PROJECT_REPONAME} -c ${CIRCLE_SHA1} -delete ${CIRCLE_TAG} /go/src/github.com/${CIRCLE_PROJECT_USERNAME}/${CIRCLE_PROJECT_REPONAME}/bin/${CIRCLE_TAG}/
workflows:
version: 2
commit:
jobs:
- build:
filters:
branches:
only: master
tags:
ignore: /^v.*/
build_release:
jobs:
- build:
- build_release:
filters:
tags:
only: /^v.*/
branches:
ignore: /.*/
tags:
only: /v.*/
- build_release:
requires:
- build
filters:
branches:
ignore: /.*/
tags:
only: /v.*/

4
.gitignore vendored
View File

@@ -1,2 +1,6 @@
bin
.vscode
cmd/qliksense/__debug_bin
pkg/qliksense/crds
pkg/qliksense/packrd
pkg/qliksense/qliksense-packr.go

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -4,13 +4,15 @@ PKG = github.com/qlik-oss/sense-installer
MAKE_OPTS ?= --no-print-directory
LDFLAGS = -w -X $(PKG)/pkg.Version=$(VERSION) -X $(PKG)/pkg.Commit=$(COMMIT) -X "$(PKG)/pkg.CommitDate=$(COMMIT_DATE)"
XBUILD = CGO_ENABLED=0 go build -a -tags netgo -ldflags '$(LDFLAGS)'
XBUILD = CGO_ENABLED=0 go build -a -tags "$(BUILDTAGS)" -ldflags '$(LDFLAGS)'
BINDIR = bin
COMMIT ?= $(shell git rev-parse --short HEAD)
COMMIT_DATE ?= $(shell git show --no-patch --no-notes --pretty='%cd' $(COMMIT) --date=iso)
VERSION ?= $(shell git describe --tags 2> /dev/null || echo v0)
PERMALINK ?= $(shell git describe --tags --exact-match &> /dev/null && echo latest || echo canary)
BUILDTAGS = netgo containers_image_ostree_stub exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp
CLIENT_PLATFORM ?= $(shell go env GOOS)
CLIENT_ARCH ?= $(shell go env GOARCH)
@@ -31,18 +33,51 @@ FILE_EXT=
endif
.PHONY: build
build:
build: clean generate
mkdir -p $(BINDIR)
go build -ldflags '$(LDFLAGS)' -o $(BINDIR)/$(MIXIN)$(FILE_EXT) ./cmd/$(MIXIN)
go build -ldflags '$(LDFLAGS)' -tags "$(BUILDTAGS)" -o $(BINDIR)/$(MIXIN)$(FILE_EXT) ./cmd/$(MIXIN)
$(MAKE) clean
xbuild-all:
.PHONY: test
test:
go test -short -count=1 -tags "$(BUILDTAGS)" -v ./...
xbuild-all: clean generate
$(foreach OS, $(SUPPORTED_PLATFORMS), \
$(foreach ARCH, $(SUPPORTED_ARCHES), \
$(MAKE) $(MAKE_OPTS) CLIENT_PLATFORM=$(OS) CLIENT_ARCH=$(ARCH) MIXIN=$(MIXIN) xbuild; \
))
$(MAKE) clean
xbuild: $(BINDIR)/$(VERSION)/$(MIXIN)-$(CLIENT_PLATFORM)-$(CLIENT_ARCH)$(FILE_EXT)
$(BINDIR)/$(VERSION)/$(MIXIN)-$(CLIENT_PLATFORM)-$(CLIENT_ARCH)$(FILE_EXT):
mkdir -p $(dir $@)
GOOS=$(CLIENT_PLATFORM) GOARCH=$(CLIENT_ARCH) $(XBUILD) -o $@ ./cmd/$(MIXIN)
tar -czvf $(BINDIR)/$(VERSION)/$(MIXIN)-$(CLIENT_PLATFORM)-$(CLIENT_ARCH).tar.gz -C $(BINDIR)/$(VERSION)/ $(MIXIN)-$(CLIENT_PLATFORM)-$(CLIENT_ARCH)$(FILE_EXT)
#tar -C $(BINDIR)/$(VERSION)/ -cvf $(BINDIR)/$(VERSION)/$(MIXIN)-$(CLIENT_PLATFORM)-$(CLIENT_ARCH).tar.gz $(MIXIN)-$(CLIENT_PLATFORM)-$(CLIENT_ARCH)$(FILE_EXT)
generate: get-crds packr2
go generate ./...
HAS_PACKR2 := $(shell packr2)
packr2:
ifndef HAS_PACKR2
go get -u github.com/gobuffalo/packr/v2/packr2
endif
clean: clean-packr
-rm -rf /tmp/operator
-rm -fr pkg/qliksense/crds
clean-packr: packr2
cd pkg/qliksense && packr2 clean
get-crds:
$(eval TMP := $(shell mktemp -d))
git clone git@github.com:qlik-oss/qliksense-operator.git -b ms-3 $(TMP)/operator
mkdir -p pkg/qliksense/crds/cr
mkdir -p pkg/qliksense/crds/crd
mkdir -p pkg/qliksense/crds/crd-deploy
cp $(TMP)/operator/deploy/*.yaml pkg/qliksense/crds/crd-deploy
cp $(TMP)/operator/deploy/crds/*_crd.yaml pkg/qliksense/crds/crd
cp $(TMP)/operator/deploy/crds/*_cr.yaml pkg/qliksense/crds/cr

View File

@@ -6,6 +6,7 @@
- [Getting Started](#getting-started)
- [Requirements](#requirements)
- [Download](#download)
- [Porter CLI](#porter-cli)
- [Generate Credentials from published bundle](#generate-credentials-from-published-bundle)
- [Qlik Sense version and image list](#qliksense-version-and-image-list)
- [Optional: Pulling images in manifest locally, "air gap"](#optional-pulling-images-in-manifest-locally-%22air-gap%22)
@@ -17,11 +18,11 @@
## About
The Qlik Sense installer CLI (sense-installer) provides an imperitive interface to many of the configurations that need to be applied against the declaritive structure described in (https://github.com/qlik-oss/qliksense-k8s)[https://github.com/qlik-oss/qliksense-k8s].
The Qlik Sense installer CLI (sense-installer) provides an imperitive interface to many of the configurations that need to be applied against the declaritive structure described in [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s).
This is a technology preview that uses (porter)[https://porter.sh] to execute "actions" (operations) and bundle versions of the (qliksense-k8s)[https://github.com/qlik-oss/qliksense-k8s] repository.
This is a technology preview that uses [porter](https://porter.sh) to execute "actions" (operations) and bundle versions of the [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s) repository.
These bundles are posted to (docker hub)[https://hub.docker.com/] at the following location: (https://hub.docker.com/r/qlik/qliksense-cnab-bundle)[https://hub.docker.com/r/qlik/qliksense-cnab-bundle/tags].
These bundles are posted to [docker hub](https://hub.docker.com/) at the following location: [qliksense-cnab-bundle](https://hub.docker.com/r/qlik/qliksense-cnab-bundle/tags).
For each version of a qliksense sense edge build there should be a corresponding release current posted on docker hub. ex. `qlik/qliksense-cnab-bundle:v1.21.23-edge` for `v1.21.23` edge release of qliksense. The latest version posted will also be labelled as `latest`
@@ -36,51 +37,83 @@ For each version of a qliksense sense edge build there should be a corresponding
### Requirements
- Docker Client connected to a docker engine into which images can built, pulled and pushed.
- Docker Client connected to a docker engine into which images can built, pulled and pushed.
- (Docker Desktop setup tested for these instructions)
### Download
- Download the appropriate executable for your platform from the [releases page](https://github.com/qlik-oss/sense-installer/releases).
- Two environment variables will need to be set so as not to conflict with an existing porter installation:
- Download the appropriate executable for your platform from the [releases page](https://github.com/qlik-oss/sense-installer/releases).
- To allow the CLI to download and initialize dependencies (including porter and it's associated mixins), simply execute `qliksense` with no arguments
- `qliksense`
#### Porter CLI
- *Optional*: If wanting to use porter CLI directly, two environment variables will need to be set so as not to conflict with an existing porter installation:
- _Bash_
```shell
bash# export PORTER_HOME="$HOME\.qliksense"
bash# export PATH="$HOME\.qliksense;$PATH"
```
- _PowerShell_
```shell
PS> $Env:PORTER_HOME="$Env:USERPROFILE\.qliksense"
PS> $Env:PATH="$Env:USERPROFILE\.qliksense;$Env:PATH"
```
- To allow the CLI to download and initialize dependencies (including porter and it's associated mixins), simply execute `qliksense` with no arguments
- `qliksense`
### Generate Credentials from published bundle
- Ensure connectivity to the target cluster create a kubeconfig credential for a target bundle run ex.
- `porter cred generate <credential_name> --tag qlik/qliksense-cnab-bundle:v1.21.23-edge`
, replace `<credential_name>` with a name of your choosing.
- Select `file path` and specify full path to a kube config file ex. _Bash_:
- `/home/user/.kube/config` or _PowerShell_ `C:\Users\user\.kube\config `
- Ensure connectivity to the target cluster create a kubeconfig credential for a target bundle.
- generating a file as follows, replace `<credential_name>` with a name of your choosing.
- _Bash_
```shell
bash# CREDENTIAL_NAME=<credential_name>
bash# cat <<EOF > $HOME/.qliksense/credentials/$CREDENTIAL_NAME.yaml
name: $CREDENTIAL_NAME
credentials:
- name: kubeconfig
source:
path: $HOME/.kube/config
EOF
```
- _PowerShell_
```shell
PS> $CREDENTIAL_NAME="<credential_name>"
PS> Add-Content -Value @"
name: $CREDENTIAL_NAME
credentials:
- name: kubeconfig
source:
path: $Env:USERPROFILE\.kube\config
"@ -Path $Env:USERPROFILE\.qliksense\credentials\$CREDENTIAL_NAME".yaml"
```
- credentials can also be created using the [porter](https://porter.sh) CLI *(the correct environmental variable need to have been set up as shown in [Porter CLI](#porter-cli) above)*
- `porter cred generate <credential_name> --tag qlik/qliksense-cnab-bundle:v1.21.23-edge`, replace `<credential_name>` with a name of your choosing.
- Select `file path` and specify full path to a kube config file ex. _Bash_: `/home/user/.kube/config` or _PowerShell_ `C:\Users\user\.kube\config`
### Qlik Sense version and image list
It is possible verify the version of the (qliksense-k8s)[https://github.com/qlik-oss/qliksense-k8s] repository bundled into the `qlik/qliksense-cnab-bundle` image and retreive the list of images included in that release. (This operation can take a minute or so)<https://github.com/qlik-oss/kustomize/issues/13> as the entire manifests needs to be rendered:
It is possible verify the version of the [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s) repository bundled into the `qlik/qliksense-cnab-bundle` image and retreive the list of images included in that release. (This operation can take a minute or so)<https://github.com/qlik-oss/kustomize/issues/13> as the entire manifests needs to be rendered:
- `qliksense about --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
### Optional: Pulling images in manifest locally, "air gap"
If the `dockerRegistry` parameter is specified as the private docker registry to be used by the kubernetes cluster hosting qliksense, it is possible to pull images to the local docker engine for an eventual push during a `qliksense install` or `qliksense upgrade`
- `qliksense pull --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
### Running Preflight checks
You can run preflight checks to ensure that the cluster is in a healthy state before installing Qliksense.
You can run preflight checks to ensure that the cluster is in a healthy state before installing Qliksense.
- `qliksense preflight -c <credential_name> --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
The above command runs the checks in the default namespace. If you want to specify the namespace to run preflight checks on:
- `qliksense preflight --param namespace=<value> -c <credential_name> --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
### Installation
- Install the bundle : `qliksense install --param acceptEULA=yes -c <credential_name> --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
#### Supported Parameters during install
@@ -107,17 +140,27 @@ idpConfigs=[{"discoveryUrl":"http://keycloak-insecure:8089/keycloak/realms/maste
Then pass that file during install command like this
```shell
porter install --param acceptEULA=yes -c <credential_name> --param-file idpconfigs.txt --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
qliksense install --param acceptEULA=yes -c <credential_name> --param-file idpconfigs.txt --tag qlik/qliksense-cnab-bundle:<qliksense_version>`
```
## Packaging a Custom bundle
## Packaging a Custom bundle
If files need to be added to the (qliksense-k8s repository)[https://github.com/qlik-oss/qliksense-k8s] in order to perform advanced configuration outside the scope of the what the operator provides, a custom bundle needs to be built.
If files need to be added to the [qliksense-k8s repository](https://github.com/qlik-oss/qliksense-k8s) in order to perform advanced configuration outside the scope of the what the operator provides, a custom bundle needs to be built.
Packaging of Qlik Sense on Kubernetes is done through a [Porter](https://porter.sh/) definition in the [Qlik Sense on Kubernetes configuration repository](https://github.com/qlik-oss/qliksense-k8s/blob/master/porter.yaml), the resulting bundle published on DockerHub as a [Cloud Natvie Application Bundle](https://cnab.io/) called [qliksense-cnab-bundle](https://hub.docker.com/r/qlik/qliksense-cnab-bundle)
To start, clone (https://github.com/qlik-oss/qliksense-k8s)[https://github.com/qlik-oss/qliksense-k8s] and modify the repo as desired, once finished make sure to be in the `qliksense-k8s` directory from which the porter bundle can be built:
- `porter build`
To start, clone [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s) and modify the repo as desired, once finished make sure to be in the `qliksense-k8s` directory from which the porter bundle can be built:
```shell
git clone git@github.com:qlik-oss/qliksense-k8s.git
cd qliksense-k8s
qliksense build
```
Once built, all of the `porter` command that were used with `--tag` can be now be used without this flag provided that porter is executed with the `qliksense-k8s` directory. `porter` will automatically use the qliksense-k8s (and the porter.yaml) in the current directory.
## List of Commands
- [qliksense about](action_about.md)

41
action_about.md Normal file
View File

@@ -0,0 +1,41 @@
# qliksense about
About action will display inside information regarding [qliksense-k8](https://github.com/qlik-oss/qliksense-k8s) release.
it will support following flags
- `qliksense about 1.0.0` display default profile for tag `1.0.0`.
- `qliksense about 1.0.0 --profile=docker-desktop`
- `qliksense about`
- assuming current directory has `manifests/docker-desktop`
- or get version information from pull of `qliksense-k8s` `master`
using other supported commands user might have built the CR into the location `~/.qliksense/myqliksense.yaml`
```yaml
apiVersion: qlik.com/v1
kind: QlikSense
metadata:
name: myqliksense
spec:
profile: docker-desktop
manifestsRoot: /Usr/ddd/my-k8-repo/manifests
namespace: myqliksense
storageClassName: efs
configs:
qliksense:
- name: acceptEULA
value: "yes"
secrets:
qliksense:
- name: mongoDbUri
value: "mongo://mongo:3307"
- name: messagingPassword
valueFromKey: messagingPassword
```
In that case the command would be
- `qliksense about`
- display from `/Usr/ddd/my-k8-repo/manifests/docker-desktop` location
- pull from `master` if directory invalid/empty

31
action_config.md Normal file
View File

@@ -0,0 +1,31 @@
# qliksense config
Config action will perform operations on configurations and contexts regarding the [qliksense-k8](https://github.com/qlik-oss/qliksense-k8s) release.
it will support following commands:
- `qliksense config apply` - generate the patchs and apply manifests to k8s
- `qliksense config list-contexts` - retrieves the contexts and lists them
- `qliksense config set` - configure a key value pair into the current context
- `qliksense config set-configs` - set configurations into the qliksense context
- `qliksense config set-context` - sets the context in which the Kubernetes cluster and resources live in
- `qliksense config set-secrets` - set secrets configurations into the qliksense context
- `qliksense config view` - view the qliksense operator CR
the global file that abstracts all the contexts is `config.yaml`, located at: `~/.qliksense/config.yaml`:
```yaml
apiVersion: config.qlik.com/v1
kind: QliksenseConfig
metadata:
name: QliksenseConfigMetadata
spec:
contexts:
- name: qlik-default
crFile: /Users/fff/.qliksense/contexts/qlik-default/qlik-default.yaml
- name: myqliksense
crFile: /Users/fff/.qliksense/contexts/myqliksense/myqliksense.yaml
- name: hello
crFile: /Users/fff/.qliksense/contexts/hello/hello.yaml
currentContext: hello
```

66
cmd/qliksense/about.go Normal file
View File

@@ -0,0 +1,66 @@
package main
import (
"errors"
"fmt"
"strings"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
"gopkg.in/yaml.v2"
)
type aboutCommandOptions struct {
Profile string
}
func about(q *qliksense.Qliksense) *cobra.Command {
opts := &aboutCommandOptions{}
c := &cobra.Command{
Use: "about ref",
Short: "About Qlik Sense",
Long: "Gives the version of QLik Sense on Kubernetes and versions of images.",
Example: `
qliksense about 1.0.0
- display default profile (docker-desktop) for Git ref 1.0.0 in the qliksense-k8s repo
qliksense about 1.0.0 --profile=docker-desktop
- specifying profile
qliksense about
qliksense about --profile=test
- if no Git ref is provided, then get version information from the configuration on disk:
- if user's current directory has a subdirectory "manifests/${profile}",
then get version information from that
- if using other supported commands the user has built a CR in ~/.qliksense,
then get version information based on the path derived like so:
- ${spec.manifestsRoot}/${spec.profile} # if no profile flag provided
- ${spec.manifestsRoot}/${profile} # if profile is provided using the --profile command flag
- if no config found on disk in locations described above,
then get version information based on the default profile in the qliksense-k8s repo master
`,
RunE: func(cmd *cobra.Command, args []string) error {
if gitRef, err := getAboutCommandGitRef(args); err != nil {
return err
} else if vout, err := q.About(gitRef, opts.Profile); err != nil {
return err
} else if out, err := yaml.Marshal(vout); err != nil {
return err
} else if _, err := fmt.Println(string(out)); err != nil {
return err
}
return nil
},
}
f := c.Flags()
f.StringVar(&opts.Profile, "profile", "", "Configuration profile")
return c
}
func getAboutCommandGitRef(args []string) (string, error) {
if len(args) > 1 {
return "", errors.New("too many arguments, only 1 expected")
} else if len(args) == 1 {
return strings.TrimSpace(args[0]), nil
}
return "", nil
}

View File

@@ -1,412 +0,0 @@
package main
import (
"bufio"
"os"
"strings"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func buildAliasCommands(porterCmd *cobra.Command, q *qliksense.Qliksense) []*cobra.Command {
return []*cobra.Command{
buildBuildAlias(porterCmd),
buildInstallAlias(porterCmd, q),
buildUpgradeAlias(porterCmd, q),
buildAboutAlias(porterCmd),
buildPreflightAlias(porterCmd, q),
buildUninstallAlias(porterCmd, q),
}
}
func buildBuildAlias(porterCmd *cobra.Command) *cobra.Command {
var (
c *cobra.Command
)
c = &cobra.Command{
Use: "build",
Short: "Build a bundle",
Long: "Builds the bundle in the current directory by generating a Dockerfile and a CNAB bundle.json, and then building the invocation image.",
DisableFlagParsing: true,
RunE: func(cmd *cobra.Command, args []string) error {
return porterCmd.RunE(porterCmd, append([]string{"build"}, args...))
},
Annotations: map[string]string{
"group": "alias",
},
}
return c
}
type paramOptions struct {
aboutOptions
Params []string
ParamFiles []string
Name string
InsecureRegistry bool
// CredentialIdentifiers is a list of credential names or paths to make available to the bundle.
CredentialIdentifiers []string
Driver string
Force bool
Insecure bool
}
func buildInstallAlias(porterCmd *cobra.Command, q *qliksense.Qliksense) *cobra.Command {
var (
c *cobra.Command
opts *paramOptions
registry *string
)
opts = &paramOptions{}
c = &cobra.Command{
Use: "install [INSTANCE]",
Short: "Install qliksense",
Long: `Install a new instance of a bundle.
The first argument is the bundle instance name to create for the installation. This defaults to the name of the bundle.
Porter uses the Docker driver as the default runtime for executing a bundle's invocation image, but an alternate driver may be supplied via '--driver/-d'.
For example, the 'debug' driver may be specified, which simply logs the info given to it and then exits.`,
Example: ` qliksense install
qliksense install --version v1.0.0
qliksense install --insecure
qliksense install qliksense --file qliksense/bundle.json
qliksense install --param-file base-values.txt --param-file dev-values.txt --param test-mode=true --param header-color=blue
qliksense install --cred kubernetes
qliksense install --driver debug
qliksense install MyAppFromTag --tag qlik/qliksense-cnab-bundle:v1.0.0
`,
//DisableFlagParsing: true,
RunE: func(cmd *cobra.Command, args []string) error {
// Push images here.
// TODO: Need to get the private reg from params
args = append(os.Args[2:], opts.getTagValue(args)...)
if registry = opts.findKey("dockerRegistry"); registry != nil {
if len(*registry) > 0 {
q.TagAndPushImages(*registry)
}
}
return porterCmd.RunE(porterCmd, append([]string{"install"}, args...))
},
Annotations: map[string]string{
"group": "alias",
},
}
f := c.Flags()
f.StringVarP(&opts.Version, "version", "v", "latest",
"Version of Qlik Sense to install")
f.BoolVar(&opts.Insecure, "insecure", true,
"Allow working with untrusted bundles")
f.StringVarP(&opts.File, "file", "f", "",
"Path to the porter manifest file. Defaults to the bundle in the current directory.")
f.StringVar(&opts.CNABFile, "cnab-file", "",
"Path to the CNAB bundle.json file.")
f.StringSliceVar(&opts.ParamFiles, "param-file", nil,
"Path to a parameters definition file for the bundle, each line in the form of NAME=VALUE. May be specified multiple times.")
f.StringSliceVar(&opts.Params, "param", nil,
"Define an individual parameter in the form NAME=VALUE. Overrides parameters set with the same name using --param-file. May be specified multiple times.")
f.StringSliceVarP(&opts.CredentialIdentifiers, "cred", "c", nil,
"Credential to use when installing the bundle. May be either a named set of credentials or a filepath, and specified multiple times.")
f.StringVarP(&opts.Driver, "driver", "d", "docker",
"Specify a driver to use. Allowed values: docker, debug")
f.StringVarP(&opts.Tag, "tag", "t", "",
"Use a bundle in an OCI registry specified by the given tag")
f.BoolVar(&opts.InsecureRegistry, "insecure-registry", false,
"Don't require TLS for the registry")
f.BoolVar(&opts.Force, "force", false,
"Force a fresh pull of the bundle and all dependencies")
return c
}
func buildUpgradeAlias(porterCmd *cobra.Command, q *qliksense.Qliksense) *cobra.Command {
var (
c *cobra.Command
opts *paramOptions
registry *string
)
opts = &paramOptions{}
c = &cobra.Command{
Use: "upgrade [INSTANCE]",
Short: "Upgrade qliksense",
Long: `Upgrade to a new instance of a bundle.
The first argument is the bundle instance name to upgrade for the installation. This defaults to the name of the bundle.
Porter uses the Docker driver as the default runtime for executing a bundle's invocation image, but an alternate driver may be supplied via '--driver/-d'.
For example, the 'debug' driver may be specified, which simply logs the info given to it and then exits.`,
Example: ` qliksense upgrade
qliksense upgrade --version v1.0.0
qliksense upgrade --insecure
qliksense upgrade qliksense --file qliksense/bundle.json
qliksense upgrade --param-file base-values.txt --param-file dev-values.txt --param test-mode=true --param header-color=blue
qliksense upgrade --cred kubernetes
qliksense upgrade --driver debug
qliksense upgrade MyAppFromTag --tag qlik/qliksense-cnab-bundle:v1.0.0
`,
//DisableFlagParsing: true,
RunE: func(cmd *cobra.Command, args []string) error {
// Push images here.
// TODO: Need to get the private reg from params
args = append(os.Args[2:], opts.getTagValue(args)...)
if registry = opts.findKey("dockerRegistry"); registry != nil {
if len(*registry) > 0 {
q.TagAndPushImages(*registry)
}
}
return porterCmd.RunE(porterCmd, append([]string{"upgrade"}, args...))
},
Annotations: map[string]string{
"group": "alias",
},
}
f := c.Flags()
f.StringVarP(&opts.Version, "version", "v", "latest",
"Version of Qlik Sense to upgrade to")
f.BoolVar(&opts.Insecure, "insecure", true,
"Allow working with untrusted bundles")
f.StringVarP(&opts.File, "file", "f", "",
"Path to the porter manifest file. Defaults to the bundle in the current directory.")
f.StringVar(&opts.CNABFile, "cnab-file", "",
"Path to the CNAB bundle.json file.")
f.StringSliceVar(&opts.ParamFiles, "param-file", nil,
"Path to a parameters definition file for the bundle, each line in the form of NAME=VALUE. May be specified multiple times.")
f.StringSliceVar(&opts.Params, "param", nil,
"Define an individual parameter in the form NAME=VALUE. Overrides parameters set with the same name using --param-file. May be specified multiple times.")
f.StringSliceVarP(&opts.CredentialIdentifiers, "cred", "c", nil,
"Credential to use when installing the bundle. May be either a named set of credentials or a filepath, and specified multiple times.")
f.StringVarP(&opts.Driver, "driver", "d", "docker",
"Specify a driver to use. Allowed values: docker, debug")
f.StringVarP(&opts.Tag, "tag", "t", "",
"Use a bundle in an OCI registry specified by the given tag")
f.BoolVar(&opts.InsecureRegistry, "insecure-registry", false,
"Don't require TLS for the registry")
f.BoolVar(&opts.Force, "force", false,
"Force a fresh pull of the bundle and all dependencies")
return c
}
func buildUninstallAlias(porterCmd *cobra.Command, q *qliksense.Qliksense) *cobra.Command {
var (
c *cobra.Command
opts *paramOptions
)
opts = &paramOptions{}
c = &cobra.Command{
Use: "uninstall [INSTANCE]",
Short: "Uninstall a bundle instance",
Long: `Uninstall a bundle instance
The first argument is the bundle instance name to uninstall. This defaults to the name of the bundle.
Porter uses the Docker driver as the default runtime for executing a bundle's invocation image, but an alternate driver may be supplied via '--driver/-d'.
For example, the 'debug' driver may be specified, which simply logs the info given to it and then exits.`,
Example: ` qliksense uninstall
qliksense uninstall --insecure
qliksense uninstall MyAppInDev --file myapp/bundle.json
qliksense uninstall --param-file base-values.txt --param-file dev-values.txt --param test-mode=true --param header-color=blue
qliksense uninstall --cred azure --cred kubernetes
qliksense uninstall --driver debug
qliksense uninstall MyAppFromTag --tag deislabs/porter-kube-bundle:v1.0
`,
RunE: func(cmd *cobra.Command, args []string) error {
return porterCmd.RunE(porterCmd, append([]string{"uninstall"}, os.Args[2:]...))
},
Annotations: map[string]string{
"group": "alias",
},
}
f := c.Flags()
f.BoolVar(&opts.Insecure, "insecure", true,
"Allow working with untrusted bundles")
f.StringVarP(&opts.File, "file", "f", "",
"Path to the porter manifest file. Defaults to the bundle in the current directory. Optional unless a newer version of the bundle should be used to uninstall the bundle.")
f.StringVar(&opts.CNABFile, "cnab-file", "",
"Path to the CNAB bundle.json file.")
f.StringSliceVar(&opts.ParamFiles, "param-file", nil,
"Path to a parameters definition file for the bundle, each line in the form of NAME=VALUE. May be specified multiple times.")
f.StringSliceVar(&opts.Params, "param", nil,
"Define an individual parameter in the form NAME=VALUE. Overrides parameters set with the same name using --param-file. May be specified multiple times.")
f.StringSliceVarP(&opts.CredentialIdentifiers, "cred", "c", nil,
"Credential to use when uninstalling the bundle. May be either a named set of credentials or a filepath, and specified multiple times.")
f.StringVarP(&opts.Driver, "driver", "d", "docker",
"Specify a driver to use. Allowed values: docker, debug")
f.StringVarP(&opts.Tag, "tag", "t", "",
"Use a bundle in an OCI registry specified by the given tag")
f.BoolVar(&opts.InsecureRegistry, "insecure-registry", false,
"Don't require TLS for the registry")
f.BoolVar(&opts.Force, "force", false,
"Force a fresh pull of the bundle and all dependencies")
return c
}
func (o *aboutOptions) getTagDefaults(args []string) []string {
args = append(args, o.getTagValue(args)...)
return args
}
func (o *aboutOptions) getTagValue(args []string) []string {
tagArr := []string{}
if len(o.Tag) > 1 {
tagArr = []string{"--tag", o.Tag}
}
if len(o.Tag) <= 0 && len(o.File) <= 0 && len(o.CNABFile) <= 0 {
if _, err := os.Stat("porter.yaml"); err != nil {
tagArr = []string{"--tag", "qlik/qliksense-cnab-bundle:" + o.Version}
}
}
return tagArr
}
type aboutOptions struct {
Version string
Tag string
File string
CNABFile string
}
func buildAboutAlias(porterCmd *cobra.Command) *cobra.Command {
var (
c *cobra.Command
opts *aboutOptions
)
opts = &aboutOptions{}
c = &cobra.Command{
Use: "about",
Short: "About Qlik Sense",
Long: "Gives the verion of QLik Sense on Kuberntetes and versions of images.",
RunE: func(cmd *cobra.Command, args []string) error {
args = opts.getTagDefaults(args)
return porterCmd.RunE(porterCmd, append([]string{"invoke", "--action", "about"}, args...))
},
Annotations: map[string]string{
"group": "alias",
},
}
f := c.Flags()
f.StringVarP(&opts.Version, "version", "v", "latest",
"Version of Qlik Sense to install")
f.StringVarP(&opts.Tag, "tag", "t", "",
"Use a bundle in an OCI registry specified by the given tag")
f.StringVarP(&opts.File, "file", "f", "",
"Path to the porter manifest file. Defaults to the bundle in the current directory.")
f.StringVar(&opts.CNABFile, "cnab-file", "",
"Path to the CNAB bundle.json file.")
return c
}
func buildPreflightAlias(porterCmd *cobra.Command, q *qliksense.Qliksense) *cobra.Command {
var (
c *cobra.Command
opts *paramOptions
)
opts = &paramOptions{}
c = &cobra.Command{
Use: "preflight",
Short: "Preflight Checks",
Long: "Perform Preflight Checks",
RunE: func(cmd *cobra.Command, args []string) error {
args = append(os.Args[1:], opts.getTagDefaults(args)...)
return porterCmd.RunE(porterCmd, append([]string{"invoke", "--action", "preflight"}, args...))
},
Annotations: map[string]string{
"group": "alias",
},
}
f := c.Flags()
f.StringSliceVar(&opts.Params, "param", nil,
"Define an individual parameter in the form NAME=VALUE. Overrides parameters set with the same name using --param-file. May be specified multiple times.")
f.StringSliceVar(&opts.ParamFiles, "param-file", nil,
"Path to a parameters definition file for the bundle, each line in the form of NAME=VALUE. May be specified multiple times.")
f.StringVarP(&opts.Tag, "tag", "t", "",
"Use a bundle in an OCI registry specified by the given tag")
f.StringVarP(&opts.Version, "version", "v", "latest",
"Version of Qlik Sense to install")
f.StringSliceVarP(&opts.CredentialIdentifiers, "cred", "c", nil,
"Credential to use when installing the bundle. May be either a named set of credentials or a filepath, and specified multiple times.")
return c
}
func (o *paramOptions) findKey(param string) *string {
var (
value *string
)
if value = o.findParams(param); value != nil {
return value
}
if value = o.findParamFiles(param); value != nil {
return value
}
return nil
}
// parsedParams parses the variable assignments in Params.
func (o *paramOptions) findParams(param string) *string {
return o.findVariableKey(param, o.Params)
}
// parseParamFiles parses the variable assignments in ParamFiles.
func (o *paramOptions) findParamFiles(param string) *string {
var (
path string
retStr *string
)
for _, path = range o.ParamFiles {
retStr = o.findParamFile(param, path)
}
return retStr
}
func (o *paramOptions) findParamFile(param string, path string) *string {
var (
f *os.File
err error
scanner *bufio.Scanner
lines []string
retStr *string
)
if f, err = os.Open(path); err == nil {
defer f.Close()
scanner = bufio.NewScanner(f)
for scanner.Scan() {
lines = append(lines, scanner.Text())
}
retStr = o.findVariableKey(param, lines)
}
return retStr
}
func (o *paramOptions) findVariableKey(param string, params []string) *string {
var (
variable, value string
)
for _, p := range params {
parts := strings.SplitN(p, "=", 2)
if len(parts) >= 2 {
variable = strings.TrimSpace(parts[0])
if variable == param {
value = strings.TrimSpace(parts[1])
return &value
}
}
}
return nil
}

42
cmd/qliksense/config.go Normal file
View File

@@ -0,0 +1,42 @@
package main
import (
"fmt"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
var configCmd = &cobra.Command{
Use: "config",
Short: "do operations on/around CR",
Long: `do operations on/around CR`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Use like: config view or config apply")
},
}
func configApplyCmd(q *qliksense.Qliksense) *cobra.Command {
c := &cobra.Command{
Use: "apply",
Short: "generate the patchs and apply manifests to k8s",
Long: `generate patches based on CR and apply manifests to k8s`,
Example: `qliksense config apply`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.ConfigApplyQK8s()
},
}
return c
}
func configViewCmd(q *qliksense.Qliksense) *cobra.Command {
c := &cobra.Command{
Use: "view",
Short: "view the qliksense operator CR",
Long: `display the operator CR, that has been created for the current context`,
Example: `qliksense config view`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.ConfigViewCR()
},
}
return c
}

89
cmd/qliksense/context.go Normal file
View File

@@ -0,0 +1,89 @@
package main
import (
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func setContextConfigCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
)
cmd = &cobra.Command{
Use: "set-context",
Short: "Sets the context in which the Kubernetes cluster and resources live in",
Example: `qliksense config set-context <context_name>`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.SetContextConfig(args)
},
}
return cmd
}
func listContextConfigCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
)
cmd = &cobra.Command{
Use: "list-contexts",
Short: "retrieves the contexts and lists them",
Example: `qliksense config list-contexts`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.ListContextConfigs()
},
}
return cmd
}
func setOtherConfigsCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
)
cmd = &cobra.Command{
Use: "set",
Short: "configure a key value pair into the current context",
Example: `qliksense config set <key>=<value>`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.SetOtherConfigs(args)
},
}
return cmd
}
func setConfigsCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
)
cmd = &cobra.Command{
Use: "set-configs",
Short: "set configurations into the qliksense context",
Example: `qliksense config set-configs <key>=<value>`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.SetConfigs(args)
},
}
return cmd
}
func setSecretsCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
secret bool
)
cmd = &cobra.Command{
Use: "set-secrets",
Short: "set secrets configurations into the qliksense context",
Example: `qliksense config set-secrets <key>=<value> --secret=true`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.SetSecrets(args, secret)
},
}
f := cmd.Flags()
f.BoolVar(&secret, "secret", false, "Whether secrets should be encrypted as a Kubernetes Secret resource")
return cmd
}

26
cmd/qliksense/fetch.go Normal file
View File

@@ -0,0 +1,26 @@
package main
import (
"errors"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func fetchCmd(q *qliksense.Qliksense) *cobra.Command {
c := &cobra.Command{
Use: "fetch",
Short: "fetch a release from qliksense-k8s repo",
Long: `fetch a release from qliksense-k8s repo`,
Example: `qliksense fetch <version>`,
Args: func(cmd *cobra.Command, args []string) error {
if len(args) != 1 {
return errors.New("requires a version (i.e. v1.0.0)")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
return q.FetchQK8s(args[0])
},
}
return c
}

34
cmd/qliksense/install.go Normal file
View File

@@ -0,0 +1,34 @@
package main
import (
"errors"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func installCmd(q *qliksense.Qliksense) *cobra.Command {
opts := &qliksense.InstallCommandOptions{}
c := &cobra.Command{
Use: "install",
Short: "install a qliksense release",
Long: `install a qliksesne release`,
Example: `qliksense install <version>`,
Args: func(cmd *cobra.Command, args []string) error {
if len(args) != 1 {
return errors.New("requires a version (i.e. v1.0.0)")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
return q.InstallQK8s(args[0], opts)
},
}
f := c.Flags()
f.StringVarP(&opts.AcceptEULA, "acceptEULA", "a", "", "AcceptEULA for qliksense")
f.StringVarP(&opts.Namespace, "namespace", "n", "", "Namespace where to install the qliksense")
f.StringVarP(&opts.StorageClass, "storageClass", "s", "", "Storage class for qliksense")
f.StringVarP(&opts.MongoDbUri, "mongoDbUri", "m", "", "mongoDbUri for qliksense (i.e. mongodb://qlik-default-mongodb:27017/qliksense?ssl=false)")
f.StringVarP(&opts.RotateKeys, "rotateKeys", "r", "", "Rotate JWT keys for qliksense (yes:rotate keys/ no:use exising keys from cluster/ None: use default EJSON_KEY from env")
return c
}

28
cmd/qliksense/operator.go Normal file
View File

@@ -0,0 +1,28 @@
package main
import (
"fmt"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
var operatorCmd = &cobra.Command{
Use: "operator",
Short: "Configuration for operator",
Long: `Configuration for operator`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("User like: operator view")
},
}
func operatorViewCmd(q *qliksense.Qliksense) *cobra.Command {
c := &cobra.Command{
Use: "view",
Short: "View Configuration for operator",
Long: `View Configuration for operator`,
Run: func(cmd *cobra.Command, args []string) {
q.ViewOperatorCrd()
},
}
return c
}

View File

@@ -1,31 +0,0 @@
package main
import (
"fmt"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
"strings"
)
func porter(q *qliksense.Qliksense) *cobra.Command {
return &cobra.Command{
Use: "porter",
Short: "Execute a porter command",
DisableFlagParsing: true,
RunE: func(cobCmd *cobra.Command, args []string) error {
var (
err error
)
if _, err = q.CallPorter(args,
func(x string) (out *string) {
out = new(string)
*out = strings.ReplaceAll(x, "porter", "qliksense porter")
fmt.Println(*out)
return
}); err != nil {
return err
}
return nil
},
}
}

View File

@@ -1,33 +1,27 @@
package main
import (
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func pullQliksenseImages(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
opts *aboutOptions
)
opts = &aboutOptions{}
cmd = &cobra.Command{
Use: "pull",
Short: "Pull docke images for offline install",
Example: `qliksense pull`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.PullImages(opts.getTagDefaults(args))
},
}
f := cmd.Flags()
f.StringVarP(&opts.Version, "version", "v", "latest",
"From version of Qlik Sense The images will be pulled")
f.StringVarP(&opts.Tag, "tag", "t", "",
"Use a bundle in an OCI registry specified by the given tag")
f.StringVarP(&opts.File, "file", "f", "",
"Path to the porter manifest file. Defaults to the bundle in the current directory.")
f.StringVar(&opts.CNABFile, "cnab-file", "",
"Path to the CNAB bundle.json file.")
return cmd
}
package main
import (
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func pullQliksenseImages(q *qliksense.Qliksense) *cobra.Command {
opts := &aboutCommandOptions{}
cmd := &cobra.Command{
Use: "pull",
Short: "Pull docke images for offline install",
Example: `qliksense pull`,
RunE: func(cmd *cobra.Command, args []string) error {
if gitRef, err := getAboutCommandGitRef(args); err != nil {
return err
} else if err = q.PullImages(gitRef, opts.Profile, false); err != nil {
return err
}
return nil
},
}
f := cmd.Flags()
f.StringVar(&opts.Profile, "profile", "", "Configuration profile")
return cmd
}

View File

@@ -3,71 +3,62 @@ package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"github.com/mitchellh/go-homedir"
"github.com/qlik-oss/sense-installer/pkg"
"github.com/qlik-oss/sense-installer/pkg/api"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
// To run this project in ddebug mode, run:
// export QLIKSENSE_DEBUG=true
// qliksense <command>
const (
// porterURLBase = "https://deislabs.blob.core.windows.net/porter"
porterURLBase = "https://github.com/qlik-oss/sense-installer/releases/download"
porterHomeVar = "PORTER_HOME"
qlikSenseHomeVar = "QLIKSENSE_HOME"
qlikSenseDirVar = ".qliksense"
mixinDirVar = "mixins"
porterRuntime = "porter-runtime"
)
func initAndExecute() error {
var (
porterExe string
err error
qlikSenseHome string
err error
)
if porterExe, err = installPorter(); err != nil {
qlikSenseHome, err = setUpPaths()
if err != nil {
log.Fatal(err)
}
// create dirs and appropriate files for setting up contexts
api.LogDebugMessage("QliksenseHomeDir: %s", qlikSenseHome)
qliksenseClient, err := qliksense.New(qlikSenseHome)
if err != nil {
return err
}
if err := rootCmd(qliksense.New(porterExe)).Execute(); err != nil {
qliksenseClient.SetUpQliksenseDefaultContext()
if err := rootCmd(qliksenseClient).Execute(); err != nil {
return err
}
return nil
}
func installPorter() (string, error) {
func setUpPaths() (string, error) {
var (
porterPermaLink = pkg.Version
//porterPermaLink = "v0.3.0"
destination, homeDir, mixin, mixinOpts, qlikSenseHome, porterExe, ext string
mixinsVar = map[string]string{
"kustomize": "-v 0.2-beta-3-0e19ca4 --url https://github.com/donmstewart/porter-kustomize/releases/download",
"qliksense": "-v v0.14.0 --url https://github.com/qlik-oss/porter-qliksense/releases/download",
"exec": "-v latest",
"kubernetes": "-v latest",
"helm": "-v latest",
"azure": "-v latest",
"terraform": "-v latest",
"az": "-v latest",
"aws": "-v latest",
"gcloud": "-v latest",
}
downloadMixins map[string]string
downloadPorter bool
err error
cmd *exec.Cmd
homeDir, qlikSenseHome string
err error
)
porterExe = "porter"
if runtime.GOOS == "windows" {
porterExe = porterExe + ".exe"
}
if qlikSenseHome = os.Getenv(qlikSenseHomeVar); qlikSenseHome == "" {
if homeDir, err = homedir.Dir(); err != nil {
return "", err
@@ -77,75 +68,12 @@ func installPorter() (string, error) {
}
qlikSenseHome = filepath.Join(homeDir, qlikSenseDirVar)
}
os.Setenv(porterHomeVar, qlikSenseHome)
//TODO: Check if porter version is one alreadu is one for this build
porterExe = filepath.Join(qlikSenseHome, porterExe)
if _, err = os.Stat(qlikSenseHome); err != nil {
if os.IsNotExist(err) {
downloadPorter = true
} else {
return "", err
}
} else {
if _, err = os.Stat(porterExe); err != nil {
if os.IsNotExist(err) {
downloadPorter = true
} else {
return "", err
}
}
}
if downloadPorter {
os.Mkdir(qlikSenseHome, os.ModePerm)
destination = filepath.Join(qlikSenseHome, porterRuntime)
if err = downloadFile(porterURLBase+"/"+porterPermaLink+"/porter-linux-amd64", destination); err != nil {
return "", err
}
os.Chmod(destination, 0755)
if runtime.GOOS == "linux" && runtime.GOARCH == "amd64" {
if _, err = copy(filepath.Join(qlikSenseHome, porterRuntime), porterExe); err != nil {
return "", err
}
os.Chmod(porterExe, 0755)
} else {
if runtime.GOOS == "windows" {
ext = ".exe"
}
if err = downloadFile(porterURLBase+"/"+porterPermaLink+"/"+"porter-"+runtime.GOOS+"-"+runtime.GOARCH+ext, porterExe); err != nil {
return "", err
}
os.Chmod(porterExe, 0755)
}
if err := os.MkdirAll(qlikSenseHome, os.ModePerm); err != nil {
return "", err
}
if _, err = os.Stat(filepath.Join(qlikSenseHome, mixinDirVar)); err != nil {
if os.IsNotExist(err) {
downloadMixins = mixinsVar
} else {
return "", err
}
} else {
downloadMixins = make(map[string]string)
for mixin, mixinOpts = range mixinsVar {
if _, err = os.Stat(filepath.Join(qlikSenseHome, mixinDirVar, mixin)); err != nil {
if os.IsNotExist(err) {
downloadMixins[mixin] = mixinOpts
} else {
return "", err
}
}
}
}
for mixin, mixinOpts = range downloadMixins {
cmd = exec.Command(porterExe, append([]string{"mixin", "install", mixin}, strings.Split(mixinOpts, " ")...)...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err = cmd.Run(); err != nil {
return "", err
}
}
return porterExe, nil
return qlikSenseHome, nil
}
var versionCmd = &cobra.Command{
@@ -159,7 +87,7 @@ var versionCmd = &cobra.Command{
func rootCmd(p *qliksense.Qliksense) *cobra.Command {
var (
cmd, porterCmd, alias *cobra.Command
cmd *cobra.Command
)
cmd = &cobra.Command{
@@ -178,15 +106,46 @@ func rootCmd(p *qliksense.Qliksense) *cobra.Command {
// For qliksense overrides/commands
cmd.AddCommand(pullQliksenseImages(p))
porterCmd = porter(p)
cmd.AddCommand(porterCmd)
for _, alias = range buildAliasCommands(porterCmd, p) {
cmd.AddCommand(alias)
}
cmd.AddCommand(about(p))
// add version command
cmd.AddCommand(versionCmd)
// add operator command
cmd.AddCommand(operatorCmd)
operatorCmd.AddCommand(operatorViewCmd(p))
//add fetch command
cmd.AddCommand(fetchCmd(p))
// add install command
cmd.AddCommand(installCmd(p))
// add config command
cmd.AddCommand(configCmd)
configCmd.AddCommand(configApplyCmd(p))
configCmd.AddCommand(configViewCmd(p))
viper.SetEnvKeyReplacer(strings.NewReplacer("-", "_"))
//add upgrade command
cmd.AddCommand(upgradeCmd(p))
// add the set-context config command as a sub-command to the app config command
configCmd.AddCommand(setContextConfigCmd(p))
// add the set profile/namespace/storageClassName/git-repository config command as a sub-command to the app config command
configCmd.AddCommand(setOtherConfigsCmd(p))
// add the set ### config command as a sub-command to the app config sub-command
configCmd.AddCommand(setConfigsCmd(p))
// add the set ### config command as a sub-command to the app config sub-command
configCmd.AddCommand(setSecretsCmd(p))
// add the list config command as a sub-command to the app config sub-command
configCmd.AddCommand(listContextConfigCmd(p))
// add uninstall command
cmd.AddCommand(uninstallCmd(p))
return cmd
}

View File

@@ -0,0 +1,23 @@
package main
import (
qapi "github.com/qlik-oss/sense-installer/pkg/api"
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func uninstallCmd(q *qliksense.Qliksense) *cobra.Command {
c := &cobra.Command{
Use: "uninstall",
Short: "Uninstall the deployed qliksense with release name [ " + qapi.NewQConfig(q.QliksenseHome).Spec.CurrentContext + " ]",
Long: `Uninstall the deployed qliksense. By default uninstall the current context`,
Example: `qliksense uninstall <context-name>`,
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
return q.UninstallQK8s(args[0])
}
return q.UninstallQK8s("")
},
}
return c
}

20
cmd/qliksense/upgrade.go Normal file
View File

@@ -0,0 +1,20 @@
package main
import (
"github.com/qlik-oss/sense-installer/pkg/qliksense"
"github.com/spf13/cobra"
)
func upgradeCmd(q *qliksense.Qliksense) *cobra.Command {
c := &cobra.Command{
Use: "upgrade",
Short: "upgrade qliksense release",
Long: `upgrade qliksesne release`,
Example: `qliksense upgrade <version>`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.UpgradeQK8s()
},
}
return c
}

77
go.mod
View File

@@ -3,74 +3,73 @@ module github.com/qlik-oss/sense-installer
go 1.13
replace (
github.com/Sirupsen/logrus v1.0.5 => github.com/sirupsen/logrus v1.0.5
github.com/Sirupsen/logrus v1.3.0 => github.com/Sirupsen/logrus v1.0.6
github.com/Sirupsen/logrus v1.4.0 => github.com/sirupsen/logrus v1.0.6
// github.com/containerd/containerd v1.3.0-0.20190507210959-7c1e88399ec0 => github.com/containerd/containerd v1.3.2
github.com/docker/docker => github.com/moby/moby v0.7.3-0.20190826074503-38ab9da00309
// github.com/jaguilar/vt100 => github.com/tonistiigi/vt100 v0.0.0-20190402012908-ad4c4a574305
// golang.org/x/crypto v0.0.0-20190129210102-0709b304e793 => golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
golang.org/x/sys => golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a
k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20191004115801-a2eda9f80ab8
k8s.io/client-go => k8s.io/client-go v0.0.0-20191016111102-bec269661e48
k8s.io/kubectl => k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
sigs.k8s.io/kustomize/api => github.com/qlik-oss/kustomize/api v0.3.3-0.20200206224201-2e697eccbad9
)
require (
get.porter.sh/porter v0.22.0-beta.1
github.com/Masterminds/semver v1.5.0 // indirect
github.com/Microsoft/hcsshim v0.8.7 // indirect
github.com/PaesslerAG/jsonpath v0.1.1 // indirect
github.com/PuerkitoBio/goquery v1.5.0 // indirect
github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d // indirect
cloud.google.com/go v0.52.0 // indirect
cloud.google.com/go/storage v1.5.0 // indirect
github.com/Shopify/ejson v1.2.1
github.com/agl/ed25519 v0.0.0-20170116200512-5312a6153412 // indirect
github.com/aws/aws-sdk-go v1.28.9 // indirect
github.com/bitly/go-hostpool v0.0.0-20171023180738-a3a6125de932 // indirect
github.com/bitly/go-simplejson v0.5.0 // indirect
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 // indirect
github.com/bugsnag/bugsnag-go v1.5.3 // indirect
github.com/bugsnag/panicwrap v1.2.0 // indirect
github.com/carolynvs/datetime-printer v0.2.0 // indirect
github.com/cbroglie/mustache v1.0.1 // indirect
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
github.com/cloudflare/cfssl v1.4.1 // indirect
github.com/containerd/containerd v1.3.2 // indirect
github.com/containerd/continuity v0.0.0-20191214063359-1097c8bae83b // indirect
github.com/deislabs/cnab-go v0.7.1-beta1 // indirect
github.com/containers/image/v5 v5.1.0
github.com/docker/cli v0.0.0-20191212191748-ebca1413117a
github.com/docker/cnab-to-oci v0.3.0-beta2 // indirect
github.com/docker/distribution v2.7.1+incompatible
github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
github.com/docker/go v1.5.1-1 // indirect
github.com/docker/go-metrics v0.0.1 // indirect
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 // indirect
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8 // indirect
github.com/gobuffalo/packr/v2 v2.7.1 // indirect
github.com/gofrs/uuid v3.2.0+incompatible // indirect
github.com/google/go-containerregistry v0.0.0-20191216221554-74b082017bc4 // indirect
github.com/gophercloud/gophercloud v0.7.0 // indirect
github.com/gobuffalo/envy v1.9.0 // indirect
github.com/gobuffalo/logger v1.0.3 // indirect
github.com/gobuffalo/packd v1.0.0 // indirect
github.com/gobuffalo/packr/v2 v2.7.1
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
github.com/golang/protobuf v1.3.3 // indirect
github.com/google/uuid v1.1.1
github.com/gorilla/mux v1.7.3 // indirect
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed // indirect
github.com/hashicorp/go-hclog v0.10.0 // indirect
github.com/hashicorp/go-plugin v1.0.1 // indirect
github.com/imdario/mergo v0.3.8 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/jinzhu/copier v0.0.0-20190924061706-b57f9002281a
github.com/jinzhu/gorm v1.9.11 // indirect
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/lib/pq v1.2.0 // indirect
github.com/mattn/go-sqlite3 v2.0.1+incompatible // indirect
github.com/miekg/pkcs11 v1.0.3 // indirect
github.com/mitchellh/go-homedir v1.1.0
github.com/mmcdole/gofeed v1.0.0-beta2 // indirect
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf // indirect
github.com/olekukonko/tablewriter v0.0.4 // indirect
github.com/opencontainers/runc v0.1.1 // indirect
github.com/pivotal/image-relocation v0.0.0-20191111101224-e94aff6df06c // indirect
github.com/qri-io/jsonschema v0.1.1 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/pkg/errors v0.8.1
github.com/qlik-oss/k-apis v0.0.8
github.com/rogpeppe/go-internal v1.5.2 // indirect
github.com/spf13/cobra v0.0.5
github.com/spf13/viper v1.6.1
github.com/theupdateframework/notary v0.6.1 // indirect
github.com/xlab/handysort v0.0.0-20150421192137-fb3537ed64a1 // indirect
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553
gopkg.in/AlecAivazis/survey.v1 v1.8.7 // indirect
golang.org/x/crypto v0.0.0-20200214034016-1d94cc7ab1c6 // indirect
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a // indirect
golang.org/x/net v0.0.0-20200202094626-16171245cfb2
golang.org/x/sys v0.0.0-20200219091948-cb0a6d8edb6c // indirect
golang.org/x/tools v0.0.0-20200219054238-753a1d49df85 // indirect
google.golang.org/genproto v0.0.0-20200128133413-58ce757ed39b // indirect
google.golang.org/grpc v1.27.0 // indirect
gopkg.in/dancannon/gorethink.v3 v3.0.5 // indirect
gopkg.in/fatih/pool.v2 v2.0.0 // indirect
gopkg.in/gorethink/gorethink.v3 v3.0.5 // indirect
gopkg.in/yaml.v2 v2.2.7
gopkg.in/yaml.v2 v2.2.8
gopkg.in/yaml.v3 v3.0.0-20190924164351-c8b7dadae555
sigs.k8s.io/kustomize/api v0.3.2
vbom.ml/util v0.0.0-20180919145318-efcd4e0f9787 // indirect
)
exclude github.com/Azure/go-autorest v12.0.0+incompatible

967
go.sum

File diff suppressed because it is too large Load Diff

159
pkg/api/apis.go Normal file
View File

@@ -0,0 +1,159 @@
package api
import (
"errors"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"github.com/jinzhu/copier"
"gopkg.in/yaml.v2"
)
// NewQConfig create QliksenseConfig object from file ~/.qliksense/config.yaml
func NewQConfig(qsHome string) *QliksenseConfig {
configFile := filepath.Join(qsHome, "config.yaml")
data, err := ioutil.ReadFile(configFile)
if err != nil {
fmt.Println("Cannot read config file from: "+configFile, err)
os.Exit(1)
}
qc := &QliksenseConfig{}
err = yaml.Unmarshal(data, qc)
if err != nil {
fmt.Println("yaml unmarshalling error ", err)
os.Exit(1)
}
qc.QliksenseHomePath = qsHome
return qc
}
// GetCR create a QliksenseCR object for a particular context
// from file ~/.qliksense/contexts/<contx-name>/<contx-name>.yaml
func (qc *QliksenseConfig) GetCR(contextName string) (*QliksenseCR, error) {
crFilePath := qc.getCRFilePath(contextName)
if crFilePath == "" {
return nil, errors.New("context name " + contextName + " not found")
}
return getCRObject(crFilePath)
}
// GetCurrentCR create a QliksenseCR object for current context
func (qc *QliksenseConfig) GetCurrentCR() (*QliksenseCR, error) {
return qc.GetCR(qc.Spec.CurrentContext)
}
// SetCrLocation sets the CR location for a context. Helpful during test
func (qc *QliksenseConfig) SetCrLocation(contextName, filepath string) (*QliksenseConfig, error) {
tempQc := &QliksenseConfig{}
copier.Copy(tempQc, qc)
found := false
tempQc.Spec.Contexts = []Context{}
for _, c := range qc.Spec.Contexts {
if c.Name == contextName {
c.CrFile = filepath
found = true
}
tempQc.Spec.Contexts = append(tempQc.Spec.Contexts, []Context{c}...)
}
if found {
return tempQc, nil
}
return nil, errors.New("cannot find the context")
}
func getCRObject(crfile string) (*QliksenseCR, error) {
data, err := ioutil.ReadFile(crfile)
if err != nil {
fmt.Println("Cannot read config file from: "+crfile, err)
return nil, err
}
cr := &QliksenseCR{}
err = yaml.Unmarshal(data, cr)
if err != nil {
fmt.Println("cannot unmarshal cr ", err)
return nil, err
}
return cr, nil
}
func (qc *QliksenseConfig) getCRFilePath(contextName string) string {
crFilePath := ""
for _, ctx := range qc.Spec.Contexts {
if ctx.Name == contextName {
crFilePath = ctx.CrFile
break
}
}
return crFilePath
}
func (qc *QliksenseConfig) IsRepoExist(contextName, version string) bool {
if _, err := os.Lstat(qc.BuildRepoPathForContext(contextName, version)); err != nil {
return false
}
return true
}
func (qc *QliksenseConfig) IsRepoExistForCurrent(version string) bool {
if _, err := os.Lstat(qc.BuildRepoPath(version)); err != nil {
return false
}
return true
}
func (qc *QliksenseConfig) BuildRepoPath(version string) string {
return qc.BuildRepoPathForContext(qc.Spec.CurrentContext, version)
}
func (qc *QliksenseConfig) BuildRepoPathForContext(contextName, version string) string {
return filepath.Join(qc.QliksenseHomePath, "contexts", contextName, "qlik-k8s", version)
}
func (qc *QliksenseConfig) BuildCurrentManifestsRoot(version string) string {
return filepath.Join(qc.BuildRepoPath(version), "manifests")
}
func (qc *QliksenseConfig) WriteCR(cr *QliksenseCR, contextName string) error {
crf := qc.getCRFilePath(contextName)
if crf == "" {
return errors.New("context name " + contextName + " not found")
}
by, err := yaml.Marshal(cr)
if err != nil {
fmt.Println("cannot marshal cr ", err)
return err
}
ioutil.WriteFile(crf, by, 0644)
return nil
}
func (qc *QliksenseConfig) WriteCurrentContextCR(cr *QliksenseCR) error {
return qc.WriteCR(cr, qc.Spec.CurrentContext)
}
func (qc *QliksenseConfig) IsContextExist(ctxName string) bool {
for _, ct := range qc.Spec.Contexts {
if ct.Name == ctxName {
return true
}
}
return false
}
func (cr *QliksenseCR) AddLabelToCr(key, value string) error {
if cr.Metadata.Labels == nil {
cr.Metadata.Labels = make(map[string]string)
}
cr.Metadata.Labels[key] = value
return nil
}
func (cr *QliksenseCR) GetString() (string, error) {
out, err := yaml.Marshal(cr)
if err != nil {
fmt.Println("cannot unmarshal cr ", err)
return "", err
}
return string(out), nil
}

85
pkg/api/apis_test.go Normal file
View File

@@ -0,0 +1,85 @@
package api
import (
"io/ioutil"
"os"
"path/filepath"
"testing"
)
const tempPermissionCode os.FileMode = 0777
func setup() (func(), string) {
dir, _ := ioutil.TempDir("", "testing_path")
config :=
`
apiVersion: config.qlik.com/v1
kind: QliksenseConfig
metadata:
name: whatever
spec:
contexts:
- name: contx1
crLocation: /Users/mqb/.qliksense/contexts/contx1
- name: cotx2
crLocation: /root/.qliksense/contexts/cotx2.yaml
currentContext: contx1
`
configFile := filepath.Join(dir, "config.yaml")
ioutil.WriteFile(configFile, []byte(config), tempPermissionCode)
tearDown := func() {
os.RemoveAll(dir)
}
return tearDown, dir
}
func createCRFile(homeDir string) {
cr :=
`
apiVersion: qlik.com/v1
kind: QlikSense
metadata:
name: contx1
labels:
version: v1.0.0
spec:
profile: docker-desktop
manifestsRoot: /Users/mqb/.qliksense/contexts/contx1/qlik-k8s/v0.0.1/manifests
namespace: myqliksense
storageClassName: efs
configs:
qliksense:
- name: acceptEULA
value: "yes"
`
ctx1Dir := filepath.Join(homeDir, "contexts", "contx1")
crFile := filepath.Join(ctx1Dir, "contx1.yaml")
os.MkdirAll(ctx1Dir, tempPermissionCode)
ioutil.WriteFile(crFile, []byte(cr), tempPermissionCode)
}
func TestGetCR(t *testing.T) {
td, dir := setup()
qc := NewQConfig(dir)
if qc.Spec.CurrentContext != "contx1" {
t.Fail()
}
// create CR
createCRFile(dir)
crFile := filepath.Join(dir, "contexts", "contx1", "contx1.yaml")
qct, e := qc.SetCrLocation("contx1", crFile)
if e != nil {
t.Fail()
t.Log(e)
}
qcr, err := qct.GetCurrentCR()
if err != nil {
t.Fail()
t.Log(err)
}
if qcr.Spec.Profile != "docker-desktop" {
t.Fail()
}
td()
}

94
pkg/api/context_apis.go Normal file
View File

@@ -0,0 +1,94 @@
package api
import (
"fmt"
"io/ioutil"
"log"
"github.com/qlik-oss/k-apis/pkg/config"
"gopkg.in/yaml.v2"
)
const (
QliksenseConfigApiVersion = "config.qlik.com/v1"
QliksenseConfigKind = "QliksenseConfig"
QliksenseContextApiVersion = "qlik.com/v1"
QliksenseContextKind = "Qliksense"
QliksenseDefaultProfile = "docker-desktop"
DefaultRotateKeys = "yes"
QliksenseMetadataName = "QliksenseConfigMetadata"
)
// AddCommonConfig adds common configs into CRs
func (qliksenseCR *QliksenseCR) AddCommonConfig(contextName string) {
qliksenseCR.ApiVersion = QliksenseContextApiVersion
qliksenseCR.Kind = QliksenseContextKind
if qliksenseCR.Metadata == nil {
qliksenseCR.Metadata = &Metadata{}
}
if qliksenseCR.Metadata.Name == "" {
qliksenseCR.Metadata.Name = contextName
}
qliksenseCR.Spec = &config.CRSpec{}
qliksenseCR.Spec.Profile = QliksenseDefaultProfile
qliksenseCR.Spec.ReleaseName = contextName
qliksenseCR.Spec.RotateKeys = DefaultRotateKeys
}
// AddBaseQliksenseConfigs adds configs into config.yaml
func (qliksenseConfig *QliksenseConfig) AddBaseQliksenseConfigs(defaultQliksenseContext string) {
qliksenseConfig.ApiVersion = QliksenseConfigApiVersion
qliksenseConfig.Kind = QliksenseConfigKind
if qliksenseConfig.Metadata == nil {
qliksenseConfig.Metadata = &Metadata{}
}
qliksenseConfig.Metadata.Name = QliksenseMetadataName
if defaultQliksenseContext != "" {
if qliksenseConfig.Spec == nil {
qliksenseConfig.Spec = &ContextSpec{}
}
qliksenseConfig.Spec.CurrentContext = defaultQliksenseContext
}
}
// WriteToFile (content, targetFile) writes content into specified file
func WriteToFile(content interface{}, targetFile string) error {
if content == nil || targetFile == "" {
return nil
}
x, err := yaml.Marshal(content)
if err != nil {
err = fmt.Errorf("An error occurred during marshalling CR: %v", err)
log.Println(err)
return err
}
// Writing content
err = ioutil.WriteFile(targetFile, x, 0644)
if err != nil {
log.Println(err)
return err
}
LogDebugMessage("Wrote content into %s", targetFile)
return nil
}
// ReadFromFile (content, targetFile) reads content from specified sourcefile
func ReadFromFile(content interface{}, sourceFile string) error {
if content == nil || sourceFile == "" {
return nil
}
contents, err := ioutil.ReadFile(sourceFile)
if err != nil {
err = fmt.Errorf("There was an error reading from file: %s, %v", sourceFile, err)
log.Println(err)
return err
}
if err := yaml.Unmarshal(contents, content); err != nil {
err = fmt.Errorf("An error occurred during unmarshalling: %v", err)
log.Println(err)
return err
}
return nil
}

View File

@@ -0,0 +1,94 @@
package api
import (
"reflect"
"testing"
"github.com/qlik-oss/k-apis/pkg/config"
)
var (
testDir = "./tests"
)
func TestAddCommonConfig(t *testing.T) {
type args struct {
qliksenseCR *QliksenseCR
contextName string
}
tests := []struct {
name string
args args
want *QliksenseCR
}{
{
name: "valid case",
args: args{
qliksenseCR: &QliksenseCR{},
contextName: "myqliksense",
},
want: &QliksenseCR{
CommonConfig: CommonConfig{
ApiVersion: QliksenseContextApiVersion,
Kind: QliksenseContextKind,
Metadata: &Metadata{
Name: "myqliksense",
},
},
Spec: &config.CRSpec{
Profile: QliksenseDefaultProfile,
ReleaseName: "myqliksense",
RotateKeys: DefaultRotateKeys,
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tt.args.qliksenseCR.AddCommonConfig(tt.args.contextName)
if !reflect.DeepEqual(tt.args.qliksenseCR, tt.want) {
t.Errorf("AddCommonConfig() = %+v, want %+v", tt.args.qliksenseCR, tt.want)
}
})
}
}
func TestAddBaseQliksenseConfigs(t *testing.T) {
type args struct {
qliksenseConfig *QliksenseConfig
defaultQliksenseContext string
}
tests := []struct {
name string
args args
want *QliksenseConfig
}{
{
name: "valid case",
args: args{
qliksenseConfig: &QliksenseConfig{},
defaultQliksenseContext: "qlik-default",
},
want: &QliksenseConfig{
CommonConfig: CommonConfig{
ApiVersion: QliksenseConfigApiVersion,
Kind: QliksenseConfigKind,
Metadata: &Metadata{
Name: QliksenseMetadataName,
},
},
Spec: &ContextSpec{
CurrentContext: "qlik-default",
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tt.args.qliksenseConfig.AddBaseQliksenseConfigs(tt.args.defaultQliksenseContext)
if !reflect.DeepEqual(tt.args.qliksenseConfig, tt.want) {
t.Errorf("AddBaseQliksenseConfigs() = %+v, want %+v", tt.args.qliksenseConfig, tt.want)
}
})
}
}

171
pkg/api/encryption.go Normal file
View File

@@ -0,0 +1,171 @@
package api
import (
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"fmt"
"io/ioutil"
"log"
"path/filepath"
)
const (
RSA_KEY_LENGTH = 4096
privateKeyPath = "privKey"
publicKeyPath = "pubKey"
QliksensePublicKey = "qliksensePub"
QliksensePrivateKey = "qliksensePriv"
)
// GenerateAndStoreSecretKeypair generates and stores key pairs
func GenerateAndStoreSecretKeypair(secretsPath string) error {
LogDebugMessage("%s exists", secretsPath)
// creating contexts/qlik-default/secrets/qliksensePub and contexts/qlik-default/secrets/qliksensePriv files
publicKeyFilePath := filepath.Join(secretsPath, QliksensePublicKey)
privateKeyFilePath := filepath.Join(secretsPath, QliksensePrivateKey)
LogDebugMessage("Generating public-private key pair.....")
GenerateRSAEncryptionKeys(publicKeyFilePath, privateKeyFilePath)
LogDebugMessage("Generated public-private key pairs")
return nil
}
// GenerateRSAEncryptionKeys is used to generate a new public-private key pair
func GenerateRSAEncryptionKeys(publicKeyFilePath, privateKeyFilePath string) error {
LogDebugMessage("Generating new RSA key pair")
privateKey, err := rsa.GenerateKey(rand.Reader, RSA_KEY_LENGTH)
if err != nil {
log.Printf("error generating RSA private key: %v\n", err)
return err
}
privateKeyPEM := EncodePrivateKey(privateKey)
if err := writeContentToFile(privateKeyPEM, privateKeyFilePath); err != nil {
return err
}
pubKeyPEM, err2 := EncodePublicKey(&privateKey.PublicKey)
if err2 != nil {
log.Printf("error occurred when encoding public key: %v\n", err2)
return err2
}
if err := writeContentToFile(pubKeyPEM, publicKeyFilePath); err != nil {
return err
}
return nil
}
// writeContentToFile writes keys to a file
func writeContentToFile(keyData []byte, fileName string) error {
err := ioutil.WriteFile(fileName, keyData, 0600)
if err != nil {
log.Printf("error writing to file (%s): %v", fileName, err)
return err
}
LogDebugMessage("Key saved: %s", fileName)
return nil
}
// Encrypt encrypts data with public key
func Encrypt(pt []byte, pub *rsa.PublicKey) ([]byte, error) {
// hash := sha512.New()
// ciphertext, err := rsa.EncryptOAEP(hash, rand.Reader, pub, msg, nil)
ct, err := rsa.EncryptPKCS1v15(rand.Reader, pub, pt)
if err != nil {
log.Println(err)
return nil, err
}
return ct, nil
}
// Decrypt decrypts data with private key
func Decrypt(ct []byte, priv *rsa.PrivateKey) ([]byte, error) {
// hash := sha512.New()
// plaintext, err := rsa.DecryptOAEP(hash, rand.Reader, priv, ciphertext, nil)
pt, err := rsa.DecryptPKCS1v15(rand.Reader, priv, ct)
if err != nil {
log.Println(err)
return nil, err
}
return pt, nil
}
// EncodePrivateKey private key to bytes
func EncodePrivateKey(priv *rsa.PrivateKey) []byte {
privBytes := pem.EncodeToMemory(
&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(priv),
},
)
return privBytes
}
// EncodePublicKey public key to bytes
func EncodePublicKey(pub *rsa.PublicKey) ([]byte, error) {
pubASN1, err := x509.MarshalPKIXPublicKey(pub)
if err != nil {
log.Println(err)
return nil, err
}
pubBytes := pem.EncodeToMemory(&pem.Block{
Type: "RSA PUBLIC KEY",
Bytes: pubASN1,
})
return pubBytes, nil
}
// DecodeToPrivateKey bytes to private key
func DecodeToPrivateKey(priv []byte) (*rsa.PrivateKey, error) {
block, _ := pem.Decode(priv)
enc := x509.IsEncryptedPEMBlock(block)
b := block.Bytes
var err error
if enc {
log.Println("is encrypted pem block")
b, err = x509.DecryptPEMBlock(block, nil)
if err != nil {
log.Println(err)
return nil, err
}
}
key, err := x509.ParsePKCS1PrivateKey(b)
if err != nil {
log.Println(err)
return nil, err
}
return key, nil
}
// DecodeToPublicKey bytes to public key
func DecodeToPublicKey(pub []byte) (*rsa.PublicKey, error) {
block, _ := pem.Decode(pub)
enc := x509.IsEncryptedPEMBlock(block)
b := block.Bytes
var err error
if enc {
log.Println("is encrypted pem block")
b, err = x509.DecryptPEMBlock(block, nil)
if err != nil {
log.Println(err)
return nil, err
}
}
iface, err := x509.ParsePKIXPublicKey(b)
if err != nil {
log.Println(err)
return nil, err
}
key, ok := iface.(*rsa.PublicKey)
if !ok {
err := fmt.Errorf("Unable to decode public key")
log.Println(err)
return nil, err
}
return key, nil
}

129
pkg/api/encryption_test.go Normal file
View File

@@ -0,0 +1,129 @@
package api
import (
"encoding/base64"
"log"
"os"
"testing"
)
func Test_generateRSAEncryptionKeys(t *testing.T) {
tests := []struct {
name string
wantErr bool
}{
{
name: "valid case",
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if err := GenerateAndStoreSecretKeypair(os.TempDir()); (err != nil) != tt.wantErr {
t.Errorf("generateRSAEncryptionKeys() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func Test_encryption_decryption(t *testing.T) {
privKeyBytes := []byte(`-----BEGIN RSA PRIVATE KEY-----
MIIJJwIBAAKCAgEAwUCimKCidbF3UxEHPy8K+hvhklRB9JYhj5sJy0if4lTVibkK
1MrYCykOnmC40pPU9GLY1b8HxAg9tvyRn0YHUxOra6vVQaVcOVJhTM8D18d+lSr3
Lp1yiX+UGT4nzWI9+R1CCbwXrqeQVoZs6QZKynEXMkFI9/wNMOwPOvQFOSTuoEoC
O+zyTyUWEkNbUq825ELUQdIsjgmlWUOONudxsAr7ESRXW9QTHVh6uWmr3VRKZHby
1JdU3I/wjdlGg5M2dDuXy5nQO9w/nYLjJXiw+zzOetZ/+t7/VOkOpNTeJQhwTM1W
F7Y2VLetbi9FHgyzHatrduh07+XEiTbgDf3GIx2bp2p6oh0G3N2zpiLcK/aZj8ro
uWWydfFfsU3MZ4FfJDP8I6b9awxjmKYqIr6hiPQCJaLBED8mwK+I5evIbnKv6E6u
K+BApWA/R7ElragoFYbqQ1VpvntVMtJt9Dy5ZrI+IQARdXD3bb34oh0IPBhClnvv
MUc1cWxDoXEX6oJ4I+LzxE87Zkwnan9qOwengolMVKFwPx1o37qrbmrXID21kKt7
FL6xN4HxHLkItr1fKzdyWDFRHgASTAWfx5BIwvPuUW0vZHkvO80VyV2L63whVhPn
PASmFkbviomrBttYfpr2aGQqF/qR1Nlxe834MFxk1pS9LMa/WnzvFr0gWakCAwEA
AQKCAgARSp9B2N2wejibDiL/3E23I1eDqFZedDB8kPrHXbAwqDaTJCN79spt9TaB
pVXkQaYEV/Pe7EDdoX8kKGU/QxzUqiXkdHOYdBtUZbKfFMbbP9ZrsnR7j0r4UpoF
yDH3hprU93E5PcNAtW2M0GpeT1nR01yn+n908PCdOAIE3GC7RDq1zOl2QzVLL55R
9ATv2Q2oTvJ/ETc7XlGVMx4+e2cIwXLFjeLjLI6pSYlxnarrGuetJZeEviWxto9n
odFVZI6yx8JFTXX8ZTCr/1IjwDDVyhMPmrHI2Lsv9cqBpSpbVe32cUkKxhsGaYjz
GvesQKamOPhco2ATNxPm0yopFlPsGKMfVl0BK0J6BqFh1BvU/SYJmXfnFuUNO3vV
4u2Saa0q1iddxV0rXDwIqUfn+S6rwzK0G7y8bH2yvpB2VwiG3TFPnULep4wsefNq
Fj92kqFBjacGpQLEEslUY0CMgeZ2+NuBQSUTscP3wBRsottMR6YXJtINdvfHBx+e
EcN71z8D00w3mYqIQ7qb4Ml6HOqknunn58g43L9sACMUMTlEBXa9pUnScNYgWBAz
W2q2mH37cIydM2JRZPpA8B4yTHt5ugJmChwyNFM7941arjKrebH+6AzLkofGedOP
zg+vZQuPEXWs+3MBBnkWoyJW3Y0fbQdjsuQTtnd+7iyoxoBroQKCAQEA4dIiFlIS
MDfRhQQWSiDvaw9aneDEJ3uo63ZRH5tm/IynLgtjYgEm/ZxlBCQgqRKLYELBxhu8
SaF0uPK8pmpFJt0mIwSlsdeVhuE2obQeKUCczaqrKeaHS3PdWLjTlwph81BGRkHy
qfqtNylyyMxrdEbnR51EtsWgFq6anTUAui1Q09JMuMNZRMOzDs1F4gExgD22rc0V
c9YQ+jHJRxBGtNKMpMEqc8cvaxBidbItrN9SMTSWog7uYPBuEuaJ6K9vpgyJMOzJ
SYcQEFGqgIqIDCg+ABE4d/4YROMKZ1DV/bJCind9brUHSx6XALsF0nC5c1Q9TnUL
qI2khOwts4KYKwKCAQEA2xRC6Az97Vkdzu7BjLJ1FKmx4S2nEEgVS12ds82U+5Xf
BHKAJnjqlqmmpzzJG+d77IYktz0+mey1QCNkqlm2fhuKs8LZMnpZRf0l8VcoBsUP
/xKz7wfiE7RRFZtLJhPp4hhe43GzX5/JFMWMnC6UykwQbj4t1E/GNM/Suqwvg12M
wktAJ6nqLgfhjQSO4xWo+nPzcbX+fNtrPCZVrBhYXihhcwRRNImWUCGJ6J4LMdPY
Y9Z59qhOvE9cReH/Xw1av46omyiSyAqlgPyZ/kzA2IJSqYCjiQR/2+RD/g13jpcJ
jatXLVZ8MJSL5OTS40G/HHTNNpNHbKKh0GOyxBA3ewKCAQBAn8UXhCcmW2L/YPsL
/b7mcX9qPP+FmRLvR23R0MQ5M/tH5wRq8I969n3GIJykJeVzB8eybQ+GNslTgEvS
iAkAJTubu+G7MkndTqg2wHf9MDtvdA8Fr646Po8yq7oJuHPtkKR7yLWsRUu6xIbP
xgheP0hCq1QVxhqZQyCGKrvpi7xc0gsYuPbcAfFFJCOCmPrUi1SzCkTAYJt9LjA+
wP6rErIjGBCRD4iXaBn1OqdtmH9KC5WsDP/VCBlIGWeQCly2NVIxiSHVg+xp7yUP
IhXq/L05gbQaSsIhPKQmivCiaJg4The8TdwneDqYf+0bmxzHT203/bD3bImPbJNr
ksz/AoIBAEwu4Y1cZzkQUmNRd5D7xecnk6ngfEYXKwCIT3zlMrfCSEl9n77BMaKu
4Dsr0iuX9eosQ7xM2eYhAG6LYEg05lc4MKWOToVVMpI6E+W3Dz47bPKgiF3I+f8s
Jz5CQIG/TwfGvciOE3hfUkec4ua09BzdEqGjkcBQ9XYMBxXPJr6h2379OBQS7FKR
fwfQ2/dv4tElXTTfut2kV8gU9Jnh5Wjo1epvR+XjKpg28YQo4W+0YX1magcyRB8L
4eSTUIC3XiVa8Jr0IwbZXPBb5xkdi7o+p4w2JahSHjxTRqmj+T1mnHXdbXVgq9Mg
9Pzl7cgFZvX4UBx4XtASRf73jITNtt0CggEADH9K+O7FrIOSQly0sMvsRCMtejp3
o+MDh1Q+vEg2kEgNXjS4ZFVljUpM2kg1OdUz7feS4dLXUJiIQ8ZWtZPedcq7wjHd
02he5+s06l0jPifN3tX1ADfXGpXg5R2fbkrIzakkPP5/RO/aDxIUo7qhklNsVTXO
VlGGfWLdk0ekA4upKm02Q1+YOlbIcAicEYYY8K7IffUwnohzKwL9yfuGi1VKTXpE
4fzdegsHI03FSqR7V+LvtBpIupQ7RO4kuBmCEyI4E9FVknchg4te4gO3qwd9y0rJ
Gu7HNIOrwOHzviI7J6Nd/l9MmeKqklHSgJvko/f5TmiXuQQ8xDZf84rcjQ==
-----END RSA PRIVATE KEY-----
`)
publicKeyBytes := []byte(`-----BEGIN RSA PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwUCimKCidbF3UxEHPy8K
+hvhklRB9JYhj5sJy0if4lTVibkK1MrYCykOnmC40pPU9GLY1b8HxAg9tvyRn0YH
UxOra6vVQaVcOVJhTM8D18d+lSr3Lp1yiX+UGT4nzWI9+R1CCbwXrqeQVoZs6QZK
ynEXMkFI9/wNMOwPOvQFOSTuoEoCO+zyTyUWEkNbUq825ELUQdIsjgmlWUOONudx
sAr7ESRXW9QTHVh6uWmr3VRKZHby1JdU3I/wjdlGg5M2dDuXy5nQO9w/nYLjJXiw
+zzOetZ/+t7/VOkOpNTeJQhwTM1WF7Y2VLetbi9FHgyzHatrduh07+XEiTbgDf3G
Ix2bp2p6oh0G3N2zpiLcK/aZj8rouWWydfFfsU3MZ4FfJDP8I6b9awxjmKYqIr6h
iPQCJaLBED8mwK+I5evIbnKv6E6uK+BApWA/R7ElragoFYbqQ1VpvntVMtJt9Dy5
ZrI+IQARdXD3bb34oh0IPBhClnvvMUc1cWxDoXEX6oJ4I+LzxE87Zkwnan9qOwen
golMVKFwPx1o37qrbmrXID21kKt7FL6xN4HxHLkItr1fKzdyWDFRHgASTAWfx5BI
wvPuUW0vZHkvO80VyV2L63whVhPnPASmFkbviomrBttYfpr2aGQqF/qR1Nlxe834
MFxk1pS9LMa/WnzvFr0gWakCAwEAAQ==
-----END RSA PUBLIC KEY-----
`)
origStr := "Value1234"
pubKey, err := DecodeToPublicKey(publicKeyBytes)
if err != nil {
t.Error(err)
t.FailNow()
}
privKey, err := DecodeToPrivateKey(privKeyBytes)
if err != nil {
t.Error(err)
t.FailNow()
}
encData, err := Encrypt([]byte(origStr), pubKey)
if err != nil {
t.Error(err)
t.FailNow()
}
encDataStr := base64.StdEncoding.EncodeToString(encData)
log.Printf("encrypted data: %s\n", encDataStr)
dec, _ := base64.StdEncoding.DecodeString(encDataStr)
data, err := Decrypt(dec, privKey)
if err != nil {
t.Error(err)
t.FailNow()
}
if string(data) != origStr {
t.Error("original string and decrypted string don't match")
t.FailNow()
}
log.Printf("decrypted data: %s\n", data)
}

43
pkg/api/kubectl.go Normal file
View File

@@ -0,0 +1,43 @@
package api
import (
"fmt"
"io/ioutil"
"os"
"os/exec"
)
func KubectlApply(manifests string) error {
return kubectlOperation(manifests, "apply")
}
func KubectlDelete(manifests string) error {
return kubectlOperation(manifests, "delete")
}
func kubectlOperation(manifests string, oprName string) error {
tempYaml, err := ioutil.TempFile("", "")
if err != nil {
fmt.Println("cannot create file ", err)
return err
}
tempYaml.WriteString(manifests)
var cmd *exec.Cmd
if oprName == "apply" {
cmd = exec.Command("kubectl", oprName, "-f", tempYaml.Name(), "--validate=false")
} else {
cmd = exec.Command("kubectl", oprName, "-f", tempYaml.Name())
}
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err = cmd.Run()
if err != nil {
fmt.Printf("kubectl apply failed with %s\n", err)
fmt.Println("temp CRD file: " + tempYaml.Name())
return err
}
os.Remove(tempYaml.Name())
return nil
}

48
pkg/api/types.go Normal file
View File

@@ -0,0 +1,48 @@
package api
import "github.com/qlik-oss/k-apis/pkg/config"
// CommonConfig is exported
type CommonConfig struct {
ApiVersion string `json:"apiVersion" yaml:"apiVersion"`
Kind string `json:"kind" yaml:"kind"`
Metadata *Metadata `json:"metadata" yaml:"metadata"`
}
// QliksenseConfig is exported
type QliksenseConfig struct {
CommonConfig `json:",inline" yaml:",inline"`
Spec *ContextSpec `json:"spec" yaml:"spec"`
QliksenseHomePath string `json:"-" yaml:"-"`
}
// QliksenseCR is exported
type QliksenseCR struct {
CommonConfig `json:",inline" yaml:",inline"`
Spec *config.CRSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
}
// ContextSpec is exported
type ContextSpec struct {
Contexts []Context `json:"contexts" yaml:"contexts"`
CurrentContext string `json:"currentContext" yaml:"currentContext"`
}
// Context is exported
type Context struct {
Name string `json:"name,omitempty" yaml:"name,omitempty"`
CrFile string `json:"crFile,omitempty" yaml:"crFile,omitempty"`
}
// Metadata is exported
type Metadata struct {
Name string `json:"name,omitempty" yaml:"name,omitempty"`
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
}
// ServiceKeyValue holds the combination of service, key and value
type ServiceKeyValue struct {
SvcName string
Key string
Value string
}

88
pkg/api/utils.go Normal file
View File

@@ -0,0 +1,88 @@
package api
import (
"fmt"
"io/ioutil"
"log"
"os"
"regexp"
)
func checkExists(filename string, isFile bool) os.FileInfo {
info, err := os.Stat(filename)
if os.IsNotExist(err) {
if isFile {
LogDebugMessage("File does not exist")
} else {
LogDebugMessage("Dir does not exist")
}
return nil
}
LogDebugMessage("File exists")
return info
}
// FileExists checks if a file exists
func FileExists(filename string) bool {
if fe := checkExists(filename, true); fe != nil && !fe.IsDir() {
return true
}
return false
}
// DirExists checks if a directory exists
func DirExists(dirname string) bool {
if fe := checkExists(dirname, false); fe != nil && fe.IsDir() {
return true
}
return false
}
// LogDebugMessage logs a debug message
func LogDebugMessage(strMessage string, args ...interface{}) {
if os.Getenv("QLIKSENSE_DEBUG") == "true" {
log.Printf(strMessage, args...)
}
}
// ReadKeys reads key file from disk
func ReadKeys(keyFile string) ([]byte, error) {
keybyteArray, err := ioutil.ReadFile(keyFile)
if err != nil {
err = fmt.Errorf("There was an error reading from file: %s, %v", keyFile, err)
log.Println(err)
return nil, err
} else {
LogDebugMessage("Read key as byte[]: %+v", keybyteArray)
}
return keybyteArray, nil
}
// ProcessConfigArgs processes args and returns an service, key, value slice
func ProcessConfigArgs(args []string) ([]*ServiceKeyValue, error) {
// prepare received args
// split args[0] into key and value
if len(args) == 0 {
err := fmt.Errorf("No args were provided. Please provide args to configure the current context")
log.Println(err)
return nil, err
}
resultSvcKV := make([]*ServiceKeyValue, len(args))
re1 := regexp.MustCompile(`(\w{1,})\[name=(\w{1,})\]=("*[\w\-_/:0-9]+"*)`)
for i, arg := range args {
LogDebugMessage("Arg received: %s", arg)
result := re1.FindStringSubmatch(arg)
// check if result array's length is == 4 (index 0 - is the full match & indices 1,2,3- are the fields we need)
if len(result) != 4 {
err := fmt.Errorf("Please provide valid args for this command")
log.Println(err)
return nil, err
}
resultSvcKV[i] = &ServiceKeyValue{
SvcName: result[1],
Key: result[2],
Value: result[3],
}
}
return resultSvcKV, nil
}

253
pkg/qliksense/about.go Normal file
View File

@@ -0,0 +1,253 @@
package qliksense
import (
"bytes"
"errors"
"io"
"io/ioutil"
"os"
"path"
"path/filepath"
"reflect"
"sort"
kapis_git "github.com/qlik-oss/k-apis/pkg/git"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
"gopkg.in/yaml.v2"
)
type patch struct {
Target struct {
Kind string `yaml:"kind"`
LabelSelector string `yaml:"labelSelector"`
} `yaml:"target"`
Patch string `yaml:"patch"`
}
type selectivePatch struct {
APIVersion string `yaml:"apiVersion"`
Metadata struct {
Name string `yaml:"name"`
} `yaml:"metadata"`
Enabled bool `yaml:"enabled"`
Patches []patch `yaml:"patches"`
}
type helmChart struct {
APIVersion string `yaml:"apiVersion"`
Kind string `yaml:"kind"`
Metadata struct {
Name string `yaml:"name"`
} `yaml:"metadata"`
ReleaseNamespace string `yaml:"releaseNamespace"`
ChartHome string `yaml:"chartHome"`
ChartRepo string `yaml:"chartRepo"`
ChartName string `yaml:"chartName"`
ChartVersion string `yaml:"chartVersion"`
}
type VersionOutput struct {
QliksenseVersion string `yaml:"qlikSenseVersion"`
Images []string `yaml:"images"`
}
type nullWriter struct {
}
func (nw *nullWriter) Write(p []byte) (n int, err error) {
return len(p), nil
}
const (
defaultProfile = "docker-desktop"
defaultGitUrl = "https://github.com/qlik-oss/qliksense-k8s"
)
func (q *Qliksense) About(gitRef, profile string) (*VersionOutput, error) {
configDirectory, isTemporary, profile, err := q.getConfigDirectory(defaultGitUrl, gitRef, profile)
if err != nil {
return nil, err
}
if isTemporary {
defer os.RemoveAll(configDirectory)
}
return q.AboutDir(configDirectory, profile)
}
func (q *Qliksense) AboutDir(configDirectory, profile string) (*VersionOutput, error) {
chartVersion, err := getChartVersion(filepath.Join(configDirectory, "transformers", "qseokversion.yaml"), "qliksense")
if err != nil {
return nil, err
}
kuzManifest, err := executeKustomizeBuild(filepath.Join(configDirectory, "manifests", profile))
if err != nil {
return nil, err
}
images, err := getImageList(kuzManifest)
if err != nil {
return nil, err
}
return &VersionOutput{
QliksenseVersion: chartVersion,
Images: images,
}, nil
}
func (q *Qliksense) getConfigDirectory(gitUrl, gitRef, profileEntered string) (dir string, isTemporary bool, profile string, err error) {
profile = profileEntered
if profile == "" {
profile = defaultProfile
}
if gitRef != "" {
if dir, err = downloadFromGitRepoToTmpDir(gitUrl, gitRef); err != nil {
return "", false, "", err
} else {
return dir, true, profile, nil
}
}
var exists bool
exists, dir, err = configExistsInCurrentDirectory(profile)
if err != nil {
return "", false, "", err
} else if exists {
return dir, false, profile, nil
}
var profileFromCurrentContext string
exists, dir, profileFromCurrentContext, err = q.ConfigExistsInCurrentContext()
if err != nil {
return "", false, "", err
} else if exists {
if profileEntered == "" {
profile = profileFromCurrentContext
}
return dir, false, profile, nil
}
if dir, err = downloadFromGitRepoToTmpDir(gitUrl, "master"); err != nil {
return "", false, "", err
} else {
return dir, true, profile, nil
}
}
func downloadFromGitRepoToTmpDir(gitUrl, gitRef string) (string, error) {
if tmpDir, err := ioutil.TempDir("", ""); err != nil {
return "", err
} else {
downloadPath := path.Join(tmpDir, "repo")
if err := downloadFromGitRepo(gitUrl, gitRef, downloadPath); err != nil {
_ = os.RemoveAll(tmpDir)
return "", err
} else {
return downloadPath, nil
}
}
}
func downloadFromGitRepo(gitUrl, gitRef, destDir string) error {
if repo, err := kapis_git.CloneRepository(destDir, gitUrl, nil); err != nil {
return err
} else {
return kapis_git.Checkout(repo, gitRef, "", nil)
}
}
func configExistsInCurrentDirectory(profile string) (exists bool, currentDirectory string, err error) {
currentDirectory, err = os.Getwd()
if err == nil {
info, err := os.Stat(path.Join(currentDirectory, "manifests", profile))
if err == nil && info.IsDir() {
exists = true
}
}
return exists, currentDirectory, err
}
func (q *Qliksense) ConfigExistsInCurrentContext() (exists bool, directory string, profile string, err error) {
qConfig := qapi.NewQConfig(q.QliksenseHome)
if currentCr, err := qConfig.GetCurrentCR(); err != nil {
return false, "", "", err
} else if currentCr.Spec.ManifestsRoot == "" {
return false, "", "", nil
} else if path.Base(currentCr.Spec.ManifestsRoot) != "manifests" {
return false, "", "", errors.New("currentCr.Spec.ManifestsRoot path should terminate with manifests/")
} else {
return true, path.Join(currentCr.Spec.ManifestsRoot, "../"), currentCr.Spec.Profile, nil
}
}
func getImageList(yamlContent []byte) ([]string, error) {
decoder := yaml.NewDecoder(bytes.NewReader(yamlContent))
var resource map[string]interface{}
imageMap := make(map[string]bool)
for {
err := decoder.Decode(&resource)
if err != nil {
if err != io.EOF {
return nil, err
}
break
}
traverseYamlDecodedMapRecursively(reflect.ValueOf(resource), []string{}, func(path []string, val interface{}) {
if len(path) >= 2 && path[len(path)-1] == "image" &&
(path[len(path)-2] == "containers" || path[len(path)-2] == "initContainers") {
if image, ok := val.(string); ok {
imageMap[image] = true
}
}
})
}
var sortedImageList []string
for image, _ := range imageMap {
sortedImageList = append(sortedImageList, image)
}
sort.Strings(sortedImageList)
return sortedImageList, nil
}
func traverseYamlDecodedMapRecursively(val reflect.Value, path []string, visitorFunc func(path []string, val interface{})) {
kind := val.Kind()
switch kind {
case reflect.Interface:
traverseYamlDecodedMapRecursively(val.Elem(), path, visitorFunc)
case reflect.Slice:
for i := 0; i < val.Len(); i++ {
traverseYamlDecodedMapRecursively(val.Index(i), path, visitorFunc)
}
case reflect.Map:
for _, key := range val.MapKeys() {
traverseYamlDecodedMapRecursively(val.MapIndex(key), append(path, key.Interface().(string)), visitorFunc)
}
default:
if kind != reflect.Invalid {
visitorFunc(path, val.Interface())
}
}
}
func getChartVersion(versionFile, chartName string) (string, error) {
var patchInst patch
var selPatch selectivePatch
var chart helmChart
if bytes, err := ioutil.ReadFile(versionFile); err != nil {
return "", err
} else if err = yaml.Unmarshal(bytes, &selPatch); err != nil {
return "", err
}
for _, patchInst = range selPatch.Patches {
if err := yaml.Unmarshal([]byte(patchInst.Patch), &chart); err == nil {
if chart.ChartName == chartName {
return chart.ChartVersion, nil
}
}
}
return "", nil
}

579
pkg/qliksense/about_test.go Normal file
View File

@@ -0,0 +1,579 @@
package qliksense
import (
"fmt"
"io/ioutil"
"os"
"path"
"reflect"
"testing"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
)
func Test_About_getImageList(t *testing.T) {
var testCases = []struct {
name string
k8sYaml string
expectedImages []string
}{
{
name: "base",
k8sYaml: `
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: nginx-deployment-1
spec:
selector:
matchLabels:
app: nginx
replicas: 2 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
---
apiVersion: v1
kind: Secret
metadata:
creationTimestamp: 2018-11-15T20:46:46Z
name: mysecret
namespace: default
resourceVersion: "7579"
uid: 91460ecb-e917-11e8-98f2-025000000001
type: Opaque
data:
username: YWRtaW5pc3RyYXRvcg==
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: nginx-deployment-2
spec:
selector:
matchLabels:
app: nginx
replicas: 2 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx # has to match .spec.template.metadata.labels
serviceName: "nginx"
replicas: 3 # by default is 1
template:
metadata:
labels:
app: nginx # has to match .spec.selector.matchLabels
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: k8s.gcr.io/nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "my-storage-class"
resources:
requests:
storage: 1Gi
---
apiVersion: batch/v1
kind: Job
metadata:
name: pi
spec:
template:
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
backoffLimit: 4
---
apiVersion: v1
kind: Pod
metadata:
name: init-demo
spec:
containers:
- name: nginx
image: nginx
env:
- name: FOO
value: null
ports:
- containerPort: 80
volumeMounts:
- name: workdir
mountPath: /usr/share/nginx/html
# These containers are run during pod initialization
initContainers:
- name: install
image: busybox
command:
- wget
- "-O"
- "/work-dir/index.html"
- http://kubernetes.io
volumeMounts:
- name: workdir
mountPath: "/work-dir"
dnsPolicy: Default
volumes:
- name: workdir
emptyDir: {}
`,
expectedImages: []string{"busybox", "k8s.gcr.io/nginx-slim:0.8", "nginx", "nginx:1.7.9", "perl"},
},
{
name: "works for custom resources and CronJobs",
k8sYaml: `
apiVersion: "qixmanager.qlik.com/v1"
kind: "Engine"
metadata:
name: release-name-engine-reload
spec:
metadata:
labels:
qix-engine: qix-engine
annotations:
prometheus.io/scrape: "true"
workloadType: "reload"
podSpec:
imagePullSecrets:
- name: artifactory-docker-secret
dnsConfig:
options:
- name: timeout
value: "1"
- name: single-request-reopen
containers:
- name: engine-reload
image: another-engine
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox2
args:
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
restartPolicy: OnFailure
`,
expectedImages: []string{"another-engine", "busybox2"},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
images, err := getImageList([]byte(testCase.k8sYaml))
if err != nil {
t.Fatalf("unexpected error: %v\n", err)
}
if !reflect.DeepEqual(images, testCase.expectedImages) {
t.Fatalf("expected %v, but got: %v\n", testCase.expectedImages, images)
}
})
}
}
func Test_About_getConfigDirectory(t *testing.T) {
verifyAsdBranch := func(configDir string) (ok bool, reason string) {
tmpDir := os.TempDir()
if (path.Clean(path.Dir(path.Dir(configDir))) != path.Clean(tmpDir)) || (path.Base(configDir) != "repo") {
return false, fmt.Sprintf("expected config directory path: %v to be under: %v and terminate with repo", configDir, tmpDir)
}
if info, err := os.Stat(path.Join(configDir, "asdczxc")); err != nil || !info.Mode().IsRegular() {
return false, fmt.Sprintf(`expected to find file: "asdczxc" under directory: %v`, configDir)
}
return true, ""
}
verifyMasterBranch := func(configDir string) (ok bool, reason string) {
tmpDir := os.TempDir()
if (path.Clean(path.Dir(path.Dir(configDir))) != path.Clean(tmpDir)) || (path.Base(configDir) != "repo") {
return false, fmt.Sprintf("expected config directory path: %v to be under: %v and terminate with repo", configDir, tmpDir)
}
if _, err := os.Stat(path.Join(configDir, "asdczxc")); err == nil || !os.IsNotExist(err) {
return false, fmt.Sprintf(`expected to NOT find file: "asdczxc"" under directory: %v`, configDir)
}
if info, err := os.Stat(path.Join(configDir, "sad")); err != nil || !info.Mode().IsRegular() {
return false, fmt.Sprintf(`expected to find file: "sad"" under directory: %v`, configDir)
}
return true, ""
}
var testCases = []struct {
name string
setup func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string)
verify func(q *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error)
cleanup func(q *Qliksense, configDir string) error
}{
{
name: "config in current directory and default profile",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
currentDirectory, err := os.Getwd()
if err != nil {
t.Fatalf("error obtaining current directory: %v\n", err)
}
defaultProfilePath := path.Join(currentDirectory, "manifests", "docker-desktop")
err = os.MkdirAll(defaultProfilePath, os.ModePerm)
if err != nil {
t.Fatalf("error making path: %v, err: %v\n", defaultProfilePath, err)
}
return &Qliksense{}, "no-clone-for-you", "", ""
},
verify: func(_ *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
currentDirectory, err := os.Getwd()
if err != nil {
return false, "", err
}
if configDir != currentDirectory {
return false, fmt.Sprintf("expected config directory: %v to equal current directory: %v", configDir, currentDirectory), nil
}
if isTemporary {
return false, "expected isTemporary to be false", nil
}
if profile != "docker-desktop" {
return false, fmt.Sprintf("expected profile to be: docker-desktop, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(_ *Qliksense, configDir string) error {
if currentDirectory, err := os.Getwd(); err != nil {
return err
} else if err := os.RemoveAll(path.Join(currentDirectory, "manifests")); err != nil {
return err
}
return nil
},
},
{
name: "config in current directory and profile specified",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
currentDirectory, err := os.Getwd()
if err != nil {
t.Fatalf("error obtaining current directory: %v\n", err)
}
profileEntered = "foo"
defaultProfilePath := path.Join(currentDirectory, "manifests", profileEntered)
err = os.MkdirAll(defaultProfilePath, os.ModePerm)
if err != nil {
t.Fatalf("error making path: %v, err: %v\n", defaultProfilePath, err)
}
return &Qliksense{}, "no-clone-for-you", "", profileEntered
},
verify: func(_ *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
currentDirectory, err := os.Getwd()
if err != nil {
return false, "", err
}
if configDir != currentDirectory {
return false, fmt.Sprintf("expected config directory: %v to equal current directory: %v", configDir, currentDirectory), nil
}
if isTemporary {
return false, "expected isTemporary to be false", nil
}
if profile != "foo" {
return false, fmt.Sprintf("expected profile to be: foo, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(_ *Qliksense, configDir string) error {
if currentDirectory, err := os.Getwd(); err != nil {
return err
} else if err := os.RemoveAll(path.Join(currentDirectory, "manifests")); err != nil {
return err
}
return nil
},
},
{
name: "config downloaded from git based on specific git ref and default profile used",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
return &Qliksense{}, "https://github.com/test/HelloWorld", "asd", ""
},
verify: func(_ *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
ok, reason = verifyAsdBranch(configDir)
if !ok {
return ok, reason, nil
}
if !isTemporary {
return false, "expected isTemporary to be true", nil
}
if profile != "docker-desktop" {
return false, fmt.Sprintf("expected profile to be: docker-desktop, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(_ *Qliksense, configDir string) error {
tmpDir := os.TempDir()
if path.Clean(path.Dir(path.Dir(configDir))) == path.Clean(tmpDir) && path.Base(configDir) == "repo" {
tmpTmpDir := path.Dir(configDir)
if err := os.RemoveAll(tmpTmpDir); err != nil {
return err
}
}
return nil
},
},
{
name: "config downloaded from git based on specific git ref and profile specified",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
return &Qliksense{}, "https://github.com/test/HelloWorld", "asd", "foo"
},
verify: func(_ *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
ok, reason = verifyAsdBranch(configDir)
if !ok {
return ok, reason, nil
}
if !isTemporary {
return false, "expected isTemporary to be true", nil
}
if profile != "foo" {
return false, fmt.Sprintf("expected profile to be: foo, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(_ *Qliksense, configDir string) error {
tmpDir := os.TempDir()
if path.Clean(path.Dir(path.Dir(configDir))) == path.Clean(tmpDir) && path.Base(configDir) == "repo" {
tmpTmpDir := path.Dir(configDir)
if err := os.RemoveAll(tmpTmpDir); err != nil {
return err
}
}
return nil
},
},
{
name: "config downloaded from git from master branch and default profile used",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
if qliksenseHome, err := ioutil.TempDir("", ""); err != nil {
t.Fatalf("error creating tmp qliksenseHome directory: %v\n", err)
return nil, "", "", ""
} else {
q := &Qliksense{QliksenseHome: qliksenseHome}
if err := q.SetUpQliksenseDefaultContext(); err != nil {
t.Fatalf("error setting up default context in the tmp dir: %v\n", err)
return nil, "", "", ""
} else {
return q, "https://github.com/test/HelloWorld", "", ""
}
}
},
verify: func(_ *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
ok, reason = verifyMasterBranch(configDir)
if !ok {
return ok, reason, nil
}
if !isTemporary {
return false, "expected isTemporary to be true", nil
}
if profile != "docker-desktop" {
return false, fmt.Sprintf("expected profile to be: docker-desktop, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(q *Qliksense, configDir string) error {
tmpDir := os.TempDir()
if path.Clean(path.Dir(path.Dir(configDir))) == path.Clean(tmpDir) && path.Base(configDir) == "repo" {
tmpTmpDir := path.Dir(configDir)
if err := os.RemoveAll(tmpTmpDir); err != nil {
return err
}
}
if err := os.RemoveAll(q.QliksenseHome); err != nil {
return err
}
return nil
},
},
{
name: "config downloaded from git from master branch and profile specified",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
if qliksenseHome, err := ioutil.TempDir("", ""); err != nil {
t.Fatalf("error creating tmp qliksenseHome directory: %v\n", err)
return nil, "", "", ""
} else {
q := &Qliksense{QliksenseHome: qliksenseHome}
if err := q.SetUpQliksenseDefaultContext(); err != nil {
t.Fatalf("error setting up default context in the tmp dir: %v\n", err)
return nil, "", "", ""
} else {
return q, "https://github.com/test/HelloWorld", "", "foo"
}
}
},
verify: func(_ *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
ok, reason = verifyMasterBranch(configDir)
if !ok {
return ok, reason, nil
}
if !isTemporary {
return false, "expected isTemporary to be true", nil
}
if profile != "foo" {
return false, fmt.Sprintf("expected profile to be: foo, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(q *Qliksense, configDir string) error {
tmpDir := os.TempDir()
if path.Clean(path.Dir(path.Dir(configDir))) == path.Clean(tmpDir) && path.Base(configDir) == "repo" {
tmpTmpDir := path.Dir(configDir)
if err := os.RemoveAll(tmpTmpDir); err != nil {
return err
}
}
if err := os.RemoveAll(q.QliksenseHome); err != nil {
return err
}
return nil
},
},
{
name: "config loaded from current context",
setup: func(t *testing.T) (q *Qliksense, gitUrl, gitRef, profileEntered string) {
if qliksenseHome, err := ioutil.TempDir("", ""); err != nil {
t.Fatalf("error creating tmp qliksenseHome directory: %v\n", err)
return nil, "", "", ""
} else {
q := &Qliksense{QliksenseHome: qliksenseHome}
if err := q.SetUpQliksenseDefaultContext(); err != nil {
t.Fatalf("error setting up default context in the tmp dir: %v\n", err)
return nil, "", "", ""
} else if err := q.FetchQK8s("master"); err != nil {
t.Fatalf("error fetching master config to the tmp dir: %v\n", err)
return nil, "", "", ""
} else {
return q, "no-git-clone-for-you", "", ""
}
}
},
verify: func(q *Qliksense, configDir string, isTemporary bool, profile string) (ok bool, reason string, err error) {
qConfig := qapi.NewQConfig(q.QliksenseHome)
expectedConfigDir := qConfig.BuildRepoPath("master")
if configDir != expectedConfigDir {
return false, fmt.Sprintf("expected configDir to be %v", expectedConfigDir), nil
}
if isTemporary {
return false, "expected isTemporary to be false", nil
}
if profile != "docker-desktop" {
return false, fmt.Sprintf("expected profile to be: docker-desktop, but it was: %v", profile), nil
}
return true, "", nil
},
cleanup: func(q *Qliksense, configDir string) error {
if err := os.RemoveAll(q.QliksenseHome); err != nil {
return err
}
return nil
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
q, gitUrl, gitRef, profileEntered := testCase.setup(t)
configDirectory, isTemporary, profile, err := q.getConfigDirectory(gitUrl, gitRef, profileEntered)
if err != nil {
t.Fatalf("unexpected error: %v\n", err)
}
if ok, reason, err := testCase.verify(q, configDirectory, isTemporary, profile); err != nil {
t.Fatalf("unexpected verification error: %v\n", err)
} else if !ok {
t.Fatalf("verification failed: %v\n", reason)
} else if err := testCase.cleanup(q, configDirectory); err != nil {
t.Fatalf("unexpected cleanup error: %v\n", err)
}
})
}
}

103
pkg/qliksense/config.go Normal file
View File

@@ -0,0 +1,103 @@
package qliksense
import (
"fmt"
"os"
"path"
"path/filepath"
"github.com/mitchellh/go-homedir"
"github.com/qlik-oss/k-apis/pkg/cr"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
)
const (
Q_INIT_CRD_PATH = "manifests/base/manifests/qliksense-init"
)
func (q *Qliksense) ConfigApplyQK8s() error {
//get the current context cr
qConfig := qapi.NewQConfig(q.QliksenseHome)
qcr, err := qConfig.GetCurrentCR()
if err != nil {
fmt.Println("cannot get the current-context cr", err)
return err
}
return q.applyConfigToK8s(qcr, "")
}
func (q *Qliksense) applyConfigToK8s(qcr *qapi.QliksenseCR, cmd string) error {
// apply qliksense-init crd first
mroot := qcr.Spec.GetManifestsRoot()
qInitMsPath := filepath.Join(mroot, Q_INIT_CRD_PATH)
if qcr.Spec.RotateKeys != "None" {
if err := os.Unsetenv("EJSON_KEY"); err != nil {
fmt.Printf("error unsetting EJSON_KEY environment variable: %v\n", err)
return err
}
if err := os.Setenv("EJSON_KEYDIR", q.QliksenseEjsonKeyDir); err != nil {
fmt.Printf("error setting EJSON_KEYDIR environment variable: %v\n", err)
return err
}
}
if cmd != "upgrade" {
qInitByte, err := executeKustomizeBuild(qInitMsPath)
if err != nil {
fmt.Println("cannot generate crds for qliksense-init", err)
return err
}
if err = qapi.KubectlApply(string(qInitByte)); err != nil {
return err
}
}
userHomeDir, err := homedir.Dir()
if err != nil {
fmt.Printf(`error fetching user's home directory: %v\n`, err)
return err
}
// generate patches
cr.GeneratePatches(qcr.Spec, path.Join(userHomeDir, ".kube", "config"))
// apply generated manifests
profilePath := filepath.Join(qcr.Spec.ManifestsRoot, qcr.Spec.Profile)
mByte, err := executeKustomizeBuild(profilePath)
if err != nil {
fmt.Println("cannot generate manifests for "+profilePath, err)
return err
}
if err = qapi.KubectlApply(string(mByte)); err != nil {
return err
}
return nil
}
func (q *Qliksense) ConfigViewCR() error {
//get the current context cr
r, err := q.getCurrentCRString()
if err != nil {
return err
}
fmt.Println(r)
return nil
}
func (q *Qliksense) getCurrentCRString() (string, error) {
qConfig := qapi.NewQConfig(q.QliksenseHome)
return q.getCRString(qConfig.Spec.CurrentContext)
}
func (q *Qliksense) getCRString(contextName string) (string, error) {
qConfig := qapi.NewQConfig(q.QliksenseHome)
qcr, err := qConfig.GetCR(contextName)
if err != nil {
fmt.Println("cannot get the context cr", err)
return "", err
}
return qcr.GetString()
}

View File

@@ -0,0 +1,354 @@
package qliksense
import (
"crypto/rsa"
"fmt"
"log"
"os"
"path/filepath"
"strings"
"text/tabwriter"
b64 "encoding/base64"
ansi "github.com/mattn/go-colorable"
"github.com/qlik-oss/sense-installer/pkg/api"
"github.com/ttacon/chalk"
)
const (
// Below are some constants to support qliksense context setup
QliksenseConfigHome = "/.qliksense"
QliksenseConfigContextHome = "/.qliksense/contexts"
QliksenseConfigFile = "config.yaml"
QliksenseContextsDir = "contexts"
DefaultQliksenseContext = "qlik-default"
MaxContextNameLength = 17
QliksenseSecretsDir = "secrets"
)
// SetSecrets - set-secrets <key>=<value> commands
func (q *Qliksense) SetSecrets(args []string, isK8sSecret bool) error {
api.LogDebugMessage("Args received: %v\n", args)
api.LogDebugMessage("isK8sSecret: %v\n", isK8sSecret)
// retieve current context from config.yaml
qliksenseCR, qliksenseContextsFile, err := retrieveCurrentContextInfo(q)
if err != nil {
return err
}
secretKeyPairLocation := filepath.Join(q.QliksenseHome, QliksenseSecretsDir, QliksenseContextsDir, qliksenseCR.Metadata.Name, QliksenseSecretsDir)
api.LogDebugMessage("SecretKeyLocation to store key pair: %s", secretKeyPairLocation)
if os.Getenv("QLIKSENSE_KEY_LOCATION") != "" {
api.LogDebugMessage("Env variable: QLIKSENSE_KEY_LOCATION= %s", os.Getenv("QLIKSENSE_KEY_LOCATION"))
secretKeyPairLocation = os.Getenv("QLIKSENSE_KEY_LOCATION")
}
// Env var: QLIKSENSE_KEY_LOCATION hasn't been set, so dropping key pair in the location:
// /.qliksense/secrets/contexts/<current-context>/secrets/
api.LogDebugMessage("Using default location to store keys: %s", secretKeyPairLocation)
publicKeyFilePath := filepath.Join(secretKeyPairLocation, api.QliksensePublicKey)
privateKeyFilePath := filepath.Join(secretKeyPairLocation, api.QliksensePrivateKey)
// try to create the dir if it doesn't exist
if !api.FileExists(publicKeyFilePath) || !api.FileExists(privateKeyFilePath) {
api.LogDebugMessage("Qliksense secretKeyLocation dir does not exist, creating it now: %s", secretKeyPairLocation)
if err := os.MkdirAll(secretKeyPairLocation, os.ModePerm); err != nil {
err = fmt.Errorf("Not able to create %s dir: %v", secretKeyPairLocation, err)
log.Println(err)
return err
}
// generating and storing key-pair
err1 := api.GenerateAndStoreSecretKeypair(secretKeyPairLocation)
if err1 != nil {
err1 = fmt.Errorf("Not able to generate and store key pair for encryption")
log.Println(err1)
return err1
}
}
var rsaPublicKey *rsa.PublicKey
var e1 error
// Read Public Key
publicKeybytes, err2 := api.ReadKeys(publicKeyFilePath)
if err2 != nil {
api.LogDebugMessage("Not able to read public key")
return err2
}
// api.LogDebugMessage("PublicKey: %+v", publicKeybytes)
// convert []byte into RSA public key object
rsaPublicKey, e1 = api.DecodeToPublicKey(publicKeybytes)
if e1 != nil {
return e1
}
resultArgs, err := api.ProcessConfigArgs(args)
if err != nil {
return err
}
for _, ra := range resultArgs {
// Metadata name in qliksense CR is the name of the current context
api.LogDebugMessage("Trying to retreive current context: %+v ----- %s", qliksenseCR.Metadata.Name, qliksenseContextsFile)
// encrypt value with RSA key pair
valueBytes := []byte(ra.Value)
cipherText, e2 := api.Encrypt(valueBytes, rsaPublicKey)
if e2 != nil {
return e2
}
api.LogDebugMessage("Returned cipher text: %s", b64.StdEncoding.EncodeToString(cipherText))
if isK8sSecret {
// TODO: store the key value in k8s as secret
api.LogDebugMessage("Need to create a Kubernetes secret")
}
// TODO: Extend AddToSecrets to support adding k8s key ref. (OR) create a separate method to support that
// store the encrypted value in the file (OR)
// TODO: k8s secret name in the file
qliksenseCR.Spec.AddToSecrets(ra.SvcName, ra.Key, string(cipherText))
}
// write modified content into context.yaml
api.WriteToFile(&qliksenseCR, qliksenseContextsFile)
return nil
}
// SetConfigs - set-configs <key>=<value> commands
func (q *Qliksense) SetConfigs(args []string) error {
// retieve current context from config.yaml
qliksenseCR, qliksenseContextsFile, err := retrieveCurrentContextInfo(q)
if err != nil {
return err
}
resultArgs, err := api.ProcessConfigArgs(args)
if err != nil {
return err
}
for _, ra := range resultArgs {
qliksenseCR.Spec.AddToConfigs(ra.SvcName, ra.Key, ra.Value)
}
// write modified content into context.yaml
api.WriteToFile(&qliksenseCR, qliksenseContextsFile)
return nil
}
func retrieveCurrentContextInfo(q *Qliksense) (api.QliksenseCR, string, error) {
var qliksenseConfig api.QliksenseConfig
qliksenseConfigFile := filepath.Join(q.QliksenseHome, QliksenseConfigFile)
api.ReadFromFile(&qliksenseConfig, qliksenseConfigFile)
currentContext := qliksenseConfig.Spec.CurrentContext
api.LogDebugMessage("Current-context from config.yaml: %s", currentContext)
if currentContext == "" {
// current-context is empty
err := fmt.Errorf(`Please run the "qliksense config set-context <context-name>" first before viewing the current context info`)
log.Println(err)
return api.QliksenseCR{}, "", err
}
// read the context.yaml file
var qliksenseCR api.QliksenseCR
if currentContext == "" {
// current-context is empty
err := fmt.Errorf(`Please run the "qliksense config set-context <context-name>" first before viewing the current context info`)
log.Println(err)
return api.QliksenseCR{}, "", err
}
qliksenseContextsFile := filepath.Join(q.QliksenseHome, QliksenseContextsDir, currentContext, currentContext+".yaml")
if !api.FileExists(qliksenseContextsFile) {
err := fmt.Errorf("Context file does not exist.\nPlease try re-running `qliksense config set-context <context-name>` and then `qliksense config view` again")
log.Println(err)
return api.QliksenseCR{}, "", err
}
api.ReadFromFile(&qliksenseCR, qliksenseContextsFile)
api.LogDebugMessage("Read context file: %s, Read QliksenseCR: %v", qliksenseContextsFile, qliksenseCR)
return qliksenseCR, qliksenseContextsFile, nil
}
// SetOtherConfigs - set profile/namespace/storageclassname/git.repository commands
func (q *Qliksense) SetOtherConfigs(args []string) error {
// retieve current context from config.yaml
qliksenseCR, qliksenseContextsFile, err := retrieveCurrentContextInfo(q)
if err != nil {
return err
}
// modify appropriate fields
if len(args) == 0 {
err := fmt.Errorf("No args were provided. Please provide args to configure the current context")
log.Println(err)
return err
}
for _, arg := range args {
argsString := strings.Split(arg, "=")
switch argsString[0] {
case "profile":
qliksenseCR.Spec.Profile = argsString[1]
api.LogDebugMessage("Current profile after modification: %s ", qliksenseCR.Spec.Profile)
case "namespace":
qliksenseCR.Spec.NameSpace = argsString[1]
api.LogDebugMessage("Current namespace after modification: %s ", qliksenseCR.Spec.NameSpace)
case "git.repository":
qliksenseCR.Spec.Git.Repository = argsString[1]
api.LogDebugMessage("Current git repository after modification: %s ", qliksenseCR.Spec.Git.Repository)
case "storageClassName":
qliksenseCR.Spec.StorageClassName = argsString[1]
api.LogDebugMessage("Current StorageClassName after modification: %s ", qliksenseCR.Spec.StorageClassName)
case "rotateKeys":
rotateKeys, err := validateInput(argsString[1])
if err != nil {
return err
}
qliksenseCR.Spec.RotateKeys = rotateKeys
api.LogDebugMessage("Current rotateKeys after modification: %s ", qliksenseCR.Spec.RotateKeys)
default:
log.Println("As part of the `qliksense config set` command, please enter one of: profile, namespace, storageClassName,rotateKeys or git.repository arguments")
}
}
// write modified content into context.yaml
api.WriteToFile(&qliksenseCR, qliksenseContextsFile)
return nil
}
// SetContextConfig - set the context for qliksense kubernetes resources to live in
func (q *Qliksense) SetContextConfig(args []string) error {
if len(args) == 1 {
q.SetUpQliksenseContext(args[0], false)
} else {
err := fmt.Errorf("Please provide a name to configure the context with")
log.Println(err)
return err
}
return nil
}
func (q *Qliksense) ListContextConfigs() error {
qliksenseConfigFile := filepath.Join(q.QliksenseHome, QliksenseConfigFile)
var qliksenseConfig api.QliksenseConfig
api.ReadFromFile(&qliksenseConfig, qliksenseConfigFile)
out := ansi.NewColorableStdout()
w := tabwriter.NewWriter(out, 5, 8, 0, '\t', 0)
fmt.Fprintln(w, chalk.Underline.TextStyle("Context Name"), "\t", chalk.Underline.TextStyle("CR File Location"))
w.Flush()
if len(qliksenseConfig.Spec.Contexts) > 0 {
for _, cont := range qliksenseConfig.Spec.Contexts {
fmt.Fprintln(w, cont.Name, "\t", cont.CrFile, "\t")
}
w.Flush()
fmt.Fprintln(out, "")
fmt.Fprintln(out, chalk.Bold.TextStyle("Current Context : "), qliksenseConfig.Spec.CurrentContext)
} else {
fmt.Fprintln(out, "No Contexts Available")
}
return nil
}
// SetUpQliksenseDefaultContext - to setup dir structure for default qliksense context
func (q *Qliksense) SetUpQliksenseDefaultContext() error {
return q.SetUpQliksenseContext(DefaultQliksenseContext, true)
}
// SetUpQliksenseContext - to setup qliksense context
func (q *Qliksense) SetUpQliksenseContext(contextName string, isDefaultContext bool) error {
// check the length of the context name entered by the user, it should not exceed 17 chars
if len(contextName) > MaxContextNameLength {
err := fmt.Errorf("Please enter a context-name with utmost 17 characters")
log.Println(err)
return err
}
qliksenseConfigFile := filepath.Join(q.QliksenseHome, QliksenseConfigFile)
var qliksenseConfig api.QliksenseConfig
configFileTrack := false
if !api.FileExists(qliksenseConfigFile) {
qliksenseConfig.AddBaseQliksenseConfigs(contextName)
} else {
api.ReadFromFile(&qliksenseConfig, qliksenseConfigFile)
if isDefaultContext { // if config file exits but a default context is requested, we want to prevent writing to config file
configFileTrack = true
}
}
// creating a file in the name of the context if it does not exist/ opening it to append/modify content if it already exists
qliksenseContextsDir1 := filepath.Join(q.QliksenseHome, QliksenseContextsDir)
if !api.DirExists(qliksenseContextsDir1) {
if err := os.Mkdir(qliksenseContextsDir1, os.ModePerm); err != nil {
err = fmt.Errorf("Not able to create %s dir: %v", qliksenseContextsDir1, err)
log.Println(err)
return err
}
}
api.LogDebugMessage("%s exists", qliksenseContextsDir1)
// creating contexts/qlik-default/qlik-default.yaml file
qliksenseContextFile := filepath.Join(qliksenseContextsDir1, contextName, contextName+".yaml")
var qliksenseCR api.QliksenseCR
defaultContextsDir := filepath.Join(qliksenseContextsDir1, contextName)
if !api.DirExists(defaultContextsDir) {
if err := os.Mkdir(defaultContextsDir, os.ModePerm); err != nil {
err = fmt.Errorf("Not able to create %s: %v", defaultContextsDir, err)
log.Println(err)
return err
}
}
api.LogDebugMessage("%s exists", defaultContextsDir)
if !api.FileExists(qliksenseContextFile) {
qliksenseCR.AddCommonConfig(contextName)
api.LogDebugMessage("Added Context: %s", contextName)
} else {
api.ReadFromFile(&qliksenseCR, qliksenseContextFile)
}
api.WriteToFile(&qliksenseCR, qliksenseContextFile)
ctxTrack := false
if len(qliksenseConfig.Spec.Contexts) > 0 {
for _, ctx := range qliksenseConfig.Spec.Contexts {
if ctx.Name == contextName {
ctx.CrFile = qliksenseContextFile
ctxTrack = true
break
}
}
}
if !ctxTrack {
qliksenseConfig.Spec.Contexts = append(qliksenseConfig.Spec.Contexts, api.Context{
Name: contextName,
CrFile: qliksenseContextFile,
})
}
qliksenseConfig.Spec.CurrentContext = contextName
if !configFileTrack {
api.WriteToFile(&qliksenseConfig, qliksenseConfigFile)
}
return nil
}
func validateInput(input string) (string, error) {
var err error
validInputs := []string{"yes", "no", "None"}
isValid := false
for _, elem := range validInputs {
if input == elem {
isValid = true
break
}
}
if !isValid {
err = fmt.Errorf("Please enter one of: yes, no or None")
log.Println(err)
}
return input, err
}

View File

@@ -0,0 +1,221 @@
package qliksense
import (
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
"testing"
)
var (
testDir = "./tests"
)
func setup() func() {
// create tests dir
if err := os.Mkdir(testDir, 0777); err != nil {
log.Printf("\nError occurred: %v", err)
}
config :=
`
apiVersion: config.qlik.com/v1
kind: QliksenseConfig
metadata:
name: qliksenseConfig
spec:
contexts:
- name: qlik-default
crLocation: /root/.qliksense/contexts/qlik-default.yaml
currentContext: qlik-default
`
configFile := filepath.Join(testDir, "config.yaml")
// tests/config.yaml exists
ioutil.WriteFile(configFile, []byte(config), 0777)
contextYaml :=
`
apiVersion: qlik.com/v1
kind: Qliksense
metadata:
name: qlik-default
spec:
profile: docker-desktop
rotateKeys: "yes"
releaseName: qlik-default
`
qlikDefaultContext := "qlik-default"
// create contexts/qlik-default/ under tests/
contexts := "contexts"
contextsDir := filepath.Join(testDir, contexts, qlikDefaultContext)
if err := os.MkdirAll(contextsDir, 0777); err != nil {
err = fmt.Errorf("Not able to create directories")
log.Fatal(err)
}
contextFile := filepath.Join(contextsDir, qlikDefaultContext+".yaml")
ioutil.WriteFile(contextFile, []byte(contextYaml), 0777)
tearDown := func() {
os.RemoveAll(testDir)
}
return tearDown
}
func Test_retrieveCurrentContextInfo(t *testing.T) {
tearDown := setup()
defer tearDown()
q := &Qliksense{
QliksenseHome: testDir,
}
_, _, err := retrieveCurrentContextInfo(q)
if err != nil {
t.FailNow()
}
}
func TestSetUpQliksenseContext(t *testing.T) {
type args struct {
qlikSenseHome string
contextName string
isDefaultContext bool
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid contextname",
args: args{
qlikSenseHome: testDir,
contextName: "testContext1",
isDefaultContext: false,
},
wantErr: false,
},
{
name: "invalid contextname",
args: args{
qlikSenseHome: testDir,
contextName: "testContext_abcdefgh",
isDefaultContext: false,
},
wantErr: true,
},
}
tearDown := setup()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
q, err := New(tt.args.qlikSenseHome)
if err != nil {
t.Errorf("unable to create a qliksense instance")
return
}
if err := q.SetUpQliksenseContext(tt.args.contextName, tt.args.isDefaultContext); (err != nil) != tt.wantErr {
t.Errorf("SetUpQliksenseContext() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestSetUpQliksenseDefaultContext(t *testing.T) {
type args struct {
qlikSenseHome string
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid case",
args: args{
qlikSenseHome: testDir,
},
wantErr: false,
},
}
tearDown := setup()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
q, err := New(tt.args.qlikSenseHome)
if err != nil {
t.Errorf("unable to create a qliksense instance")
return
}
if err := q.SetUpQliksenseDefaultContext(); (err != nil) != tt.wantErr {
t.Errorf("SetUpQliksenseDefaultContext() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestSetOtherConfigs(t *testing.T) {
type args struct {
q *Qliksense
args []string
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid case",
args: args{
q: &Qliksense{
QliksenseHome: testDir,
},
args: []string{"profile=minikube", "namespace=qliksense", "storageClassName=efs"},
},
wantErr: false,
},
}
tearDown := setup()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if err := tt.args.q.SetOtherConfigs(tt.args.args); (err != nil) != tt.wantErr {
t.Errorf("SetOtherConfigs() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestSetConfigs(t *testing.T) {
type args struct {
q *Qliksense
args []string
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid case",
args: args{
q: &Qliksense{
QliksenseHome: testDir,
},
args: []string{"qliksense[name=acceptEULA]=\"yes\"", "qliksense[name=mongoDbUri]=\"mongo://mongo:3307\""},
},
wantErr: false,
},
}
tearDown := setup()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if err := tt.args.q.SetConfigs(tt.args.args); (err != nil) != tt.wantErr {
t.Errorf("SetConfigs() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

View File

@@ -1,253 +1,411 @@
package qliksense
import (
"fmt"
"io"
"io/ioutil"
"os"
"github.com/docker/cli/cli/command"
cliflags "github.com/docker/cli/cli/flags"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/term"
"github.com/docker/docker/registry"
"strings"
"golang.org/x/net/context"
yaml "gopkg.in/yaml.v2"
)
// Images ...
type Images struct {
Images []string `yaml:"images"`
}
// PullImages ...
func (p *Qliksense) PullImages(args []string) error {
var (
image string
err error
yamlVersion string
valid bool
images Images
)
println("getting images list...")
if yamlVersion, err = p.CallPorter(append([]string{"invoke", "--action", "about"}, args...),
func(x string) (out *string) {
if strings.HasPrefix(x, "qlikSenseVersion") {
valid = true
}
if strings.HasPrefix(x, "execution") {
valid = false
}
if valid {
return &x
}
return nil
}); err != nil {
return err
}
if err = yaml.Unmarshal([]byte(yamlVersion), &images); err != nil {
return err
}
for _, image = range images.Images {
if err = p.PullImage(image); err != nil {
fmt.Print(err)
}
println("---")
}
return nil
}
// PullImage ...
func (p *Qliksense) PullImage(imageName string) error {
var (
cli *command.DockerCli
dockerOutput io.Writer
response io.ReadCloser
pullOptions types.ImagePullOptions
ctx context.Context
ref reference.Named
repoInfo *registry.RepositoryInfo
authConfig types.AuthConfig
encodedAuth string
termFd uintptr
err error
)
// TODO: Create a real cli config context
ctx = context.Background()
if cli, err = command.NewDockerCli(); err != nil {
return err
}
if err = cli.Initialize(cliflags.NewClientOptions()); err != nil {
return err
}
if ref, err = reference.ParseNormalizedNamed(imageName); err != nil {
return err
}
if repoInfo, err = registry.ParseRepositoryInfo(ref); err != nil {
return err
}
authConfig = command.ResolveAuthConfig(ctx, cli, repoInfo.Index)
if encodedAuth, err = command.EncodeAuthToBase64(authConfig); err != nil {
return err
}
pullOptions = types.ImagePullOptions{
RegistryAuth: encodedAuth,
}
if response, err = cli.Client().ImagePull(ctx, imageName, pullOptions); err != nil {
return err
}
defer response.Close()
dockerOutput = ioutil.Discard
// if b.IsVerbose() {
// dockerOutput = b.Out
// }
dockerOutput = os.Stdout
termFd, _ = term.GetFdInfo(dockerOutput)
// Setting this to false here because Moby os.Exit(1) all over the place and this fails on WSL (only)
// when Term is true.
isTerm := false
if err = jsonmessage.DisplayJSONMessagesStream(response, dockerOutput, termFd, isTerm, nil); err != nil {
return err
}
return nil
}
// PullImage ...
func (p *Qliksense) TagAndPushImages(registry string) error {
var (
image string
err error
yamlVersion string
valid bool
images Images
)
if yamlVersion, err = p.CallPorter([]string{"invoke", "--action", "about"},
func(x string) (out *string) {
if strings.HasPrefix(x, "qlikSenseVersion") {
valid = true
}
if strings.HasPrefix(x, "execution") {
valid = false
}
if valid {
return &x
}
return nil
}); err != nil {
return err
}
if err = yaml.Unmarshal([]byte(yamlVersion), &images); err != nil {
return err
}
for _, image = range images.Images {
if err = p.TagAndPush(image, registry); err != nil {
fmt.Print(err)
}
println("---")
}
return nil
}
// PullImage ...
func (p *Qliksense) TagAndPush(image string, registryName string) error {
var (
cli *command.DockerCli
dockerOutput io.Writer
response io.ReadCloser
pushOptions types.ImagePushOptions
ctx context.Context
newName string
segments []string
imageList []types.ImageSummary
imageListOptions types.ImageListOptions
filterArgs filters.Args
ref reference.Named
repoInfo *registry.RepositoryInfo
authConfig types.AuthConfig
encodedAuth string
termFd uintptr
err error
)
// TODO: Create a real cli config context
ctx = context.Background()
if cli, err = command.NewDockerCli(); err != nil {
return err
}
if err = cli.Initialize(cliflags.NewClientOptions()); err != nil {
return err
}
segments = strings.Split(image, "/")
if segments[0] == "docker.io" {
image = strings.Join(segments[1:], "/")
}
newName = registryName + "/" + segments[len(segments)-1]
filterArgs = filters.NewArgs()
filterArgs.Add("reference", image)
imageListOptions = types.ImageListOptions{
Filters: filterArgs,
}
if imageList, err = cli.Client().ImageList(ctx, imageListOptions); err != nil {
return err
}
if imageList == nil || len(imageList) <= 0 {
fmt.Printf("Use `qliksense pull`, to pull %v for an air gap push", newName)
return nil
}
if err = cli.Client().ImageTag(ctx, image, newName); err != nil {
return err
}
if ref, err = reference.ParseNormalizedNamed(image); err != nil {
return err
}
if repoInfo, err = registry.ParseRepositoryInfo(ref); err != nil {
return err
}
authConfig = command.ResolveAuthConfig(ctx, cli, repoInfo.Index)
if encodedAuth, err = command.EncodeAuthToBase64(authConfig); err != nil {
return err
}
pushOptions = types.ImagePushOptions{
All: true,
RegistryAuth: encodedAuth,
}
if response, err = cli.Client().ImagePush(ctx, newName, pushOptions); err != nil {
return err
}
defer response.Close()
dockerOutput = ioutil.Discard
// if b.IsVerbose() {
// dockerOutput = b.Out
// }
dockerOutput = os.Stdout
termFd, _ = term.GetFdInfo(dockerOutput)
// Setting this to false here because Moby os.Exit(1) all over the place and this fails on WSL (only)
// when Term is true.
isTerm := false
if err = jsonmessage.DisplayJSONMessagesStream(response, dockerOutput, termFd, isTerm, nil); err != nil {
return err
}
return nil
}
package qliksense
import (
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"
"time"
"github.com/containers/image/v5/copy"
"github.com/containers/image/v5/signature"
"github.com/containers/image/v5/transports/alltransports"
imageTypes "github.com/containers/image/v5/types"
"github.com/docker/cli/cli/command"
cliflags "github.com/docker/cli/cli/flags"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/term"
"github.com/docker/docker/registry"
homedir "github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"golang.org/x/net/context"
yaml "gopkg.in/yaml.v2"
)
// PullImages ...
func (q *Qliksense) PullImages(gitRef, profile string, engine bool) error {
var (
image, versionFile, imagesDir, homeDir string
err error
versionOut *VersionOutput
)
println("getting images list...")
// TODO: get getref and profile from config/cr for About function call
if versionOut, err = q.About(gitRef, profile); err != nil {
return err
}
if homeDir, err = homedir.Dir(); err != nil {
return err
}
imagesDir = filepath.Join(homeDir, ".qliksense", "images")
os.MkdirAll(imagesDir, os.ModePerm)
versionFile = filepath.Join(imagesDir, versionOut.QliksenseVersion)
if _, err = os.Stat(versionFile); err != nil {
if os.IsNotExist(err) {
if yamlVersion, err := yaml.Marshal(versionOut); err != nil {
return err
} else if err = ioutil.WriteFile(versionFile, yamlVersion, os.ModePerm); err != nil {
return err
}
} else {
return errors.Errorf("Unable to determine About file %v exists", versionFile)
}
}
for _, image = range versionOut.Images {
if _, err = q.PullImage(image, engine); err != nil {
fmt.Print(err)
}
println("---")
}
return nil
}
// PullImage ...
func (q *Qliksense) PullImage(imageName string, engine bool) (map[string]string, error) {
if engine {
return q.pullDockerImage(imageName)
}
return q.pullImage(imageName)
}
func (q *Qliksense) commandTimeoutContext(commandTimeout time.Duration) (context.Context, context.CancelFunc) {
ctx := context.Background()
var cancel context.CancelFunc = func() {}
if commandTimeout > 0 {
ctx, cancel = context.WithTimeout(ctx, commandTimeout)
}
return ctx, cancel
}
func (q *Qliksense) pullImage(imageName string) (map[string]string, error) {
var (
ctx context.Context
cancel context.CancelFunc
srcRef, destRef imageTypes.ImageReference
blobDir, targetDir, homeDir string
segments []string
nameTag []string
err error
policyContext *signature.PolicyContext
)
ctx, cancel = q.commandTimeoutContext(0)
defer cancel()
if srcRef, err = alltransports.ParseImageName("docker://" + imageName); err != nil {
return nil, err
}
segments = strings.Split(imageName, "/")
nameTag = strings.Split(segments[len(segments)-1], ":")
if len(nameTag) < 2 {
nameTag = append(nameTag, "latest")
}
if homeDir, err = homedir.Dir(); err != nil {
return nil, err
}
targetDir = filepath.Join(homeDir, ".qliksense", "images", nameTag[0], nameTag[1])
fmt.Printf("==> Pulling image %v:%v", nameTag[0], nameTag[1])
fmt.Println()
os.MkdirAll(targetDir, os.ModePerm)
blobDir = filepath.Join(homeDir, ".qliksense", "blobs")
os.MkdirAll(blobDir, os.ModePerm)
if destRef, err = alltransports.ParseImageName("oci:" + targetDir); err != nil {
return nil, err
}
if policyContext, err = signature.NewPolicyContext(&signature.Policy{Default: []signature.PolicyRequirement{signature.NewPRInsecureAcceptAnything()}}); err != nil {
return nil, err
}
defer policyContext.Destroy()
_, err = copy.Image(ctx, policyContext, destRef, srcRef, &copy.Options{
ReportWriter: os.Stdout,
SourceCtx: &imageTypes.SystemContext{
ArchitectureChoice: "amd64",
OSChoice: "linux",
},
DestinationCtx: &imageTypes.SystemContext{
OCISharedBlobDirPath: blobDir,
},
})
return nil, err
}
func (q *Qliksense) pullDockerImage(imageName string) (map[string]string, error) {
var (
cli *command.DockerCli
dockerOutput io.Writer
response io.ReadCloser
pullOptions types.ImagePullOptions
ctx context.Context
cancel context.CancelFunc
ref reference.Named
repoInfo *registry.RepositoryInfo
authConfig types.AuthConfig
encodedAuth string
termFd uintptr
err error
)
ctx, cancel = q.commandTimeoutContext(0)
defer cancel()
if cli, err = command.NewDockerCli(); err != nil {
return nil, err
}
if err = cli.Initialize(cliflags.NewClientOptions()); err != nil {
return nil, err
}
if ref, err = reference.ParseNormalizedNamed(imageName); err != nil {
return nil, err
}
if repoInfo, err = registry.ParseRepositoryInfo(ref); err != nil {
return nil, err
}
authConfig = command.ResolveAuthConfig(ctx, cli, repoInfo.Index)
if encodedAuth, err = command.EncodeAuthToBase64(authConfig); err != nil {
return nil, err
}
pullOptions = types.ImagePullOptions{
RegistryAuth: encodedAuth,
}
if response, err = cli.Client().ImagePull(ctx, imageName, pullOptions); err != nil {
return nil, err
}
defer response.Close()
dockerOutput = ioutil.Discard
// if b.IsVerbose() {
// dockerOutput = b.Out
// }
dockerOutput = os.Stdout
termFd, _ = term.GetFdInfo(dockerOutput)
// Setting this to false here because Moby os.Exit(1) all over the place and this fails on WSL (only)
// when Term is true.
isTerm := false
if err = jsonmessage.DisplayJSONMessagesStream(response, dockerOutput, termFd, isTerm, nil); err != nil {
return nil, err
}
inspectData, _, err := cli.Client().ImageInspectWithRaw(ctx, imageName)
if err != nil {
return nil, err
}
return inspectData.ContainerConfig.Labels, nil
}
//TagAndPushImages ...
func (q *Qliksense) TagAndPushImages(registry string, engine bool) error {
var (
image string
err error
yamlVersion string
images VersionOutput
)
if err = yaml.Unmarshal([]byte(yamlVersion), &images); err != nil {
return err
}
for _, image = range images.Images {
if err = q.TagAndPush(image, registry, engine); err != nil {
fmt.Print(err)
}
println("---")
}
return nil
}
func (q *Qliksense) directoryExists(path string) (exists bool, err error) {
if info, err := os.Stat(path); err != nil && os.IsNotExist(err) {
exists = false
err = nil
} else if err != nil && !os.IsNotExist(err) {
exists = false
} else if err == nil && info.IsDir() {
exists = true
} else if err == nil && !info.IsDir() {
exists = false
err = fmt.Errorf("path: %v is occupied by a file instead of a directory", path)
}
return exists, err
}
//TagAndPush ...
func (q *Qliksense) TagAndPush(image string, registryName string, engine bool) error {
if engine {
return q.tagAndDockerPush(image, registryName)
}
return q.tagAndPush(image, registryName)
}
func (q *Qliksense) tagAndPush(image string, registryName string) error {
var (
ctx context.Context
cancel context.CancelFunc
srcRef, destRef imageTypes.ImageReference
blobDir, srcDir, homeDir, newName string
segments []string
nameTag []string
err error
policyContext *signature.PolicyContext
srcExists bool
)
ctx, cancel = q.commandTimeoutContext(0)
defer cancel()
segments = strings.Split(image, "/")
nameTag = strings.Split(segments[len(segments)-1], ":")
if len(nameTag) < 2 {
nameTag = append(nameTag, "latest")
}
if homeDir, err = homedir.Dir(); err != nil {
return err
}
srcDir = filepath.Join(homeDir, ".qliksense", "images", nameTag[0], nameTag[1])
if srcExists, err = q.directoryExists(srcDir); err != nil {
return err
}
if !srcExists {
if _, err = q.PullImage(image, false); err != nil {
return err
}
}
if srcRef, err = alltransports.ParseImageName("oci:" + srcDir); err != nil {
return err
}
if segments[0] == "docker.io" {
image = strings.Join(segments[1:], "/")
}
newName = "//" + registryName + "/" + segments[len(segments)-1]
fmt.Printf("==> Tag and push image to %v", newName)
fmt.Println()
if destRef, err = alltransports.ParseImageName("docker:" + newName); err != nil {
return err
}
if policyContext, err = signature.NewPolicyContext(&signature.Policy{Default: []signature.PolicyRequirement{signature.NewPRInsecureAcceptAnything()}}); err != nil {
return err
}
defer policyContext.Destroy()
blobDir = filepath.Join(homeDir, ".qliksense", "blobs")
os.MkdirAll(blobDir, os.ModePerm)
_, err = copy.Image(ctx, policyContext, destRef, srcRef, &copy.Options{
ReportWriter: os.Stdout,
SourceCtx: &imageTypes.SystemContext{
OCISharedBlobDirPath: blobDir,
},
DestinationCtx: &imageTypes.SystemContext{
DockerDaemonInsecureSkipTLSVerify: true,
},
})
return err
}
// PullImage ...
func (q *Qliksense) tagAndDockerPush(image string, registryName string) error {
var (
cli *command.DockerCli
dockerOutput io.Writer
response io.ReadCloser
pushOptions types.ImagePushOptions
ctx context.Context
cancel context.CancelFunc
newName string
segments []string
imageList []types.ImageSummary
imageListOptions types.ImageListOptions
filterArgs filters.Args
ref reference.Named
repoInfo *registry.RepositoryInfo
authConfig types.AuthConfig
encodedAuth string
termFd uintptr
err error
)
// TODO: Create a real cli config context
ctx, cancel = q.commandTimeoutContext(0)
defer cancel()
if cli, err = command.NewDockerCli(); err != nil {
return err
}
if err = cli.Initialize(cliflags.NewClientOptions()); err != nil {
return err
}
segments = strings.Split(image, "/")
if segments[0] == "docker.io" {
image = strings.Join(segments[1:], "/")
}
newName = registryName + "/" + segments[len(segments)-1]
filterArgs = filters.NewArgs()
filterArgs.Add("reference", image)
imageListOptions = types.ImageListOptions{
Filters: filterArgs,
}
if imageList, err = cli.Client().ImageList(ctx, imageListOptions); err != nil {
return err
}
if imageList == nil || len(imageList) <= 0 {
fmt.Printf("Use `qliksense pull`, to pull %v for an air gap push", newName)
return nil
}
if err = cli.Client().ImageTag(ctx, image, newName); err != nil {
return err
}
if ref, err = reference.ParseNormalizedNamed(image); err != nil {
return err
}
if repoInfo, err = registry.ParseRepositoryInfo(ref); err != nil {
return err
}
authConfig = command.ResolveAuthConfig(ctx, cli, repoInfo.Index)
if encodedAuth, err = command.EncodeAuthToBase64(authConfig); err != nil {
return err
}
pushOptions = types.ImagePushOptions{
All: true,
RegistryAuth: encodedAuth,
}
if response, err = cli.Client().ImagePush(ctx, newName, pushOptions); err != nil {
return err
}
defer response.Close()
dockerOutput = ioutil.Discard
// if b.IsVerbose() {
// dockerOutput = b.Out
// }
dockerOutput = os.Stdout
termFd, _ = term.GetFdInfo(dockerOutput)
// Setting this to false here because Moby os.Exit(1) all over the place and this fails on WSL (only)
// when Term is true.
isTerm := false
if err = jsonmessage.DisplayJSONMessagesStream(response, dockerOutput, termFd, isTerm, nil); err != nil {
return err
}
return nil
}

40
pkg/qliksense/fetch.go Normal file
View File

@@ -0,0 +1,40 @@
package qliksense
import (
"fmt"
"github.com/google/uuid"
kapis_git "github.com/qlik-oss/k-apis/pkg/git"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
)
const (
QLIK_GIT_REPO = "https://github.com/qlik-oss/qliksense-k8s"
)
func (q *Qliksense) FetchQK8s(version string) error {
qConfig := qapi.NewQConfig(q.QliksenseHome)
return fetchAndUpdateCR(qConfig, version)
}
func fetchAndUpdateCR(qConfig *qapi.QliksenseConfig, version string) error {
qcr, err := qConfig.GetCurrentCR()
if err != nil {
fmt.Println("cannot get the current-context cr", err)
return err
}
if qConfig.IsRepoExistForCurrent(version) {
return nil
}
destDir := qConfig.BuildRepoPath(version)
fmt.Printf("fetching version [%s] from %s\n", version, QLIK_GIT_REPO)
if repo, err := kapis_git.CloneRepository(destDir, QLIK_GIT_REPO, nil); err != nil {
return err
} else if err = kapis_git.Checkout(repo, version, fmt.Sprintf("%v-by-operator-%v", version, uuid.New().String()), nil); err != nil {
return err
}
qcr.Spec.ManifestsRoot = qConfig.BuildCurrentManifestsRoot(version)
qcr.AddLabelToCr("version", version)
return qConfig.WriteCurrentContextCR(qcr)
}

74
pkg/qliksense/install.go Normal file
View File

@@ -0,0 +1,74 @@
package qliksense
import (
"fmt"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
)
type InstallCommandOptions struct {
AcceptEULA string
Namespace string
StorageClass string
MongoDbUri string
RotateKeys string
}
func (q *Qliksense) InstallQK8s(version string, opts *InstallCommandOptions) error {
// step1: fetch 1.0.0 # pull down qliksense-k8s@1.0.0
// step2: operator view | kubectl apply -f # operator manifest (CRD)
// step3: config apply | kubectl apply -f # generates patches (if required) in configuration directory, applies manifest
// step4: config view | kubectl apply -f # generates Custom Resource manifest (CR)
// fetch the version
qConfig := qapi.NewQConfig(q.QliksenseHome)
fetchAndUpdateCR(qConfig, version)
//TODO: may need to check if CRD already installed, but doing apply does not hurt for now
//install crd into cluster
fmt.Println("Installing operator CRD")
if err := qapi.KubectlApply(q.GetCRDString()); err != nil {
fmt.Println("cannot do kubectl apply on opeartor CRD", err)
return err
}
// install generated manifests into cluster
fmt.Println("Installing generated manifests into cluster")
qcr, err := qConfig.GetCurrentCR()
if err != nil {
fmt.Println("cannot get the current-context cr", err)
return err
}
if opts.AcceptEULA != "" {
qcr.Spec.AddToConfigs("qliksense", "acceptEULA", opts.AcceptEULA)
}
if opts.MongoDbUri != "" {
qcr.Spec.AddToSecrets("qliksense", "mongoDbUri", opts.MongoDbUri)
}
if opts.StorageClass != "" {
qcr.Spec.StorageClassName = opts.StorageClass
}
if opts.Namespace != "" {
qcr.Spec.NameSpace = opts.Namespace
}
if opts.RotateKeys != "" {
qcr.Spec.RotateKeys = opts.RotateKeys
}
qConfig.WriteCurrentContextCR(qcr)
if err := q.applyConfigToK8s(qcr, "install"); err != nil {
fmt.Println("cannot do kubectl apply on manifests")
return err
}
// install operator cr into cluster
//get the current context cr
fmt.Println("Install operator CR into cluster")
r, err := q.getCurrentCRString()
if err != nil {
return err
}
if err := qapi.KubectlApply(r); err != nil {
fmt.Println("cannot do kubectl apply on operator CR")
}
return nil
}

32
pkg/qliksense/kuz.go Normal file
View File

@@ -0,0 +1,32 @@
package qliksense
import (
"log"
"os"
"sigs.k8s.io/kustomize/api/filesys"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/api/krusty"
"sigs.k8s.io/kustomize/api/types"
)
func executeKustomizeBuild(directory string) ([]byte, error) {
log.SetOutput(&nullWriter{})
defer func() {
log.SetOutput(os.Stderr)
}()
fSys := filesys.MakeFsOnDisk()
options := &krusty.Options{
DoLegacyResourceSort: false,
LoadRestrictions: types.LoadRestrictionsNone,
DoPrune: false,
PluginConfig: konfig.DisabledPluginConfig(),
}
k := krusty.MakeKustomizer(fSys, options)
resMap, err := k.Run(directory)
if err != nil {
return nil, err
}
return resMap.AsYaml()
}

141
pkg/qliksense/kuz_test.go Normal file
View File

@@ -0,0 +1,141 @@
package qliksense
import (
"bytes"
"encoding/base64"
"io"
"io/ioutil"
"log"
"os"
"path"
"strings"
"testing"
"gopkg.in/yaml.v3"
"github.com/Shopify/ejson"
"github.com/qlik-oss/k-apis/pkg/config"
"github.com/qlik-oss/k-apis/pkg/qust"
kapis_git "github.com/qlik-oss/k-apis/pkg/git"
)
func Test_executeKustomizeBuild(t *testing.T) {
tmpDir, err := ioutil.TempDir("", "")
if err != nil {
t.Fatalf("unexpected error: %v\n", err)
}
defer os.RemoveAll(tmpDir)
kustomizationYamlFilePath := path.Join(tmpDir, "kustomization.yaml")
kustomizationYaml := `
generatorOptions:
disableNameSuffixHash: true
configMapGenerator:
- name: foo-config
literals:
- foo=bar
`
err = ioutil.WriteFile(kustomizationYamlFilePath, []byte(kustomizationYaml), os.ModePerm)
if err != nil {
t.Fatalf("error writing kustomization file to path: %v error: %v\n", kustomizationYamlFilePath, err)
}
result, err := executeKustomizeBuild(tmpDir)
if err != nil {
t.Fatalf("unexpected kustomize error: %v\n", err)
}
expectedK8sYaml := `apiVersion: v1
data:
foo: bar
kind: ConfigMap
metadata:
name: foo-config
`
if string(result) != expectedK8sYaml {
t.Fatalf("expected k8s yaml: [%v] but got: [%v]\n", expectedK8sYaml, string(result))
}
}
func Test_executeKustomizeBuild_onQlikConfig_regenerateKeys(t *testing.T) {
tmpDir, err := ioutil.TempDir("", "")
if err != nil {
t.Fatalf("unexpected error: %v\n", err)
}
defer os.RemoveAll(tmpDir)
configPath := path.Join(tmpDir, "config")
if repo, err := kapis_git.CloneRepository(configPath, defaultGitUrl, nil); err != nil {
t.Fatalf("unexpected error: %v\n", err)
} else if err := kapis_git.Checkout(repo, "v1.21.23-edge", "", nil); err != nil {
t.Fatalf("unexpected error: %v\n", err)
}
cr := &config.CRSpec{
ManifestsRoot: configPath,
}
if err := os.Setenv("EJSON_KEYDIR", tmpDir); err != nil {
t.Fatalf("unexpected error setting EJSON_KEYDIR environment variable: %v\n", err)
}
if err := os.Unsetenv("EJSON_KEY"); err != nil {
t.Fatalf("unexpected error unsetting EJSON_KEY: %v\n", err)
}
generateKeys(cr, "won't-use")
yamlResources, err := executeKustomizeBuild(path.Join(configPath, "manifests", "base", "resources", "users"))
if err != nil {
t.Fatalf("unexpected kustomize error: %v\n", err)
}
decoder := yaml.NewDecoder(bytes.NewReader(yamlResources))
var resource map[string]interface{}
keyIdBase64 := ""
for {
err := decoder.Decode(&resource)
if err != nil {
if err != io.EOF {
t.Fatalf("unexpected yaml decode error: %v\n", err)
}
break
}
if resource["kind"].(string) == "Secret" && strings.Contains(resource["metadata"].(map[string]interface{})["name"].(string), "users-secrets-") {
keyIdBase64 = resource["data"].(map[string]interface{})["tokenAuthPrivateKeyId"].(string)
break
}
}
untransformedKeyId := `(( (ds "data").kid ))`
if keyIdBase64 == "" {
t.Fatalf("expected keyIdBase64 for users secret to be non empty:\n")
} else if keyId, err := base64.StdEncoding.DecodeString(keyIdBase64); err != nil {
t.Fatalf("unexpected base64 decode error: %v\n", err)
} else if string(keyId) == untransformedKeyId {
t.Fatalf("unexpected users keyId: %v\n", untransformedKeyId)
}
}
func generateKeys(cr *config.CRSpec, defaultKeyDir string) {
log.Println("rotating all keys")
keyDir := getEjsonKeyDir(defaultKeyDir)
if ejsonPublicKey, ejsonPrivateKey, err := ejson.GenerateKeypair(); err != nil {
log.Printf("error generating an ejson key pair: %v\n", err)
} else if err := qust.GenerateKeys(cr, ejsonPublicKey); err != nil {
log.Printf("error generating application keys: %v\n", err)
} else if err := os.MkdirAll(keyDir, os.ModePerm); err != nil {
log.Printf("error makeing sure private key storage directory: %v exists, error: %v\n", keyDir, err)
} else if err := ioutil.WriteFile(path.Join(keyDir, ejsonPublicKey), []byte(ejsonPrivateKey), os.ModePerm); err != nil {
log.Printf("error storing ejson private key: %v\n", err)
}
}
func getEjsonKeyDir(defaultKeyDir string) string {
ejsonKeyDir := os.Getenv("EJSON_KEYDIR")
if ejsonKeyDir == "" {
ejsonKeyDir = defaultKeyDir
}
return ejsonKeyDir
}

41
pkg/qliksense/operator.go Normal file
View File

@@ -0,0 +1,41 @@
package qliksense
import (
"fmt"
"path/filepath"
"io"
"os"
"strings"
)
func (q *Qliksense) ViewOperatorCrd() {
io.WriteString(os.Stdout, q.GetCRDString())
}
// this will return crd,deployment,role, rolebinding,serviceaccount for operator
func (q *Qliksense) GetCRDString() string {
result := ""
for _, v := range q.getFileList("crd") {
result = q.getYamlFromPackrFile(v)
}
for _, v := range q.getFileList("crd-deploy") {
result = result + q.getYamlFromPackrFile(v)
}
return result
}
func (q *Qliksense) getYamlFromPackrFile(packrFile string) string {
s, err := q.CrdBox.FindString(packrFile)
if err != nil {
fmt.Printf("Cannot read file %s", packrFile)
}
return fmt.Sprintln("#soruce: " + packrFile + "\n\n" + s + "\n---")
}
func (q *Qliksense) getFileList(resourceType string) []string {
var resList []string
for _, v := range q.CrdBox.List() {
if strings.Contains(v, filepath.Join(resourceType, "")) {
resList = append(resList, []string{v}...)
}
}
return resList
}

View File

@@ -1,60 +0,0 @@
package qliksense
import (
"bufio"
"fmt"
"io"
"os"
"os/exec"
)
// ProcessLine ...
type ProcessLine func(string) *string
// CallPorter ...
func (p *Qliksense) CallPorter(args []string, processor ProcessLine) (string, error) {
var (
outText string
cmd *exec.Cmd
err error
output io.ReadCloser
scanner *bufio.Scanner
done chan struct{}
)
cmd = exec.Command(p.porterExe, args[:]...)
if output, err = cmd.StdoutPipe(); err != nil {
return "", err
}
cmd.Stderr = os.Stderr
done = make(chan struct{})
scanner = bufio.NewScanner(output)
go func() {
for scanner.Scan() {
var text string
var newText *string
text = scanner.Text()
if processor != nil {
newText = processor(text)
if newText != nil {
outText = outText + fmt.Sprintln(*newText)
}
} else {
outText = outText + fmt.Sprintln(text)
}
}
done <- struct{}{}
}()
if err = cmd.Start(); err != nil {
return "", err
}
<-done
if err = cmd.Wait(); err != nil {
return "", err
}
if err = scanner.Err(); err != nil {
return "", err
}
return outText, nil
}

View File

@@ -1,13 +1,30 @@
package qliksense
// Qliksense is the logic behind the qliksense client
type Qliksense struct {
porterExe string
}
// New qliksense client, initialized with useful defaults.
func New(porterExe string) *Qliksense {
return &Qliksense{
porterExe,
}
}
//go:generate packr2
package qliksense
import (
"os"
"path"
"github.com/gobuffalo/packr/v2"
)
// Qliksense is the logic behind the qliksense client
type Qliksense struct {
QliksenseHome string
QliksenseEjsonKeyDir string
CrdBox *packr.Box ``
}
// New qliksense client, initialized with useful defaults.
func New(qliksenseHome string) (*Qliksense, error) {
qliksenseClient := &Qliksense{
QliksenseHome: qliksenseHome,
CrdBox: packr.New("crds", "./crds"),
}
qliksenseClient.QliksenseEjsonKeyDir = path.Join(qliksenseHome, "ejson", "keys")
if err := os.MkdirAll(qliksenseClient.QliksenseEjsonKeyDir, os.ModePerm); err != nil {
return nil, err
}
return qliksenseClient, nil
}

View File

@@ -0,0 +1,21 @@
package qliksense
import (
"errors"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
)
func (q *Qliksense) UninstallQK8s(contextName string) error {
qConfig := qapi.NewQConfig(q.QliksenseHome)
if contextName == "" {
contextName = qConfig.Spec.CurrentContext
} else if !qConfig.IsContextExist(contextName) {
return errors.New("context name [ " + contextName + " ] not found")
}
str, err := q.getCRString(contextName)
if err != nil {
return err
}
return qapi.KubectlDelete(str)
}

39
pkg/qliksense/upgrade.go Normal file
View File

@@ -0,0 +1,39 @@
package qliksense
import (
"fmt"
qapi "github.com/qlik-oss/sense-installer/pkg/api"
)
func (q *Qliksense) UpgradeQK8s() error {
// step1: get CR
// step2: run kustomize
// step3: run kubectl apply
// fetch the version
qConfig := qapi.NewQConfig(q.QliksenseHome)
qcr, err := qConfig.GetCurrentCR()
if err != nil {
fmt.Println("cannot get the current-context cr", err)
return err
}
qcr.Spec.RotateKeys = "no"
if err := q.applyConfigToK8s(qcr, "upgrade"); err != nil {
fmt.Println("cannot do kubectl apply on manifests")
return err
}
fmt.Println("Install operator CR into cluster")
r, err := qcr.GetString()
if err != nil {
return err
}
if err := qapi.KubectlApply(r); err != nil {
fmt.Println("cannot do kubectl apply on operator CR")
}
return nil
}

View File

@@ -1,8 +1,8 @@
package pkg
// These are build-time values, set during an official release
var (
Commit string
Version string
CommitDate string
)
package pkg
// These are build-time values, set during an official release
var (
Commit string
Version string
CommitDate string
)

View File

@@ -1,39 +0,0 @@
FROM quay.io/deis/lightweight-docker-go:v0.2.0
FROM debian:stretch as base
# This is a template Dockerfile for the bundle's invocation image
# You can customize it to use different base images, install tools and copy configuration files.
#
# Porter will use it as a template and append lines to it for the mixins
# and to set the CMD appropriately for the CNAB specification.
#
# Add the following line to porter.yaml to instruct Porter to use this template
# dockerfile: Dockerfile.tmpl
# You can control where the mixin's Dockerfile lines are inserted into this file by moving "# PORTER_MIXINS" line
# another location in this file. If you remove that line, the mixins generated content is appended to this file.
# PORTER_MIXINS
COPY --from=0 /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ARG BUNDLE_DIR
ARG HELM_VERSION=v2.16.0
RUN apt-get update && \
apt-get install -y curl && \
curl -o helm.tgz https://get.helm.sh/helm-$HELM_VERSION-linux-amd64.tar.gz && \
tar -xzf helm.tgz && \
mv linux-amd64/helm /usr/local/bin && \
rm helm.tgz
RUN helm init --client-only
# CI job will update this version and make a PR
ARG QSEOK_VERSION=1.12.89
# This is likely not needed
ARG QLIKSENSE_INIT_VERSION=1.1.0
RUN mkdir -p /tmp/.chartcache
RUN helm repo add qlik-edge https://qlik.bintray.com/edge
RUN helm fetch qlik-edge/qliksense-init --version $QLIKSENSE_INIT_VERSION --untar -d /tmp/.chartcache/
# This is now done by qliksense mixin
#RUN helm fetch qlik-edge/qliksense --version $QSEOK_VERSION --untar -d /tmp/.chartcache/
# Use the BUNDLE_DIR build argument to copy files into the bundle
COPY . $BUNDLE_DIR

View File

@@ -1,42 +0,0 @@
name: test-version
version: 0.1.0
description: "An example Porter bundle demonstrating exec mixin outputs"
invocationImage: qlik/test-version:0.1.0
tag: qlik/test-version-bundle:0.1.0
dockerfile: Dockerfile.tmpl
mixins:
- exec
- qliksense
customActions:
about:
description: "About QLiksense"
stateless: true
modifies: true
install:
- exec:
description: "Get the qliksense version"
command: bash
arguments:
- -c
- "echo 1.2.13"
outputs:
- name: version
regex: '.*'
uninstall:
- exec:
description: "Get the qliksense version"
command: bash
arguments:
- -c
- "echo 1.2.13"
outputs:
- name: version
regex: '.*'
about:
- qliksense:
description: "Use bundled version by default"
version: "bundled"