Compare commits

..

2 Commits

Author SHA1 Message Date
bearium
f78aca62b6 added testing 2020-03-19 12:26:53 -04:00
bearium
4b4348df38 unset and unset-config commands 2020-03-19 11:59:06 -04:00
12 changed files with 218 additions and 448 deletions

View File

@@ -1,21 +0,0 @@
name: Publish docs via GitHub Pages
on:
push:
branches:
- master
paths:
- docs/
- mkdocs.yml
jobs:
build:
name: Deploy docs
runs-on: ubuntu-latest
steps:
- name: Checkout master
uses: actions/checkout@v1
- name: Deploy docs
uses: mhausenblas/mkdocs-deploy-gh-pages@1.11
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,23 +0,0 @@
# Qlik Sense installer documentation
## Local development of documentation
Documentation is built using [mkdocs](https://www.mkdocs.org/) and uses [Material for MKDocs theme](https://squidfunk.github.io/mkdocs-material/)
Requirements: Python and PIP or Docker
```console
pip install mkdocs
pip install mkdocs-material
```
View live changes locally at http://localhost:8000
```console
mkdocs serve
```
### Docker
```console
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
```

View File

@@ -64,6 +64,25 @@ qliksense config set <key>=<value>
return cmd
}
func unsetOtherConfigsCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
)
cmd = &cobra.Command{
Use: "un-set",
Short: "remove a key value pair in the current context",
Example: `
qliksense config unset <key>
- The above configuration will be removed in the CR
`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.UnsetOtherConfigs(args)
},
}
return cmd
}
func setConfigsCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
@@ -82,6 +101,24 @@ qliksense config set-configs <service_name>.<attribute>="<value>"
}
return cmd
}
func unsetConfigsCmd(q *qliksense.Qliksense) *cobra.Command {
var (
cmd *cobra.Command
)
cmd = &cobra.Command{
Use: "unset-configs",
Short: "unset configurations in the qliksense context as key-value pairs",
Example: `
qliksense config unset-configs <service_name>.<attribute>="<value>"
- The above configuration will be removed in the CR
`,
RunE: func(cmd *cobra.Command, args []string) error {
return q.UnsetConfigs(args)
},
}
return cmd
}
func setSecretsCmd(q *qliksense.Qliksense) *cobra.Command {
var (

View File

View File

@@ -1,83 +0,0 @@
# How qliksense cli works
At the initialization `qliksense` cli create few files in the director `~/.qliksene` and it contains following files
```console
.qliksense
├── config.yaml
├── contexts
│   └── qlik-default
│   └── qlik-default.yaml
└── ejson
└── keys
```
`qlik-default.yaml` is a default CR has been created with some default values like this
```yaml
apiVersion: qlik.com/v1
kind: Qliksense
metadata:
name: qlik-default
spec:
profile: docker-desktop
secrets:
qliksense:
- name: mongoDbUri
value: mongodb://qlik-default-mongodb:27017/qliksense?ssl=false
rotateKeys: "yes"
releaseName: qlik-default
```
The `qliksense` cli creates a default qliksense context (different from kubectl context) named `qlik-default` which will be the prefix for all kubernetes resources created by the cli under this context latter on. New context and configuration can be created by the cli.
```console
$ qliksense config -h
do operations on/around CR
Usage:
qliksense config [command]
Available Commands:
apply generate the patchs and apply manifests to k8s
list-contexts retrieves the contexts and lists them
set configure a key value pair into the current context
set-configs set configurations into the qliksense context as key-value pairs
set-context Sets the context in which the Kubernetes cluster and resources live in
set-secrets set secrets configurations into the qliksense context as key-value pairs
view view the qliksense operator CR
Flags:
-h, --help help for config
Use "qliksense config [command] --help" for more information about a command.
```
`qliksense` cli works in two modes
- with a git repo fork/clone of [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s)
- without git repo
## Without git repo
In this mode `qliksense` CLI download the specified version from [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s) and put it into folder `~/.qliksense/contexts/<context-name>/qlik-k8s`.
The qliksense cli create a CR for the qliksense operator and all the config operations are peformed to edit the CR. So when `qliksense install` or `qliksense config apply` both generate patches in local file system (i.e `~/.qliksense/contexts/<context-name>/qlik-k8s`) and install those manifests into the cluster and create a custom resoruce (CR) for the `qliksene operator` then the operator make association to the isntalled resoruces so that when `qliksenes uninstall` is performed the operator can delete all those kubernetes resources related to QSEoK for the current context.
## With a git repo
User has to create fork or clone of [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s) and push it to their own git server. When user perform `qliksense install` or `qliksene config apply` the qliksense operator do these tasks
- downloads the corresponding version of manifests from the user's git repo.
- generate kustomize patches
- install kubernetes resoruces
- push those generated patches into a new branch in the provided git repo. so that user user can merge those patches into their master branch.
- spinup a cornjob to monitor master branch. If user modifies anything in the master branch those changes will be applied into the cluster. This is a light weight `git-ops` model
This is how repo info is provided into the CR
```console
qliksense config set git.repository="https://github.com/my-org/qliksense-k8s"
qliksense config set git.accessToken=blablalaala
```

View File

@@ -1,37 +0,0 @@
# Getting started
## Requirements
- `kubectl` need to be installed and configured properly so that `kubectl` can connect to the kubernetes cluser. The `qliksense` CLI uses `kubectl` under the hood to perform operations on cluster
- (Docker Desktop setup tested for these instructions)
## Download
- Download the appropriate executable for your platform from the [releases page](https://github.com/qlik-oss/sense-installer/releases) and rename it to `qliksense`. All the examples below uses `qliksense`.
### Linux
```console
curl -fsSL https://raw.githubusercontent.com/qlik-oss/sense-installer/ibiqlik/installscript/scripts/install.sh -o install-sense-cli.sh
sh install-sense-cli.sh
```
## Quick start
- To download the version `v0.0.2` from qliksense-k8s [releases](https://github.com/qlik-oss/qliksense-k8s/releases).
```shell
$qliksense fetch v0.0.2
```
- To install CRDs for QSEoK and qliksense operator into the kubernetes cluster.
```shell
$qliksense crds install --all
```
- To install QSEoK into a namespace in the kubernetes cluster where `kubectl` is pointing to.
```shell
$qliksense install --acceptEULA="yes"
```

View File

@@ -1,19 +0,0 @@
# Overview
The Qlik Sense installer CLI (qliksense) provides an imperative interface to many of the configurations that need to be applied against the declarative structure described in [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s). This cli facilitates:
- installation of QSEoK
- installation of qliksense operator to manage QSEoK
- air gapped installation of QSEoK
This is a technology preview that uses Qlik modified [kustomize](https://github.com/qlik-oss/kustomize) to kubernetes manifests of the versions of the [qliksense-k8s](https://github.com/qlik-oss/qliksense-k8s) repository.
For each version of a qliksense edge build there should be a corresponding release in [qliksense-k8s] repository under [releases](https://github.com/qlik-oss/qliksense-k8s/releases)
## Future Direction
- More operations:
- Expand preflight checks
- backup/restore operations
- fully support airgap installation of QSEoK
- restore unwanted deletion of kubernetes resources

View File

@@ -1,18 +0,0 @@
site_name: Qlik Sense installer
repo_url: 'https://github.com/qlik-oss/sense-installer'
strict: true
theme:
name: "material"
palette:
primary: 'green'
accent: 'indigo'
markdown_extensions:
- toc:
permalink: true
- admonition
nav:
- Overview: index.md
- getting_started.md
- concepts.md
- air_gap.md
- Releases ⧉: https://github.com/qlik-oss/sense-installer/releases

View File

@@ -80,6 +80,31 @@ func ProcessConfigArgs(args []string) ([]*ServiceKeyValue, error) {
return resultSvcKV, nil
}
// ProcessUnsetConfigArgs processes args and returns an service, key, nil slice
func ProcessUnsetConfigArgs(args []string) ([]*ServiceKeyValue, error) {
if len(args) == 0 {
err := fmt.Errorf("No args were provided. Please provide args to configure the current context")
return nil, err
}
resultSvcKV := make([]*ServiceKeyValue, len(args))
re1 := regexp.MustCompile(`(\w{1,}).(\w{1,})`)
for i, arg := range args {
LogDebugMessage("Arg received: %s", arg)
result := re1.FindStringSubmatch(arg)
// check if result array's length is == 3 (index 0 - is the full match & indices 1,2,- are the fields we need)
if len(result) != 3 {
err := fmt.Errorf("Please provide valid args for this command")
return nil, err
}
resultSvcKV[i] = &ServiceKeyValue{
SvcName: result[1],
Key: result[2],
Value: "",
}
}
return resultSvcKV, nil
}
func ExecuteTaskWithBlinkingStdoutFeedback(task func() (interface{}, error), feedback string) (result interface{}, err error) {
taskDone := make(chan bool)
go func() {

View File

@@ -13,6 +13,7 @@ import (
"github.com/qlik-oss/k-apis/pkg/config"
b64 "encoding/base64"
ansi "github.com/mattn/go-colorable"
"github.com/qlik-oss/sense-installer/pkg/api"
"github.com/ttacon/chalk"
@@ -146,6 +147,27 @@ func (q *Qliksense) SetConfigs(args []string) error {
return nil
}
// SetConfigs - set-configs <key>=<value> commands
func (q *Qliksense) UnsetConfigs(args []string) error {
// retieve current context from config.yaml
qliksenseCR, qliksenseContextsFile, err := retrieveCurrentContextInfo(q)
if err != nil {
return err
}
resultArgs, err := api.ProcessUnsetConfigArgs(args)
if err != nil {
return err
}
for _, ra := range resultArgs {
qliksenseCR.Spec.AddToConfigs(ra.SvcName, ra.Key, ra.Value)
}
// write modified content into context.yaml
api.WriteToFile(&qliksenseCR, qliksenseContextsFile)
return nil
}
func retrieveCurrentContextInfo(q *Qliksense) (*api.QliksenseCR, string, error) {
var qliksenseConfig api.QliksenseConfig
qliksenseConfigFile := filepath.Join(q.QliksenseHome, QliksenseConfigFile)
@@ -237,6 +259,49 @@ func (q *Qliksense) SetOtherConfigs(args []string) error {
return nil
}
// SetOtherConfigs - set profile/storageclassname/git.repository/manifestRoot commands
func (q *Qliksense) UnsetOtherConfigs(args []string) error {
// retieve current context from config.yaml
qliksenseCR, qliksenseContextsFile, err := retrieveCurrentContextInfo(q)
if err != nil {
return err
}
// modify appropriate fields
if len(args) == 0 {
err := fmt.Errorf("No args were provided. Please provide args to configure the current context")
log.Println(err)
return err
}
for _, arg := range args {
switch arg {
case "profile":
qliksenseCR.Spec.Profile = ""
api.LogDebugMessage("Current profile after modification: %s ", qliksenseCR.Spec.Profile)
case "git.repository":
qliksenseCR.Spec.Git.Repository = ""
api.LogDebugMessage("Current git repository after modification: %s ", qliksenseCR.Spec.Git.Repository)
case "storageClassName":
qliksenseCR.Spec.StorageClassName = ""
api.LogDebugMessage("Current StorageClassName after modification: %s ", qliksenseCR.Spec.StorageClassName)
case "manifestsRoot":
qliksenseCR.Spec.ManifestsRoot = ""
case "rotateKeys":
qliksenseCR.Spec.RotateKeys = ""
api.LogDebugMessage("Current rotateKeys after modification: %s ", qliksenseCR.Spec.RotateKeys)
default:
err := fmt.Errorf("Please enter one of: profile, storageClassName,rotateKeys, manifestRoot or git.repository arguments to configure the current context")
log.Println(err)
return err
}
}
// write modified content into context.yaml
api.WriteToFile(&qliksenseCR, qliksenseContextsFile)
return nil
}
// SetContextConfig - set the context for qliksense kubernetes resources to live in
func (q *Qliksense) SetContextConfig(args []string) error {
if len(args) == 1 {
@@ -284,12 +349,10 @@ func (q *Qliksense) DeleteContextConfig(args []string) error {
out := ansi.NewColorableStdout()
switch args[0] {
case qliksenseConfig.Spec.CurrentContext:
fmt.Fprintln(out, chalk.Red.Color("Error: Cannot delete current context -"), chalk.Bold.TextStyle(qliksenseConfig.Spec.CurrentContext))
fmt.Fprintln(out, chalk.Yellow.Color("Please switch contexts to be able to delete this context."))
err := fmt.Errorf(chalk.Red.Color("Cannot delete current context - %s"), chalk.White.Color(chalk.Bold.TextStyle(qliksenseConfig.Spec.CurrentContext)))
return err
case DefaultQliksenseContext:
err := fmt.Errorf(chalk.Red.Color("Cannot delete default qliksense context"))
return err
fmt.Fprintln(out, chalk.Red.Color("Error: Cannot delete default qliksense context"))
default:
qliksenseContextsDir1 := filepath.Join(q.QliksenseHome, QliksenseContextsDir)
qliksenseContextFile := filepath.Join(qliksenseContextsDir1, args[0])
@@ -322,8 +385,7 @@ func (q *Qliksense) DeleteContextConfig(args []string) error {
fmt.Fprintln(out, chalk.Yellow.Color(chalk.Underline.TextStyle("Warning: Active resources may still be running in-cluster")))
fmt.Fprintln(out, chalk.Green.Color("Successfully deleted context: "), chalk.Bold.TextStyle(args[0]))
} else {
err := fmt.Errorf(chalk.Red.Color("Context not found"))
return err
fmt.Fprintln(out, chalk.Red.Color("Error: Context not found"))
}
}
}

View File

@@ -370,6 +370,60 @@ func TestSetOtherConfigs(t *testing.T) {
}
}
func TestUnsetOtherConfigs(t *testing.T) {
type args struct {
q *Qliksense
args []string
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid case",
args: args{
q: &Qliksense{
QliksenseHome: testDir,
},
args: []string{"profile", "rotateKeys", "storageClassName"},
},
wantErr: false,
},
{
name: "invalid configs",
args: args{
q: &Qliksense{
QliksenseHome: testDir,
},
args: []string{"someconfig"},
},
wantErr: true,
},
{
name: "empty configs",
args: args{
q: &Qliksense{
QliksenseHome: testDir,
},
args: []string{},
},
wantErr: true,
},
}
tearDown := setup()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
//set arguments to be removed
_ = tt.args.q.SetOtherConfigs([]string{"profile=minikube", "rotateKeys=yes", "storageClassName=efs"})
if err := tt.args.q.UnsetOtherConfigs(tt.args.args); (err != nil) != tt.wantErr {
t.Errorf("SetOtherConfigs() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestSetConfigs(t *testing.T) {
type args struct {
q *Qliksense
@@ -402,6 +456,40 @@ func TestSetConfigs(t *testing.T) {
}
}
func TestUnsetConfigs(t *testing.T) {
type args struct {
q *Qliksense
args []string
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid case",
args: args{
q: &Qliksense{
QliksenseHome: testDir,
},
args: []string{"qliksense.acceptEULA", "qliksense.mongoDbUri"},
},
wantErr: false,
},
}
tearDown := setup()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
//set arguments to be removed
_ = tt.args.q.SetConfigs([]string{"qliksense.acceptEULA=\"yes\"", "qliksense.mongoDbUri=\"mongo://mongo:3307\""})
if err := tt.args.q.UnsetConfigs(tt.args.args); (err != nil) != tt.wantErr {
t.Errorf("SetConfigs() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestSetImageRegistry(t *testing.T) {
getQlikSense := func(tmpQlikSenseHome string) (*Qliksense, error) {
if err := ioutil.WriteFile(path.Join(tmpQlikSenseHome, "config.yaml"), []byte(fmt.Sprintf(`
@@ -792,157 +880,3 @@ func getValueToBeDecodedForSetSecrets(item config.NameValue, qliksenseCR *api.Ql
err := fmt.Errorf("Both Value and ValueFrom are empty")
return "", err
}
func setupDeleteContext() func() {
if err := os.Mkdir(testDir, 0777); err != nil {
log.Printf("\nError occurred: %v", err)
}
config :=
`
apiVersion: config.qlik.com/v1
kind: QliksenseConfig
metadata:
name: qliksenseConfig
spec:
contexts:
- name: qlik-default
crFile: /root/.qliksense/contexts/qlik-default.yaml
- name: qlik1
crFile: /root/.qliksense/contexts/qlik1.yaml
- name: qlik2
crFile: /root/.qliksense/contexts/qlik2.yaml
currentContext: qlik1
`
configFile := filepath.Join(testDir, "config.yaml")
// tests/config.yaml exists
ioutil.WriteFile(configFile, []byte(config), 0777)
contextYaml :=
`
apiVersion: qlik.com/v1
kind: Qliksense
metadata:
name: qlik-default
spec:
profile: docker-desktop
rotateKeys: "yes"
releaseName: qlik-default
`
qlikDefaultContext := "qlik-default"
// create contexts/qlik-default/ under tests/
contexts := "contexts"
contextsDir1 := filepath.Join(testDir, contexts, qlikDefaultContext)
if err := os.MkdirAll(contextsDir1, 0777); err != nil {
err = fmt.Errorf("Not able to create directories")
log.Fatal(err)
}
contextYaml1 :=
`
apiVersion: qlik.com/v1
kind: Qliksense
metadata:
name: qlik1
spec:
profile: docker-desktop
rotateKeys: "yes"
releaseName: qlik1`
contextYaml2 :=
`
apiVersion: qlik.com/v1
kind: Qliksense
metadata:
name: qlik2
spec:
profile: docker-desktop
rotateKeys: "yes"
releaseName: qlik2`
contextsDir := filepath.Join(testDir, contexts, "qlik1")
if err := os.MkdirAll(contextsDir, 0777); err != nil {
err = fmt.Errorf("Not able to create directories")
log.Fatal(err)
}
contextsDir2 := filepath.Join(testDir, contexts, "qlik2")
if err := os.MkdirAll(contextsDir2, 0777); err != nil {
err = fmt.Errorf("Not able to create directories")
log.Fatal(err)
}
contextFile := filepath.Join(contextsDir, "qlik1.yaml")
ioutil.WriteFile(contextFile, []byte(contextYaml1), 0777)
contextFile2 := filepath.Join(contextsDir2, "qlik2.yaml")
ioutil.WriteFile(contextFile2, []byte(contextYaml2), 0777)
contextFile1 := filepath.Join(contextsDir1, "qlik-default.yaml")
ioutil.WriteFile(contextFile1, []byte(contextYaml), 0777)
tearDown := func() {
os.RemoveAll(testDir)
}
return tearDown
}
func TestDeleteContexts(t *testing.T) {
type args struct {
qlikSenseHome string
contextName string
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "valid context",
args: args{
qlikSenseHome: testDir,
contextName: "qlik2",
},
wantErr: false,
},
{
name: "default context",
args: args{
qlikSenseHome: testDir,
contextName: "qlik-default",
},
wantErr: true,
},
{
name: "non-existent context",
args: args{
qlikSenseHome: testDir,
contextName: "qlik3",
},
wantErr: true,
},
{
name: "current context",
args: args{
qlikSenseHome: testDir,
contextName: "qlik1",
},
wantErr: true,
},
}
tearDown := setupDeleteContext()
defer tearDown()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
q, err := New(tt.args.qlikSenseHome)
if err != nil {
t.Errorf("unable to create a qliksense instance")
return
}
var arg []string
arg = append(arg, tt.args.contextName)
if err := q.DeleteContextConfig(arg); (err != nil) != tt.wantErr {
t.Errorf("DeleteContext() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

View File

@@ -1,87 +0,0 @@
#!/bin/sh
set -e
# Sense installer for Linux/MacOS script
#
# See https://github.com/qlik-oss/sense-installer for the installation steps.
#
# This script is meant for quick & easy install via:
# $ curl -fsSL https://raw.githubusercontent.com/qlik-oss/sense-installer/scripts/install.sh | sh -
RELEASES_URI="https://api.github.com/repos/qlik-oss/sense-installer/releases/latest"
REPO_URL="https://github.com/qlik-oss/sense-installer"
BIN_DIR="/usr/local/bin"
FILE_NAME="kubectl-qliksense"
command_exists() {
command -v "$@" > /dev/null 2>&1
}
do_install() {
echo "\n==> Executing qliksense install script\n"
if ! command_exists kubectl; then
echo "\n==> ERROR: kubectl is required for $FILE_NAME to work"
echo "See https://github.com/qlik-oss/sense-installer#requirements for more information\n"
exit 1
fi
if ! command_exists curl; then
echo "==> ERROR: curl is missing"
exit 1
fi
user="$(id -un 2>/dev/null || true)"
SUDO='sh -c'
if [ "$(id -u)" != "0" ]; then
root_msg="\tNext operation might ask for root password to place\n\t$FILE_NAME in $BIN_DIR and set executable permission\n"
if command_exists sudo; then
echo $root_msg
SUDO='sudo -E sh -c'
elif command_exists su; then
echo $root_msg
SUDO='su -c'
else
cat >&2 <<-'EOF'
Error: this installer needs the ability to run commands as root.
We are unable to find either "sudo" or "su" available to make this happen.
EOF
exit 1
fi
fi
if command_exists curl; then
releases=$(mktemp)
if [ -n "$GITHUB_TOKEN" ]; then
curl -v -H "Authorization: token $GITHUB_TOKEN" $RELEASES_URI > $releases 2>&1
else
curl -v $RELEASES_URI > $releases 2>&1
fi
if ! grep -q "Status: 200" $releases; then
echo "==> ERROR: cannot get qliksense-installer"
echo "GitHub:" $(grep "message" $releases | cut -d '"' -f 4) "\n"
echo "Use: GITHUB_TOKEN=token install-sense-cli.sh"
exit 1
fi
download_url=$(grep -E "browser_download_url.*linux" $releases | grep -v "tar.gz" | cut -d '"' -f 4)
echo "==> Installing to $BIN_DIR/$FILE_NAME\n"
if [ -n "$GITHUB_TOKEN" ]; then
$SUDO "curl -fSL -H \"Authorization: token $GITHUB_TOKEN\" $download_url -o $BIN_DIR/$FILE_NAME"
else
$SUDO "curl -fSL $download_url -o $BIN_DIR/$FILE_NAME"
fi
$SUDO "chmod +x $BIN_DIR/$FILE_NAME"
fi
if command_exists $FILE_NAME; then
echo "\n==> Success: You can now start using $FILE_NAME\n"
else
echo "\n==> ERROR: Something went wrong, try again"
fi
}
do_install