Test DevOps stack using k3s on libvirt

Setting up extra terraform provider

Due to an issue in the terraform provider system, you need to download and place the libvirt provider at a very specific spot in your home directory before deploying k3s. You can find the libvirt provider version in modules/k3s/libvirt/versions.tf to keep things simple this version will be referred as $LIBVIRT_PROVIDER_VERSION from now on. Go to https://github.com/dmacvicar/terraform-provider-libvirt/releases/tag/v$LIBVIRT_PROVIDER_VERSION and find the correct link for your OS/CPU_ARCH (an example is "linux_amd64"; now refered as $OS_CPU_ARCH).

mkdir -p ~/.local/share/terraform/plugins/registry.terraform.io/dmacvicar/libvirt/$LIBVIRT_PROVIDER_VERSION/$OS_CPU_ARCH/
mv terraform-provider-libvirt ~/.local/share/terraform/plugins/registry.terraform.io/dmacvicar/libvirt/$LIBVIRT_PROVIDER_VERSION/$OS_CPU_ARCH/terraform-provider-libvirt

Finding the flavor

In the tests folder at the root of the repository, you can find all flavors that are ready to use. The name of the folders describe the flavor of the DevOps stack that will be installed. For example:

$ ls tests/
k3s-docker
k3s-libvirt

You can see that, here, both flavors are k3s and one installs it using docker while the other uses libvirt.

Deploying using Makefile

The folder of the flavor you are trying to test contains a terraform repository. Typing terraform init && terraform apply in that folder is sufficient to deploy the DevOps stack.

Cleaning up

terraform destroy will delete everything that was deployed for the test.

Finding the URLs of the services

The Access URLs are found as output of terraform, look in your console for something like:

Outputs:

admin_password = "dEzq88Zx0SY2FSFQ"
alertmanager_url = "https://alertmanager.apps.10-17-3-127.nip.io"
app_of_apps_values = <sensitive>
argocd_auth_token = <sensitive>
argocd_url = "https://argocd.apps.10-17-3-127.nip.io"
base_domain = "10-17-3-127.nip.io"
grafana_url = "https://grafana.apps.10-17-3-127.nip.io"
kubeconfig = <sensitive>
prometheus_url = "https://prometheus.apps.10-17-3-127.nip.io"
repo_url = "https://github.com/.git"
target_revision = "master"

Finding the kubeconfig

The kubeconfig file is written on disk by terraform the path is terraform/kubeconfig.yaml in the flavor directory.

Exporting the KUBECONFIG variable to point to that file will allow you to use the kubectl and other kubernetes tools to manipulate your cluster.

Testing a terraform change

By simply running the Makefile again all changes to the terraform code will be applied.

Making a change to an Application deployed by ArgoCD

Due to how ArgoCD works, you need to commit and push your change to a fork of the project for it to be visible. ArgoCD should be in auto-sync by default, if it is not, you can visit the administration page of ArgoCD which url was given in the terraform ouput and manually sync.