K3s on Libvirt Quickstart

Setting up extra terraform provider

Due to an issue in the terraform provider system, you need to download and place the libvirt provider at a very specific location in your home directory before deploying K3s. You can find the libvirt provider version in modules/k3s/libvirt/versions.tf. To keep things simple this version will be referred to as $LIBVIRT_PROVIDER_VERSION in this documentation. Go to https://github.com/dmacvicar/terraform-provider-libvirt/releases/tag/v$LIBVIRT_PROVIDER_VERSION and find the correct link for your OS/CPU_ARCH (an example is linux_amd64, referred to as $OS_CPU_ARCH in the rest of this documentation).

mkdir -p ~/.local/share/terraform/plugins/registry.terraform.io/dmacvicar/libvirt/$LIBVIRT_PROVIDER_VERSION/$OS_CPU_ARCH/
mv terraform-provider-libvirt ~/.local/share/terraform/plugins/registry.terraform.io/dmacvicar/libvirt/$LIBVIRT_PROVIDER_VERSION/$OS_CPU_ARCH/terraform-provider-libvirt

Prerequisites

Access to a functional Libvirt daemon
Knowledge of Terraform basics

Create your Terraform root module

Camptocamp’s DevOps Stack is instantiated using a Terraform composition module.

Here is a minimal working example:

module "cluster" {
  source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/libvirt?ref=master"

  cluster_name = "my-cluster"
  node_count   = 2
}

Terraform Outputs

Define outputs:

# terraform/outputs.tf

output "argocd_auth_token" {
  sensitive = true
  value     = module.cluster.argocd_auth_token
}

output "kubeconfig" {
  sensitive = true
  value     = module.cluster.kubeconfig
}

output "argocd_server" {
  value = module.cluster.argocd_server
}

output "grafana_admin_password" {
  sensitive = true
  value     = module.cluster.grafana_admin_password
}

Deploy the cluster

$ terraform init
$ terraform apply

You should see the services URL as Terraform outputs.

Get kubeconfig and admin password

Retrieve the Kubeconfig file:

$ terraform output -json kubeconfig | jq -r .

Retrieve the Keycloak password for the admin user of the kubernetes realm:

$ terraform output admin_password

You will use this user and password to log in to applications.

Wait for Keycloak to be ready

$ kubectl -n keycloak get sts
NAME       READY   AGE
keycloak   1/1     8m58s

Wait until the READY column says 1/1.

Inspect the DevOps Stack Applications

You can view the ingress routes for the various DevOps Stack Applications with:

$ kubectl get ingress --all-namespaces

Access the URLs in https, and use the OIDC/OAuth2 to log in, using the admin account with the password previously retrieved.

Access the Keycloak dashboard

The keycloak dashboard uses the kubernetes realm. You can log in to it using the /auth/realms/kubernetes/account/ path with the Keycloak ingress.

there is currently an issue when accessing applications and login details in Keycloak.

Destroy the cluster

$ terraform destroy

Reference

See the K3s Libvirt reference page.