I'm trying to set up a K8s cluster on Tumbleweed [TW] to learn some stuff, but I can't get it to work.

I have 5 VMs [3 control-plane-nodes to be, and 2 workers-to-be].

On the first control node|VM, I'm doing the following to bootstrap the cluster:

bash control@cp01:~> sudo kubeadm init phase preflight [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action beforehand using 'kubeadm config images pull' control@cp01:~> sudo kubeadm init --config /etc/kubernetes/kubeadm-config.yaml --upload-certs [init] Using Kubernetes version: v1.36.3 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [cp01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.paap.local] and IPs [10.96.0.1 10.10.30.10 10.10.30.5] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [cp01 localhost] and IPs [10.10.30.10 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [cp01 localhost] and IPs [10.10.30.10 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "super-admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml" [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet 👀 error: error execution phase wait-control-plane: cannot obtain client without bootstrap: could not bootstrap the admin user in file admin.conf: unable to create ClusterRoleBinding: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline To see the stack trace of this error execute with --v=5 or higher

Now, before running sudo kuebeadm init, I set up the kube-vip manifest as

bash control@cp01:~> cat /etc/kubernetes/manifests/kube-vip.yaml apiVersion: v1 kind: Pod metadata: name: kube-vip namespace: kube-system spec: containers: - args: - manager env: - name: vip_arp value: "true" - name: port value: "6443" - name: vip_nodename valueFrom: fieldRef: fieldPath: spec.nodeName - name: vip_interface value: enp1s0 - name: vip_subnet value: "32" - name: dns_mode value: first - name: dhcp_mode value: ipv4 - name: cp_enable value: "true" - name: cp_namespace value: kube-system - name: vip_leaderelection value: "true" - name: vip_leasename value: plndr-cp-lock - name: vip_leaseduration value: "15" - name: vip_renewdeadline value: "10" - name: vip_retryperiod value: "2" - name: address value: 10.10.30.5 - name: prometheus_server value: :2112 image: ghcr.io/kube-vip/kube-vip:v1.2.2 imagePullPolicy: IfNotPresent name: kube-vip resources: {} securityContext: capabilities: add: - NET_ADMIN - NET_RAW drop: - ALL volumeMounts: - mountPath: /etc/kubernetes/admin.conf name: kubeconfig hostAliases: - hostnames: - kubernetes ip: 127.0.0.1 hostNetwork: true volumes: - hostPath: path: /etc/kubernetes/admin.conf # tried `super-admin.conf` 2 name: kubeconfig status: {}

And to sudo kubeadm init, I passed the following config

“`bash control@cp01:~> cat /etc/kubernetes/kubeadm-config.yaml apiVersion: kubeadm.k8s.io/v1beta4 kind: InitConfiguration skipPhases: – addon/kube-proxy nodeRegistration: criSocket: "unix:///var/run/crio/crio.sock" localAPIEndpoint: advertiseAddress: "10.10.30.10"

bindPort: 6443

apiVersion: kubeadm.k8s.io/v1beta4 kind: ClusterConfiguration kubernetesVersion: "v1.36.3" controlPlaneEndpoint: "10.10.30.5:6443" networking: podSubnet: "10.244.0.0/16" serviceSubnet: "10.96.0.0/12" dnsDomain: "paap.local" proxy:

disabled: true

apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd clusterDomain: "paap.local" “`

I noted that kube-vip didn't get created, somehow; I mean, I can't see it here:

“`bash control@cp01:~> sudo kubeadm init –config /etc/kubernetes/kubeadm-config.yaml –upload-certs

…

[control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler"

…

“`

And also, crictl returns nothing on kube-vip, meaning kube-vip wasn't picked up at all?!!🤔
And yes, since kube-vip didn't get properly set up, the apiserver somehow wasn't accessible at the VIP 10.10.30.5?

I've tried everything I found online.

PS:
– please don't tell me to use another Distro; there's a particular reason why testing this on SUSE
– yes, container-selinux is installed; getenforce returns Enforcing
– I thought it was a kube-vip problem, but if I sudo setenforce 0, the cluster bootstrapping process finishes successfully¹, though this isn't what really what one wants?! I'm sure there's another way to get this working? On Fedora 44 Cloud it works even with SELinux enforcing.
– the Tumbleweed image I'm using is openSUSE-Tumbleweed-Minimal-VM.x86_64-Cloud.qcow2 from OpenSUSE Appliances

Thanks for any help.

¹

“`bash

…

[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml" [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet 👀 [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests" [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 279.684µs [control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s [control-plane-check] Checking kube-apiserver at https://10.10.30.10:6443/livez [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez [control-plane-check] kube-scheduler is healthy after 1.710405ms [control-plane-check] kube-controller-manager is healthy after 2.322778ms [control-plane-check] kube-apiserver is healthy after 1.501584092s [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace [upload-certs] Using certificate key: bafec1e64cdc807aee82d97ba2bd0631d435b3c7968a3b99af26abd23319548d [mark-control-plane] Marking the node cp01 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node cp01 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: 4wocea.ryvletddn47fuwqk [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS

Your Kubernetes control-plane has initialized successfully! “`