개요
최초 Kubespray글을 작성했을 때의 쿠버네티스 버전은 1.29.5였습니다.
[root@k8smaster1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smaster1 Ready control-plane 74d v1.29.5
k8smaster2 Ready control-plane 74d v1.29.5
k8smaster3 Ready control-plane 74d v1.29.5
k8sworker1 Ready worker 74d v1.29.5
k8sworker2 Ready worker 74d v1.29.5
k8sworker3 Ready worker 74d v1.29.5Kubespray도 2.26으로 업데이트 되었기 때문에 이를 이용해 1.30.4로 클러스터를 업데이트 해 보겠습니다.
제 환경은 아래와 같습니다.
Ansible 전용 VM 1대 + Master 노드(VM) 3대 + Worker 노드(VM) 3대
모든 OS는 Rocky Linux입니다.
Kubespray 업데이트
현재 Kubespray버전은 2.26입니다.
패치노트에서 지원하는 쿠버네티스 버전을 확인할 수 있습니다.
1.30.4버전 기본화 및 1.27버전대 삭제
kubespray를 설치했던 VM에서 git clone으로 생성했던 kubespray폴더로 진입해서 아래와 같이 입력하면 2.26 버전의 변경사항에 대해 fetch, merge과정을 거칩니다.
[root@localhost kubespray]# git pull https://github.com/kubernetes-sigs/kubespray.git release-2.26
Cluster 업그레이드
주의사항
Cluster를 구성 후 추가된 노드가 있을 경우 inventory.ini에 반영해야 하며, addons.yml을 통해 구성했던 애드온들의 값이 변경되었을 경우 Kubespray에서도 이를 반영해야 합니다.
업그레이드 수행
지난 글에서 작성했던 파이썬 가상환경에 진입한 후, ping을 통해 정상적으로 연결되는지 확인합니다.
(kubespray-venv) [root@localhost kubespray]# ansible all -m ping -i inventory/k8s_fenta/inventory.ini
정상적으로 연결이 이루어진 것을 확인했다면 아래 명령어를 통해 업그레이드를 수행합니다.
ansible-playbook upgrade-cluster.yml -i inventory/k8s_fenta/inventory.ini -e ansible_ssh_timeout=50 -vvv -b --become-user root쿠버네티스 특정 버전을 지정하기 위해선 환경 변수(-e kube_version)를 추가합니다.
ansible-playbook upgrade-cluster.yml -i inventory/k8s_fenta/inventory.ini -e ansible_ssh_timeout=50 -e kube_version=v1.30.4 -vvv -b --become-user root이후 무시무시한 양의 로그가 지나갑니다. 업데이트 되는 동안 노드에서 버전이 변경되는 것을 실시간으로 확인할 수 있습니다(upgrade-cluster는 클러스터 한 대씩 업데이트를 수행합니다).
[root@k8smaster1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smaster1 Ready control-plane 74d v1.30.4
k8smaster2 NotReady,SchedulingDisabled control-plane 74d v1.30.4
k8smaster3 Ready control-plane 74d v1.29.5
k8sworker1 Ready worker 74d v1.29.5
k8sworker2 Ready worker 74d v1.29.5
k8sworker3 Ready worker 74d v1.29.5최종적으로 아래처럼 출력되며 모든 클러스터의 업그레이드가 끝납니다.
제 사양 기준 50분 정도가 소요되었으니, 느긋하게 기다리시면 됩니다.
[root@k8smaster1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smaster1 Ready control-plane 74d v1.30.4
k8smaster2 Ready control-plane 74d v1.30.4
k8smaster3 Ready control-plane 74d v1.30.4
k8sworker1 Ready worker 74d v1.30.4
k8sworker2 Ready worker 74d v1.30.4
k8sworker3 Ready worker 74d v1.30.4비정상작동하는 파드가 있는지도 살펴보겠습니다.
[root@k8smaster1 ~]# kubectl get pods -ANAMESPACE NAME READY STATUS RESTARTS AGE
authentik authentik-postgresql-0 1/1 Running 0 19m
authentik authentik-redis-master-0 1/1 Running 0 31m
authentik authentik-server-56c8ccfcbf-lh246 1/1 Running 0 24m
authentik authentik-server-56c8ccfcbf-v2tzh 1/1 Running 1 (24m ago) 31m
authentik authentik-worker-785cfff9ff-kwvdg 1/1 Running 0 31m
authentik authentik-worker-785cfff9ff-rr76l 1/1 Running 6 (18m ago) 24m
authentik authentik-worker-785cfff9ff-w2x88 1/1 Running 6 (18m ago) 24m
cloudflare cloudflare-cloudflare-tunnel-remote-6d6fc5d96d-srnjx 1/1 Running 0 31m
cloudflare cloudflare-cloudflare-tunnel-remote-6d6fc5d96d-wg5qn 1/1 Running 0 24m
collabora collabora-collabora-online-868c7dbb44-79xrw 1/1 Running 0 31m
collabora collabora-collabora-online-868c7dbb44-lfwqw 1/1 Running 0 24m
immich immich-machine-learning-6cdc78c65d-scmz4 1/1 Running 0 24m
immich immich-postgresql-0 1/1 Running 0 24m
immich immich-redis-master-0 1/1 Running 0 31m
immich immich-server-76cb48f56d-d4nt7 1/1 Running 5 (18m ago) 24m
ingress-nginx ingress-nginx-controller-cff95bf8b-nn9h4 1/1 Running 0 31m
komga komga-bdb54df5d-qw6tx 1/1 Running 0 31m
kube-system calico-kube-controllers-86884b6c49-xl5kp 1/1 Running 0 40m
kube-system calico-node-d26cn 1/1 Running 0 40m
kube-system calico-node-d9pbf 1/1 Running 0 39m
kube-system calico-node-hzcpl 1/1 Running 0 39m
kube-system calico-node-v7z74 1/1 Running 0 39m
kube-system calico-node-x86t9 1/1 Running 0 39m
kube-system calico-node-xt426 1/1 Running 0 40m
kube-system coredns-565f455cf4-5xhdz 1/1 Running 0 17m
kube-system coredns-565f455cf4-tkqlw 1/1 Running 0 17m
kube-system csi-nfs-controller-7d5b7c6584-bnwgp 4/4 Running 0 31m
kube-system csi-nfs-node-6dr9c 3/3 Running 12 (10d ago) 56d
kube-system csi-nfs-node-j5vl9 3/3 Running 9 (10d ago) 56d
kube-system csi-nfs-node-l44td 3/3 Running 9 (10d ago) 56d
kube-system csi-nfs-node-rnjw8 3/3 Running 12 (10d ago) 56d
kube-system csi-nfs-node-xmhrx 3/3 Running 12 (10d ago) 56d
kube-system csi-nfs-node-zt49g 3/3 Running 9 (10d ago) 56d
kube-system csi-smb-controller-675f7b5458-2sfx7 3/3 Running 0 24m
kube-system csi-smb-node-2jjqb 3/3 Running 9 (10d ago) 56d
kube-system csi-smb-node-jrx82 3/3 Running 12 (10d ago) 56d
kube-system csi-smb-node-mfbh2 3/3 Running 12 (10d ago) 56d
kube-system csi-smb-node-mgb4h 3/3 Running 9 (10d ago) 56d
kube-system csi-smb-node-n6rlr 3/3 Running 12 (10d ago) 56d
kube-system csi-smb-node-w52sk 3/3 Running 9 (10d ago) 56d
kube-system dns-autoscaler-6f4b597d8c-nch2d 1/1 Running 0 45m
kube-system keel-79dc84dbc8-s625c 1/1 Running 0 31m
kube-system kube-apiserver-k8smaster1 1/1 Running 0 52m
kube-system kube-apiserver-k8smaster2 1/1 Running 0 47m
kube-system kube-apiserver-k8smaster3 1/1 Running 0 42m
kube-system kube-controller-manager-k8smaster1 1/1 Running 0 51m
kube-system kube-controller-manager-k8smaster2 1/1 Running 0 46m
kube-system kube-controller-manager-k8smaster3 1/1 Running 0 41m
kube-system kube-proxy-b5z5j 1/1 Running 0 31m
kube-system kube-proxy-kwmzd 1/1 Running 0 31m
kube-system kube-proxy-mggh4 1/1 Running 0 31m
kube-system kube-proxy-vcwfs 1/1 Running 0 31m
kube-system kube-proxy-vqjtq 1/1 Running 0 31m
kube-system kube-proxy-zm474 1/1 Running 0 31m
kube-system kube-scheduler-k8smaster1 1/1 Running 0 51m
kube-system kube-scheduler-k8smaster2 1/1 Running 0 46m
kube-system kube-scheduler-k8smaster3 1/1 Running 0 41m
kube-system kubernetes-dashboard-759b858758-fkl8b 1/1 Running 0 31m
kube-system kubernetes-metrics-scraper-7b78d8fdd8-7l8xd 1/1 Running 0 45m
kube-system nginx-proxy-k8sworker1 1/1 Running 0 31m
kube-system nginx-proxy-k8sworker2 1/1 Running 0 24m
kube-system nginx-proxy-k8sworker3 1/1 Running 0 18m
kube-system nodelocaldns-4pnh8 1/1 Running 2 (51m ago) 51m
kube-system nodelocaldns-bzg7c 1/1 Running 0 50m
kube-system nodelocaldns-fcl4z 1/1 Running 2 (51m ago) 51m
kube-system nodelocaldns-jktk6 1/1 Running 1 (50m ago) 50m
kube-system nodelocaldns-nb4nb 1/1 Running 0 50m
kube-system nodelocaldns-twrjr 1/1 Running 1 (50m ago) 50m
kube-system snapshot-controller-5b45bc8c58-7t49s 1/1 Running 0 31m
kured kured-27d24 1/1 Running 0 10d
kured kured-7s65c 1/1 Running 0 10d
kured kured-97zdz 1/1 Running 0 10d
kured kured-d4c9n 1/1 Running 0 10d
kured kured-gxrrm 1/1 Running 0 10d
kured kured-mth6b 1/1 Running 0 10d
metallb metallb-controller-665d96757f-fkmsw 1/1 Running 0 24m
metallb metallb-speaker-7fdsv 4/4 Running 1 (10d ago) 10d
metallb metallb-speaker-bt4zx 4/4 Running 0 10d
metallb metallb-speaker-cqd7q 4/4 Running 0 10d
metallb metallb-speaker-mmzsg 4/4 Running 0 10d
metallb metallb-speaker-whgrh 4/4 Running 0 10d
metallb metallb-speaker-xdbnl 4/4 Running 0 10d
monitoring alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 24m
monitoring prometheus-grafana-849dcb8b9f-ksrhr 3/3 Running 0 31m
monitoring prometheus-kube-prometheus-operator-594fb4d6d6-m95j6 1/1 Running 0 31m
monitoring prometheus-kube-state-metrics-6bd66c46df-nrzvb 1/1 Running 0 24m
monitoring prometheus-prometheus-kube-prometheus-prometheus-0 2/2 Running 0 31m
monitoring prometheus-prometheus-node-exporter-52t7n 1/1 Running 0 10d
monitoring prometheus-prometheus-node-exporter-8wnch 1/1 Running 0 10d
monitoring prometheus-prometheus-node-exporter-fhvg2 1/1 Running 0 10d
monitoring prometheus-prometheus-node-exporter-kcnrm 1/1 Running 0 10d
monitoring prometheus-prometheus-node-exporter-mzxmx 1/1 Running 0 10d
monitoring prometheus-prometheus-node-exporter-vrdks 1/1 Running 0 10d
nextcloud nextcloud-7c8d4c5d67-zs2q6 2/2 Running 0 24m
nextcloud nextcloud-metrics-685cfb46f6-jkjgm 1/1 Running 0 24m
nextcloud nextcloud-redis-master-0 1/1 Running 0 31m
nextcloud nextcloud-redis-replicas-0 1/1 Running 0 31m
postgresql postgresql-0 1/1 Running 0 24m
selenium selenium-chrome-node-55c7f65b6d-lmrx2 2/2 Running 0 24m
selenium selenium-chrome-node-55c7f65b6d-qm7xj 2/2 Running 0 31m
selenium selenium-chrome-node-55c7f65b6d-s5d2s 2/2 Running 0 31m
selenium selenium-chrome-node-55c7f65b6d-ttfrh 2/2 Running 0 38m
selenium selenium-chrome-node-55c7f65b6d-z2t42 2/2 Running 0 31m
selenium selenium-edge-node-7b95fdcf5f-5rxgk 2/2 Running 0 24m
selenium selenium-firefox-node-56d56fdd89-tcj7w 2/2 Running 0 24m
selenium selenium-hub-5c7d99c6d8-pv67q 1/1 Running 0 31m
stirlingpdf stirlingpdf-754d765759-l2782 1/1 Running 2 (18m ago) 24m
wopiserver wopiserver-7fb9855d8-4bkx4 1/1 Running 0 31m
모두 정상작동중이네요!
관련 글
Kubespray로 쿠버네티스(Kubernetes, K8S) 설치하기
