[root@k8s-master ~]# kubeadm upgrade plan [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [preflight] Running pre-flight checks. [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: v1.18.0 [upgrade/versions] kubeadm version: v1.18.0-dirty I0906 09:48:57.817361 3822642 version.go:252] remote version is much newer: v1.22.1; falling back to: stable-1.18 [upgrade/versions] Latest stable version: v1.18.20 [upgrade/versions] Latest stable version: v1.18.20 [upgrade/versions] Latest version in the v1.18 series: v1.18.20 [upgrade/versions] Latest version in the v1.18 series: v1.18.20
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply': COMPONENT CURRENT AVAILABLE Kubelet 3 x v1.18.0 v1.18.20
Upgrade to the latest version in the v1.18 series:
COMPONENT CURRENT AVAILABLE API Server v1.18.0 v1.18.20 Controller Manager v1.18.0 v1.18.20 Scheduler v1.18.0 v1.18.20 Kube Proxy v1.18.0 v1.18.20 CoreDNS 1.6.7 1.6.7 Etcd 3.4.3 3.4.3-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.18.20
Note: Before you can perform this upgrade, you have to update kubeadm to v1.18.20.
[root@k8s-master ~]# kubeadm upgrade apply v1.18.20 [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [preflight] Running pre-flight checks. [upgrade] Running cluster health checks [upgrade/version] You have chosen to change the cluster version to "v1.18.20" [upgrade/versions] Cluster version: v1.18.0 [upgrade/versions] kubeadm version: v1.18.20 [upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y [upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd] [upgrade/prepull] Prepulling image for component etcd. [upgrade/prepull] Prepulling image for component kube-apiserver. [upgrade/prepull] Prepulling image for component kube-controller-manager. [upgrade/prepull] Prepulling image for component kube-scheduler. [apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver [apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler [apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager [apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd [apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler [apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd [upgrade/prepull] Prepulled image for component etcd. [upgrade/prepull] Prepulled image for component kube-controller-manager. [upgrade/prepull] Prepulled image for component kube-scheduler. [upgrade/prepull] Prepulled image for component kube-apiserver. [upgrade/prepull] Successfully prepulled the images for all the control plane components [upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.18.20"... Static pod: kube-apiserver-k8s-master hash: 19d5d1ced032267ed77feb4529dd89b6 Static pod: kube-controller-manager-k8s-master hash: c99654aba4d0627786d6f7f6ca3cf8a4 Static pod: kube-scheduler-k8s-master hash: b901723f22729fcf965378cb64525117 [upgrade/etcd] Upgrading to TLS for etcd [upgrade/etcd] Non fatal issue encountered during upgrade: the desired etcd version for this Kubernetes version "v1.18.20" is "3.4.3-0", but the current etcd version is "3.4.3". Won't downgrade etcd, instead just continue [upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests350292254" W0906 12:02:13.995741 3880622 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" [upgrade/staticpods] Preparing for "kube-apiserver" upgrade [upgrade/staticpods] Renewing apiserver certificate [upgrade/staticpods] Renewing apiserver-kubelet-client certificate [upgrade/staticpods] Renewing front-proxy-client certificate [upgrade/staticpods] Renewing apiserver-etcd-client certificate [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2021-09-06-12-02-12/kube-apiserver.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) Static pod: kube-apiserver-k8s-master hash: 19d5d1ced032267ed77feb4529dd89b6 Static pod: kube-apiserver-k8s-master hash: 4b88bf8fbacc7b265a069eb9c4b66b28 [apiclient] Found 1 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component "kube-apiserver" upgraded successfully! [upgrade/staticpods] Preparing for "kube-controller-manager" upgrade [upgrade/staticpods] Renewing controller-manager.conf certificate [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2021-09-06-12-02-12/kube-controller-manager.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) Static pod: kube-controller-manager-k8s-master hash: c99654aba4d0627786d6f7f6ca3cf8a4 Static pod: kube-controller-manager-k8s-master hash: 36c14c513b299c15f058e6c085d03822 [apiclient] Found 1 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component "kube-controller-manager" upgraded successfully! [upgrade/staticpods] Preparing for "kube-scheduler" upgrade [upgrade/staticpods] Renewing scheduler.conf certificate [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2021-09-06-12-02-12/kube-scheduler.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) Static pod: kube-scheduler-k8s-master hash: b901723f22729fcf965378cb64525117 Static pod: kube-scheduler-k8s-master hash: f8fbdc8109ec0a5dd59acbd52a6c05ca [apiclient] Found 1 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component "kube-scheduler" upgraded successfully! [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.18.20". Enjoy!
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
[root@k8s-master ~]# kubeadm upgrade node [upgrade] Reading configuration from the cluster... [upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade] Upgrading your Static Pod-hosted control plane instance to version "v1.18.20"... Static pod: kube-apiserver-k8s-master hash: 4b88bf8fbacc7b265a069eb9c4b66b28 Static pod: kube-controller-manager-k8s-master hash: 36c14c513b299c15f058e6c085d03822 Static pod: kube-scheduler-k8s-master hash: f8fbdc8109ec0a5dd59acbd52a6c05ca [upgrade/etcd] Upgrading to TLS for etcd [upgrade/etcd] Non fatal issue encountered during upgrade: the desired etcd version for this Kubernetes version "v1.18.20" is "3.4.3-0", but the current etcd version is "3.4.3". Won't downgrade etcd, instead just continue [upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests324894338" W0906 13:40:16.646872 3923374 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" [upgrade/staticpods] Preparing for "kube-apiserver" upgrade [upgrade/staticpods] Current and new manifests of kube-apiserver are equal, skipping upgrade [upgrade/staticpods] Preparing for "kube-controller-manager" upgrade [upgrade/staticpods] Current and new manifests of kube-controller-manager are equal, skipping upgrade [upgrade/staticpods] Preparing for "kube-scheduler" upgrade [upgrade/staticpods] Current and new manifests of kube-scheduler are equal, skipping upgrade [upgrade] The control plane instance for this node was successfully updated! [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [upgrade] The configuration for this node was successfully updated! [upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.
[root@k8s-master ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready master 6d19h v1.18.0 k8s-node1 Ready <none> 6d19h v1.18.0 k8s-node2 Ready <none> 6d19h v1.18.0
kubectl版本
1 2 3 4
[root@k8s-master ~]# kubectl version Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.20", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:50:46Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"} WARNING: version difference between client (1.18) and server (1.18) exceeds the supported minor version skew of +/-1
[root@k8s-master ~]# kubeadm upgrade plan [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [preflight] Running pre-flight checks. [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: v1.18.20 [upgrade/versions] kubeadm version: v1.18.20 I0906 13:48:20.988439 3926779 version.go:255] remote version is much newer: v1.22.1; falling back to: stable-1.18 [upgrade/versions] Latest stable version: v1.18.20 [upgrade/versions] Latest stable version: v1.18.20 [upgrade/versions] Latest version in the v1.18 series: v1.18.20 [upgrade/versions] Latest version in the v1.18 series: v1.18.20
Awesome, you're up-to-date! Enjoy!
最后发现群集已经为1.18.20
1.6. 更新群集证书
更新所有证书
1
kubeadm alpha certs renew all
输出如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14
[root@k8s-master ~]# kubeadm alpha certs renew all [renew] Reading configuration from the cluster... [renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed certificate for serving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificate for the API server to connect to kubelet renewed certificate embedded in the kubeconfig file for the controller manager to use renewed certificate for liveness probes to healthcheck etcd renewed certificate for etcd nodes to communicate with each other renewed certificate for serving etcd renewed certificate for the front proxy client renewed certificate embedded in the kubeconfig file for the scheduler manager to use renewed
[root@k8s-master ~]# kubeadm alpha certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Sep 06, 2022 05:51 UTC 364d no apiserver Sep 06, 2022 05:51 UTC 364d ca no apiserver-etcd-client Sep 06, 2022 05:51 UTC 364d etcd-ca no apiserver-kubelet-client Sep 06, 2022 05:51 UTC 364d ca no controller-manager.conf Sep 06, 2022 05:51 UTC 364d no etcd-healthcheck-client Sep 06, 2022 05:51 UTC 364d etcd-ca no etcd-peer Sep 06, 2022 05:51 UTC 364d etcd-ca no etcd-server Sep 06, 2022 05:51 UTC 364d etcd-ca no front-proxy-client Sep 06, 2022 05:51 UTC 364d front-proxy-ca no scheduler.conf Sep 06, 2022 05:51 UTC 364d no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Aug 28, 2031 07:52 UTC 9y no etcd-ca Aug 28, 2031 07:53 UTC 9y no front-proxy-ca Aug 28, 2031 07:53 UTC 9y no