TauCeti.blog
Posts Tags
TauCeti.blog

Contents

Kubernetes upgrade notes: 1.32.x to 1.33.x

     2051 words   9 minutes 
Upgrade notes to Kubernetes 1.33 (esp. if you use my Kubernetes Ansible roles)

If you used my Kubernetes the Not So Hard Way With Ansible blog posts to setup a Kubernetes (K8s) cluster this notes might be helpful for you (and maybe for others too that manage a K8s cluster on their own e.g.). I’ll only mention changes that might be relevant because they will either be interesting for most K8s administrators anyways (even in case they run a fully managed Kubernetes deployment) or if it’s relevant if you manage your own bare-metal/VM based on-prem Kubernetes deployment. I normally skip changes that are only relevant for GKE, AWS EKS, Azure or other cloud providers.

I’ve a general upgrade guide Kubernetes the Not So Hard Way With Ansible - Upgrading Kubernetes that worked quite well for me for the last past K8s upgrades. So please read that guide if you want to know HOW the components are updated. This post here is esp. for the 1.32.x to 1.33.x upgrade and WHAT was interesting for me.

As usual I don’t update a production system before the .2 release of a new major version is released. In my experience the .0 and .1 are just too buggy (well, it got way better meanwhile but you don’t want do experiments in production, right? 😉). Nevertheless it’s important to test new releases (and even beta or release candidates if possible) already in development environments and report bugs!

I only upgrade from the latest version of the former major release. At the time writing this blog post 1.32.9 was the latest 1.33.x release. After reading the 1.32 CHANGELOG to figure out if any important changes where made between the current 1.32.x and latest 1.32.9 release I didn’t see anything that prevented me updating and I don’t needed to change anything.

So I did the 1.32.9 update first. If you use my Ansible roles that basically only means to change k8s_ctl_release variable from 1.32.x to 1.32.9 (for the controller nodes) and the same for k8s_worker_release (for the worker nodes). Deploy the changes for the control plane and worker nodes as described in my upgrade guide.

Hint: To save some time, IMHO it should be good enough to only update the controller nodes to the latest 1.32.x release as it’s mostly the kube-apiserver that stores the state of the Kubernetes cluster in etcd and that state is quite important. That’s what I normally do. Upgrading to the next major release can then be done for all nodes as usual. But if you want to be absolutely sure just upgrade the whole cluster to the latest 1.32.x release first.

After that everything still worked as expected, I continued with the next step.

As it’s normally no problem (and actually the supported method) to have a newer kubectl utility that is only one major version ahead of the server version I updated kubectl from 1.32.x to latest 1.33.x using my kubectl Ansible role.

While my roles are not using kubeadm to manage my K8s cluster, it’s recommended to have at least etcd 3.5.16 running. I updated my etcd role to current 3.5.24 and updated my etcd deployment accordingly. See Upgrading Kubernetes - etcd for more information how to upgrade etcd. Also Kubernetes v1.31: Accelerating Cluster Performance with Consistent Reads from Cache.

Note: There is already etcd 3.6.x available. But for K8s 1.33 I’ll stay with etcd 3.5 for now. etcd 3.6.x has some breaking changes that needs to be addressed first.

containerd was updated from 2.1.3 to 2.1.4. I updated my containerd role accordingly. Please read the CHANGELOG for potential breaking changes. From my experience the upgrade “just works”.

containerd >= 2.0 is also a prerequisite to use user namespace isolation which got enabled by default as of Kubernetes 1.33.

runc was upgraded from 1.3.0 to 1.3.2. I’ve updated my runc role accordingly. Release notes for runc are in runc releases but shouldn’t be that interesting.

With runc >= 1.2 and containerd >= 2.0 (as mentioned above) User Namespaces are finally supported. Together with Kubernetes release 1.33 user namespace isolation finally arrived out of the box and you should consider using it.

And finally the CNI plugins are updated from 1.7.1 to 1.8.0. Again I updated my CNI role accordingly. The release notes for CNI 1.8.0 might be worth a read but only if you want to go deeper 😉

Since K8s 1.14 there are also searchable release notes available. You can specify the K8s version and a K8s area/component (e.g. kubelet, apiserver, …) and immediately get an overview what changed in that regard. Quite nice! 😉

I guess most users wont be affected by any Urgent Upgrade Notes. Here are the potentially important ones:

  • Action Required: Added the ability to reduce both the initial delay and the maximum delay accrued between container restarts for a node for containers in CrashLoopBackOff across the cluster to the recommended values of 1s initial delay and 60s maximum delay. To set this for a node, turn on the feature gate ReduceDefaultCrashLoopBackOffDecay. If you are also using the feature gate KubeletCrashLoopBackOffMax with a configured per-node CrashLoopBackOff.MaxContainerRestartPeriod, the effective kubelet configuration will follow the conflict resolution policy described in the documentation.

  • Action Required: CSI drivers that call IsLikelyNotMountPoint should not assume false means that the path is a mount point. Each CSI driver needs to make sure correct usage of return value of IsLikelyNotMountPoint.

  • Fixed the behavior of the KUBE_PROXY_NFTABLES_SKIP_KERNEL_VERSION_CHECK environment variable in the nftables proxier. The kernel version check is now skipped only when this variable is explicitly set to a non-empty value. To skip the check, set the KUBE_PROXY_NFTABLES_SKIP_KERNEL_VERSION_CHECK environment variable.

All important stuff is listed in the Kubernetes v1.33: Octarine release announcement.

The following list of changes and features only contains stuff that I found useful and interesting. That means I’m normally not mentioning any Kubernetes internals that have changed but mostly stuff that is interesting for administrators and operation. This is mainly to remember myself what changed 😉 See the full Kubernetes v1.33 Changelog for all changes.

  • Field status.hostIPs added for Pod: This feature adds support for dual-stack IP addresses for Pods, making migration from IPv4 to IPv6 smoother. Applications that originally used IPv4 can now work with both IPv4 and IPv6 during the transition phase.

  • Support to size memory backed volumes: Kubernetes now supports dynamically sizing memory-backed volumes (tmpfs) based on Pod resource limits, improving workload portability and overall node resource utilization.

  • Add support for a drop-in kubelet configuration directory: Kubernetes now supports a drop-in configuration directory for the kubelet (e.g., /etc/kubernetes/kubelet.conf.d/), allowing different components to manage their own configuration files without conflicts.

  • Kubelet OpenTelemetry Tracing: The kubelet now has GA support for OpenTelemetry tracing of gRPC and HTTP API requests, providing better observability for node-level operations.

  • Custom Resource field selectors: Developers can now add field selectors to custom resources, similar to built-in Kubernetes objects, enabling more efficient and precise filtering.

  • Bound service account token improvement: The inclusion of the node name in service account token claims allows users to use this information during authorization and admission (ValidatingAdmissionPolicy).

  • Structured authorization configuration: Structured authorization configuration is now stable, providing a more flexible and maintainable way to configure Kubernetes authorization.

  • Always Honor PersistentVolume Reclaim Policy: The reclaim policy associated with a PersistentVolume is now always honored, regardless of the deletion order of PV and PVC.

  • Support recovery from volume expansion failure: Users can now recover from volume expansion failures by retrying with values that may succeed, making volume management more resilient.

  • Volume group snapshot: This feature introduces a VolumeGroupSnapshot API that lets users take snapshots of multiple volumes together, ensuring data consistency across volumes.

  • Job API managed-by mechanism: This feature enables external controllers (like Kueue) to manage Job synchronization, offering greater flexibility and integration with advanced workload management systems.

  • Only allow anonymous auth for configured endpoints: Kubernetes now allows more fine-grained control over which endpoints accept anonymous authentication, improving security.

  • KEP-3104: Introduce kuberc: An optional kuberc file has been introduced to separate cluster credentials and server configuration from user preferences, improving kubectl configuration management.

  • Pod-level resource specifications: Currently, resource allocation in PodSpec is container-centric. This feature allows users to set resource constraints at the pod level, simplifying resource management for pods as a whole.

  • New statusz and flagz endpoints for core components: New HTTP endpoints /statusz and /flagz have been added for core components (kube-apiserver, kube-controller-manager, kube-scheduler, kubelet), enhancing cluster debuggability by providing version information, uptime, and command-line flags.

  • The pod status.resize field is now deprecated and will no longer be set. The status of a pod resize will be exposed under two new conditions: PodResizeInProgress and PodResizePending instead.

  • The v1 Endpoints API is now officially deprecated (though still fully supported). The API will not be removed, but all users should use the EndpointSlice API instead. Also see Kubernetes v1.33: Continuing the transition from Endpoints to EndpointSlices

  • The WatchFromStorageWithoutResourceVersion feature flag is deprecated and can no longer be enabled.

  • DRA API changes: The maximum number of pods that can use the same ResourceClaim is now 256 instead of 32. Downgrading to 1.32.0 is not supported if this relaxed limit is in use.

  • DRA Device taints: Device taints enable DRA drivers or admins to mark devices as unusable, preventing their allocation. Pods may also get evicted at runtime if a device becomes unusable.

  • DRA admin access control: Starting Kubernetes 1.33, only users with access to an admin namespace with the kubernetes.io/dra-admin-access label are authorized to create ResourceClaim or ResourceClaimTemplate objects with the adminAccess field.

  • DRA “All” devices semantic change: When asking for “All” devices on a node, Kubernetes 1.33 now only picks nodes which have at least one device, whereas Kubernetes ≤ 1.32 proceeded to schedule pods onto nodes with no devices.

  • Improved kubelet credential provider: The kubelet credential provider configuration now allows an optional tokenAttribute field, enabling the kubelet to provision tokens for credential provider plugins.

  • In-place Pod vertical scaling improvements: The API server will no longer set the resize status to Proposed upon receiving a resize request. Memory limits cannot be decreased unless the memory resize restart policy is set to RestartContainer. Container resizePolicy is no longer mutable.

  • User namespaces support enabled by default: As mentioned above already, User namespace isolation for pods is now enabled by default, significantly improving container security. You can find more in KEP-127: Support User Namespaces in pods.

  • Sidecar containers feature GA: The SidecarContainers feature has graduated to GA, and the feature gate will be removed in v1.36. The sidecar pattern involves deploying separate auxiliary container(s) to handle extra capabilities in areas such as networking, logging, and metrics gathering. Sidecar containers graduate to stable in v1.33. To learn more, read Sidecar Containers.

  • Improved scheduling performance: Calculated pod resources are now cached when adding pods to NodeInfo in the scheduler framework, improving performance when processing unschedulable pods.

  • New CPU Manager static policy option: A new strict-cpu-reservation option for CPU Manager static policy enables strict CPU core reservation for system daemons.

  • Added a /flagz and /status endpoint for kube-proxy

  • Added a /statusz endpoint for kube-scheduler

  • Added a /statusz endpoint for kube-controller-manager

  • Added a /statusz HTTP endpoint to the kubelet

  • etcd: Updated to v3.5.24

If you use CSI then also check the CSI Sidecar Containers documentation. Every sidecar container contains a matrix which version you need at a minimum, maximum and which version is recommend to use with whatever K8s version.

Nevertheless if your K8s update to v1.33 worked fine I would recommend to also update the CSI sidecar containers sooner or later.

Now I finally upgraded the K8s controller and worker nodes to version 1.33.x as described in Kubernetes the Not So Hard Way With Ansible - Upgrading Kubernetes.

That’s it for today! Happy upgrading! 😉

Updated on 2025-10-27
 Kubernetes, Update, Upgrade, Ansible
 | Home