这是本节的多页打印视图。点击此处打印.

通过 kubeadm 安装 kubenetes

通过 kubeadm 安装 kubenetes 集群

1: 在 debian12 上安装 kubenetes

1.1: 在线安装 kubenetes

1.1.1: 准备工作
1.1.2: 安装命令行
1.1.3: 初始化集群
1.1.4: 安装 dashboard
1.1.5: 安装 metrics server
1.1.6: 安装监控

1.2: 预热安装 kubenetes
1.3: 离线安装 kubenetes

1.3.1: 在 debian12 上离线安装 kubeadmin
1.3.2: 在 debian12 上离线安装 k8s

1 - 在 debian12 上安装 kubenetes

在 debian12 上用 kubeadm 安装 kubenetes

有三种安装方式：

在线安装：最标准的安装方法，最大的问题就是需要联网+科学上网，速度慢，中途有被墙/被dns污染的风险
预热安装：在在线安装的基础上，提前准备好安装文件和镜像文件，速度快，而且不需要用到镜像仓库。需要充分的提前准备，最好结合 pve 模板一起使用
离线安装：需要提前下载好所有需要的文件到本地或者本地镜像仓库，速度快，但是同样需要充分的提前准备，而且需要用到 harbor 之类的镜像仓库

1.1 - 在线安装 kubenetes

在 debian12 上用 kubeadm 在线安装 kubenetes

参考官方文档：

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

1.1.1 - 准备工作

在 debian12 上安装 kubenetes 之前的准备工作

系统更新

确保更新debian系统到最新，移除不再需要的软件，清理无用的安装包：

sudo apt update && sudo apt full-upgrade -y
sudo apt autoremove
sudo apt autoclean

如果更新了内核，最好重启一下。

swap 分区

安装 Kubernetes 要求机器不能有 swap 分区。

参考：

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#swap-configuration

开启模块

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

container runtime

Kubernetes 支持多种 container runtime，这里暂时继续使用 docker engine + cri-dockerd。

参考：

https://kubernetes.io/docs/setup/production-environment/container-runtimes/

安装 docker + cri-dockerd

docker 的安装参考：

https://skyao.io/learning-docker/docs/installation/debian12/

cri-dockerd 的安装参考：

https://mirantis.github.io/cri-dockerd/usage/install/

从 release 页面下载：

https://github.com/Mirantis/cri-dockerd/releases

debian 12 选择下载文件

https://github.com/Mirantis/cri-dockerd/releases/download/v0.4.0/cri-dockerd_0.4.0.3-0.debian-bookworm_amd64.deb

下载后安装：

sudo dpkg -i ./cri-dockerd_0.4.0.3-0.debian-bookworm_amd64.deb

安装后会提示：

Selecting previously unselected package cri-dockerd.
(Reading database ... 68005 files and directories currently installed.)
Preparing to unpack .../cri-dockerd_0.4.0.3-0.debian-bookworm_amd64.deb ...
Unpacking cri-dockerd (0.4.0~3-0~debian-bookworm) ...
Setting up cri-dockerd (0.4.0~3-0~debian-bookworm) ...
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /lib/systemd/system/cri-docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/cri-docker.socket → /lib/systemd/system/cri-docker.socket.

安装后查看状态：

sudo systemctl status cri-docker.service

如果成功则状态为：

● cri-docker.service - CRI Interface for Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/cri-docker.service; enabled; preset: enabled)
     Active: active (running) since Sat 2025-05-10 20:39:41 CST; 19s ago
TriggeredBy: ● cri-docker.socket
       Docs: https://docs.mirantis.com
   Main PID: 8294 (cri-dockerd)
      Tasks: 9
     Memory: 13.1M
        CPU: 18ms
     CGroup: /system.slice/cri-docker.service
             └─8294 /usr/bin/cri-dockerd --container-runtime-endpoint fd://

May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Hairpin mode is set to none"
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="The binary conntrack is not installed, this can cau>
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="The binary conntrack is not installed, this can cau>
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Loaded network plugin cni"
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Docker cri networking managed by network plugin cni"
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Setting cgroupDriver systemd"
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Docker cri received runtime config &RuntimeConfig{N>
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Starting the GRPC backend for the Docker CRI interf>
May 10 20:39:41 debian12 cri-dockerd[8294]: time="2025-05-10T20:39:41+08:00" level=info msg="Start cri-dockerd grpc backend"
May 10 20:39:41 debian12 systemd[1]: Started cri-docker.service - CRI Interface for Docker Application Container Engine.

安装 containerd

TODO：后面考虑换 containerd

安装 helm

参考：

https://helm.sh/docs/intro/install/#from-apt-debianubuntu

安装：

curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
sudo apt-get install apt-transport-https --yes
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm

安装后取消 helm 的自动更新：

sudo vi /etc/apt/sources.list.d/helm-stable-debian.list

查看安装的版本：

$ helm version
version.BuildInfo{Version:"v3.17.3", GitCommit:"e4da49785aa6e6ee2b86efd5dd9e43400318262b", GitTreeState:"clean", GoVersion:"go1.23.7"}

1.1.2 - 安装命令行

在 debian12 上安装 kubeadm / kubelet / kubectl

参考： https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

安装 kubeadm / kubelet / kubectl

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

假定要安装的 kubernetes 版本为 1.33:

export K8S_VERSION=1.33

# sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v${K8S_VERSION}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v${K8S_VERSION}/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

开始安装 kubelet kubeadm kubectl：

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl

禁止这三个程序的自动更新：

sudo apt-mark hold kubelet kubeadm kubectl

验证安装：

kubectl version --client && echo && kubeadm version

输出为：

Client Version: v1.33.0
Kustomize Version: v5.6.0

kubeadm version: &version.Info{Major:"1", Minor:"33", EmulationMajor:"", EmulationMinor:"", MinCompatibilityMajor:"", MinCompatibilityMinor:"", GitVersion:"v1.33.0", GitCommit:"60a317eadfcb839692a68eab88b2096f4d708f4f", GitTreeState:"clean", BuildDate:"2025-04-23T13:05:48Z", GoVersion:"go1.24.2", Compiler:"gc", Platform:"linux/amd64"}

在运行 kubeadm 之前，先启动 kubelet 服务：

sudo systemctl enable --now kubelet

安装后配置

优化 zsh

vi ~/.zshrc

增加以下内容:

# k8s auto complete
alias k=kubectl
complete -F __start_kubectl k

执行：

source ~/.zshrc

之后即可使用，此时用 k 这个别名来执行 kubectl 命令时也可以实现自动完成，非常的方便。

取消更新

kubeadm / kubelet / kubectl 的版本没有必要升级到最新，因此可以取消他们的自动更新。

sudo vi /etc/apt/sources.list.d/kubernetes.list

注释掉里面的内容。

备注：前面执行 apt-mark hold 后已经不会再更新了，但依然会拖慢 apt update 的速度，因此还是需要手动注释。

常见问题

prod-cdn.packages.k8s.io 无法访问

偶然会遇到 prod-cdn.packages.k8s.io 无法访问的问题，此时的报错如下：

sudo apt-get update
Hit:1 http://mirrors.ustc.edu.cn/debian bookworm InRelease
Hit:2 http://mirrors.ustc.edu.cn/debian bookworm-updates InRelease
Hit:3 http://security.debian.org/debian-security bookworm-security InRelease
Ign:4 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.32/deb  InRelease
Ign:4 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.32/deb  InRelease
Ign:4 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.32/deb  InRelease
Err:4 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.32/deb  InRelease
  Could not connect to prod-cdn.packages.k8s.io:443 (221.228.32.13), connection timed out
Reading package lists... Done
W: Failed to fetch https://pkgs.k8s.io/core:/stable:/v1.32/deb/InRelease  Could not connect to prod-cdn.packages.k8s.io:443 (221.228.32.13), connection timed out
W: Some index files failed to download. They have been ignored, or old ones used instead.

首先排除是网络问题，因为实际配好网络代理，也依然无法访问。

后来发现，在不同地区的机器上 ping prod-cdn.packages.k8s.io 的 ip 地址是不一样的，

$ ping prod-cdn.packages.k8s.io

Pinging dkhzw6k7x6ord.cloudfront.net [108.139.10.84] with 32 bytes of data:
Reply from 108.139.10.84: bytes=32 time=164ms TTL=242
Reply from 108.139.10.84: bytes=32 time=166ms TTL=242
......

# 这个地址无法访问
$ ping prod-cdn.packages.k8s.io
PING dkhzw6k7x6ord.cloudfront.net (221.228.32.13) 56(84) bytes of data.
64 bytes from 221.228.32.13 (221.228.32.13): icmp_seq=1 ttl=57 time=9.90 ms
64 bytes from 221.228.32.13 (221.228.32.13): icmp_seq=2 ttl=57 time=11.4 ms
......

因此考虑通过修改 /etc/hosts 文件来避开 dns 解析的问题：

sudo vi /etc/hosts

添加如下内容：

108.139.10.84 prod-cdn.packages.k8s.io

这样在出现问题的这台机器上，强制将 prod-cdn.packages.k8s.io 解析到 108.139.10.84 这个 ip 地址，这样就可以访问了。

1.1.3 - 初始化集群

在 debian12 上初始化 kubernetes 集群

参考官方文档：

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/

初始化集群

pod-network-cidr 尽量用 10.244.0.0/16 这个范围，不然有些网络插件会需要额外的配置。

cri-socket 的配置参考：

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime

因为前面用的 Docker Engine 和 cri-dockerd ，因此这里的 cri-socket 需要指定为 “unix:///var/run/cri-dockerd.sock”。

apiserver-advertise-address 需要指定为当前节点的 IP 地址，因为当前节点是单节点，因此这里指定为 192.168.3.179。

sudo kubeadm init --pod-network-cidr 10.244.0.0/16 --cri-socket unix:///var/run/cri-dockerd.sock --apiserver-advertise-address=192.168.3.179 --image-repository=192.168.3.91:5000/k8s-proxy

输出为：

W0511 22:22:37.653053    1276 version.go:109] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://dl.k8s.io/release/stable-1.txt": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
W0511 22:22:37.653104    1276 version.go:110] falling back to the local client version: v1.33.0
[init] Using Kubernetes version: v1.33.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [debian12 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.3.179]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [debian12 localhost] and IPs [192.168.3.179 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [debian12 localhost] and IPs [192.168.3.179 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 501.341993ms
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.3.179:6443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-controller-manager is healthy after 1.002560331s
[control-plane-check] kube-scheduler is healthy after 1.156287353s
[control-plane-check] kube-apiserver is healthy after 2.500905726s
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node debian12 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node debian12 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: 5dlwbv.j26vzkb6uvf9yqv6
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.3.179:6443 --token 5dlwbv.j26vzkb6uvf9yqv6 \
	--discovery-token-ca-cert-hash sha256:0d1be37706d728f6c09dbcff86614c6fe04c536d969371400f4d3551f197c6e4

根据提示操作：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

对于测试用的单节点，去除 master/control-plane 的污点：

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

执行：

kubectl get node

能看到此时节点的状态会是 NotReady：

NAME       STATUS     ROLES           AGE     VERSION
debian12   NotReady   control-plane   3m49s   v1.32.2

执行：

kubectl describe node debian12

能看到节点的错误信息：

Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Tue, 04 Mar 2025 20:28:00 +0800   Tue, 04 Mar 2025 20:23:53 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Tue, 04 Mar 2025 20:28:00 +0800   Tue, 04 Mar 2025 20:23:53 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Tue, 04 Mar 2025 20:28:00 +0800   Tue, 04 Mar 2025 20:23:53 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            False   Tue, 04 Mar 2025 20:28:00 +0800   Tue, 04 Mar 2025 20:23:53 +0800   KubeletNotReady              container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

需要继续安装网络插件。

安装网络插件

安装 flannel

参考官方文档： https://github.com/flannel-io/flannel#deploying-flannel-with-kubectl

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

如果一切正常，就能看到 k8s 集群内的 pod 都启动完成状态为 Running：

k get pods -A
NAMESPACE      NAME                               READY   STATUS    RESTARTS        AGE
kube-flannel   kube-flannel-ds-ts6n8              1/1     Running   7 (9m27s ago)   15m
kube-system    coredns-668d6bf9bc-rbkzb           1/1     Running   0               3h55m
kube-system    coredns-668d6bf9bc-vbltg           1/1     Running   0               3h55m
kube-system    etcd-debian12                      1/1     Running   0               3h55m
kube-system    kube-apiserver-debian12            1/1     Running   1 (5h57m ago)   3h55m
kube-system    kube-controller-manager-debian12   1/1     Running   0               3h55m
kube-system    kube-proxy-95ccr                   1/1     Running   0               3h55m
kube-system    kube-scheduler-debian12            1/1     Running   1 (6h15m ago)   3h55m

如果发现 kube-flannel-ds pod 的状态总是 CrashLoopBackOff：

 k get pods -A
NAMESPACE      NAME                               READY   STATUS              RESTARTS        AGE
kube-flannel   kube-flannel-ds-ts6n8              0/1     CrashLoopBackOff    2 (22s ago)     42s

继续查看 pod 的具体错误信息：

k describe pods -n kube-flannel kube-flannel-ds-ts6n8

发现报错 “Back-off restarting failed container kube-flannel in pod kube-flannel”：

Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  117s                default-scheduler  Successfully assigned kube-flannel/kube-flannel-ds-ts6n8 to debian12
  Normal   Pulled     116s                kubelet            Container image "ghcr.io/flannel-io/flannel-cni-plugin:v1.6.2-flannel1" already present on machine
  Normal   Created    116s                kubelet            Created container: install-cni-plugin
  Normal   Started    116s                kubelet            Started container install-cni-plugin
  Normal   Pulled     115s                kubelet            Container image "ghcr.io/flannel-io/flannel:v0.26.4" already present on machine
  Normal   Created    115s                kubelet            Created container: install-cni
  Normal   Started    115s                kubelet            Started container install-cni
  Normal   Pulled     28s (x5 over 114s)  kubelet            Container image "ghcr.io/flannel-io/flannel:v0.26.4" already present on machine
  Normal   Created    28s (x5 over 114s)  kubelet            Created container: kube-flannel
  Normal   Started    28s (x5 over 114s)  kubelet            Started container kube-flannel
  Warning  BackOff    2s (x10 over 110s)  kubelet            Back-off restarting failed container kube-flannel in pod kube-flannel-ds-ts6n8_kube-flannel(1e03c200-2062-4838

此时应该去检查准备工作中 “开启模块” 一节的内容是不是有疏漏。

补救之后，就能看到 kube-flannel-ds 这个 pod 正常运行了：

k get pods -A
NAMESPACE      NAME                               READY   STATUS    RESTARTS        AGE
kube-flannel   kube-flannel-ds-ts6n8              1/1     Running   7 (9m27s ago)   15m

安装 Calico

https://docs.tigera.io/calico/latest/getting-started/kubernetes/self-managed-onprem/onpremises#install-calico

查看最新版本，当前最新版本是 v3.29.2:

kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.29.2/manifests/tigera-operator.yaml

TODO：用了 flannel， Calico 后面再验证。

1.1.4 - 安装 dashboard

安装 kubernetes 的 dashboard

安装 dashboard

参考：https://github.com/kubernetes/dashboard/#installation

在下面地址上查看当前 dashboard 的版本：

https://github.com/kubernetes/dashboard/releases

根据对 kubernetes 版本的兼容情况选择对应的 dashboard 的版本：

kubernetes-dashboard-7.11.0 ，兼容 k8s 1.32

最新版本需要用 helm 进行安装：

helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard

输出为：

"kubernetes-dashboard" already exists with the same configuration, skipping
Release "kubernetes-dashboard" does not exist. Installing it now.
NAME: kubernetes-dashboard
LAST DEPLOYED: Wed Mar  5 00:53:17 2025
NAMESPACE: kubernetes-dashboard
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
*************************************************************************************************
*** PLEASE BE PATIENT: Kubernetes Dashboard may need a few minutes to get up and become ready ***
*************************************************************************************************

Congratulations! You have just installed Kubernetes Dashboard in your cluster.

To access Dashboard run:
  kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443

NOTE: In case port-forward command does not work, make sure that kong service name is correct.
      Check the services in Kubernetes Dashboard namespace using:
        kubectl -n kubernetes-dashboard get svc

Dashboard will be available at:
  https://localhost:8443

此时 dashboard 的 service 和 pod 情况：

kubectl -n kubernetes-dashboard get services

输出为：

NAME                                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
kubernetes-dashboard-api               ClusterIP   10.108.225.190   <none>        8000/TCP   2m5s
kubernetes-dashboard-auth              ClusterIP   10.99.205.102    <none>        8000/TCP   2m5s
kubernetes-dashboard-kong-proxy        ClusterIP   10.96.247.162    <none>        443/TCP    2m5s
kubernetes-dashboard-metrics-scraper   ClusterIP   10.103.222.22    <none>        8000/TCP   2m5s
kubernetes-dashboard-web               ClusterIP   10.108.219.9     <none>        8000/TCP   2m5s

查看 pod 的情况：

kubectl -n kubernetes-dashboard get pods

等待两三分钟之后，pod 启动完成，输出为：

NAME                                                    READY   STATUS    RESTARTS   AGE
kubernetes-dashboard-api-7d8567b8f-9ksk2                1/1     Running   0          3m8s
kubernetes-dashboard-auth-6877bf44b9-9qfmg              1/1     Running   0          3m8s
kubernetes-dashboard-kong-79867c9c48-rzlhp              1/1     Running   0          3m8s
kubernetes-dashboard-metrics-scraper-794c587449-6phjv   1/1     Running   0          3m8s
kubernetes-dashboard-web-75576c76b-sm2wj                1/1     Running   0          3m8s

为了方便，使用 node port 来访问 dashboard，需要执行：

kubectl -n kubernetes-dashboard edit service kubernetes-dashboard-kong-proxy

然后修改 type: ClusterIP 为 type: NodePort。然后看一下具体分配的 node port 是哪个：

kubectl -n kubernetes-dashboard get service kubernetes-dashboard-kong-proxy

输出为：

NAME                              TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard-kong-proxy   NodePort   10.96.247.162   <none>        443:32616/TCP   17m

现在可以用浏览器直接访问：

https://192.168.3.215:32616/

创建用户并登录 dashboard

参考：Creating sample user

创建 admin-user 用户：

mkdir -p ~/work/soft/k8s
cd ~/work/soft/k8s

vi dashboard-adminuser.yaml

内容为:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

执行：

k create -f dashboard-adminuser.yaml

然后绑定角色：

vi dashboard-adminuser-binding.yaml

内容为:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

执行：

k create -f dashboard-adminuser-binding.yaml

然后创建 token ：

kubectl -n kubernetes-dashboard create token admin-user

输出为:

eyJhbGciOiJSUzI1NiIsImtpZCI6Ik9sWnJsTk5UNE9JVlVmRFMxMUpwNC1tUlVndTl5Zi1WQWtmMjIzd2hDNmcifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzQxMTEyNDg4LCJpYXQiOjE3NDExMDg4ODgsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiNDU5ZGQxNjctNWI5OS00MWIzLTgzZWEtNGIxMGY3MTc5ZjEyIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJhZG1pbi11c2VyIiwidWlkIjoiZjMxN2VhZTItNTNiNi00MGZhLWI3MWYtMzZiNDI1YmY4YWQ0In19LCJuYmYiOjE3NDExMDg4ODgsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbi11c2VyIn0.TYzOdrMFXcSEeVMbc1ewIA13JVi4FUYoRN7rSH5OstbVfKIF48X_o1RWxOGM_AurhgLxuKZHzmns3K_pX_OR3u1URfK6-gGos4iAQY-H1yntfRmzzsip_FbZh95EYFGTN43gw21jTyfem3OKBXXLgzsnVT_29uMnJzSnCDnrAciVKMoCEUP6x2RSHQhp6PrxrIrx_NMB3vojEZYq3AysQoNqYYjRDd4MnDRClm03dNvW5lvKSgNCVmZFje_EEa2EhI2X6d3X8zx6tHwT5M4-T3hMmyIpzHUwf3ixeZR85rhorMbskNVvRpH6VLH6BXP31c3NMeSgYk3BG8d7UjCYxQ

这个 token 就可以用在 kubernetes-dashboard 的登录页面上了。

为了方便，将这个 token 存储在 Secret ：

vi dashboard-adminuser-secret.yaml

内容为:

apiVersion: v1
kind: Secret
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
  annotations:
    kubernetes.io/service-account.name: "admin-user"   
type: kubernetes.io/service-account-token

执行：

k create -f dashboard-adminuser-secret.yaml

之后就可以用命令随时获取这个 token 了：

kubectl get secret admin-user -n kubernetes-dashboard -o jsonpath="{.data.token}" | base64 -d

备注：复制 token 的时候，不要复制最后的那个 % 字符，否则会报错。

1.1.5 - 安装 metrics server

安装 kubernetes 的 metrics server

参考：https://github.com/kubernetes-sigs/metrics-server/#installation

安装 metrics server

下载：

mkdir -p ~/work/soft/k8s
cd ~/work/soft/k8s
wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

修改下载下来的 components.yaml，增加 --kubelet-insecure-tls 并修改 --kubelet-preferred-address-types：

  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=4443
        - --kubelet-preferred-address-types=InternalIP   # 修改这行，默认是InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls  # 增加这行

然后安装：

k apply -f components.yaml

稍等片刻看是否启动:

kubectl get pod -n kube-system | grep metrics-server

验证一下，查看 service 信息

kubectl describe svc metrics-server -n kube-system

简单验证一下基本使用:

kubectl top nodes
kubectl top pods -n kube-system

参考资料

1.1.6 - 安装监控

安装 prometheus 和 grafana 以监控 kubernetes 集群

参考：https://github.com/prometheus-operator/prometheus-operator

https://computingforgeeks.com/setup-prometheus-and-grafana-on-kubernetes/

1.2 - 预热安装 kubenetes

在 debian12 上用 kubeadm 预热安装 kubenetes

原理

所谓预热安装，就是在在线安装的基础上，在执行 kubeadmin init 之前，提前准备好所有的安装文件和镜像文件，然后制造成 pve 模板。

之后就可以重用该模板，在需要时创建虚拟机，在虚拟机中执行 kubeadmin init 即可快速安装 kubenetes。

原则上，在执行 kubeadmin init 之前的各种准备工作都可以参考在线安装的方式。而在 kubeadmin init 之后的安装工作，就只能通过提前准备安装文件，提前下载镜像文件等方式来加速。

准备工作

安装 docker：参考 https://skyao.io/learning-docker/docs/installation/debian12/ ，在线安装和离线安装都可以。
安装 kubeadm：参考前面的在线安装方式，或者直接用后面的离线安装方式，将 cri-dockerd / helm 和kubeadm / kubelete / kubectl 安装好。

预下载镜像文件

k8s cluster

kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock

这样就可以提前下载好 kubeadm init 时需要的镜像文件：

[config/images] Pulled registry.k8s.io/kube-apiserver:v1.33.0
[config/images] Pulled registry.k8s.io/kube-controller-manager:v1.33.0
[config/images] Pulled registry.k8s.io/kube-scheduler:v1.33.0
[config/images] Pulled registry.k8s.io/kube-proxy:v1.33.0
[config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.0
[config/images] Pulled registry.k8s.io/pause:3.10
[config/images] Pulled registry.k8s.io/etcd:3.5.21-0

flannel

下载 flannel 需要的镜像文件：

docker pull ghcr.io/flannel-io/flannel-cni-plugin:v1.6.2-flannel1
docker pull ghcr.io/flannel-io/flannel:v0.26.7

参考在线安装文档准备以下 yaml 文件：

~/work/soft/k8s/menifests/kube-flannel.yml

dashboard

查看 dashboard 的最新版本：

helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm repo update
helm search repo kubernetes-dashboard -l

发现 dashboard 的最新版本是 7.12.0，所以下载 dashboard 需要的 charts 文件：

helm pull kubernetes-dashboard/kubernetes-dashboard --version 7.12.0 --untar --untardir ~/work/soft/k8s/charts

下载 dashboard 需要的镜像文件：

docker pull docker.io/kubernetesui/dashboard-api:1.12.0
docker pull docker.io/kubernetesui/dashboard-auth:1.2.4
docker pull docker.io/kubernetesui/dashboard-web:1.6.2
docker pull docker.io/kubernetesui/dashboard-metrics-scraper:1.2.2

参考在线安装文档准备以下 yaml 文件：

~/work/soft/k8s/menifests/dashboard-adminuser-binding.yaml
~/work/soft/k8s/menifests/dashboard-adminuser.yaml
~/work/soft/k8s/menifests/dashboard-adminuser-secret.yaml

metrics-server

下载 metrics-server 需要的镜像文件：

docker pull registry.k8s.io/metrics-server/metrics-server:v0.7.2
docker pull docker.io/kubernetesui/dashboard-metrics-scraper:1.2.2

参考在线安装文档准备以下 yaml 文件：

~/work/soft/k8s/menifests/metrics-server-components.yaml

安装

手工安装

执行 kubeadm init 命令，注意检查并修改 IP 地址为实际 IP 地址：

NODE_IP=192.168.3.175

sudo kubeadm init --pod-network-cidr 10.244.0.0/16 --cri-socket unix:///var/run/cri-dockerd.sock --apiserver-advertise-address=$NODE_IP

配置 kube config：

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

配置 flannel 网络：

kubectl apply -f ~/work/soft/k8s/menifests/kube-flannel.yml

去除污点：

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

安装 dashboard ：

helm upgrade --install kubernetes-dashboard \
  ~/work/soft/k8s/charts/kubernetes-dashboard \
  --create-namespace \
  --namespace kubernetes-dashboard

准备用于登录 dashboard 的 admin-user 用户：

kubectl apply -f ~/work/soft/k8s/menifests/dashboard-adminuser.yaml
kubectl apply -f ~/work/soft/k8s/menifests/dashboard-adminuser-binding.yaml

kubectl -n kubernetes-dashboard create token admin-user
kubectl apply -f ~/work/soft/k8s/menifests/dashboard-adminuser-secret.yaml

ADMIN_USER_TOKEN=$(kubectl get secret admin-user -n kubernetes-dashboard -o jsonpath="{.data.token}" | base64 -d)
echo $ADMIN_USER_TOKEN > ~/work/soft/k8s/dashboard-admin-user-token.txt
echo "admin-user token is: $ADMIN_USER_TOKEN"

将 kubernetes-dashboard-kong-proxy 设置为 NodePort 类型：

kubectl -n kubernetes-dashboard patch service kubernetes-dashboard-kong-proxy -p '{"spec":{"type":"NodePort"}}'

获取 NodePort：

NODE_PORT=$(kubectl -n kubernetes-dashboard get service kubernetes-dashboard-kong-proxy \
  -o jsonpath='{.spec.ports[0].nodePort}')
echo "url is: https://$NODE_IP:$NODE_PORT"

安装 metrics-server：

kubectl apply -f ~/work/soft/k8s/menifests/metrics-server-components.yaml

kubectl wait --namespace kube-system \
  --for=condition=Ready \
  --selector=k8s-app=metrics-server \
  --timeout=300s pod
echo "metrics-server installed, have a try:"
echo
echo "kubectl top nodes"
echo
kubectl top nodes
echo
echo "kubectl top pods -n kube-system"
echo
kubectl top pods -n kube-system

脚本自动安装

#!/usr/bin/env zsh

# Kubernetes 自动化安装脚本 (Debian 12 + Helm + Dashboard + Metrics Server)
# 使用方法: sudo ./install_k8s_prewarm.zsh <NODE_IP>

# 获取脚本所在绝对路径
K8S_INSTALL_PATH=$(cd "$(dirname "$0")"; pwd)
MANIFESTS_PATH="$K8S_INSTALL_PATH/menifests"
CHARTS_PATH="$K8S_INSTALL_PATH/charts"
echo "🔍 检测到安装文件目录: $K8S_INSTALL_PATH"

# 检查是否以 root 执行
if [[ $EUID -ne 0 ]]; then
  echo "❌ 此脚本必须以 root 身份运行" 
  exit 1
fi

# 获取节点 IP
if [[ -z "$1" ]]; then
  echo "ℹ️ 用法: $0 <节点IP>"
  exit 1
fi
NODE_IP=$1

# 安装日志
LOG_FILE="$K8S_INSTALL_PATH/k8s_install_$(date +%Y%m%d_%H%M%S).log"
exec > >(tee -a "$LOG_FILE") 2>&1

echo "📅 开始安装 Kubernetes 集群 - $(date)"
echo "🔧 节点IP: $NODE_IP"
echo "📁 资源目录: $K8S_INSTALL_PATH"

# 步骤1: kubeadm 初始化
echo "🚀 正在初始化 Kubernetes 控制平面..."
kubeadm_init() {
  kubeadm init \
    --pod-network-cidr 10.244.0.0/16 \
    --cri-socket unix:///var/run/cri-dockerd.sock \
    --apiserver-advertise-address=$NODE_IP
  
  if [[ $? -ne 0 ]]; then
    echo "❌ kubeadm init 失败"
    exit 1
  fi
}
kubeadm_init
sleep 3

# 步骤2: 配置 kubectl
echo "⚙️ 为 root 用户配置 kubectl..."
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
echo "⚙️ 为当前用户配置 kubectl..."
CURRENT_USER_HOME=$(getent passwd $SUDO_USER | cut -d: -f6)
mkdir -p $CURRENT_USER_HOME/.kube
cp -i /etc/kubernetes/admin.conf $CURRENT_USER_HOME/.kube/config
chown $(id -u $SUDO_USER):$(id -g $SUDO_USER) $CURRENT_USER_HOME/.kube/config

# 步骤3: 安装 Flannel 网络插件
echo "🌐 正在安装 Flannel 网络..."
kubectl apply -f "$MANIFESTS_PATH/kube-flannel.yml" || {
  echo "❌ Flannel 安装失败"
  exit 1
}
sleep 3

# 步骤4: 去除控制平面污点
echo "✨ 去除控制平面污点..."
kubectl taint nodes --all node-role.kubernetes.io/control-plane- || {
  echo "⚠️ 去除污点失败 (可能不影响功能)"
}

# 步骤5: 从本地安装 Dashboard
echo "📊 正在从本地安装 Kubernetes Dashboard..."
helm upgrade --install kubernetes-dashboard \
  "$CHARTS_PATH/kubernetes-dashboard" \
  --create-namespace \
  --namespace kubernetes-dashboard || {
  echo "❌ Dashboard 安装失败"
  exit 1
}
sleep 3

# 步骤6: 配置 Dashboard 管理员用户
echo "👤 创建 Dashboard 管理员用户..."
kubectl apply -f "$MANIFESTS_PATH/dashboard-adminuser.yaml" || {
  echo "❌ 创建 admin-user 失败"
  exit 1
}

kubectl apply -f "$MANIFESTS_PATH/dashboard-adminuser-binding.yaml" || {
  echo "❌ 创建 RBAC 绑定失败"
  exit 1
}

kubectl apply -f "$MANIFESTS_PATH/dashboard-adminuser-secret.yaml" || {
  echo "⚠️ 创建 Secret 失败 (可能已存在)"
}

# 获取并保存 Token
echo "🔑 获取管理员 Token..."
ADMIN_TOKEN=$(kubectl -n kubernetes-dashboard create token admin-user 2>/dev/null || \
  kubectl get secret admin-user -n kubernetes-dashboard -o jsonpath="{.data.token}" | base64 -d)

if [[ -z "$ADMIN_TOKEN" ]]; then
  echo "❌ 获取 Token 失败"
  exit 1
fi

echo "$ADMIN_TOKEN" > "$K8S_INSTALL_PATH/dashboard-admin-user-token.txt"
echo "✅ Token 已保存到: $K8S_INSTALL_PATH/dashboard-admin-user-token.txt"

# 步骤7: 修改 Dashboard Service 类型
echo "🔧 修改 Dashboard 服务类型为 NodePort..."
kubectl -n kubernetes-dashboard patch service kubernetes-dashboard-kong-proxy \
  -p '{"spec":{"type":"NodePort"}}' || {
  echo "❌ 修改服务类型失败"
  exit 1
}
sleep 3

# 获取 NodePort
NODE_PORT=$(kubectl -n kubernetes-dashboard get service kubernetes-dashboard-kong-proxy \
  -o jsonpath='{.spec.ports[0].nodePort}')

echo "🌍 Dashboard 访问地址: https://$NODE_IP:$NODE_PORT"
echo "🔑 登录 Token: $ADMIN_TOKEN"

# 步骤8: 安装 Metrics Server
echo "📈 正在安装 Metrics Server..."
kubectl apply -f "$MANIFESTS_PATH/metrics-server-components.yaml" || {
  echo "❌ Metrics Server 安装失败"
  exit 1
}

# 等待 Metrics Server 就绪
echo "⏳ 等待 Metrics Server 就绪 (最多5分钟)..."
kubectl wait --namespace kube-system \
  --for=condition=Ready \
  --selector=k8s-app=metrics-server \
  --timeout=300s pod || {
  echo "❌ Metrics Server 启动超时"
  exit 1
}

# 验证安装
echo "✅ 安装完成!"
sleep 5
echo ""
echo "🛠️ 验证命令:"
echo "kubectl top nodes"
kubectl top nodes
echo ""
echo "kubectl top pods -n kube-system"
kubectl top pods -n kube-system

echo ""
echo "📌 重要信息:"
echo "Dashboard URL: https://$NODE_IP:$NODE_PORT"
echo "Token 文件: $K8S_INSTALL_PATH/dashboard-admin-user-token.txt"
echo "安装日志: $LOG_FILE"

1.3 - 离线安装 kubenetes

在 debian12 上用 kubeadm 离线安装 kubenetes

1.3.1 - 在 debian12 上离线安装 kubeadmin

在 debian12 上用 kubeadm 离线安装 kubeadmin

制作离线安装包

mkdir -p ~/temp/k8s-offline/
cd ~/temp/k8s-offline/

cri-dockerd

下载安装包：

wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.4.0/cri-dockerd_0.4.0.3-0.debian-bookworm_amd64.deb

helm

参考在线安装的方式，同样需要先添加 helm 的 apt 仓库，

curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
sudo apt-get install apt-transport-https --yes
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update

然后找到需要安装的版本，下载离线安装包。

apt-get download helm

kubeadmin

添加 k8s 的 keyrings：

export K8S_VERSION=1.33

# sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v${K8S_VERSION}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v${K8S_VERSION}/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list


sudo apt-get update

下载安装包：

# 下载 k8s 的 .deb 包
apt-get download kubelet kubeadm kubectl

# 下载所有依赖（可能需要运行多次直到无新依赖）
apt-get download $(apt-cache depends kubelet kubeadm kubectl | grep -E 'Depends|Recommends' | cut -d ':' -f 2 | tr -d ' ' | grep -v "^kube" | sort -u)

rm -r iptables*.deb

下载 kubernetes-cni:

apt-get download kubernetes-cni

完成后的离线安装包如下：

ls -lh
total 124M
-rw-r--r-- 1 sky sky   35K Mar 23  2023 conntrack_1%3a1.4.7-1+b2_amd64.deb
-rw-r--r-- 1 sky sky   11M Apr 16 15:43 cri-dockerd_0.4.0.3-0.debian-bookworm_amd64.deb
-rw-r--r-- 1 sky sky   17M Apr 22 16:56 cri-tools_1.33.0-1.1_amd64.deb
-rw-r--r-- 1 sky sky  193K Dec 20  2022 ethtool_1%3a6.1-1_amd64.deb
-rw-r--r-- 1 sky sky   17M Apr 29 21:31 helm_3.17.3-1_amd64.deb
-rw-r--r-- 1 sky sky 1022K May 22  2023 iproute2_6.1.0-3_amd64.deb
-rw-r--r-- 1 sky sky  352K Jan 16  2023 iptables_1.8.9-2_amd64.deb
-rw-r--r-- 1 sky sky   13M Apr 24 02:07 kubeadm_1.33.0-1.1_amd64.deb
-rw-r--r-- 1 sky sky   12M Apr 24 02:07 kubectl_1.33.0-1.1_amd64.deb
-rw-r--r-- 1 sky sky   16M Apr 24 02:08 kubelet_1.33.0-1.1_amd64.deb
-rw-r--r-- 1 sky sky   37M Feb  5 17:03 kubernetes-cni_1.6.0-1.1_amd64.deb
-rw-r--r-- 1 sky sky  2.7M Mar  8 05:26 libc6_2.36-9+deb12u10_amd64.deb
-rw-r--r-- 1 sky sky  131K Dec  9 20:54 mount_2.38.1-5+deb12u3_amd64.deb
-rw-r--r-- 1 sky sky  1.2M Dec  9 20:54 util-linux_2.38.1-5+deb12u3_amd64.deb

将这个离线安装包压缩成一个 tar 包：

cd ~/temp/
tar -czvf k8s-offline-v1.33.tar.gz k8s-offline

离线安装

下载离线安装包到本地：

mkdir -p ~/temp/ && cd ~/temp/
scp sky@192.168.3.179:/home/sky/temp/k8s-offline-v1.33.tar.gz .

解压离线安装包：

tar -xvf k8s-offline-v1.33.tar.gz
cd k8s-offline

手工安装 cri-dockerd

sudo dpkg -i cri-dockerd*.deb

手工安装 helm

sudo dpkg -i helm*.deb

手工安装 kubeadm

安装 kubeadm 的依赖：

sudo dpkg -i util-linux*.deb
sudo dpkg -i conntrack*.deb
sudo dpkg -i cri-tools*.deb
sudo dpkg -i libc6*.deb
sudo dpkg -i ethtool*.deb
sudo dpkg -i mount*.deb
sudo dpkg -i iproute2*.deb
sudo dpkg -i iptables*.deb

sudo dpkg -i kubernetes-cni*.deb

安装 kubectl / kubelet / kubeadm ：

sudo dpkg -i kubectl*.deb
sudo dpkg -i kubelet*.deb
sudo dpkg -i kubeadm*.deb

修改 ~/.zshrc 文件，添加 alias ：

if ! grep -qF "# k8s auto complete" ~/.zshrc; then
    cat >> ~/.zshrc << 'EOF'

# k8s auto complete
alias k=kubectl
complete -F __start_kubectl k
EOF
fi

source ~/.zshrc

制作离线安装脚本

离线安装避免在线安装的网络问题，非常方便，考虑写一个离线安装脚本，方便以后使用。

vi install_k8s_offline.zsh

内容为：

#!/usr/bin/env zsh

# ------------------------------------------------------------
# kubeadm 离线安装脚本 (Debian 12)
# 前提条件：
# 1. 所有 .deb 文件已放在 ~/k8s-offline
# 2. 已经
# ------------------------------------------------------------

set -e  # 遇到错误立即退出

K8S_OFFLINE_DIR="./k8s-offline"

# 检查是否在 Debian 12 上运行
if ! grep -q "Debian GNU/Linux 12" /etc/os-release; then
    echo "❌ 错误：此脚本仅适用于 Debian 12！"
    exit 1
fi

# 检查是否已安装 kubeadm
if command -v kubeadm &>/dev/null; then
    echo "⚠️ kubeadm 已安装，跳过安装步骤。"
    exit 0
fi

# 检查离线目录是否存在
if [[ ! -d "$K8S_OFFLINE_DIR" ]]; then
    echo "❌ 错误：离线目录 $K8S_OFFLINE_DIR 不存在！"
    exit 1
fi

echo "🔧 开始离线安装 kubeadm..."

# ------------------------------------------------------------
# 1. 开启模块
# ------------------------------------------------------------
echo "🔧 开启模块..."
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system


# ------------------------------------------------------------
# 2. 安装 cri-dockerd
# ------------------------------------------------------------
echo "📦 安装 cri-dockerd..."

cd "$K8S_OFFLINE_DIR"
sudo dpkg -i cri-tools*.deb
sudo dpkg -i cri-dockerd*.deb

# ------------------------------------------------------------
# 3. 安装 kubeadm 的依赖
# ------------------------------------------------------------
echo "📦 安装 kubeadm 的依赖包..."

# 按顺序安装依赖（防止 dpkg 报错）
for pkg in util-linux conntrack libc6 ethtool mount iproute2 iptables helm kubernetes-cni; do
    if ls "${pkg}"*.deb &>/dev/null; then
        echo "➡️ 正在安装: ${pkg}"
        sudo dpkg -i "${pkg}"*.deb || true  # 忽略部分错误，后续用 apt-get -f 修复
    fi
done

# 修复依赖关系
echo "🛠️ 修复依赖关系..."
sudo apt-get -f install -y

# ------------------------------------------------------------
# 4. 安装 kubeadm
# ------------------------------------------------------------
# 按顺序安装 kubeadm 组件（防止 dpkg 报错）
echo "📦 安装 kubeadm 组件..."
for pkg in kubectl kubelet kubeadm; do
    if ls "${pkg}"*.deb &>/dev/null; then
        echo "➡️ 正在安装: ${pkg}"
        sudo dpkg -i "${pkg}"*.deb || true  # 忽略部分错误，后续用 apt-get -f 修复
    fi
done

# 修复依赖关系
echo "🛠️ 修复依赖关系..."
sudo apt-get -f install -y


# ------------------------------------------------------------
# 5. 配置 kubectl
# ------------------------------------------------------------
echo "⚙️ 配置 kubectl 使用 alias..."

if ! grep -qF "# k8s auto complete" ~/.zshrc; then
    cat >> ~/.zshrc << 'EOF'

# k8s auto complete
alias k=kubectl
complete -F __start_kubectl k
EOF
fi

# ------------------------------------------------------------
# 6. 验证安装
# ------------------------------------------------------------
echo "✅ 安装完成！验证版本："
kubectl version --client && echo && kubelet --version && echo && kubeadm version && echo

echo "✨ kubeadm 安装完成！"
echo "👥 然后重新登录，或者执行命令以便 k alias 立即生效： source ~/.zshrc"
echo "🟢 之后请运行测试 kubectl 的别名 k： k version --client"

1.3.2 - 在 debian12 上离线安装 k8s

在 debian12 上用 kubeadm 离线安装 k8s

指定镜像仓库进行离线安装

新建一个 kubeadm-config.yaml 文件：

vi kubeadm-config.yaml

内容设置为：

apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: "192.168.3.179"
  bindPort: 6443
nodeRegistration:
  criSocket: "unix:///var/run/containerd/containerd.sock"
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
imageRepository: "192.168.3.91:5000/k8s-proxy"
dns:
  imageRepository: "192.168.3.91:5000/k8s-proxy/coredns"
  imageTag: "v1.12.0"
etcd:
  local:
    imageRepository: "192.168.3.91:5000/k8s-proxy"
    imageTag: "3.5.21-0"

使用提前准备好的 harbor 代理仓库 192.168.3.91:5000/k8s-proxy 进行 kubeadm 的 init 操作：

sudo kubeadm init --pod-network-cidr 10.244.0.0/16 --cri-socket unix:///var/run/cri-dockerd.sock --apiserver-advertise-address=192.168.3.179 --image-repository=192.168.3.91:5000/k8s-proxy

但是报错了：

W0511 21:34:13.908906   60242 version.go:109] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://dl.k8s.io/release/stable-1.txt": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
W0511 21:34:13.908935   60242 version.go:110] falling back to the local client version: v1.33.0
[init] Using Kubernetes version: v1.33.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
W0511 21:34:13.944144   60242 checks.go:846] detected that the sandbox image "registry.k8s.io/pause:3.10" of the container runtime is inconsistent with that used by kubeadm.It is recommended to use "192.168.3.91:5000/k8s-proxy/pause:3.10" as the CRI sandbox image.
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR ImagePull]: failed to pull image 192.168.3.91:5000/k8s-proxy/coredns:v1.12.0: failed to pull image 192.168.3.91:5000/k8s-proxy/coredns:v1.12.0: Error response from daemon: unknown: repository k8s-proxy/coredns not found
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

不知道为什么 coredns 的镜像路径和其他的不一样。这是 k8s 所有的镜像的路径，可以看到正常的名称是 “registry.k8s.io/xxxx:version”，只有 coredns 是 registry.k8s.io/coredns/coredns:v1.12.0, 多一个路径：

kubeadm config images pull
[config/images] Pulled registry.k8s.io/kube-apiserver:v1.33.0
[config/images] Pulled registry.k8s.io/kube-controller-manager:v1.33.0
[config/images] Pulled registry.k8s.io/kube-scheduler:v1.33.0
[config/images] Pulled registry.k8s.io/kube-proxy:v1.33.0
[config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.0
[config/images] Pulled registry.k8s.io/pause:3.10
[config/images] Pulled registry.k8s.io/etcd:3.5.21-0

指定镜像仓库再做一次镜像下载，验证一下，发现也是同样报错：

kubeadm config images pull --image-repository=192.168.3.91:5000/k8s-proxy
[config/images] Pulled 192.168.3.91:5000/k8s-proxy/kube-apiserver:v1.33.0
[config/images] Pulled 192.168.3.91:5000/k8s-proxy/kube-controller-manager:v1.33.0
[config/images] Pulled 192.168.3.91:5000/k8s-proxy/kube-scheduler:v1.33.0
[config/images] Pulled 192.168.3.91:5000/k8s-proxy/kube-proxy:v1.33.0
failed to pull image "192.168.3.91:5000/k8s-proxy/coredns:v1.12.0": failed to pull image 192.168.3.91:5000/k8s-proxy/coredns:v1.12.0: Error response from daemon: unknown: resource not found: repo k8s-proxy/coredns, tag v1.12.0 not found
To see the stack trace of this error execute with --v=5 or higher