k8s v1.27 集群部署记录

k8s container runtime

Runtime	Path to Unix domain socket
containerd	`unix:///var/run/containerd/containerd.sock`
CRI-O	`unix:///var/run/crio/crio.sock`
Docker Engine (using cri-dockerd)	`unix:///var/run/cri-dockerd.sock`

k8s官方推荐contained，与docker解耦。用contained的话，docker无法管理k8s的镜像，必须用ctr管理。contained区分命名空间，k8s默认为k8s.io。

k8s架构

安装：containerd为容器运行时

linux环境准备

设置hostname及hosts

关闭swap

# 暂时关闭
swapoff -a
# 永久关闭，修改/etc/fstab取消swap分区
sudo vi /etc/fstab
#/swap.img      none    swap    sw      0       0

关闭firewall

systemctl stop firewalld
systemctl disable firewalld

关闭selinux

# 暂时关闭 
setenforce 0
# 永久关闭 
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

添加源

#添加aliyun密钥
curl -fsSL https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg
#添加aliyun k8s源
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

安装

sudo atp-get install kubelet kubeadm kubectl docker-ce [containerd.io]

docker最好装docker-ce，docker.io没有包含containerd，还要单儿装containerd.io

以上操作在master和node节点都要执行

初始化k8s集群

在master新建集群

# 初始化集群控制台 Control plane
# 失败了必须用 kubeadm reset 重置再重试
sudo kubeadm init \
--apiserver-advertise-address=192.168.8.41 \    #主机ip
--image-repository registry.aliyuncs.com/google_containers \
--pod-network-cidr=10.244.0.0/16                #集群网络域，采用flannel用这个默认地址

# 记得把 kubeadm join xxx 保存起来
# 忘记了重新获取：kubeadm token create --print-join-command

# 复制授权文件，以便 kubectl 可以有权限访问集群
# 如果你其他节点需要访问集群，需要从主节点复制这个文件过去其他节点
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# 在其他机器上创建 ~/.kube/config 文件也能通过 kubectl 访问到集群

成功输出结果

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.8.41:6443 --token u0ion1.nxz86lyryfeikq1l \
        --discovery-token-ca-cert-hash sha256:8de62f9c5d16506bf6d52b1545d5ccd282e7b1e60eb2c403ec53215dc0cfeddf

子节点加入集群

sudo kubeadm join 192.168.8.41:6443 --token u0ion1.nxz86lyryfeikq1l \
        --discovery-token-ca-cert-hash sha256:8de62f9c5d16506bf6d52b1545d5ccd282e7b1e60eb2c403ec53215dc0cfeddf

配置集群网络

采用flannel插件，可选还有Calico等

在主结点执行

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

Troubleshoot

ApiService 6443 没起来

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
  - 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock ps -a | grep kube | grep -v pause'
      Once you have found the failing container, you can inspect its logs with:
  - 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock logs CONTAINERID'

报错是 dial unix /var/run/cri-dockerd.sock: connect: permission denied
解决：修改cri-dockerd.sock权限，或者把当前$USER加入docker用户组

Kubeadm初始化报错：[ERROR CRI]: container runtime is not running
删阶除 rm -rf /etc/containerd/config.toml
重启 systemctl restart containerd
Flannel 组网错误 "Failed to create sandbox for pod" err="rpc error: code = DeadlineExceeded desc = failed to get sandbox image \"registry.k8s.io/pause:3.8\"
需要在ctr 里下载镜像重新tag命名，且必须在k8s.io命名空间下
kubeadm init 超时 ==》没有安装containerd