k8s部署（kubeadmin） - 容器相关

#注意：master与node节点都需先安装docker
---

![](/media//202109/2021-09-29094357231014.png)

---
#MASTER节点
####组件：docker、apiserver、scheduler、controller manager

---
##署部MASTER节点
###关闭swap:
	swapoff -a

###添加阿里云YUM软件源
	cat <<EOF > /etc/yum.repos.d/kubernetes.repo
	[kubernetes]
	name=Kubernetes
	baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
	enabled=1
	gpgcheck=1
	repo_gpgcheck=1
	gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
	EOF

#更新缓存
	yum clean all
	yum -y makecache

###安装kubeadm，kubelet和kubectl
	yum install -y kubelet-1.17.3 kubeadm-1.17.3 kubectl-1.17.3
	systemctl enable kubelet

###kubeadm配置
	#输出kubeadm init命令默认参数的内容
	kubeadm config print init-defaults
	#输出到文件
	kubeadm config print init-defaults > init-config.yaml
	
	#输出kubeadm join命令默认参数的内容
	kubeadm config print join-defaults
	
	#列出所需的镜像列表
	kubeadm config images list --config=init-config.yaml
	kubeadm config images list --image-repository registry.aliyuncs.com/google_containers
	
	#拉取所需的镜像列表
	kubeadm config images pull --config=init-config.yaml
	kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers

---
###初始化Kubernetes

kubeadm init --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16
	#一定要加--pod-network-cidr=10.244.0.0/16，不然后面的pod的coredns会一直处于ContainerCreating

###使用kubectl工具：

mkdir -p $HOME/.kube
	sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
	sudo chown $(id -u):$(id -g) $HOME/.kube/config

---
###安装Pod网络插件（CNI）
##calico插件 [推荐使用]
### [calico网络插件安装](http://baodoc.frp.bao-server.club:22080/project/35/593/ "calico网络插件")

####下载yaml文件
	wget https://projectcalico.docs.tigera.io/archive/v3.14/manifests/calico.yaml

---
####下载镜像（如无外网权限，找台有的，离线下载下来）
	grep "image:" calico.yaml
	
	#根据上面信息，下载镜像
	for i in calico/cni:v3.14.2 calico/pod2daemon-flexvol:v3.14.2 calico/node:v3.14.2 calico/kube-controllers:v3.14.2 ; do docker pull $i ; done
	
---
####修改配置
	# - name: CALICO_IPV4POOL_CIDR
	#   value: "192.168.0.0/16"
	修改为
	- name: CALICO_IPV4POOL_CIDR
	  value: "10.244.0.0/16"    #跟你初始化k8s的pod网络一致
	-----------------------------------------------------------------------

####Calico为节点分配地址段的掩码长度（可不修改）
	CALICO_IPV4POOL_BLOCK_SIZE变量的值，指定Calico为节点分配地址段的掩码长度，默认26
	添加：
	- name: CALICO_IPV4POOL_BLOCK_SIZE
	  value: 24

---
####【指定网卡】
	添加：（如有多网卡情况）【指定网卡】
	- name: IP_AUTODETECTION_METHOD
      value: "interface=ens33,eth0"     #系统上的网卡

![](/media//202306/2023-06-04121248366223.png)

---
####【排除指定网卡】
	添加：（如有多网卡情况）【排除网卡】
	- name: IP_AUTODETECTION_METHOD
      value: "skip-interface=eth1,ens32" 
---

![](/media//202306/2023-06-04121211266430.png)

---
####应用
	kubectl apply -f calico.yaml
	
---
####查看
	kubectl get pods -A

![](/media//202306/2023-06-04121805590636.png)

---
##flannel插件
	kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

####离线安装
[kube-flannel.yml文件下载](http://baowork.frp.bao-server.club:22080/KODExplorer/index.php?share/file&user=1&sid=jYRBKiup "kube-flannel.yml文件下载")
	
		kubectl apply -f kube-flannel.yml

---
##命令补全
	yum install bash-completion
	source /usr/share/bash-completion/bash_completion
	source <(kubectl completion bash)
	echo "source <(kubectl completion bash)" >> ~/.bashrc
---
#NODE节点
####组件：docker、kubelet、kube-proxy
---
##部署NODE节点
###关闭swap:
	swapoff -a

#更新缓存
	yum clean all
	yum -y makecache
---
###安装kubeadm，kubelet(NODE的节点不用安装kubectl）
	yum install -y kubelet-1.17.3 kubeadm-1.17.3 kubernetes-cni-0.7.5
	systemctl enable kubelet
---
###加入Kubernetes Node

####查看token:

kubeadm token list

####查看--discovery-token-ca-cert-hash

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

向集群添加新节点，执行在kubeadm init输出的kubeadm join命令：

![](/media//202002/2020-02-21152120322021.png)

kubeadm join 10.0.52.13:6443 --token nqg5gl.zr84xb63z2ve7dbn --discovery-token-ca-cert-hash sha256:3e43b9c41dd74ba0f46c260d0e0b6663993cf4563c3852384916d76338d6202e

---
####查看join命令
	kubeadm token create --print-join-command

---
####在master执行
	kubectl get nodes

![](/media//202002/2020-02-21152217140836.png)

###测试kubernetes集群
在Kubernetes集群中创建一个pod，验证是否正常运行：

kubectl create deployment nginx --image=nginx
	kubectl expose deployment nginx --port=80 --type=NodePort
	kubectl get pod,svc,nodes -o wide

访问地址：http://NodeIP:Port

![](/media//202002/2020-02-22121835515074.png)

##安装问题

###错误：
>error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to decode cluster configuration data: no kind "ClusterConfiguration" is registered for version "kubeadm.k8s.io/v1beta2"

###解决：
>执行 join 的 kubeadm 版本与执行 init 的 kubeadm 版本不匹配，升级 kubeadm 版本

###错误：
>####node加入master时：
>error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition

###解决：

swapoff -a    # will turn off the swap 
	kubeadm reset
	systemctl daemon-reload
	systemctl restart kubelet
	iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

###错误
>####Docker服务Cgroup驱动为"cgroupfs"问题
###解决：
####方式一：
	vim /etc/docker/daemon.json
	#添加：
	{
	"exec-opts": ["native.cgroupdriver=systemd"]
	}
	#重启docker
	systemctl restrt docker
	#查看
	docker info | grep Cgroup

![](/media//202110/2021-10-13155255001594.png)

####方式二：（如果方式一不行）
	vim /usr/lib/systemd/system/docker.service

![](/media//202110/2021-10-13155314924282.png)

#重启docker
	systemctl restrt docker
	#查看
	docker info | grep Cgroup
---
####node节点pod无法启动/节点删除网络重置"cni0" already has an IP address different from

kubeadm reset
	systemctl stop kubelet
	systemctl stop docker
	rm -rf /var/lib/cni/
	rm -rf /var/lib/kubelet/*
	rm -rf /etc/cni/
	ifconfig cni0 down
	ifconfig flannel.1 down
	ifconfig docker0 down
	ip link delete cni0
	ip link delete flannel.1
	systemctl start docker
	systemctl start kubelet

---
#参考资料：
	https://www.jianshu.com/p/b16628de567e
	https://blog.51cto.com/ckl893/2343871?source=dra