[Rocky 10] Kubernetes 를 해보자
INTRO
토막난 글입니다.
kube 1.35
cri-o 사용
설치 과정
내부 설정
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
systemctl stop firewalld && systemctl disable firewalld
swapoff -a
sed -i '/ swap /s/./#&/' /etc/fstab
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
kernel.unprivileged_bpf_disabled = 1 # CNI 설치 시
EOF
rebootkernel.unprivileged_bpf_disabled : calico 설치 시 EBPF 비활성화로 설치가 안된다. 2 -> 1로 바꾸면 설치 된다. 재부팅 필요
컨테이너 런타임
참고: Dockershim은 쿠버네티스 릴리스 1.24부터 쿠버네티스 프로젝트에서 제거되었다. 더 자세한 내용은 Dockershim 제거 FAQ를 참고한다. 파드가 노드에서 실행될 수 있도록 클러스터의 각 노드에 컨
kubernetes.io
What is eBPF? An Introduction and Deep Dive into the eBPF Technology
A detailed step by step introduction to the eBPF technology with lots of references for further reading.
ebpf.io
Documentation for /proc/sys/kernel/ — The Linux Kernel documentation
Documentation for /proc/sys/kernel/ Copyright (c) 1998, 1999, Rik van Riel Copyright (c) 2009, Shen Feng For general info and legal blurb, please look in Documentation for /proc/sys. This file contains documentation for the sysctl files in /proc/sys/kernel
docs.kernel.org
저장소 설정 및 설치
cat <<'EOF'> /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.35/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.35/rpm/repodata/repomd.xml.key
EOF
cat <<EOF | tee /etc/yum.repos.d/cri-o.repo
[cri-o]
name=CRI-O
baseurl=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/v1.35/rpm/
enabled=1
gpgcheck=1
gpgkey=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/v1.35/rpm/repodata/repomd.xml.key
EOF
dnf install -y container-selinux
dnf install -y cri-o kubelet kubeadm kubectlcri-o
Try Now Minikube: Use CRI-O as the container runtime kubeadm: Checkout this kubeadm guide to setup kubeadm to use CRI-O Kubic: Configured to use CRI-O out of the box in both its kubeadm and microOS system roles Join #crio on Kubernetes Slack Distribution P
cri-o.io
(선택) systemd cgroup 지정
# /etc/crio/crio.conf.d/10-crio.conf 파일 [crio.runtime] 아래 아래 두줄 추가
conmon_cgroup = "pod"
systemd_cgroup = true
# 아니면
sed -i '/\[crio.runtime\]/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/10-crio.conf
sed -i '/\[crio.runtime\]/a systemd_cgroup = true' /etc/crio/crio.conf.d/10-crio.conf이미 기본으로 systemd 사용한다고 한다..
컨테이너 런타임
참고: Dockershim은 쿠버네티스 릴리스 1.24부터 쿠버네티스 프로젝트에서 제거되었다. 더 자세한 내용은 Dockershim 제거 FAQ를 참고한다. 파드가 노드에서 실행될 수 있도록 클러스터의 각 노드에 컨
kubernetes.io
자동시작 활성화 및 재시작
systemctl daemon-reload
systemctl enable crio.service
systemctl enable kubelet
reboot
설치용 설정 파일 생성
cat <<EOF> kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: "<호스트 아이피>"
bindPort: 6443
nodeRegistration:
criSocket: "unix:///var/run/crio/crio.sock"
imagePullPolicy: IfNotPresent
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: "v1.35.0"
controlPlaneEndpoint: "kube-host:6443"
networking:
serviceSubnet: "10.200.0.0/12"
podSubnet: "10.0.0.0/16"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
failSwapOn: true
EOFkubeadm Configuration (v1beta3)
Overview Package v1beta3 defines the v1beta3 version of the kubeadm configuration file format. This version improves on the v1beta2 format by fixing some minor issues and adding a few new fields. A list of changes since v1beta2: The deprecated "ClusterConf
kubernetes.io
설치
kubeadm init --config kubeadm-config.yaml
## 설치 실패 시 입력
kubeadm reset -f --cri-socket=unix:///var/run/crio/crio.sock
설정 복사
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
CNI - Calico
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.31.4/manifests/tigera-operator.yaml
# 배포 상태 확인 후 완료되면 아래 진행
kubectl describe pod -n tigera-operator
# 만약, 대역을 바꿨다면 yaml 수정 후 넣어야 함
curl -O https://raw.githubusercontent.com/projectcalico/calico/v3.31.4/manifests/custom-resources.yaml
# ip 확인하기
kubectl get cm -n kube-system kubeadm-config -o yaml | grep podSubnet
# cidr: 192.168.0.0/16 를 10.0.0.0/16 으로 변경
kubectl create -f custom-resources.yamlCalico quickstart guide | Calico Documentation
Quickstart for Calico.
docs.tigera.io
headlamp
https://headlamp.dev/docs/latest/installation/in-cluster/
설치하기 (helm)
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/headlamp/main/kubernetes-headlamp.yamlhelm 은 epel-release 에 있다
# 현재 노드 상황
kubectl get nodes <-o wide>
[root@kube-host ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
kube-02 Ready <none> 98m v1.35.3
kube-host Ready control-plane 129m v1.35.3
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
calico-apiserver-9f96f4445-7xvfp 1/1 Running 1 15h 10.0.154.10 kube-host <none> <none>
calico-apiserver-9f96f4445-s88mz 1/1 Running 1 15h 10.0.154.11 kube-host <none> <none>
calico-kube-controllers-657bd55c99-b99hg 1/1 Running 1 15h 10.0.154.9 kube-host <none> <none>
calico-node-f58z9 1/1 Running 1 15h 192.168.24.190 kube-host <none> <none>
calico-node-ght5t 1/1 Running 1 15h 192.168.24.191 kube-02 <none> <none>
calico-typha-68cbdcbb55-8v7q6 1/1 Running 1 15h 192.168.24.191 kube-02 <none> <none>
csi-node-driver-dmh7c 2/2 Running 2 15h 10.0.154.12 kube-host <none> <none>
csi-node-driver-j8zt8 2/2 Running 2 15h 10.0.84.66 kube-02 <none> <none>
goldmane-58f96f7c58-52875 1/1 Running 1 15h 10.0.154.14 kube-host <none> <none>
whisker-5f8555ddf5-zjncj 2/2 Running 2 15h 10.0.154.13 kube-host <none> <none>
[root@kube-host ~]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-7d764666f9-5mcvx 1/1 Running 1 17h 10.0.154.15 kube-host <none> <none>
coredns-7d764666f9-r7rgd 1/1 Running 1 17h 10.0.154.16 kube-host <none> <none>
etcd-kube-host 1/1 Running 4 17h 192.168.24.190 kube-host <none> <none>
kube-apiserver-kube-host 1/1 Running 3 17h 192.168.24.190 kube-host <none> <none>
kube-controller-manager-kube-host 1/1 Running 3 17h 192.168.24.190 kube-host <none> <none>
kube-proxy-kfxt7 1/1 Running 3 17h 192.168.24.190 kube-host <none> <none>
kube-proxy-lf55f 1/1 Running 1 17h 192.168.24.191 kube-02 <none> <none>
kube-scheduler-kube-host 1/1 Running 4 17h 192.168.24.190 kube-host <none> <none>| kubelet | 각 노드에 실행되는 에이전트 |
| CNI(Container Network Interface) | |
| kube-apiserver | |
| etcd | |
| kube-controller-manager | |
| kube-scheduler | |
| kube-proxy | |