Блог

Я настроил 1 главного 2 рабочих. после успешной установки kubernetes. Все в порядке с присоединением worker1 к кластеру, но я не могу присоединить worker2 к кластеру, потому что служба kubelet не запущена. Похоже, что kubelet не запущен или не исправен

sudo kubectl получает узлы:

ИМЯ СТАТУС РОЛИ ВОЗРАСТ ВЕРСИЯ
master1 Готовая плоскость управления, master 23m v1.22.2
node1 не готов 4m13s v1.22.2

Я хочу знать, почему служба kubelet не запущена.

Здесь kubelet регистрируется.

 The start-up result is RESULT.
Dec 04 20:21:26 node2 kubelet[25435]: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Dec 04 20:21:26 node2 kubelet[25435]: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.659131   25435 server.go:440] "Kubelet version" kubeletVersion="v1.22.2"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.659587   25435 server.go:868] "Client rotation is on, will bootstrap in background"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.678863   25435 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.684321   25435 dynamic_cafile_content.go:155] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.728096   25435 server.go:687] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.728320   25435 container_manager_linux.go:280] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.728388   25435 container_manager_linux.go:285] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName:
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729329   25435 topology_manager.go:133] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="c
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729345   25435 container_manager_linux.go:320] "Creating device plugin manager" devicePluginEnabled=true
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729367   25435 state_mem.go:36] "Initialized new in-memory state store"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729408   25435 kubelet.go:314] "Using dockershim is deprecated, please consider using a full-fledged CRI implementation"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729430   25435 client.go:78] "Connecting to docker on the dockerEndpoint" endpoint="unix:///var/run/docker.sock"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729441   25435 client.go:97] "Start docker client with request timeout" timeout="2m0s"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.744324   25435 docker_service.go:566] "Hairpin mode is set but kubenet is not enabled, falling back to HairpinVeth" hairpinMode=promiscu
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.744354   25435 docker_service.go:242] "Hairpin mode is set" hairpinMode=hairpin-veth
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.744554   25435 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.750011   25435 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.750260   25435 docker_service.go:257] "Docker cri networking managed by the network plugin" networkPluginName="cni"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.753050   25435 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d"
Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.764080   25435 docker_service.go:264] "Docker Info" dockerInfo=amp;{ID:4UUR:AFJU:SXYE:5IRP:6G6B:SFDY:H3AA:D5ZB:JSDO:GXVQ:UYNG:POJY Containe
Dec 04 20:21:26 node2 kubelet[25435]: E1204 20:21:26.765777   25435 server.go:294] "Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: "systemd" i
Dec 04 20:21:26 node2 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Dec 04 20:21:26 node2 systemd[1]: kubelet.service: Failed with result 'exit-code'.

 

журналы соединения kubeadm:

 I1204 20:27:56.222794   29796 join.go:405] [preflight] found NodeName empty; using OS hostname as NodeName
I1204 20:27:56.223032   29796 initconfiguration.go:116] detected and using CRI socket: /var/run/dockershim.sock
[preflight] Running pre-flight checks
I1204 20:27:56.223834   29796 preflight.go:92] [preflight] Running general checks
I1204 20:27:56.225983   29796 checks.go:245] validating the existence and emptiness of directory /etc/kubernetes/manifests
I1204 20:27:56.226133   29796 checks.go:282] validating the existence of file /etc/kubernetes/kubelet.conf
I1204 20:27:56.226271   29796 checks.go:282] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I1204 20:27:56.226408   29796 checks.go:106] validating the container runtime
I1204 20:27:56.282374   29796 checks.go:132] validating if the "docker" service is enabled and active
I1204 20:27:56.300100   29796 checks.go:331] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I1204 20:27:56.300279   29796 checks.go:331] validating the contents of file /proc/sys/net/ipv4/ip_forward
I1204 20:27:56.300580   29796 checks.go:649] validating whether swap is enabled or not
I1204 20:27:56.300738   29796 checks.go:372] validating the presence of executable conntrack
I1204 20:27:56.301009   29796 checks.go:372] validating the presence of executable ip
I1204 20:27:56.301613   29796 checks.go:372] validating the presence of executable iptables
I1204 20:27:56.301801   29796 checks.go:372] validating the presence of executable mount
I1204 20:27:56.302057   29796 checks.go:372] validating the presence of executable nsenter
I1204 20:27:56.302384   29796 checks.go:372] validating the presence of executable ebtables
I1204 20:27:56.302473   29796 checks.go:372] validating the presence of executable ethtool
I1204 20:27:56.302569   29796 checks.go:372] validating the presence of executable socat
I1204 20:27:56.302610   29796 checks.go:372] validating the presence of executable tc
I1204 20:27:56.303072   29796 checks.go:372] validating the presence of executable touch
I1204 20:27:56.303472   29796 checks.go:520] running all checks
I1204 20:27:56.372402   29796 checks.go:403] checking whether the given node name is valid and reachable using net.LookupHost
I1204 20:27:56.373211   29796 checks.go:618] validating kubelet version
I1204 20:27:56.467792   29796 checks.go:132] validating if the "kubelet" service is enabled and active
I1204 20:27:56.485715   29796 checks.go:205] validating availability of port 10250
I1204 20:27:56.486624   29796 checks.go:282] validating the existence of file /etc/kubernetes/pki/ca.crt
I1204 20:27:56.487016   29796 checks.go:432] validating if the connectivity type is via proxy or direct
I1204 20:27:56.487841   29796 join.go:475] [preflight] Discovering cluster-info
I1204 20:27:56.488260   29796 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "192.168.1.53:6443"
I1204 20:27:56.520182   29796 token.go:118] [discovery] Requesting info from "192.168.1.53:6443" again to validate TLS against the pinned public key
I1204 20:27:56.530589   29796 token.go:135] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.1.53:6443"
I1204 20:27:56.530702   29796 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I1204 20:27:56.530924   29796 join.go:489] [preflight] Fetching init configuration
I1204 20:27:56.531171   29796 join.go:534] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I1204 20:27:56.549808   29796 interface.go:431] Looking for default routes with IPv4 addresses
I1204 20:27:56.549913   29796 interface.go:436] Default route transits interface "enp0s3"
I1204 20:27:56.550259   29796 interface.go:208] Interface enp0s3 is up
I1204 20:27:56.550564   29796 interface.go:256] Interface "enp0s3" has 2 addresses :[192.168.1.50/24 fe80::a00:27ff:fe7e:db8b/64].
I1204 20:27:56.550644   29796 interface.go:223] Checking addr  192.168.1.50/24.
I1204 20:27:56.550887   29796 interface.go:230] IP found 192.168.1.50
I1204 20:27:56.550955   29796 interface.go:262] Found valid IPv4 address 192.168.1.50 for interface "enp0s3".
I1204 20:27:56.551237   29796 interface.go:442] Found active IP 192.168.1.50
I1204 20:27:56.563573   29796 preflight.go:103] [preflight] Running configuration dependant checks
I1204 20:27:56.563872   29796 controlplaneprepare.go:219] [download-certs] Skipping certs download
I1204 20:27:56.565399   29796 kubelet.go:112] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I1204 20:27:56.569613   29796 kubelet.go:120] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I1204 20:27:56.572216   29796 kubelet.go:141] [kubelet-start] Checking for an existing Node in the cluster with name "node2" and status "Ready"
I1204 20:27:56.576685   29796 kubelet.go:155] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
I1204 20:28:01.956734   29796 kubelet.go:190] [kubelet-start] preserving the crisocket information for the node
I1204 20:28:01.956911   29796 patchnode.go:31] [patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "node2" as an annotation
I1204 20:28:01.957066   29796 cert_rotation.go:137] Starting client certificate rotation controller
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.

 

Сначала проверьте, включен ли swap на вашем узле, поскольку вы ДОЛЖНЫ отключить swap, чтобы kubelet работал должным образом.

 sudo swapoff -a  
sudo sed -i '/ swap / s/^/#/' /etc/fstab
 

Также проверьте, установлен ли kubernetes и драйвер docker cgroup на то же значение.
Из документации kubernetes:

Как среда выполнения контейнера, так и kubelet имеют свойство, называемое «драйвер cgroup», что важно для управления cgroups на компьютерах Linux.
Требуется сопоставление среды выполнения контейнера и драйверов kubelet cgroup, иначе процесс kubelet завершится с ошибкой.

На странице среды выполнения контейнера объясняется, что для настроек на основе kubeadm рекомендуется использовать драйвер systemd вместо драйвера cgroupfs, поскольку kubeadm управляет kubelet как службой systemd.

Для docker:

 docker info |grep -i cgroup
 

Вы можете добавить это /etc/docker/daemon.json , чтобы установить драйвер docker cgroup в systemd:

 {
    "exec-opts": ["native.cgroupdriver=systemd"]
}
 

Перезапустите службу docker после внесения каких-либо изменений с помощью

 sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl restart kubelet
 

Вы можете попытаться выполнить kubeadm join после выполнения вышеуказанных шагов.