[preflight] Running pre-flight checks [WARNING Hostname]: hostname "node01" could not be reached [WARNING Hostname]: hostname "node01": lookup node01 on 114.114.114.114:53: no such host error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2 [ERROR Mem]: the system RAM (972 MB) is less than the minimum 1700 MB [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
[control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [kubelet-check] Initial timeout of 40s passed. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
Unfortunately, an error has occurred: timed out waiting for the condition
This error is likely caused by: - The kubelet is not running - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: - 'systemctl status kubelet' - 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime. To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in docker: - 'docker ps -a | grep kube | grep -v pause' Once you have found the failing container, you can inspect its logs with: - 'docker logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster To see the stack trace of this error execute with --v=5 or higher
kubelet 没有启动起来，查看日志：tail /var/log/messages
1 2 3 4 5 6 7 8 9 10
Nov 1 15:04:51 node01 kubelet: I1101 15:04:51.235877 8809 docker_service.go:242] "Hairpin mode is set" hairpinMode=hairpin-veth Nov 1 15:04:51 node01 kubelet: I1101 15:04:51.235947 8809 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Nov 1 15:04:51 node01 kubelet: I1101 15:04:51.242254 8809 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Nov 1 15:04:51 node01 kubelet: I1101 15:04:51.242306 8809 docker_service.go:257] "Docker cri networking managed by the network plugin" networkPluginName="cni" Nov 1 15:04:51 node01 kubelet: I1101 15:04:51.242331 8809 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Nov 1 15:04:51 node01 kubelet: I1101 15:04:51.249360 8809 docker_service.go:264] "Docker Info" dockerInfo=&{ID:ZB4Z:FUQW:IXZR:H3XP:E4PL:WXGO:4ODH:A72V:BDIY:D4AJ:F6S7:M2J2 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:7 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:[] Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:[] Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6tables:false Debug:false NFd:25 OomKillDisable:true NGoroutines:34 SystemTime:2021-11-01T15:04:51.242765302+08:00 LoggingDriver:json-file CgroupDriver:cgroupfs CgroupVersion:1 NEventsListener:0 KernelVersion:3.10.0-1160.45.1.el7.x86_64 OperatingSystem:CentOS Linux 7 (Core) OSVersion:7 OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:0xc0003fa070 NCPU:2 MemTotal:2093301760 GenericResources:[] DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:node01 Labels:[] ExperimentalBuild:false ServerVersion:20.10.10 ClusterStore: ClusterAdvertise: Runtimes:map[io.containerd.runc.v2:{Path:runc Args:[] Shim:<nil>} io.containerd.runtime.v1.linux:{Path:runc Args:[] Shim:<nil>} runc:{Path:runc Args:[] Shim:<nil>}] DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:[] Nodes:0 Managers:0 Cluster:<nil> Warnings:[]} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:5b46e404f6b9f661a205e28d59c982d3634148f8 Expected:5b46e404f6b9f661a205e28d59c982d3634148f8} RuncCommit:{ID:v1.0.2-0-g52b36a2 Expected:v1.0.2-0-g52b36a2} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=default] ProductLicense: DefaultAddressPools:[] Warnings:[WARNING: bridge-nf-call-ip6tables is disabled]} Nov 1 15:04:51 node01 kubelet: E1101 15:04:51.249384 8809 server.go:294] "Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: \"cgroupfs\"" Nov 1 15:04:51 node01 systemd: kubelet.service: main process exited, code=exited, status=1/FAILURE Nov 1 15:04:51 node01 systemd: Unit kubelet.service entered failed state. Nov 1 15:04:51 node01 systemd: kubelet.service failed.
[mark-control-plane] Marking the node node01 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [bootstrap-token] Using token: 810man.kfqm2lvq8mfqq2k7 [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
[preflight] Running pre-flight checks [WARNING Hostname]: hostname "node02" could not be reached [WARNING Hostname]: hostname "node02": lookup node02 on 114.114.114.114:53: no such host [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
根据提示，在 Master 节点上执行命令 kubectl get nodes，获取到当前集群中的 node，
1 2 3 4
[root@node01 docker]# kubectl get nodes NAME STATUS ROLES AGE VERSION node01 NotReady control-plane,master 67m v1.22.3 node02 NotReady <none> 5m38s v1.22.3
这时候是没有网络的，所以节点都是 NOT READY 的状态，在查看 /var/log/messages 的日志时，一直报网络的错误：
1 2
Nov 1 16:18:50 node01 kubelet: I1101 16:18:50.622393 13344 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Nov 1 16:18:52 node01 kubelet: E1101 16:18:52.110886 13344 kubelet.go:2337] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized"
I1103 13:57:58.187573 18232 checks.go:403] checking whether the given node name is valid and reachable using net.LookupHost I1103 13:57:58.187665 18232 checks.go:618] validating kubelet version I1103 13:57:58.222792 18232 checks.go:132] validating if the "kubelet" service is enabled and active I1103 13:57:58.228624 18232 checks.go:205] validating availability of port 10250 I1103 13:57:58.228884 18232 checks.go:282] validating the existence of file /etc/kubernetes/pki/ca.crt I1103 13:57:58.228894 18232 checks.go:432] validating if the connectivity type is via proxy or direct I1103 13:57:58.228917 18232 join.go:475] [preflight] Discovering cluster-info I1103 13:57:58.228939 18232 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "192.168.170.111:6443" I1103 13:57:58.241805 18232 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "810man", will try again I1103 13:58:04.154621 18232 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "810man", will try again I1103 13:58:10.576176 18232 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "810man", will try again I1103 13:58:16.580191 18232 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "810man", will try again
