Kubernetes/K8S-kubectl
Kubernetes/K8S kubectl 命令行工具使用笔记
kubectl reference docs （页面下面有所有 kubectl 子命令的 doc 链接）
https://kubernetes.io/docs/reference/kubectl/kubectl/
kubectl 概览
Overview of kubectl
https://kubernetes.io/docs/reference/kubectl/overview/
kubectl 命令行工具用于管理 Kubernetes 集群。
kubectl 会在 $HOME/.kube 目录中查找并加载名为 config 的配置文件，也可以通过 KUBECONFIG 环境变量或 --kubeconfig 参数来指定 kubeconfig 文件。
kubectl 命令的语法格式：kubectl [command] [TYPE] [NAME] [flags]
其中：
command指定在一个或多个资源上要进行的操作，例如create,get,describe,deleteTYPE指定资源类型，资源类型不区分大小写，可以指定单数、复数或缩写形式。
例如kubectl get nodes或kubectl get node或kubectl get no都可以查看 k8s 集群中的节点信息。kubectl get pods或kubectl get pod或kubectl get po都可以查看 podNAME指定资源名，资源名不区分大小写。如果忽略资源名，会显示全部此类资源，例如kubectl get pods查看默认命名空间的全部 pod
在对多个资源执行操作时，你可以按类型和名称指定每个资源，或指定一个或多个文件，例如：
1、TYPE1 name1 name2 name<#>同一类型的多个资源，例如kubectl get pod example-pod1 example-pod2
2、TYPE1/name1 TYPE1/name2 TYPE2/name3 TYPE<#>/name<#>不同类型的资源，例如kubectl get pod/example-pod1 replicationcontroller/example-rc1
3、-f file1 -f file2 -f file<#>用一个或多个文件指定资源，例如kubectl get -f ./pod.yamlflags可选的参数，例如可以使用 -s 或 -server 参数指定 Kubernetes API 服务器的地址和端口。
通用选项
kubectl
https://kubernetes.io/docs/reference/kubectl/kubectl/
-n, --namespace string 在指定的名字空间中操作。默认使用 default 名字空间。
kubectl api-resources 列出支持的资源类型
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#api-resources
列出k8s支持的所欲api资源
kubectl api-resources -o wide 可以多打印出 VERBS 列，说明此资源支持的动词(create delete deletecollection get list patch update watch)
# kubectl api-resources
NAME SHORTNAMES APIGROUP NAMESPACED KIND
bindings true Binding
componentstatuses cs false ComponentStatus
configmaps cm true ConfigMap
endpoints ep true Endpoints
events ev true Event
limitranges limits true LimitRange
namespaces ns false Namespace
nodes no false Node
persistentvolumeclaims pvc true PersistentVolumeClaim
persistentvolumes pv false PersistentVolume
pods po true Pod
podtemplates true PodTemplate
replicationcontrollers rc true ReplicationController
resourcequotas quota true ResourceQuota
secrets true Secret
serviceaccounts sa true ServiceAccount
services svc true Service
mutatingwebhookconfigurations admissionregistration.k8s.io false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io false CustomResourceDefinition
apiservices apiregistration.k8s.io false APIService
controllerrevisions apps true ControllerRevision
daemonsets ds apps true DaemonSet
deployments deploy apps true Deployment
replicasets rs apps true ReplicaSet
statefulsets sts apps true StatefulSet
tokenreviews authentication.k8s.io false TokenReview
localsubjectaccessreviews authorization.k8s.io true LocalSubjectAccessReview
selfsubjectaccessreviews authorization.k8s.io false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io false SubjectAccessReview
horizontalpodautoscalers hpa autoscaling true HorizontalPodAutoscaler
cronjobs cj batch true CronJob
jobs batch true Job
certificatesigningrequests csr certificates.k8s.io false CertificateSigningRequest
leases coordination.k8s.io true Lease
events ev events.k8s.io true Event
ingresses ing extensions true Ingress
nodes metrics.k8s.io false NodeMetrics
pods metrics.k8s.io true PodMetrics
alertmanagers monitoring.coreos.com true Alertmanager
podmonitors monitoring.coreos.com true PodMonitor
prometheuses monitoring.coreos.com true Prometheus
prometheusrules monitoring.coreos.com true PrometheusRule
servicemonitors monitoring.coreos.com true ServiceMonitor
thanosrulers monitoring.coreos.com true ThanosRuler
ingresses ing networking.k8s.io true Ingress
networkpolicies netpol networking.k8s.io true NetworkPolicy
runtimeclasses node.k8s.io false RuntimeClass
poddisruptionbudgets pdb policy true PodDisruptionBudget
podsecuritypolicies psp policy false PodSecurityPolicy
clusterrolebindings rbac.authorization.k8s.io false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io false ClusterRole
rolebindings rbac.authorization.k8s.io true RoleBinding
roles rbac.authorization.k8s.io true Role
priorityclasses pc scheduling.k8s.io false PriorityClass
podpresets settings.k8s.io true PodPreset
apimonitors am stable.example.com false apiMonitor
csidrivers storage.k8s.io false CSIDriver
csinodes storage.k8s.io false CSINode
storageclasses sc storage.k8s.io false StorageClass
volumeattachments storage.k8s.io false VolumeAttachment
kubectl get 查看资源
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get
kubectl api-resources 可以查看 k8s 支持的所有资源类型。
常用的：
# kubectl api-resources -o wide
NAME SHORTNAMES APIGROUP NAMESPACED KIND VERBS
configmaps cm true ConfigMap [create delete deletecollection get list patch update watch]
namespaces ns false Namespace [create delete get list patch update watch]
nodes no false Node [create delete deletecollection get list patch update watch]
persistentvolumeclaims pvc true PersistentVolumeClaim [create delete deletecollection get list patch update watch]
persistentvolumes pv false PersistentVolume [create delete deletecollection get list patch update watch]
pods po true Pod [create delete deletecollection get list patch update watch]
services svc true Service [create delete get list patch update watch]
deployments deploy apps true Deployment [create delete deletecollection get list patch update watch]
replicasets rs apps true ReplicaSet [create delete deletecollection get list patch update watch]
statefulsets sts apps true StatefulSet [create delete deletecollection get list patch update watch]
serviceaccounts sa true ServiceAccount [create delete deletecollection get list patch update watch]
secrets true Secret [create delete deletecollection get list patch update watch]
jobs batch true Job [create delete deletecollection get list patch update watch]
ingresses ing extensions true Ingress [create delete deletecollection get list patch update watch]
-o, --output='' 输出格式，常用的有：-o wide 带额外信息的纯文本输出。-o yaml 输出为 yaml 格式的 api 对象。-o json 输出为 json 格式的 api 对象。
kubectl get all 查看全部资源
例如查看全部 mysql 相关资源
# kubectl get all|grep mysql
pod/mysql-ha-0 3/3 Running 0 67d
pod/mysql-init-job-zcdxx 0/1 Completed 0 75d
service/mysql-ha ClusterIP None <none> 3306/TCP 75d
service/mysql-service NodePort 127.0,0,1 <none> 3306:3306/TCP 75d
deployment.apps/mysql-ha-controller 1/1 1 1 75d
replicaset.apps/mysql-ha-controller-677456846d 0 0 0 75d
statefulset.apps/mysql-ha 3/3 75d
job.batch/mysql-init-job 1/1 3m35s 75d
kubectl get -w/–watch 查看并监控资源变化
比如 mysql pod 启动过程中可以 kubectl get pod -o wide -w|grep mysql 监控 mysql pod 的状态变化。
kubectl get pod,svc 同时查看Pod和Service
# kubectl get pod,svc -o wide|grep mysql
pod/mysql-0 1/1 Running 2 6d19h 10.200.12.123 centos <none> <none>
service/mysql-service ClusterIP 10.200.10.111 <none> 8306/TCP 20d app=mysql-my
kubectl get cs 查看组件状态(v1.19后废弃)
kubectl get componentstatus 或 kubectl get cs 查看 k8s 组件状态。
$ kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}
从 v1.19 开始， componentstatus API 被废弃。
componentstatus API 提供 etcd, kube-scheduler 和 kube-controller-manager 的状态信息，但只有在这些组件和 api-server 位于同一节点上时数据才是正确的。
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#deprecation
kubectl get node/no 查看节点信息
kubectl get nodes 或 kubectl get node 或 kubectl get no 查看 k8s 集群中的节点信息
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
linode NotReady master 11d v1.19.0
kubectl get node -o wide 查看node及IP
# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
node1 Ready master 38d v1.16.3 127.0.0.1 <none> CentOS Linux 7 (Core) 4.17.11-1.el7.elrepo.x86_64 docker://19.3.12
node2 Ready master 38d v1.16.3 127.0.0.2 <none> CentOS Linux 7 (Core) 4.17.11-1.el7.elrepo.x86_64 docker://19.3.12
node3 Ready master 38d v1.16.3 127.0.0.3 <none> CentOS Linux 7 (Core) 4.17.11-1.el7.elrepo.x86_64 docker://19.3.12
kubectl get node –show-labels=true 查看节点的标签
kubectl get node –show-labels=true 或 kubectl get node –show-labels
# kubectl get node --show-labels=true
NAME STATUS ROLES AGE VERSION LABELS
linode Ready master,node 22d v1.12.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=linode,node-role.kubernetes.io/master=,node-role.kubernetes.io/node=
kubectl get namespace/ns 查看命名空间
kubectl get namespaces 或 kubectl get namespace 或 kubectl get ns 查看命名空间
# kubectl get ns
NAME STATUS AGE
default Active 202d
ingress-nginx Active 202d
kube-node-lease Active 202d
kube-public Active 202d
kube-system Active 202d
local-path-storage Active 202d
kubectl get service/svc 查看service
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kafka-alone-headless ClusterIP None <none> 8092/TCP 36d
kafka-service ClusterIP 10.123.12.123 <none> 8092/TCP 36d
kubernetes ClusterIP 10.123.0.1 <none> 443/TCP 38d
mongo-service ClusterIP 10.123.12.123 <none> 8017/TCP 2d13h
mysql-service ClusterIP 10.123.12.12 <none> 8306/TCP 21d
redis-service ClusterIP 10.123.12.123 <none> 8179/TCP 36d
zk-service ClusterIP 10.123.12.123 <none> 8121/TCP 2d13h
zookeeper-headless ClusterIP None <none> 2181/TCP,3888/TCP,2888/TCP 2d13h
gateway-service NodePort 10.123.123.12 <none> 8081:9081/TCP 42h
rest-service NodePort 10.233.13.131 <none> 8102:9102/TCP,8488:9488/TCP 15d
可以看到
NodePort 8081:9081/TCP 的意思是：这是个 NodePort 类型的 service，将容器内部的 8081 端口映射为集群外可访问的 9081 端口
NodePort 8102:9102/TCP,8488:9488/TCP 的意思是：这是个 NodePort 类型的 service，向外暴露了两个端口，容器内的 8102 映射为 9102，容器内的 8488 映射为 9488
kubectl get service -o wide 查看service及selector
# kubectl get svc -o wide
NAME type cluster-ip external-ip port(s) age selector
da-service nodeport 10.233.53.253 <none> 8768:8768/tcp 4m18s app=da
kafka-alone-headless clusterip none <none> 8092/tcp 36d app.kubernetes.io/component=kafka-broker,app.kubernetes.io/instance=kafka-alone,app.kubernetes.io/name=kafka
kafka-service clusterip 10.233.58.186 <none> 8092/tcp 36d app.kubernetes.io/component=kafka-broker,app.kubernetes.io/instance=kafka-alone,app.kubernetes.io/name=kafka
kubernetes clusterip 10.233.0.1 <none> 443/tcp 38d <none>
mongo-service clusterip 10.233.37.193 <none> 8017/tcp 2d15h app=mongodb-ist
mysql-service clusterip 10.233.58.99 <none> 8306/tcp 21d app=mysql-alone-ist-ist
redis-service clusterip 10.233.44.137 <none> 8179/tcp 36d app=redis-ist
zk-service clusterip 10.233.16.246 <none> 8121/tcp 2d15h app=zookeeper-ist
kubectl get svc xx -o yaml 查看Service的yaml
# kubectl get svc myapp-service -o yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2021-07-14T03:02:07Z"
name: myapp-service
namespace: default
resourceVersion: "39037191"
selfLink: /api/v1/namespaces/default/services/myapp-service
uid: e958ddd5-396d-4be9-926c-df05b50d8d5a
spec:
clusterIP: 127.0.0.1
externalTrafficPolicy: Cluster
ports:
- name: myapp
nodePort: 9652
port: 8652
protocol: TCP
targetPort: 8652
selector:
app: myapp
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
kubectl get pod/po 查看pod
kubectl get pods 或 kubectl get pod 或 kubectl get po 查看 pod
输出结果中 READY 1/2 表示 已就绪容器个数/pod中总容器个数
kubectl get pods 查看默认namespace的pod
kubectl get pods 查看当前 namespace 下的所有 pod
kubectl get pod -A 查看所有namespace的pod
kubectl get pods -A -A 表示列出所有 namespace 的 pod
kubectl get pod -n kube-system 查看系统命名空间的pod
# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
cgpu-exporter-daemonset-wcfj6 1/1 Running 4 8d
coredns-5879b9c8f9-vlr27 1/1 Running 0 15d
dashboard-metrics-scraper-68746c59c9-pmfmh 1/1 Running 0 15d
dns-autoscaler-56547c6f84-l2rls 1/1 Running 0 15d
gpushare-scheduler-extender-dfcbw 1/1 Running 0 15d
kube-apiserver-centos-hostname 1/1 Running 0 6d3h
kube-controller-manager-centos-hostname 1/1 Running 0 15d
kube-flannel-ds-8rvqg 1/1 Running 0 13d
kube-flannel-ds-dd226 1/1 Running 0 15d
kube-proxy-l4m7r 1/1 Running 0 13d
kube-proxy-nw97h 1/1 Running 0 15d
kube-scheduler-centos-hostname 1/1 Running 0 15d
kubernetes-dashboard-c949889c5-74ztk 1/1 Running 0 15d
nodelocaldns-6p469 1/1 Running 0 15d
nodelocaldns-n9g7g 1/1 Running 0 13d
nvidia-device-plugin-daemonset-wchw4 1/1 Running 0 15d
nvidia-device-plugin-daemonset-x85dg 1/1 Running 0 13d
kubectl get pod -o wide 查看pod及ip
# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
prometheus-mm27q 1/1 Running 0 27d 127.0.0.1 linode <none> <none>
kubectl get pods name 查看指定pod
kubectl get pods pod-name 查看指定pod
$ kubectl get pods
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-f9fd979d6-8l6f2 0/1 Pending 0 11d
kube-system coredns-f9fd979d6-bmbnt 0/1 Pending 0 11d
kube-system etcd-linode 1/1 Running 0 11d
kube-system kube-apiserver-linode 1/1 Running 0 11d
kube-system kube-controller-manager-linode 1/1 Running 0 10d
kube-system kube-proxy-mlls6 1/1 Running 0 11d
kube-system kube-scheduler-linode 1/1 Running 0 10d
kubectl get pod name -o yaml 看pod原始yaml
kubectl get pods pod-name -o yaml 查看指定 pod 的原始 yaml
pod-name 来自 kubectl get pods 的第一列
kubectl get pod -owide -w 持续监控资源变化
kubectl get pod -owide -w |egrep "mysql|kafka" 查看并持续监控 mysql 和 kafka 的 pod
kubectl get configmap/cm 查看配置映射
kubectl get cm myapp-configmap -o yaml 查看某个 configmap 的具体内容
kubectl get cm -A coredns -o yaml 查看CoreDNS
kubectl get pvc 查看持久卷申领
# kubectl get pvc -o wide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
es-7-master-0 Bound pvc-44b8897d-506f-46ee-aed0-d259c0b7dded 100Gi RWO local-path 74d Filesystem
es-7-master-1 Bound pvc-6426963e-1896-4722-99ba-221034530331 100Gi RWO local-path 74d Filesystem
es-7-master-2 Bound pvc-51188068-0635-4f15-b1d1-d88b538690fa 100Gi RWO local-path 74d Filesystem
kubectl get pv 查看持久卷
# kubectl get pv -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
pvc-44b8897d-506f-46ee-aed0-d259c0b7dded 100Gi RWO Delete Bound default/es-7-master-0 local-path 74d Filesystem
pvc-51188068-0635-4f15-b1d1-d88b538690fa 100Gi RWO Delete Bound default/es-7-master-2 local-path 74d Filesystem
pvc-6426963e-1896-4722-99ba-221034530331 100Gi RWO Delete Bound default/es-7-master-1 local-path 74d Filesystem
kubectl get storageclass/sc 查看存储类
kubectl get storageclasses 或 kubectl get storageclass 或 kubectl get sc 查看存储类
# kubectl get sc
NAME PROVISIONER AGE
local-path rancher.io/local-path 48d
local-path-ssd cluster.local/local-path-provisioner-ssd 91m
kubectl get secret 查看密钥
kubectl get secret 查看全部密钥
查看 namespace 下的 secret 密钥
# kubectl get secret
NAME TYPE DATA AGE
apimonitor-reader-token-d7bkh kubernetes.io/service-account-token 3 2d
kubeapps-operator-token-k7v8w kubernetes.io/service-account-token 3 2d
mysql-alone-ist Opaque 2 4h35m
prometheus-prometheus-operator-prometheus-tls-assets Opaque 0 2d
sh.helm.release.v1.mysql-alone.v1 helm.sh/release.v1 1 4h35m
vitess-operator-token-nn4m5 kubernetes.io/service-account-token 3 9h
kubectl get secret xx -o yaml 查看密钥原始yaml
# kubectl get secret mysql-alone -o yaml
apiVersion: v1
data:
mysql-password: NDxxxxxxxx0tYOQ==
mysql-root-password: xxxxxxxxxxxx
kind: Secret
metadata:
creationTimestamp: "2021-06-30T23:49:17Z"
labels:
app: mysql-alone
chart: mysql-1.1.1
heritage: Helm
release: mysql-alone
name: mysql-alone
namespace: default
resourceVersion: "374990"
selfLink: /api/v1/namespaces/default/secrets/mysql-alone
uid: fc66f625-3fb6-4957-99ee-9786af38d6b0
type: Opaque
kubectl get serviceaccount/sa 查看服务账号
kubectl get serviceaccounts 或 kubectl get serviceaccount 或 kubectl get sa 查看服务账号
kubectl get serviceaccount/sa 查看全部服务账号
查看 namespace 下的服务账号
# kubectl get serviceAccounts
NAME SECRETS AGE
apimonitor-reader 1 2d
default 1 2d
kubeapps-operator 1 2d
prometheus-operator-alertmanager 1 2d
prometheus-operator-grafana 1 2d
prometheus-operator-grafana-test 1 2d
prometheus-operator-kube-state-metrics 1 2d
prometheus-operator-operator 1 2d
prometheus-operator-prometheus 1 2d
prometheus-operator-prometheus-adapter 1 2d
prometheus-operator-prometheus-node-exporter 1 2d
tianniu-default 1 32h
vitess-operator 1 9h
kubectl get serviceaccounts xx -o yaml 查看服务账号原始yaml
# kubectl get serviceaccounts vitess-operator -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"vitess-operator","namespace":"default"}}
creationTimestamp: "2021-06-30T18:50:26Z"
name: vitess-operator
namespace: default
resourceVersion: "332313"
selfLink: /api/v1/namespaces/default/serviceaccounts/vitess-operator
uid: 2868977f-cd8d-4a79-be2e-1cc77c248a23
secrets:
- name: vitess-operator-token-nn4m5
kubectl describe 查看资源详情
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#describekubectl describe 查看资源或资源组详情。
kubectl describe pod name 看pod配置和事件
pod-name 来自 kubectl get pods 的第一列
# kubectl describe pod my-app-deployment-5bf87ff7f9-gnsdb
Name: my-app-deployment-5bf87ff7f9-gnsdb
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: linode/127.0.0.1
Start Time: Sat, 20 Feb 2021 11:06:26 +0800
Labels: app=my-app
pod-template-hash=5bf87ff7f9
Annotations: podpreset.admission.kubernetes.io/podpreset-allow-tz-env: 1137
Status: Running
IP: 127.0.0.1
Controlled By: ReplicaSet/my-app-deployment-5bf87ff7f9
Containers:
my-app:
Container ID: docker://6ea436028db8d732523b395effad18efb2a1c0414de59d796c6f3dc06d9ee0cc
Image: masikkk.com/ist/my-app:20210220_1613789493178
Image ID: docker-pullable://docker.masikkk.com/my-app@sha256:6035a91620701696ecdb5e7dc638782c16c52be35540dac7db077a5fe9fadb53
Port: <none>
Host Port: <none>
Command:
python
Args:
/root/apps/start.py
--cluster_configFilesParams="[{ \"key\": \"db.datasource.url\", \"value\": \"jdbc:mysql://localhost:3306/mydb?useUnicode=true&characterEncoding=utf-8&autoReconnect=true&useSSL=false\" }, { \"key\": \"db.datasource.username\", \"value\": \"haha\" }]"
State: Running
Started: Sat, 20 Feb 2021 11:06:45 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 24
memory: 16Gi
Requests:
cpu: 100m
memory: 1Gi
Liveness: http-get http://localhost:8080/ delay=30s timeout=60s period=15s #success=1 #failure=8
Environment:
TZ: Asia/Shanghai
Mounts:
/etc/localtime:ro from ro (rw)
/home/centos/logs/my-app from log (rw)
/root/data from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-tj2ln (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
data:
Type: HostPath (bare host directory volume)
Path: /home/centos/data
HostPathType:
log:
Type: HostPath (bare host directory volume)
Path: /home/centos/logs/my-app
HostPathType:
ro:
Type: HostPath (bare host directory volume)
Path: /etc/localtime
HostPathType:
default-token-tj2ln:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-tj2ln
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
kubectl describe svc 看service的后端地址
Endpoints （该资源类别为复数）定义了网络端点的列表，通常由 Service 引用，以定义可以将流量发送到哪些 Pod。
# kubectl describe svc mysql-service
Name: mysql-service
Namespace: default
Labels: app=mysql-ha
chart=mysql-ha-5.7.34-v1.2.3.1
heritage=Helm
release=mysql-ha
Annotations: <none>
Selector: app=mysql-ha,release=mysql-ha,role=leader
Type: NodePort
IP: 192.168.1.101
Port: mysql 3306/TCP
TargetPort: mysql/TCP
NodePort: mysql 3306/TCP
Endpoints: 192.168.66.4:3306
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
kubectl describe node name 查看node详情
# kubectl describe node gpu-node1
Name: gpu-node1
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
gpu=nvidia
kubernetes.io/arch=amd64
kubernetes.io/hostname=gpu-node1
kubernetes.io/os=linux
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 23 Sep 2021 18:08:31 +0800
Taints: node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unreachable:NoSchedule
Unschedulable: false
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Tue, 04 Jan 2022 14:17:27 +0800 Tue, 04 Jan 2022 14:17:27 +0800 CalicoIsUp Calico is running on this node
MemoryPressure Unknown Tue, 04 Jan 2022 14:22:31 +0800 Tue, 04 Jan 2022 14:23:20 +0800 NodeStatusUnknown Kubelet stopped posting node status.
DiskPressure Unknown Tue, 04 Jan 2022 14:22:31 +0800 Tue, 04 Jan 2022 14:23:20 +0800 NodeStatusUnknown Kubelet stopped posting node status.
PIDPressure Unknown Tue, 04 Jan 2022 14:22:31 +0800 Tue, 04 Jan 2022 14:23:20 +0800 NodeStatusUnknown Kubelet stopped posting node status.
Ready Unknown Tue, 04 Jan 2022 14:22:31 +0800 Tue, 04 Jan 2022 14:23:20 +0800 NodeStatusUnknown Kubelet stopped posting node status.
Addresses:
InternalIP: 127.0.0.1
Hostname: gpu-node1
Capacity:
cpu: 40
ephemeral-storage: 102350Mi
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 196702560Ki
pods: 110
tpu.bitmain.com/bm1682: 16
Allocatable:
cpu: 39900m
ephemeral-storage: 96589578081
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 196350160Ki
pods: 110
tpu.bitmain.com/bm1682: 16
System Info:
Machine ID: e1ecfb37865b4e58b3064bb7dcc4e27a
System UUID: f7bf926c-f30b-03e4-b211-d21d600e2d1b
Boot ID: 05e536f6-e42e-4b07-ab2f-917cf7b7cef5
Kernel Version: 4.17.11-1.el7.elrepo.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://18.9.7
Kubelet Version: v1.16.3
Kube-Proxy Version: v1.16.3
PodCIDR: 10.233.90.0/24
PodCIDRs: 10.233.90.0/24
Non-terminated Pods: (18 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default nvidia-deviceplugin-qqvjb 0 (0%) 0 (0%) 0 (0%) 0 (0%) 98d
default nvidia-exporter-5v9bm 0 (0%) 0 (0%) 0 (0%) 0 (0%) 98d
default nvidia-my-app-gpu-0-deployment-868f79d77d-glpwn 100m (0%) 0 (0%) 1Gi (0%) 0 (0%) 3h34m
default nvidia-my-app-gpu-1-deployment-5bdbbdf774-kz849 100m (0%) 0 (0%) 1Gi (0%) 0 (0%) 3h33m
default nvidia-my-app-gpu-2-deployment-6cc4d9598b-pmftc 100m (0%) 0 (0%) 1Gi (0%) 0 (0%) 3h33m
default bvs-algo-video-server-deployment-5469d4c44b-879qg 100m (0%) 0 (0%) 1Gi (0%) 0 (0%) 3h33m
default consul-cluster-slave-26 0 (0%) 0 (0%) 0 (0%) 0 (0%) 3h36m
default ipmi-exporter-9tsbw 0 (0%) 0 (0%) 0 (0%) 0 (0%) 101d
default loki-stack-0 0 (0%) 0 (0%) 0 (0%) 0 (0%) 3h36m
default loki-stack-promtail-ffqbt 100m (0%) 1 (2%) 128Mi (0%) 1Gi (0%) 13d
default memory-storage-deployment-5d44c9b6f9-wt5np 100m (0%) 0 (0%) 1Gi (0%) 0 (0%) 3h33m
default prometheus-operator-prometheus-node-exporter-xk666 0 (0%) 0 (0%) 0 (0%) 0 (0%) 98d
ingress-nginx ingress-nginx-controller-mmz9b 0 (0%) 0 (0%) 0 (0%) 0 (0%) 102d
kube-system bml-filebeat-7l7rm 1 (2%) 2 (5%) 1Gi (0%) 2Gi (1%) 102d
kube-system calico-node-j7s76 150m (0%) 300m (0%) 64M (0%) 500M (0%) 102d
kube-system kube-proxy-wcvm8 0 (0%) 0 (0%) 0 (0%) 0 (0%) 102d
kube-system nginx-proxy-gpu-node1 25m (0%) 0 (0%) 32M (0%) 0 (0%) 102d
kube-system nodelocaldns-fn4dl 100m (0%) 0 (0%) 70Mi (0%) 170Mi (0%) 45d
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 1875m (4%) 3300m (8%)
memory 6587958Ki (3%) 3899483392 (1%)
ephemeral-storage 0 (0%) 0 (0%)
tpu.bitmain.com/bm1682 0 0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 20m kubelet, gpu-node1 Starting kubelet.
Normal NodeAllocatableEnforced 20m kubelet, gpu-node1 Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 20m (x8 over 20m) kubelet, gpu-node1 Node gpu-node1 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 20m (x8 over 20m) kubelet, gpu-node1 Node gpu-node1 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 20m (x7 over 20m) kubelet, gpu-node1 Node gpu-node1 status is now: NodeHasSufficientPID
Normal Starting 19m kube-proxy, gpu-node1 Starting kube-proxy.
Normal Starting 12m kubelet, gpu-node1 Starting kubelet.
Normal NodeHasSufficientMemory 12m (x8 over 12m) kubelet, gpu-node1 Node gpu-node1 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 12m (x8 over 12m) kubelet, gpu-node1 Node gpu-node1 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 12m (x7 over 12m) kubelet, gpu-node1 Node gpu-node1 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 12m kubelet, gpu-node1 Updated Node Allocatable limit across pods
Normal Starting 12m kube-proxy, gpu-node1 Starting kube-proxy.
kubectl describe ingress name 查看Ingress详情
kubectl describe ingress ingress-resource-backend
Name: ingress-resource-backend
Namespace: default
Address:
Default backend: APIGroup: k8s.example.com, Kind: StorageBucket, Name: static-assets
Rules:
Host Path Backends
---- ---- --------
*
/icons APIGroup: k8s.example.com, Kind: StorageBucket, Name: icon-assets
Annotations: <none>
Events: <none>
kubectl describe pvc name 查看PVC详情
# kubectl describe pvc es-7-master-0
Name: es-7-master-0
Namespace: default
StorageClass: local-path
Status: Bound
Volume: pvc-44b8897d-506f-46ee-aed0-d259c0b7dded
Labels: app=es-7-master
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: rancher.io/local-path
volume.kubernetes.io/selected-node: centos-node1
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 100Gi
Access Modes: RWO
VolumeMode: Filesystem
Mounted By: es-7-master-0
Events: <none>
Mounted By 指明了挂载此 PVC 的 pod
kubectl describe pv name 查看PV详情
# kubectl describe pv pvc-44b8897d-506f-46ee-aed0-d259c0b7dded
Name: pvc-44b8897d-506f-46ee-aed0-d259c0b7dded
Labels: <none>
Annotations: pv.kubernetes.io/provisioned-by: rancher.io/local-path
Finalizers: [kubernetes.io/pv-protection]
StorageClass: local-path
Status: Bound
Claim: default/es-7-master-0
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 100Gi
Node Affinity:
Required Terms:
Term 0: kubernetes.io/hostname in [centos-node1]
Message:
Source:
Type: HostPath (bare host directory volume)
Path: /data/local-path-provisioner/pvc-44b8897d-506f-46ee-aed0-d259c0b7dded_default_es-7-master-0
HostPathType: DirectoryOrCreate
Events: <none>
kubectl describe prometheusrule 查看全部规则
如果不知道某个告警项来自哪个 prometheusrule，可以直接 kubectl describe prometheusrule 查看全部 prometheus 规则详情，从中 grep 想找的告警项，然后顺着往上就能找到对应的 prometheusrule 名字。
kubectl create 创建资源
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#create
从文件或标准输入创建资源，这里的资源可以是 Service, Pod, ConfigMap 等等。kubectl create -f FILENAME
例如kubectl create -f helm-rbac.yaml 根据配置文件 helm-rbac.yaml 创建资源。
kubectl apply 应用配置文件
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply
应用指定的配置文件，如果指定的资源不存在会自动创建，相当于 kubectl createkubectl apply (-f FILENAME | -k DIRECTORY)
--record 记录这条 kubectl 命令到资源注解中，默认是 false 不记录。
例1、根据配置文件创建资源kubectl apply -f helm-rbac.yaml
例2、利用 <<EOF 多行输入重定向，直接在命令行中输入文件内容，创建一个 consul 的 Service
kubectl apply -f -<<EOF
apiVersion: v1
kind: Service
metadata:
name: consul-service
namespace: default
spec:
ports:
- name: consul-ui
port: 8500
protocol: TCP
targetPort: 8500
selector:
app: consul
sessionAffinity: ClientIP
type: ClusterIP
EOF
last-applied-configuration 资源的初始json
kubectl apply -f 操作会在每个对象上设置 kubectl.kubernetes.io/last-applied-configuration: '{...}' 注解。注解值中包含了用来创建对象的配置文件的内容。kubectl.kubernetes.io/last-applied-configuration 是资源的原始 JSON 表示，在对象创建时由 kubectl apply 命令写入，后续 edit 修改对象时此部分不会变。
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"apps/v1","kind":"Deployment",
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
"spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx",
"ports":[{"containerPort":80}]}]}}}}
Declarative Management of Kubernetes Objects Using Configuration Files
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/declarative-config/
kubectl run 运行容器
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#run
在 pod 中运行指定的镜像，类似 docker run
kubectl exec 在容器中执行命令
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#exec
在容器中执行命令，类似 docker exec
进入容器 kubectl exec -it pod-name sh-c, --container='' 指定容器名，如果忽略此参数，自动选择 pod 中的第一个容器
1、查找consul的pod名
# kubectl get pods |grep consul
consul-alone-0 1/1 Running 0 22d
2、进入consul容器 kubectl exec -it consul-alone-0 sh
输入完pod名前缀后按tab键可以动补全完整的pod名，很方便
kubectl exec -c 在pod中的指定容器中执行命令
对于包含多个容器的 pod, 可以通过 -c 容器名 参数指定目标容器。
1、kubectl describe pod xxx 可以看到 pod 中的容器列表，在 Containers 属性中列出。
2、kubectl exec -it pod-name -c container-name sh 在pod的指定容器中执行命令
kubectl scale 扩缩容
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#scale
扩容或缩容 Deployment, ReplicaSet, Replication Controller 或 StatefulSet 中的 Pod 数量。
scale 也可以指定多个前提条件，如：当前副本数量 --current-replicas 或 版本号 --resource-version 进行伸缩比例设置前，系统会先验证前提条件是否成立。
1、将 deployment myapp-deployment 中的 pod 个数设置为 3 个kubectl scale --replicas=3 deployment/myapp-deployment
这也是一种 声明式配置 ，可能从小于 3 的 pod 数增加到 3 个，也可能是从大于 3 的个数减少到 3 个。
2、如果 mysql 当前副本数为 2 则将其扩展至 3kubectl scale --current-replicas=2 --replicas=3 deployment/mysql
3、同时将 deployment app1 和 app2 的副本数设置为 2kubectl scale --replicas=2 deploy/app1-deployment deploy/app2-deployment
4、将由 foo.yaml 配置文件中指定的资源对象和名称标识的 Pod 资源副本设为 3kubectl scale --replicas=3 -f foo.yaml
Kubernetes kubectl scale 命令详解
http://docs.kubernetes.org.cn/664.html
kubectl label 给资源打标签
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#label
更新资源的标签。
kubectl label node master node-role.kubernetes.io/edge= 给 master node 打上 edge label
kubectl label nodes gpu-nvidia-t4 node=nvidia-t4 给节点 gpu-nvidia-t4 打上 nvidia-t4 标签
kubectl label pods foo bar- 删除 foo 上的标签 bar
kubectl drain 从节点上驱逐pod
https://kubernetes.io/docs/reference/kubectl/generated/kubectl_drain/
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#drain
kubectl drain 命令用于将节点上的工作负载（例如Pods）逐渐迁移到其他节点，以便可以对该节点进行维护或升级。
这个命令通常用于在维护或升级节点之前，确保节点上的所有 Pods 都已经被迁移到集群的其他地方。
在一个节点上执行 kubectl drain 命令时，它会：
- 标记该节点为不可调度（unschedulable），这意味着 Kubernetes 调度器不会将新的 Pods 调度到这个节点上。
- 尝试驱逐该节点上所有的 Pods。驱逐过程会尊重 Pods 的优先级和 PDB（Pod Disruption Budgets）设置。
使用 kubectl drain 时，你可以指定一些选项来控制驱逐行为，例如：
--ignore-daemonsets：忽略 DaemonSet 管理的 Pods。DaemonSet 通常用于运行每个节点上都需要运行的 Pods，如日志收集器或网络插件。--delete-local-data：允许驱逐带有本地存储的 Pods。这可能会导致数据丢失，因此应谨慎使用。--force：强制驱逐 Pods，即使它们没有违反 PDB 或其他驱逐策略。--grace-period：设置 Pod 终止的宽限期。
kubectl cordon/uncordon 将节点标为不可调度
https://kubernetes.io/docs/reference/kubectl/generated/kubectl_cordon/
kubectl cordon NODE 将节点标记为不可调度（SchedulingDisabled）kubectl uncordon NODE 将节点重新标记为可调度
当一个节点被 cordon 后，Kubernetes 将不会在该节点上调度新的 Pod。这对于需要对节点进行维护、升级或者修复等操作非常有用，因为它可以确保在维护期间不会有新的工作负载被调度到该节点上。
执行 kubectl cordon 后，已存在的 Pod 仍然会在该节点上继续运行，但不会有新的 Pod 被调度到这个节点上。
例如kubectl cordon node1 将 node1 节点标为不可调度
kubectl delete 删除资源
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#delete
根据文件名、标签等删除资源
kubectl delete pod 删除pod
kubectl delete pod myapp-deployment-64f855cd65-7dhc4 删除指定pod，输入 myapp 后按 tab 键可自动补全 pod 名
kubectl delete pod foo –force
kubectl delete pod foo –force –grace-period=0 强制删除 pod
kubectl delete -f xx.yaml 根据描述文件删除资源
kubectl delete -f 101_initial_cluster.yaml 根据描述文件 101_initial_cluster.yaml 删除资源，这是一个 CRD 自定义资源，是通过 kubectl apply -f 101_initial_cluster.yaml 命令安装的。
kubectl taint 给node加污点标记
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#taint
更新一个或多个 node 上的污点。kubectl taint NODE NAME KEY_1=VAL_1:TAINT_EFFECT_1 ... KEY_N=VAL_N:TAINT_EFFECT_N
一个污点(Taints) 包括 key, value 和 effect(效果)，形式为 key=value:effect
注意：
1、effect 目前只能是 NoSchedule, PreferNoSchedule, NoExecute 三者之一，不能是别的值。
2、目前污点只能应用于 node
3、value 是可选的，没有value也可以。
例如kubectl taint nodes foo dedicated=special-user:NoSchedule 给节点 foo 增加一个污点，它的键名是 dedicated, 键值是 special-user, 效果是 NoSchedule。 这表示只有拥有和这个污点相匹配的容忍度的 Pod 才能够被分配到 foo 这个节点。
如果在节点 foo 上键是 dedicated, 效果是 NoSchedule 的污点已经存在，则其 value 被替换为 special-user
kubectl taint nodes foo dedicated:NoSchedule- 删除节点 foo 上键是 dedicated, 效果是 NoSchedule 的污点（如果有的话）。
kubectl taint nodes foo bar:NoSchedule 给节点 foo 加上一个键是 bar, 效果是 NoSchedule 的污点，此污点没有value。
kubectl logs 查看日志
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#logs
打印 pod 的日志
先 kubectl get pods 列出所有 pods
然后 kubectl logs --tail=20 pod-name 显示 pod-name 的最近 20 行日志。
kubectl logs -c 查看pod中指定容器的日志
-c, --container='' 指定容器名
例如kubectl logs -f example-vttablet-zone1-2469782763-bfadd780 -c mysqld
对于包含多个容器的 pod, kubectl logs 命令必须用 -c 参数指定要查看的具体容器名，否则报如下错误：
# kubectl logs -f example-vttablet-zone1-2469782763-bfadd780
Error from server (BadRequest): a container name must be specified for pod example-vttablet-zone1-2469782763-bfadd780, choose one of: [vttablet mysqld mysqld-exporter] or one of the init containers: [init-vt-root init-mysql-socket]
可以看到错误提示中还会给列出各个容器的名字。
kubectl logs -p 查看pod中前一个容器的日志
-p, --previous=false 增加 -p 参数后可以打印 pod 中前一个容器的日志（如果有的话），当 pod 中容器有重启时，用此方法很容易排查前一个容器重启的原因
kubectl cp 拷贝文件
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#cp
kubectl cp <file-spec-src> <file-spec-dest>
拷贝 mysql-alone pod 中的 /usr/lib/mysql/plugin/libcompare.so 到当前目录的 libcompare.so
kubectl cp mysql-alone:usr/lib/mysql/plugin/libcompare.so libcompare.so
Cannot open: Permission denied
拷贝文件到 pod 内，报错：
tar: a: Cannot open: Permission denied
解决：
先拷贝到容器内的 /tmp/ 目录，再 exec 进入容器移动到指定目录。
kubectl edit 编辑资源
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#edit
kubectl edit cm mc-configmap 修改 configmapkubectl edit deployment/mydeployment 编辑 deployment，可直接在这里修改副本数
kubectl proxy 创建代理
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#proxy
在宿主机 localhost 和 Kubernetes API Server 之间创建一个代理服务或应用级网关。
还可以在指定的 path 上做静态资源服务器。
kubectl proxy --port=8001 在端口 8001 上创建代理。
例2、创建代理，以后可通过 8888 端口访问 k8s api server kubectl proxy --address='0.0.0.0' --accept-hosts='^*$' --port=8888 &
# kubectl proxy --address='0.0.0.0' --accept-hosts='^*$' --port=8888 &
[1] 60505
]# Starting to serve on [::]:8888
kubectl port-forward 端口转发
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#port-forward
kubectl port-forward TYPE/NAME [options] [LOCAL_PORT:]REMOTE_PORT [...[LOCAL_PORT_N:]REMOTE_PORT_N]
将一个或多个本地端口转发到 pod, 这个命令需要 node 节点上安装了 socat 工具。
通过 资源类型/资源名 来选择 pod, 例如 deployment/mydeployment, 也可只指定资源名，默认资源类型是 pod.
如果有多个 pod 匹配筛选条件，会自动选择一个 pod, 如果选中的 pod 结束，转发 session 也会自动终止。
kubectl port-forward service/vtctld 15000 15999 & 监听本地 15000 和 15999 端口转发到指定 service 的相同端口。kubectl port-forward pod/mypod 8888:5000 监听本地 8888 端口转发到指定 pod 的 5000 端口。
kubectl version 查看版本
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#version
kubectl version 可以看客户端及服务端的版本，其中 GitVersion 即为 k8s 的版本号。
kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.7", GitCommit:"132a687512d7fb058d0f5890f07d4121b3f0a2e2", GitTreeState:"clean", BuildDate:"2021-05-12T12:32:49Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}
上一篇 Linux-YUM
页面信息
location:protocol: host: hostname: origin: pathname: href: document:referrer: navigator:platform: userAgent: