| 知乎专栏 ｜ 多维度架构 |
目录
执行下面命令检查服务器是否开启虚拟化技术
egrep --color 'vmx|svm' /proc/cpuinfo
如果没有任何输出，请重启服务器进入 BIOS 启用 VT-X 或 AMD-v
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \ && install minikube-linux-amd64 /usr/local/bin/minikube
尝试运行 minikube 如果输出帮助信息表示安装成功
[root@localhost ~]# minikube version minikube version: v1.13.0 commit: 0c5e9de4ca6f9c55147ae7f90af97eff5befef5f-dirty
echo "1" > /proc/sys/net/bridge/bridge-nf-call-iptables
dnf 安装 kubectl
cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF
[root@localhost ~]# dnf install kubectl
二进制安装 kubectl
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" \ && install kubectl /usr/local/bin/kubectl
如果你不想安装虚拟机
adduser docker su - docker sudo usermod -aG docker $USER && newgrp docker
[docker@localhost ~]$ minikube start --driver=docker
* minikube v1.13.0 on Centos 8.2.2004
* Using the docker driver based on user configuration
X Requested memory allocation (1694MB) is less than the recommended minimum 2000MB. Deployments may fail.
X The requested memory allocation of 1694MiB does not leave room for system overhead (total system memory: 1694MiB). You may face stability issues.
* Suggestion: Start minikube with less memory allocated: 'minikube start --memory=1694mb'
* Starting control plane node minikube in cluster minikube
* Pulling base image ...
* Downloading Kubernetes v1.19.0 preload ...
> preloaded-images-k8s-v6-v1.19.0-docker-overlay2-amd64.tar.lz4: 486.28 MiB
检查硬件是否支持虚拟化
iMac:Linux neo$ sysctl -a | grep -E --color 'machdep.cpu.features|VMX' machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC POPCNT AES PCID XSAVE OSXSAVE TSCTMR AVX1.0
$ brew install hyperkit $ brew install minikube $ brew install kubectl $ brew install kubernetes-helm
neo@MacBook-Pro-Neo ~ % minikube start 😄 minikube v1.13.1 on Darwin 11.0 🆕 Kubernetes 1.19.2 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.19.2 ✨ Using the hyperkit driver based on existing profile 👍 Starting control plane node minikube in cluster minikube 🔄 Restarting existing hyperkit VM for "minikube" ... ❗ This VM is having trouble accessing https://k8s.gcr.io 💡 To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/ 🐳 Preparing Kubernetes v1.19.0 on Docker 19.03.12 ... 🔎 Verifying Kubernetes components... 🌟 Enabled addons: dashboard, default-storageclass, storage-provisioner 🏄 Done! kubectl is now configured to use "minikube" by default
有些老系统可能不支持 hyperkit，需要virtualbox。
$ brew cask install virtualbox $ minikube start —vm-driver=virtualbox $ minikube dashboard
检查 minikube 启动状态
Neo-iMac:~ neo$ docker container inspect minikube --format={{.State.Status}}
running
启动
minikube start
运行一个 echoserver 镜像
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080 kubectl expose deployment hello-minikube --type=NodePort minikube service hello-minikube
查询 echoserver 访问地址
minikube service hello-minikube --url
在浏览器中访问查询到的网址
停止并删除镜像
minikube stop minikube delete
例 2.1. minikube 操作演示
快速开始使用 minikube 运行一个镜像
[root@localhost ~]# kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080 kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead. deployment.apps/hello-minikube created [root@localhost ~]# kubectl expose deployment hello-minikube --type=NodePort service/hello-minikube exposed [root@localhost ~]# minikube service hello-minikube Opening kubernetes service default/hello-minikube in default browser... [root@localhost ~]# kubectl get pod NAME READY STATUS RESTARTS AGE hello-minikube-5c856cbf98-6vfvp 1/1 Running 0 6m59s [root@localhost ~]# minikube service hello-minikube --url http://172.16.0.121:30436 [root@localhost ~]# curl http://172.16.0.121:30436 CLIENT VALUES: client_address=172.17.0.1 command=GET real path=/ query=nil request_version=1.1 request_uri=http://172.16.0.121:8080/ SERVER VALUES: server_version=nginx: 1.10.0 - lua: 10001 HEADERS RECEIVED: accept=*/* host=172.16.0.121:30436 user-agent=curl/7.29.0 BODY: -no body in request-
[root@localhost ~]# minikube
Minikube is a CLI tool that provisions and manages single-node Kubernetes clusters optimized for development workflows.
Usage:
minikube [command]
Available Commands:
addons Modify minikube's kubernetes addons
cache Add or delete an image from the local cache.
completion Outputs minikube shell completion for the given shell (bash or zsh)
config Modify minikube config
dashboard Access the kubernetes dashboard running within the minikube cluster
delete Deletes a local kubernetes cluster
docker-env Sets up docker env variables; similar to '$(docker-machine env)'
help Help about any command
ip Retrieves the IP address of the running cluster
logs Gets the logs of the running instance, used for debugging minikube, not user code
mount Mounts the specified directory into minikube
profile Profile sets the current minikube profile
service Gets the kubernetes URL(s) for the specified service in your local cluster
ssh Log into or run a command on a machine with SSH; similar to 'docker-machine ssh'
ssh-key Retrieve the ssh identity key path of the specified cluster
start Starts a local kubernetes cluster
status Gets the status of a local kubernetes cluster
stop Stops a running local kubernetes cluster
tunnel tunnel makes services of type LoadBalancer accessible on localhost
update-check Print current and latest version number
update-context Verify the IP address of the running cluster in kubeconfig.
version Print the version of minikube
Flags:
--alsologtostderr log to standard error as well as files
-b, --bootstrapper string The name of the cluster bootstrapper that will set up the kubernetes cluster. (default "kubeadm")
-h, --help help for minikube
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory
--logtostderr log to standard error instead of files
-p, --profile string The name of the minikube VM being used.
This can be modified to allow for multiple minikube instances to be run independently (default "minikube")
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
-v, --v Level log level for V logs
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
Use "minikube [command] --help" for more information about a command.
[docker@localhost ~]$ minikube ip 192.168.58.2
kubectl get nodes -o jsonpath='{.items[*].status.addresses[].address}'
minikube start --registry-mirror=https://registry.docker-cn.com minikube start --image-mirror-country=cn --registry-mirror="https://docker.mirrors.ustc.edu.cn" --insecure-registry="127.0.0.1:5000" minikube start --image-mirror-country=cn --registry-mirror="https://docker.mirrors.ustc.edu.cn" --insecure-registry="192.168.0.0/24"
minikube start --image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers
# 从阿里云下载 virtualbox 镜像
minikube start --vm-driver='virtualbox' --image-mirror-country cn \
--iso-url=https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/iso/minikube-v1.9.0.iso \
--registry-mirror=https://docker.mirrors.ustc.edu.cn
minikube start --vm-driver=virtualbox \
--image-mirror-country cn \
--registry-mirror=https://docker.mirrors.ustc.edu.cn \
--image-repository=registry.aliyuncs.com/google_containers \
--insecure-registry=192.168.0.10:5000 //访问宿主机的私有docker仓库
minikube start --extra-config="kubelet.allowed-unsafe-sysctls=kernel.msg*,net.core.somaxconn".
minikube start --container-runtime=cri-o --vm-driver=none
启动演示
iMac:~ neo$ minikube start --container-runtime=cri-o 😄 Darwin 10.13.6 上的 minikube v1.15.0 🆕 Kubernetes 1.19.4 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.19.4 ✨ 根据现有的配置文件使用 hyperkit 驱动程序 👍 Starting control plane node minikube in cluster minikube 🔄 Restarting existing hyperkit VM for "minikube" ... 🎁 正在 CRI-O 1.17.3 中准备 Kubernetes v1.19.2… 🔗 Configuring bridge CNI (Container Networking Interface) ... 🔎 Verifying Kubernetes components... 🌟 Enabled addons: storage-provisioner, dashboard, default-storageclass 🏄 Done! kubectl is now configured to use "minikube" cluster and "" namespace by default
[root@localhost ~]# minikube stop Stopping local Kubernetes cluster... Machine stopped.
neo@MacBook-Pro-Neo ~ % minikube docker-env export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://192.168.64.3:2376" export DOCKER_CERT_PATH="/Users/neo/.minikube/certs" export MINIKUBE_ACTIVE_DOCKERD="minikube" # To point your shell to minikube's docker-daemon, run: # eval $(minikube -p minikube docker-env)
设置环境变量
# eval $(minikube docker-env) # eval $(minikube -p minikube docker-env)
neo@MacBook-Pro-Neo ~ % minikube ssh
_ _
_ _ ( ) ( )
___ ___ (_) ___ (_)| |/') _ _ | |_ __
/' _ ` _ `\| |/' _ `\| || , < ( ) ( )| '_`\ /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )( ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)
$
# cache a image into $HOME/.minikube/cache/images $ minikube cache add ubuntu:16.04 $ minikube cache add redis:3 # list cached images $ minikube cache list redis:3 ubuntu:16.04 # delete cached images $ minikube cache delete ubuntu:16.04 $ minikube cache delete $(minikube cache list)
Dashboard是基于Web的Kubernetes管理界面。使用下面的命令启动：
minikube dashboard
查询控制面板访问地址
$ minikube dashboard --url http://192.168.3.14:30000
列出所有服务
Neo-iMac:~ neo$ minikube service list |----------------------|------------------------------------|--------------|-----| | NAMESPACE | NAME | TARGET PORT | URL | |----------------------|------------------------------------|--------------|-----| | default | kubernetes | No node port | | default | nginx | 80 | | | ingress-nginx | ingress-nginx-controller | http/80 | | | | | https/443 | | | ingress-nginx | ingress-nginx-controller-admission | No node port | | kube-system | kube-dns | No node port | | kubernetes-dashboard | dashboard-metrics-scraper | No node port | | kubernetes-dashboard | kubernetes-dashboard | No node port | |----------------------|------------------------------------|--------------|-----|
查看指定服务
Neo-iMac:~ neo$ minikube service nginx |-----------|-------|-------------|---------------------------| | NAMESPACE | NAME | TARGET PORT | URL | |-----------|-------|-------------|---------------------------| | default | nginx | 80 | http://192.168.49.2:30330 | |-----------|-------|-------------|---------------------------| 🏃 Starting tunnel for service nginx. |-----------|-------|-------------|------------------------| | NAMESPACE | NAME | TARGET PORT | URL | |-----------|-------|-------------|------------------------| | default | nginx | | http://127.0.0.1:55815 | |-----------|-------|-------------|------------------------| 🎉 Opening service default/nginx in default browser... ❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.
查看服务的网址
[root@localhost ~]# minikube service hello-minikube --url http://172.16.0.121:30436
minikube docker-env
Neo-iMac:~ neo$ minikube docker-env export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://127.0.0.1:54734" export DOCKER_CERT_PATH="/Users/neo/.minikube/certs" export MINIKUBE_ACTIVE_DOCKERD="minikube" # To point your shell to minikube's docker-daemon, run: # eval $(minikube -p minikube docker-env)
minikube profile demo minikube start -p demo --memory=8192 --cpus=6 --disk-size=50g
iMac:registry neo$ minikube addons list |-----------------------------|----------|--------------| | ADDON NAME | PROFILE | STATUS | |-----------------------------|----------|--------------| | ambassador | minikube | disabled | | dashboard | minikube | enabled ✅ | | default-storageclass | minikube | enabled ✅ | | efk | minikube | disabled | | freshpod | minikube | disabled | | gcp-auth | minikube | disabled | | gvisor | minikube | disabled | | helm-tiller | minikube | disabled | | ingress | minikube | disabled | | ingress-dns | minikube | disabled | | istio | minikube | disabled | | istio-provisioner | minikube | disabled | | kubevirt | minikube | disabled | | logviewer | minikube | disabled | | metallb | minikube | disabled | | metrics-server | minikube | disabled | | nvidia-driver-installer | minikube | disabled | | nvidia-gpu-device-plugin | minikube | disabled | | olm | minikube | disabled | | pod-security-policy | minikube | disabled | | registry | minikube | disabled | | registry-aliases | minikube | disabled | | registry-creds | minikube | disabled | | storage-provisioner | minikube | enabled ✅ | | storage-provisioner-gluster | minikube | disabled | |-----------------------------|----------|--------------|
minikube addons enable heapster minikube addons enable ingress
启用 WebUI
[root@localhost ~]# minikube addons enable dashboard dashboard was successfully enabled [root@localhost ~]# minikube addons list | grep dashboard - dashboard: enabled
[root@localhost ~]# minikube addons list - addon-manager: enabled - dashboard: enabled - default-storageclass: enabled - efk: disabled - freshpod: disabled - gvisor: disabled - heapster: disabled - ingress: disabled - kube-dns: disabled - metrics-server: disabled - nvidia-driver-installer: disabled - nvidia-gpu-device-plugin: disabled - registry: disabled - registry-creds: disabled - storage-provisioner: enabled - storage-provisioner-gluster: disabled
Neo-iMac:~ neo$ minikube addons enable dashboard
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.7
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.3.1
💡 Some dashboard features require the metrics-server addon. To enable all features please run:
minikube addons enable metrics-server
🌟 The 'dashboard' addon is enabled
Neo-iMac:~ neo$ minikube dashboard 🤔 Verifying dashboard health ... 🚀 Launching proxy ... 🤔 Verifying proxy health ... 🎉 Opening http://127.0.0.1:62433/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...
# enable the registry addon
$ minikube addons enable registry
$ minikube start
# use the minikube docker daemon from the host
$ eval $(minikube docker-env)
# get the ip of the registry endpoint
$ kubectl -n kube-system get svc registry -o jsonpath="{.spec.clusterIP}"
10.0.0.240
{
"insecure-registries" : ["10.0.0.240"]
}
$ minikube ssh $ docker pull busybox $ docker tag busybox 10.0.0.240/busybox # or # build and push to insecure registry $ docker build -t 10.0.0.240/busybox . $ docker push 10.0.0.240/busybox
Neo-iMac:~ neo$ minikube addons enable ingress
💡 After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.0.4
🔎 Verifying ingress addon...
🌟 The 'ingress' addon is enabled
运行一个简单的demo
运行 nginx 服务
kubectl run nginx --image=nginx --port=80
暴露服务
kubectl expose deployment nginx --port=80 --target-port=80
创建ingress
yaml 定义 ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx
spec:
rules:
- host: www.netkiller.cn
http:
paths:
- path: /
backend:
serviceName: nginx
servicePort: 80
运行
kubectl apply -f ingress.yaml
配置本机host获取minikube ip
[docker@localhost ~]$ minikube ip
192.168.58.2
配置 /etc/hosts 文件
192.168.58.2 www.netkiller.cn
访问 http://www.netkiller.cn
--vm-driver=none 不支持 ssh
[root@localhost ~]# minikube ssh 'none' driver does not support 'minikube ssh' command
neo@MacBook-Pro-Neo ~ % minikube image ls registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v5 registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.4 registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.7 registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.7 registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.7 registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.7 registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0 registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.1.0 registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0 docker.io/netkiller/flask:latest
neo@MacBook-Pro-Neo ~ % minikube kubectl -- get pods -A
> kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubectl: 44.08 MiB / 44.08 MiB [--------------] 100.00% 5.30 MiB p/s 8.5s
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-admission-create-vzk2b 0/1 ImagePullBackOff 0 118d
ingress-nginx ingress-nginx-admission-patch-65b85 0/1 ImagePullBackOff 0 118d
ingress-nginx ingress-nginx-controller-7f79776f95-ncqkn 0/1 ContainerCreating 0 118d
kube-system coredns-54d67798b7-cnjgw 1/1 Running 2 121d
kube-system etcd-minikube 1/1 Running 2 121d
kube-system kube-apiserver-minikube 1/1 Running 2 121d
kube-system kube-controller-manager-minikube 1/1 Running 2 121d
kube-system kube-proxy-tr8fd 1/1 Running 2 121d
kube-system kube-scheduler-minikube 1/1 Running 2 121d
kube-system storage-provisioner 1/1 Running 2 121d
检查一下 BIOS 是否开启 VT-X/AMD-v
如果在虚拟机安装 Minikube 也会遇到这个问题。 可以使用 --vm-driver=none 参数启动。
neo@ubuntu:~$ sudo minikube start --vm-driver=none
解决方法
echo "1" > /proc/sys/net/bridge/bridge-nf-call-iptables
然后在 minikube start
[ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.1: output: 3.1: Pulling from pause Get https://k8s.gcr.io/v2/pause/manifests/sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610: net/http: TLS handshake timeout
更换镜像再重试
[root@localhost ~]# minikube start --vm-driver=none --registry-mirror=https://registry.docker-cn.com
启动提示如下错误，一般出现这种错误是因为 minikube stop, minikube delete 后再重启 minikube start
error execution phase kubeconfig/admin: a kubeconfig file "/etc/kubernetes/admin.conf" exists already but has got the wrong CA cert error execution phase kubeconfig/kubelet: a kubeconfig file "/etc/kubernetes/kubelet.conf" exists already but has got the wrong CA cert error execution phase kubeconfig/controller-manager: a kubeconfig file "/etc/kubernetes/controller-manager.conf" exists already but has got the wrong CA cert error execution phase kubeconfig/scheduler: a kubeconfig file "/etc/kubernetes/scheduler.conf" exists already but has got the wrong CA cert
解决方法
[root@localhost ~]# mv /etc/kubernetes/admin.conf /etc/kubernetes/admin.conf.backup [root@localhost ~]# mv /etc/kubernetes/kubelet.conf /etc/kubernetes/kubelet.conf.backup [root@localhost ~]# mv /etc/kubernetes/controller-manager.conf /etc/kubernetes/controller-manager.conf.backup [root@localhost ~]# mv /etc/kubernetes/scheduler.conf /etc/kubernetes/scheduler.conf.backup
现在启动 minikube start 不会再出错
[root@localhost ~]# minikube start --vm-driver=none Starting local Kubernetes v1.13.2 cluster... Starting VM... Getting VM IP address... Moving files into cluster... Setting up certs... Connecting to cluster... Setting up kubeconfig... Stopping extra container runtimes... Starting cluster components... Verifying kubelet health ... Verifying apiserver health ... Kubectl is now configured to use the cluster. =================== WARNING: IT IS RECOMMENDED NOT TO RUN THE NONE DRIVER ON PERSONAL WORKSTATIONS The 'none' driver will run an insecure kubernetes apiserver as root that may leave the host vulnerable to CSRF attacks When using the none driver, the kubectl config and credentials generated will be root owned and will appear in the root home directory. You will need to move the files to the appropriate location and then set the correct permissions. An example of this is below: sudo mv /root/.kube $HOME/.kube # this will write over any previous configuration sudo chown -R $USER $HOME/.kube sudo chgrp -R $USER $HOME/.kube sudo mv /root/.minikube $HOME/.minikube # this will write over any previous configuration sudo chown -R $USER $HOME/.minikube sudo chgrp -R $USER $HOME/.minikube This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true Loading cached images from config file. Everything looks great. Please enjoy minikube!
问题原因，使用私有 registry 由于没有 HTTPS 导致 kubectl 使用 https 去访问私有 registry.
Failed to pull image "192.168.3.85:5000/netkiller/config:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://192.168.3.85:5000/v2/: http: server gave HTTP response to HTTPS client
minikube 并不会使用 docker 配置文件中的 insecure-registry 配置项
解决办法
minikube start --insecure-registry=127.0.0.1:5000
或指定网段
minikube start --insecure-registry "10.0.0.0/24"
iMac:kubernetes neo$ kubectl create -f redis/redis.yml configmap/redis-config created deployment.apps/redis created The Service "redis" is invalid: spec.ports[0].nodePort: Invalid value: 6379: provided port is not in the valid range. The range of valid ports is 30000-32767
编辑kube-apiserver.yaml文件
$ minikube ssh $ sudo vi /etc/kubernetes/manifests/kube-apiserver.yaml
增加kube-apiserver的启动配置项
--service-node-port-range=1024-65535
$ sudo cat /etc/kubernetes/manifests/kube-apiserver.yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.64.5:8443
creationTimestamp: null
labels:
component: kube-apiserver
tier: control-plane
name: kube-apiserver
namespace: kube-system
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=192.168.64.5
- --allow-privileged=true
- --authorization-mode=Node,RBAC
- --client-ca-file=/var/lib/minikube/certs/ca.crt
- --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota
- --enable-bootstrap-token-auth=true
- --etcd-cafile=/var/lib/minikube/certs/etcd/ca.crt
- --etcd-certfile=/var/lib/minikube/certs/apiserver-etcd-client.crt
- --etcd-keyfile=/var/lib/minikube/certs/apiserver-etcd-client.key
- --etcd-servers=https://127.0.0.1:2379
- --insecure-port=0
- --kubelet-client-certificate=/var/lib/minikube/certs/apiserver-kubelet-client.crt
- --kubelet-client-key=/var/lib/minikube/certs/apiserver-kubelet-client.key
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --proxy-client-cert-file=/var/lib/minikube/certs/front-proxy-client.crt
- --proxy-client-key-file=/var/lib/minikube/certs/front-proxy-client.key
- --requestheader-allowed-names=front-proxy-client
- --requestheader-client-ca-file=/var/lib/minikube/certs/front-proxy-ca.crt
- --requestheader-extra-headers-prefix=X-Remote-Extra-
- --requestheader-group-headers=X-Remote-Group
- --requestheader-username-headers=X-Remote-User
- --secure-port=8443
- --service-account-key-file=/var/lib/minikube/certs/sa.pub
- --service-cluster-ip-range=10.10.0.0/24
- --service-node-port-range=1024-65535
- --tls-cert-file=/var/lib/minikube/certs/apiserver.crt
- --tls-private-key-file=/var/lib/minikube/certs/apiserver.key
image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.19.2
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 8
httpGet:
host: 192.168.64.5
path: /livez
port: 8443
scheme: HTTPS
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 15
name: kube-apiserver
readinessProbe:
failureThreshold: 3
httpGet:
host: 192.168.64.5
path: /readyz
port: 8443
scheme: HTTPS
periodSeconds: 1
timeoutSeconds: 15
resources:
requests:
cpu: 250m
startupProbe:
failureThreshold: 24
httpGet:
host: 192.168.64.5
path: /livez
port: 8443
scheme: HTTPS
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 15
volumeMounts:
- mountPath: /etc/ssl/certs
name: ca-certs
readOnly: true
- mountPath: /var/lib/minikube/certs
name: k8s-certs
readOnly: true
- mountPath: /usr/share/ca-certificates
name: usr-share-ca-certificates
readOnly: true
hostNetwork: true
priorityClassName: system-node-critical
volumes:
- hostPath:
path: /etc/ssl/certs
type: DirectoryOrCreate
name: ca-certs
- hostPath:
path: /var/lib/minikube/certs
type: DirectoryOrCreate
name: k8s-certs
- hostPath:
path: /usr/share/ca-certificates
type: DirectoryOrCreate
name: usr-share-ca-certificates
status: {}
sudo systemctl restart kubelet
iMac:~ neo$ minikube addons enable registry 🔎 Verifying registry addon... ❌ Exiting due to MK_ENABLE: run callbacks: running callbacks: [verifying registry addon pods : timed out waiting for the condition: timed out waiting for the condition] 😿 If the above advice does not help, please let us know: 👉 https://github.com/kubernetes/minikube/issues/new/choose
minikube dashboard --alsologtostderr -v=1
[docker@localhost ~]$ kubectl get pods --all-namespaces | grep dashboard kubernetes-dashboard dashboard-metrics-scraper-6f7955cd98-xjzkq 0/1 ImagePullBackOff 0 11d kubernetes-dashboard kubernetes-dashboard-7bf64fd654-ckr7v 0/1 ImagePullBackOff 0 11d
[docker@localhost ~]$ kubectl logs --namespace=kubernetes-dashboard kubernetes-dashboard-7bf64fd654-ckr7v Error from server (BadRequest): container "kubernetes-dashboard" in pod "kubernetes-dashboard-7bf64fd654-ckr7v" is waiting to start: trying and failing to pull image
minikube start --image-mirror-country=cn --insecure-registry="registry.netkiller.cn" --cache-images=true
Neo-iMac:~ neo$ kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE ingress-nginx-admission-create--1-qpckk 0/1 Completed 0 18h ingress-nginx-admission-patch--1-5x94l 0/1 Completed 0 18h ingress-nginx-controller-78d858bdc7-nrszs 1/1 Running 1 18h Neo-iMac:~ neo$ kubectl create deployment web --image=nginx:latest deployment.apps/web created Neo-iMac:~ neo$ kubectl expose deployment web --type=NodePort --port=80 service/web exposed Neo-iMac:~ neo$ kubectl get service web NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE web NodePort 10.109.55.204 <none> 8080:30857/TCP 19s Neo-iMac:~ neo$ minikube service web --url 🏃 Starting tunnel for service web. |-----------|------|-------------|------------------------| | NAMESPACE | NAME | TARGET PORT | URL | |-----------|------|-------------|------------------------| | default | web | | http://127.0.0.1:62956 | |-----------|------|-------------|------------------------| http://127.0.0.1:62956 ❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.
ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: www.netkiller.cn
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 80
http://www.netkiller.cn 无法访问，解决方案 minikube tunnel
Neo-iMac:~ neo$ minikube tunnel ❗ The service/ingress example-ingress requires privileged ports to be exposed: [80 443] 🔑 sudo permission will be asked for it. 🏃 Starting tunnel for service example-ingress. Password:
如果注意观察，在启动的时候系统已经提示：After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"
Neo-iMac:nginx neo$ minikube start --image-mirror-country=cn --insecure-registry="registry.netkiller.cn" --cache-images=true
😄 minikube v1.24.0 on Darwin 12.0.1
✨ Using the docker driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🔄 Restarting existing docker container for "minikube" ...
🐳 Preparing Kubernetes v1.22.3 on Docker 20.10.8 ...
🔎 Verifying Kubernetes components...
💡 After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.3.1
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v5
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.0.4
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.7
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
🔎 Verifying ingress addon...
🌟 Enabled addons: dashboard, storage-provisioner, default-storageclass, ingress
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default