#docker #kubernetes
#docker #kubernetes
Вопрос:
С k8s deployment, service, and ingress этого момента я практикую GKE реализацию. Просмотр симулятора помогает мне быстро понять концепцию, но когда я делаю это на практике. Я застрял default backend - 404 . Вот мои файлы манифеста и bash описание.
deployment.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: tunnel-deployment
labels:
app: tunnel
spec:
replicas: 1
selector:
matchLabels:
app: tunnel
template:
metadata:
labels:
app: tunnel
spec:
containers:
- name: tunnel
image: gcr.io/k8s-v1-235608/tunnel:latest
imagePullPolicy: Always
ports:
- containerPort: 8080
env:
- name: MONGODB_HOST
value: moon-mongodb-replicaset-client
- name: RABBIT_HOST
value: rodent-rabbitmq-headless
- name: RABBIT_PASSWORD
valueFrom:
secretKeyRef:
name: rodent-rabbitmq
key: rabbitmq-password
- name: REDIS_HOST
value: ninja-redis-ha
readinessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
Смотрите deployments в bash
$ kubectl get deployments -o wide
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
tunnel-deployment 1 1 1 1 11m tunnel gcr.io/k8s-v1-235608/tunnel:latest app=tunnel
Смотрите po в bash
$ kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
bbox 1/1 Running 0 1d 10.4.0.6 gke-doom-default-pool-4b763b09-pxnz <none>
moon-mongodb-replicaset-0 1/1 Running 0 1d 10.4.2.9 gke-doom-default-pool-4b763b09-lcs6 <none>
moon-mongodb-replicaset-1 1/1 Running 0 1d 10.4.0.8 gke-doom-default-pool-4b763b09-pxnz <none>
moon-mongodb-replicaset-2 1/1 Running 0 1d 10.4.2.10 gke-doom-default-pool-4b763b09-lcs6 <none>
ninja-redis-ha-server-0 2/2 Running 0 1d 10.4.0.9 gke-doom-default-pool-4b763b09-pxnz <none>
ninja-redis-ha-server-1 2/2 Running 0 1d 10.4.1.11 gke-doom-default-pool-4b763b09-85ch <none>
ninja-redis-ha-server-2 2/2 Running 0 1d 10.4.2.11 gke-doom-default-pool-4b763b09-lcs6 <none>
rodent-rabbitmq-0 1/1 Running 0 1d 10.4.2.12 gke-doom-default-pool-4b763b09-lcs6 <none>
tunnel-deployment-fddf78dcc-lpq8l 1/1 Running 0 11m 10.4.1.37 gke-doom-default-pool-4b763b09-85ch <none>
service.yaml . Я использую NodePort со своим tunnel-service .
С моим selection.app
apiVersion: v1
kind: Service
metadata:
name: tunnel-service
labels:
app: tunnel
spec:
type: NodePort
ports:
- name: tunnel-port
port: 80
targetPort: 8080
selector:
app: tunnel
Тогда у меня есть один IP адрес для представления моего service .
$ kubectl describe svc tunnel-service
Name: tunnel-service
Namespace: default
Labels: app=tunnel
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"tunnel"},"name":"tunnel-service","namespace":"default"},...
Selector: app=tunnel
Type: NodePort
IP: 10.7.242.110
Port: tunnel-port 80/TCP
TargetPort: 8080/TCP
NodePort: tunnel-port 32713/TCP
Endpoints: 10.4.1.37:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
ingress.yaml . I route the / to my tunnel-service
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
spec:
rules:
- host: abc.hbot.io
http:
paths:
- path: /
backend:
serviceName: tunnel-service
servicePort: 80
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 16m loadbalancer-controller default/tunnel-ingress
Normal CREATE 15m loadbalancer-controller ip: 35.244.186.216
In my browser. I had routed my hostname to my correct DNS and got default backend - 404
I confirm my pod in deployment works. My endpoint does response on my request.
bash-4.3# curl http://localhost:8080/api
Hi I'm API Interface
Update on Attempt #1:
Add * after / and apply
$ kubectl apply -f ingress.yaml
ingress.extensions/tunnel-ingress configured
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/* tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/*"}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 42m loadbalancer-controller default/tunnel-ingress
Normal CREATE 40m loadbalancer-controller ip: 35.244.186.216
Not work.
Attempt #2
Add annotations
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: abc.hbot.io
http:
paths:
- path: /
backend:
serviceName: tunnel-service
servicePort: 80
Check the ingress
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 54m loadbalancer-controller default/tunnel-ingress
Normal CREATE 53m loadbalancer-controller ip: 35.244.186.216
Error: Server Error
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.
Attemp #3
Install ingress-nginx to cluster. I follow the cert-manager doc. Because I used it once with former cluster.
Follow the same name.
$ helm install stable/nginx-ingress --name quickstart
NAME: quickstart
LAST DEPLOYED: Wed Mar 27 16:09:15 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
quickstart-nginx-ingress-controller 1 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
quickstart-nginx-ingress-controller-5b8d54d964-rnvw6 0/1 ContainerCreating 0 0s
quickstart-nginx-ingress-default-backend-57bdfdcd46-vtf4h 0/1 Pending 0 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
quickstart-nginx-ingress-controller LoadBalancer 10.7.241.190 <pending> 80:32341/TCP,443:32762/TCP 0s
quickstart-nginx-ingress-default-backend ClusterIP 10.7.254.207 <none> 80/TCP 0s
==> v1/ServiceAccount
NAME SECRETS AGE
quickstart-nginx-ingress 1 0s
==> v1beta1/ClusterRole
NAME AGE
quickstart-nginx-ingress 0s
==> v1beta1/ClusterRoleBinding
NAME AGE
quickstart-nginx-ingress 0s
==> v1beta1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-nginx-ingress-controller 0/1 1 0 0s
quickstart-nginx-ingress-default-backend 0/1 1 0 0s
==> v1beta1/Role
NAME AGE
quickstart-nginx-ingress 0s
==> v1beta1/RoleBinding
NAME AGE
quickstart-nginx-ingress 0s
NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w quickstart-nginx-ingress-controller'
An example Ingress that makes use of the controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
describe ingress
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.244.186.216
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-31768--d2232907436d0807":"HEALTHY","k8s-be-32713--d2232907436d0807":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/target-proxy: k8s-tp-default-tunnel-ingress--d2232907436d0807
ingress.kubernetes.io/url-map: k8s-um-default-tunnel-ingress--d2232907436d0807
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 5m12s nginx-ingress-controller Ingress default/tunnel-ingress
Normal CREATE 7s (x7 over 92m) loadbalancer-controller ip: 35.244.186.216
Normal UPDATE 7s (x13 over 5m12s) nginx-ingress-controller Ingress default/tunnel-ingress
Test with curl on my laptop
$ curl -H 'Host: abc.hbot.io' 35.244.186.216/api
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>
Attempt 4th:
Solve insufficient cpu otherwise my new pods will not run
values.yaml
controller:
resources:
limits:
cpu: 100m
memory: 64Mi
requests:
cpu: 100m
memory: 64Mi
defaultBackend:
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
Установить ingress-nginx
helm install --values values.yaml stable/nginx-ingress --name quickstart
ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: abc.hbot.io
http:
paths:
- backend:
serviceName: tunnel-service
servicePort: 80
path: /
describe ingress
$ kubectl describe ingress
Name: tunnel-ingress
Namespace: default
Address: 35.240.162.185
Default backend: default-http-backend:80 (10.4.1.9:8080)
Rules:
Host Path Backends
---- ---- --------
abc.hbot.io
/ tunnel-service:80 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tunnel-ingress","namespace":"default"},"spec":{"rules":[{"host":"abc.hbot.io","http":{"paths":[{"backend":{"serviceName":"tunnel-service","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 14m nginx-ingress-controller Ingress default/tunnel-ingress
Normal CREATE 14m nginx-ingress-controller Ingress default/tunnel-ingress
Normal UPDATE 14m nginx-ingress-controller Ingress default/tunnel-ingress
Normal CREATE 9m27s nginx-ingress-controller Ingress default/tunnel-ingress
Я удаляю и снова применяю ingress. Изменен IP-адрес
$ kubectl get ingress -o wide
NAME HOSTS ADDRESS PORTS AGE
tunnel-ingress abc.hbot.io 35.240.162.185 80 15m
Моя ситуация еще хуже.
$ curl -H 'Host: abc.hbot.io' http://35.240.162.185/api
curl: (7) Failed to connect to 35.240.162.185 port 80: Connection refused
Последняя попытка # 5:
Я отказался от использования простой http настройки. И начните сначала с https подхода. Выполнив следующие действия, cert-manager а затем заменив kuard сервис на мой сервис. Тогда это сделано!
Вопрос:
Где я не прав?
Комментарии:
1. Привет, возможно, вам нужно использовать эту аннотацию
kubernetes.io/ingress.class: "nginx", если вы используете nginx-ingress controller или имя ingress-controller в GKE2. Спасибо за ваш комментарий. Я обновил свой вопрос, ссылаясь на ваш комментарий.
3. Вы на самом деле используете контроллер входа nginx? (вы его установили?) или используете механизм входа GKE по умолчанию? Если вы не устанавливали nginx-ingress controller, вам не следует указывать эту аннотацию
4. @PierreB. Нет. Я не устанавливал
nginx-ingress. Нужно ли мне это?5. Я думаю, что это вызывает ваш,
502когда вы устанавливаетеnginx-ingress.default-backendОшибка может быть вызвана тем, что ваш браузер плохо обрабатывает переопределение DNS (у меня была такая же проблема), более надежным было бы выполнить тестcurl -H 'Host: abc.hbot.io' 35.244.186.216— это позволит убедиться, что правила входа обрабатывают определение вашего хоста
Ответ №1:
Я никогда не использовал GKE, но в AWS и ALB путь должен включать подстановочный знак. Похоже, что это то же самое на GKE. https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#multiple_backend_services
https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer#step_6_optional_serving_multiple_applications_on_a_load_balancer
Попробуйте настроить свой вход следующим образом:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tunnel-ingress
spec:
rules:
- host: abc.hbot.io
http:
paths:
- path: /*
backend:
serviceName: tunnel-service
servicePort: 80
Комментарии:
1. Спасибо за вашу попытку. Я обновил свою попытку, обратитесь к вашему ответу.