kubernetes ingress(三)： traefik: 多域名及证书配置

目标：

部署三个服务traefik-ui，grafana，prometheus，并通过traefik 反向代理。

service namespaces domain name https traefik-ui traefik traefik.qyd.com Y grafana kube-system grafana.dfb.com N prometheus kube-system prometheus.qyd.com Y

步骤：

1、部署traefik

相关资源yml

• https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/rbac.yml
• https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/deployment.yml
  
• https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/configmap.yml
• https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/prometheus-ingress.yml
• https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/grafana-ingress.yml
• https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/traefik-web-ui.yml

创建traefik 这个命名空间，使用configmap 挂载配置。

kubectl create cm -n traefik  traefik-config --from-file=traefik.toml

apiVersion: v1
items:
- apiVersion: v1
  data:
    traefik.toml: |
      graceTimeOut = 10
      traefikLogsFile = "/log/traefik.log"
      accessLogsFile = "/log/access.log"
      logLevel = "INFO"
      MaxIdleConnsPerHost = 60
      InsecureSkipVerify = true
      defaultEntryPoints = ["https","http"]
      [entryPoints]
        [entryPoints.http]
        address = ":80"
              [entryPoints.http.redirect]
              regex = "^http://(.*).qyd.com/(.*)"
              replacement = "https://$1.qyd.com/$2"

      [entryPoints.https]
      address = ":443"
        [entryPoints.https.tls]
          [[entryPoints.https.tls.certificates]]
          certFile = "/ssl/qyd/tls.crt"
          keyFile = "/ssl/qyd/tls.key"
          [[entryPoints.https.tls.certificates]]
          certFile = "/ssl/dfb/tls.crt"
          keyFile = "/ssl/dfb/tls.key"
      [metrics]
        [metrics.prometheus]
          entryPoint = "traefik"


  kind: ConfigMap
  metadata:
    name: traefik-config
    namespace: traefik
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

获取 qyd.com 和dfb.com 两个域名的证书，并创建secret。

kubectl create secret generic dfb-tls-cert --from-file=dfb/tls.crt --from-file=dfb/tls.key -n traefik
kubectl create secret generic qyd-tls-cert --from-file=qyd/tls.crt --from-file=qyd/tls.key -n traefik

部署traefik-ingreess-controller

kubectl app -f rbac.yml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - extensions
    resources:
    - ingresses/status
    verbs:
    - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: traefik
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: traefik

kubectl apply -f deployment.yml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    k8s-app: traefik-ingress-lb
  name: traefik-ingress-controller
  namespace: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      containers:
      - args:
        - --configFile=/etc/traefik/traefik.yml
        - --api
        - --kubernetes
        image: itanony.com/repository/docker-hosted/test/treafik:v1.7.10
        imagePullPolicy: IfNotPresent
        name: traefik-ingress-lb
        ports:
        - containerPort: 80
          hostPort: 80
          name: http
          protocol: TCP
        - containerPort: 8080
          hostPort: 8080
          name: admin
          protocol: TCP
        - containerPort: 443
          hostPort: 443
          name: https
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/traefik/
          name: config
        - mountPath: /ssl/qyd/
          name: qyd-cert
        - mountPath: /ssl/dfb/
          name: dfb-cert
        - mountPath: /log/
          name: logs
      dnsPolicy: ClusterFirst
      hostNetwork: true
      nodeSelector:
        cpu: high
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: traefik-ingress-controller
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      volumes:
      - name: qyd-cert
        secret:
          defaultMode: 420
          secretName: qyd-tls-cert
      - name: dfb-cert
        secret:
          defaultMode: 420
          secretName: dfb-tls-cert
      - configMap:
          defaultMode: 420
          name: traefik-config
        name: config
      - hostPath:
          path: /var/log/traefik
          type: ""
        name: logs

注意deployment.yml 中修改images地址。另外因为是测试，故采用nodeselector 只部署到一台固定的node节点，采用宿主机网络模式。ingress controller 的高可用留在以后研究。
查看pod 状态

kubectl get pods -n traefik

traefik 启动后会监控一个8080 的端口提供一个管理的web-ui，可以查看frontend 和backend 的对应关系，及一些基本的监控数据
我们创建一个ClusterIP 的service，并创建ingress，通过traefik 使用traefik.qyd.com 域名来反向代理

kubectl apply -f traefik-web-ui.yml

apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: traefik
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: traefik
spec:
  rules:
  - host: traefik.qyd.com
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

在本机hosts中添加 traefik.qyd.com 的hosts 记录解析到traefik 部署的node节点。
通过浏览器访问。页面正常显示，并且使用http 访问时会自动跳转到https。

部署prometheus 和grafana 代理

这里只讨论通过traefik-ingres 代理prometheus 和grafan。部署过程请Google。

创建prometheus 和 grafana 的ingress 。 通过traefik 分别使用 prometheus.yd.com 和grafana.dfb.com 反向代理。

注意yml 中namespace，serviceName，servicePort 与自己集群中服务的名称一致。

kubectl apply -f grafana-ingress.yml
kubectl apply -f prometheus-ingress.yml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: grafana
  namespace: kube-system
spec:
  rules:
  - host: grafana.dfb.com
    http:
      paths:
      - backend:
          serviceName: monitoring-grafana
          servicePort: 80
        path: /

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: prometheus
  namespace: kube-system
spec:
  rules:
  - host: prometheus.qyd.com
    http:
      paths:
      - backend:
          serviceName: prometheus
          servicePort: prometheus
        path: /

同样在本机hosts 中添加两个域名的解析记录。通过浏览器访问正常，prometheus.qyd.com访问http 会rewrite到https，grafana.dfb.com不会做rewrite。至此部署部分结束

配置解析

多域名 配置https，我们不需要对每一个域名指定证书， 只需要在entrypoints 中指定证书路径。traefik 会自动根据请求中的主机头和证书中的CN进行匹配。
生产中可能遇到同一个反向代理下。 有的域名需要启用https 的强制rewrite。 有些则不能做强制rewrite。traefik 提供entryPoints.http.redirect 通过正则来对需要rewrite 的域名进行正则匹配。 这里感觉有点不灵活。 也可能还有更好的方式。