本地K8S集群基于MetalLB工具实现LoadBalancer

概述

MetalLB 是裸机的负载均衡器实现库伯内斯集群，使用标准路由协议

官方文档：https://metallb.universe.tf/

安装

准备前提

在 IPVS 模式下使用 kube-proxy，则从 Kubernetes v1.14.2 开始，必须启用严格 ARP 模式

# 二进制部署的K8S集群，kube-proxy的配置文件，是在本地，而不是在configmap
$ vim /var/lib/kube-proxy/kube-proxy-config.yaml
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
  kubeconfig: "/etc/kubernetes/kube-proxy.kubeconfig"
# 根据clusterCIDR 判断集群内部和外部流量，配置clusterCIDR选项后，kube-proxy 会对访问 Service IP 的请求做 SNAT
clusterCIDR: "172.20.0.0/16"
conntrack:
  maxPerCore: 32768
  min: 131072
  tcpCloseWaitTimeout: 1h0m0s
  tcpEstablishedTimeout: 24h0m0s
healthzBindAddress: 0.0.0.0:10256
# hostnameOverride 值必须与 kubelet 的对应一致，否则 kube-proxy 启动后会找不到该 Node，从而不会创建任何 iptables 规则
hostnameOverride: "worker-63"
metricsBindAddress: 0.0.0.0:10249
mode: "ipvs"
ipvs:
  excludeCIDRs: null
  minSyncPeriod: 0s
  scheduler: ""
  strictARP: true   # 修改为true
  syncPeriod: 30s
  tcpFinTimeout: 0s
  tcpTimeout: 0s
  udpTimeout: 0s

安装

根据官方安装指南，通过 manifest 的方式安装

$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.3/config/manifests/metallb-native.yaml

配置L2 地址池

配置文档：https://metallb.universe.tf/configuration/#layer-2-configuration

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.90-192.168.1.99

测试

# mentallb/whoami.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  labels:
    app: containous
    name: whoami
spec:
  replicas: 2
  selector:
    matchLabels:
      app: containous
      task: whoami
  template:
    metadata:
      labels:
        app: containous
        task: whoami
    spec:
      containers:
        - name: containouswhoami
          image: registry.cn-hangzhou.aliyuncs.com/qiqios/whoami_test:v1.5.0
          resources:
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: whoami
  type: LoadBalancer

查看

$ kubectl get all
NAME                            READY   STATUS    RESTARTS   AGE
pod/whoami-cd7cbb856-wbrjq      1/1     Running   0          14m
pod/whoami-cd7cbb856-wdf45      1/1     Running   0          14m

NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
service/kubernetes   ClusterIP      10.68.0.1       <none>         443/TCP        2d18h
service/whoami       LoadBalancer   10.68.58.219    192.168.1.90   80:32075/TCP   14m

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/whoami     2/2     2            2           14m

NAME                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/whoami-cd7cbb856      2         2         2       14m

可以查看到分配 EXTERNAL-IP 为 192.168.1.90

测试

$ curl 192.168.1.90
Hostname: whoami-cd7cbb856-wdf45
IP: 127.0.0.1
IP: ::1
IP: 172.20.92.198
IP: fe80::1863:66ff:febe:d1bb
RemoteAddr: 172.20.234.192:64535
GET / HTTP/1.1
Host: 192.168.1.90
User-Agent: curl/7.68.0
Accept: */* 

清理测试数据

$ kubectl delete -f whoami.yaml