使用Helm安装Ingress-nginx-Controller
概述
Nginx Ingress Controller 是使用 Kubernetes Ingress 资源对象构建的，用ConfigMap 来存储 Nginx 配置的一种 Ingress Controller 实现。
安装 ingress-nginx 有多种方式，本文使用 helm 3.12 方式进行安装。
其它安装方式可以参考文档：https://kubernetes.github.io/ingress-nginx/deploy/
本文 K8S 集群采用 kubeasz 工具搭建的二进制单 master 集群
$ kubectl get node
NAME STATUS ROLES AGE VERSION
master-60 Ready,SchedulingDisabled master 43h v1.28.1
worker-61 Ready node 43h v1.28.1
worker-62 Ready node 43h v1.28.1
worker-63 Ready node 43h v1.28.1Helm 介绍
Helm 是一个用于对需要在 K8S 上部署的复杂应用进行定义、安装和更新。 Helm 以 char 的方式对应用软件进行描述，可以方便地创建、版本化、共享和发布复杂的应用软件。
- 官方英文文档：https://helm.sh/docs/
- 官方中文文档：https://helm.sh/zh/docs/
helm的主要概念
- Chart
Helm 的应用包，采用tgz格式。类似于 Yum 的 RPM 包，其包含了一组定义 Kubernetes 资源相关的 YAML 文件，也称为应用 Chart。
- Repoistory
Helm 的应用仓库，Repository 本质上是一个 Web 服务器，该服务器保存了一系列的 Chart 应用包以供用户下载，并且提供了一个该 Repository 的 Chart 包的清单文件以供查询，Helm 可以同时管理多个不同的 Repository。
Helm 社区官方提供了 stable 和 incubator 仓库，但 Helm 社区没有打算独占仓库，而是允许其他人和组织也可以搭建仓库。仓库可以是公共仓库，也可以是私有仓库。
- Release
在 Kubernetes 集群上运行的 Chart 的一个实例。在同一个集群上，一个 Chart 可以安装很多次。每次安装都会创建一个新的 Release。例如一个 MySQL Chart，如果想在服务器上运行两个 MySQL 数据库，就可以把这个 Chart 安装两次。每次安装都会生成一个新的 Release。
helm 安装
Helm安装官方文档：https://helm.sh/docs/intro/install/
# 下载
[root@k8s-master ~]# wget https://get.helm.sh/helm-v3.6.0-linux-amd64.tar.gz
# 解压
[root@k8s-master ~]# tar -zxvf helm-v3.6.0-linux-amd64.tar.gz
# 移动到环境变量目录里面即可
[root@k8s-master ~]# mv linux-amd64/helm /usr/local/bin/helm
# 输出版本
[root@k8s-master ~]# helm version
version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}Chart包的目录结构
# 创建一个chart，名称叫helm-test
$ helm create helm-test
$ tree helm-test/
helm-test/
├── charts
├── Chart.yaml
├── templates
│ ├── deployment.yaml
│ ├── _helpers.tpl
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── serviceaccount.yaml
│ ├── service.yaml
│ └── tests
│ └── test-connection.yaml
└── values.yaml
3 directories, 10 fileschart 是 Helm 的应用打包格式。chart 由一些列文件组成。这些文件描述了 Kubernetes 部署应用时所需要的资源。上面通过helm 命令创建一个 chart 包，目录结构说明如下：
helm-test: chart 包的名称charts目录: 保存依赖文件的目录，如果依赖其它的chart，则会保存在这里Chart.yaml文件：用于描述 chart 信息的yaml文件，如版本信息等values.yaml文件：chart 支持在安装的时候根据参数进行定制化配置，而 values.yaml 则提供了这些配置参数的默认值，可以在安装前根据需要修改 values.yaml 的参数进行定制化部署templates目录：各类 Kubernetes 资源的配置模版都放置在这里。Helm 会将 values.yaml 中的参数值注入到模版中生成标准的 YAML 配置文件。
Helm 安装 Ingress-nginx
添加 ingress-nginx 官方 helm 仓库
$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ingress-nginx" chart repository
Update Complete. ⎈Happy Helming!⎈
$ helm repo list
NAME URL
ingress-nginx https://kubernetes.github.io/ingress-nginx下载 ingress-nginx 的 chart 包
# 查找ingress-nginx 的 chart 包
$ helm search repo ingress-nginx
NAME CHART VERSION APP VERSION DESCRIPTION
ingress-nginx/ingress-nginx 4.10.0 1.10.0 Ingress controller for Kubernetes using NGINX a...
$ helm pull ingress-nginx/ingress-nginx
$ tar -xf ingress-nginx-4.10.0.tgz
$ tree ingress-nginx -L 1
ingress-nginx
├── changelog
├── Chart.yaml
├── ci
├── OWNERS
├── README.md
├── README.md.gotmpl
├── templates
├── tests
└── values.yaml
4 directories, 5 files修改 values.yaml 文件
下载的chart 包，需要修改一下资源清单配置文件，符合需求，修改 values.yaml 文件如下：
修改 ingress-nginx-controller 的镜像仓库地址，默认是 k8s.gcr.io
controller:
name: controller
enableAnnotationValidations: false
image:
## Keep false as default for now!
chroot: false
registry: docker.io
image: willdockerhub/ingress-nginx-controller # 修改镜像仓库地址
## for backwards compatibility consider setting the full image url via the repository value below
## use *either* current default registry/image or repository format or installing chart by providing the values.yaml will fail
## repository:
tag: "v1.1.3"
#digest: sha256:42b3f0e5d0846876b1791cd3afeb5f1cbbe4259d6f35651dcc1b5c980925379c
#digestChroot: sha256:7eb46ff733429e0e46892903c7394aff149ac6d284d92b3946f3baf7ff26a096
# 注释掉 digest 信息修改 hostNetwork 的值为 true
# since CNI and hostport don't mix yet. Can be deprecated once https://github.com/kubernetes/kubernetes/issues/23920
# is merged
hostNetwork: true
## Use host ports 80 and 443
## Disabled by defaultnodeSelector 添加标签 ingress: "true"，用于部署 ingrss-controller 到指定节点
## Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
##
nodeSelector:
kubernetes.io/os: linux
ingress: "true"kind 类型为 DaemonSet
# -- Use a `DaemonSet` or `Deployment`
kind: DaemonSet
# -- Annotations to be added to the controller Deployment or DaemonSetkube-webhook-certgen 的镜像地址修改为 docker仓库 dyrnq/kube-webhook-certgen:v1.4.0
patch:
enabled: true
image:
registry: docker.io
image: dyrnq/kube-webhook-certgen
## for backwards compatibility consider setting the full image url via the repository value below
## use *either* current default registry/image or repository format or installing chart by providing the values.yaml will fail
## repository:
tag: v1.4.0
#digest: sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334
pullPolicy: IfNotPresent
# -- Provide a priority class name to the webhook patching job执行安装
资源清单文件修改完成后，执行 helm 安装
# 创建一个命名空间
$ kubectl create ns ingress-nginx
namespace/ingress-nginx created
# 给指定节点打标签，调度ingress-ontroller pod指定节点
$ kubectl label nodes worker-63 ingress=true
node/worker-63 labeled
$ kubectl get node worker-63 --show-labels
NAME STATUS ROLES AGE VERSION LABELS
worker-63 Ready node 2d1h v1.28.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker-63,kubernetes.io/os=linux,kubernetes.io/role=node
# helm 安装
$ helm install ingress-nginx -n ingress-nginx .
NAME: ingress-nginx
LAST DEPLOYED: Thu Mar 7 10:36:31 2024
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the load balancer IP to be available.
You can watch the status by running 'kubectl get service --namespace ingress-nginx ingress-nginx-controller --output wide --watch'
An example Ingress that makes use of the controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
namespace: foo
spec:
ingressClassName: nginx
rules:
- host: www.example.com
http:
paths:
- pathType: Prefix
backend:
service:
name: exampleService
port:
number: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls安装完成后，检查相关资源
$ kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-controller-zqm64 1/1 Running 0 9m42s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller LoadBalancer 10.68.81.137 <pending> 80:32744/TCP,443:30620/TCP 9m42s
service/ingress-nginx-controller-admission ClusterIP 10.68.175.14 <none> 443/TCP 9m42s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/ingress-nginx-controller 1 1 1 1 1 ingress=true,kubernetes.io/os=linux 9m42s
# 查看Ingrss 的ClassName,部署ingress的时候需要绑定
$ kubectl get ingressclasses.networking.k8s.io
NAME CONTROLLER PARAMETERS AGE
nginx k8s.io/ingress-nginx <none> 52m测试 ingress-nginx
创建后端的nginx 的 pod 和 Service
apiVersion: apps/v1
kind: Deployment
metadata:
name: svc-demo
spec:
replicas: 2
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- image: swr.cn-north-4.myhuaweicloud.com/qiqios/nginx:1.25.2-alpine
name: svc-demo
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: svc-demo
spec:
selector:
app: myapp
ports:
- targetPort: 80 # 后端Pod的端口
port: 8080 # 服务要暴露的端口部署完成检查资源
$ kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/svc-demo-84d4dbc776-b462p 1/1 Running 0 19s
pod/svc-demo-84d4dbc776-hgx4x 1/1 Running 0 19s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 2d14h
service/svc-demo ClusterIP 10.68.113.155 <none> 8080/TCP 19s创建 ingress 规则，ingress-nginx.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-wildcard-host
spec:
ingressClassName: nginx
rules:
- host: "test.qiqios.com"
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: svc-demo
port:
number: 8080创建 ingress
$ kubectl apply -f test-ingess-nginx.yaml
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-wildcard-host nginx test.qiqios.com 80 31m配置host解析，循环访问测试
192.168.1.63 test.qiqios.com
$ for i in {1..10};do curl test.qiqios.com;sleep 1 ; done
This is pod1 Test.
This is pod2 Test.
This is pod1 Test.
This is pod1 Test.
This is pod2 Test.
This is pod2 Test.
This is pod1 Test.
This is pod2 Test.
This is pod1 Test.
This is pod1 Test.
# 说明访问正常，后端svc 是轮询访问后端的方式使用Helm安装Ingress-nginx-Controller
http://www.qiqios.cn/2024/03/07/使用Helm安装Ingress-nginx-Controller/