Skip to content

The Ultimate Guide to Kubernetes Load Balancers in 2026 (K3s Edition) ​

TL;DR β€” Running K3s on bare metal or edge? This guide dissects every major Kubernetes load balancer β€” NGINX, Traefik, MetalLB, HAProxy, Envoy, Cilium, Istio, Linkerd, and K3s's own Klipper β€” across architecture, performance, K3s compatibility, and real-world use cases. Pick the right one for your stack, once and for all.


🧭 Why This Guide Exists ​

Kubernetes load balancers are one of the most confusing corners of the cloud-native ecosystem. Search for "best Kubernetes load balancer" and you'll find a dozen blog posts each recommending something different, often without context. When you throw K3s β€” the lightweight, single-binary Kubernetes distribution from Rancher β€” into the mix, the confusion compounds further.

K3s ships with its own built-in load balancer (Klipper/ServiceLB) and its own ingress controller (Traefik). But is that the right choice for your production workload? What if you need BGP routing, service mesh capabilities, or sub-millisecond latency?

This guide covers every serious option in the market today, with real benchmarks, architecture diagrams, and clear K3s-specific guidance.


πŸ—ΊοΈ The Landscape: What Are We Even Comparing? ​

Before diving in, let's clarify the terminology. "Load balancer" in Kubernetes refers to multiple layers:

LayerWhat It DoesExample Tools
L4 LoadBalancer (IP/TCP)Assigns external IPs to ServicesMetalLB, Klipper, Kube-VIP
L7 Ingress ControllerRoutes HTTP/HTTPS traffic by host/pathNGINX, Traefik, HAProxy
Reverse Proxy / Edge ProxyAdvanced traffic shaping, retries, circuit breakingEnvoy, HAProxy
Service MeshEast-west (pod-to-pod) traffic management + securityIstio, Linkerd, Cilium

Most real deployments combine tools from multiple layers. For K3s, a typical production stack might be: MetalLB (L4) + Traefik (L7 Ingress) + optionally Linkerd (mesh).


πŸ”¬ Competitor Deep-Dive ​

1. 🏠 Klipper ServiceLB (K3s Built-In) ​

What it is: K3s's embedded load balancer, enabled by default. Uses host ports and iptables rules to forward traffic.

Architecture:

External Traffic
      β”‚
      β–Ό
[Node HostPort] ──iptables──► [ClusterIP] ──► [Pod]
      β–²
[DaemonSet: svc-* pods on each node]

How it works: For each LoadBalancer Service, Klipper creates a DaemonSet with svc- prefixed pods that bind to the host port. The node's own external IP is reported as the EXTERNAL-IP. There is no IP announcement to the network β€” it simply binds ports.

K3s-specific note: Klipper is enabled by default. To run MetalLB or any other LB controller, you must disable it:

bash
# During K3s install
curl -sfL https://get.k3s.io | sh -s - --disable servicelb

# Or in K3s config file
disable:
  - servicelb
FeatureRating
Zero configβœ… Built-in
True IP announcement❌ No
BGP support❌ No
Multi-node HA⚠️ Failover only
Production-readiness⚠️ Dev/small clusters
Resource usageβœ… Minimal

Best for: Local dev, single-node K3s, homelab, quick demos.


2. 🟒 NGINX Ingress Controller ​

What it is: The most widely deployed Kubernetes Ingress controller, based on the battle-tested NGINX reverse proxy. Two major variants exist: the community ingress-nginx and the commercial NGINX Inc. version (nginx-ingress).

Architecture:

Internet
   β”‚
   β–Ό
[NGINX Pod]
   β”‚  Reads Ingress rules + Annotations
   β”œβ”€β”€β–Ί /app-a  ──► Service A ──► Pods
   β”œβ”€β”€β–Ί /app-b  ──► Service B ──► Pods
   └──► /api    ──► Service C ──► Pods
        β”‚
   [ConfigMap / Annotations drive nginx.conf]

Key features:

  • Annotation-driven configuration (granular control via nginx.ingress.kubernetes.io/*)
  • SSL termination, wildcard certs, HSTS
  • Rate limiting, IP allowlisting, custom error pages
  • WebSocket support, gRPC proxying
  • Prometheus metrics out of the box
  • ModSecurity WAF support (community build)

K3s installation:

bash
# First, disable K3s's default Traefik if you want NGINX instead
curl -sfL https://get.k3s.io | sh -s - --disable traefik

# Install NGINX Ingress via Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace

Sample Ingress resource:

yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/rate-limit: "100"
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-app-svc
            port:
              number: 80

Performance: NGINX processes ~30,000–40,000 RPS per instance in typical Kubernetes ingress scenarios. Config reloads happen on Ingress updates (brief traffic disruption is possible on busy clusters).

FeatureRating
Community & docsβœ… Massive
Annotation flexibilityβœ… Excellent
Auto TLS (Let's Encrypt)⚠️ Needs cert-manager
Dynamic config (no reload)❌ Requires reload
Performanceβœ… Very good
K3s compatibilityβœ… Excellent
Learning curveβœ… Low

Best for: Teams migrating from traditional NGINX setups, production HTTP/HTTPS workloads, teams needing extensive annotation-based customization.


3. 🐹 Traefik (K3s Default) ​

What it is: A cloud-native reverse proxy and ingress controller written in Go. K3s ships Traefik v2 by default (upgraded to v3 in recent K3s releases). It auto-discovers services via Kubernetes CRDs and annotations.

Architecture:

Internet
   β”‚
   β–Ό
[Traefik Proxy]
   β”‚  Watches: IngressRoutes, Ingress, Services
   β”‚  Providers: Kubernetes CRD, Kubernetes Ingress
   β”‚
   β”œβ”€[Routers]──[Middlewares]──[Services]──► Pods
   β”‚     β”‚            β”‚
   β”‚  Host/Path    RateLimit
   β”‚  rules        Auth
   β”‚               Retry
   β”‚
   └─[Dashboard: :8080]  [Metrics: Prometheus]

Key features:

  • Zero-config service discovery β€” annotate a Service and Traefik picks it up instantly, no config file reloads
  • Automatic Let's Encrypt TLS with ACME challenge support
  • Middleware system: auth, rate limiting, headers, circuit breakers, retry
  • Native IngressRoute CRDs for full power
  • Built-in dashboard and Prometheus metrics
  • TCP/UDP routing support (not just HTTP)

K3s-specific note: Traefik is bundled and managed by K3s. To customize it, use a HelmChartConfig:

yaml
# /var/lib/rancher/k3s/server/manifests/traefik-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    dashboard:
      enabled: true
    additionalArguments:
      - "--entrypoints.websecure.http.tls"
    ports:
      web:
        redirectTo: websecure

Sample IngressRoute:

yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: my-app
spec:
  entryPoints:
    - websecure
  routes:
  - match: Host(`myapp.example.com`)
    kind: Rule
    services:
    - name: my-app-svc
      port: 80
    middlewares:
    - name: rate-limit
  tls:
    certResolver: letsencrypt

Performance: Traefik handles ~19,000 RPS with very stable resource consumption and zero-reload dynamic config β€” a key advantage over NGINX for fast-moving microservices.

FeatureRating
K3s integrationβœ… Native, bundled
Auto TLS (Let's Encrypt)βœ… Built-in ACME
Dynamic config (no reload)βœ… Real-time
Dashboardβœ… Built-in
TCP/UDP routingβœ… Yes
Performance vs NGINX⚠️ Slightly lower RPS
Enterprise features⚠️ Enterprise version needed

Best for: K3s default stack, teams wanting zero-touch TLS, GitOps-friendly pipelines, dev-friendly environments.


4. πŸ”· MetalLB ​

What it is: A bare-metal L4 load balancer for Kubernetes. It gives LoadBalancer type Services an actual external IP from a pool you define, using either Layer 2 (ARP) or BGP protocols.

Architecture (Layer 2 mode):

External Network
      β”‚
      β”‚  ARP: "Who has 192.168.1.100?" β†’ Leader Node replies
      β–Ό
[Leader Node] ──► kube-proxy ──► Service Pods (all nodes)
      β”‚
[MetalLB Speaker DaemonSet] on every node
[MetalLB Controller] handles IP assignment

Architecture (BGP mode):

[Router/Switch]
      β”‚  BGP peering
      β–Ό
[MetalLB Speaker] on each K3s node
      β”‚  Announces /32 routes per service IP
      β–Ό
[Direct packet routing to node]

K3s installation:

bash
# Step 1: Disable Klipper
curl -sfL https://get.k3s.io | sh -s - --disable servicelb

# Step 2: Install MetalLB
helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb -n metallb-system --create-namespace

# Step 3: Configure IP pool
kubectl apply -f - <<EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: k3s-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.200-192.168.1.220
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: k3s-l2
  namespace: metallb-system
EOF

Important caveat: In L2 mode, MetalLB doesn't truly load-balance at L4 β€” it elects a leader node that handles ARP for a given IP, and kube-proxy does the actual pod distribution. It's more of a failover mechanism than a true LB. BGP mode provides real per-node distribution but requires BGP-capable routers.

FeatureRating
Bare-metal IP assignmentβœ… Core purpose
BGP modeβœ… Yes
Layer 2 modeβœ… Yes (ARP/NDP)
True L4 load balancing⚠️ BGP only
K3s compatibilityβœ… Excellent (disable Klipper first)
Resource usageβœ… Very lightweight
Requires routers⚠️ BGP mode does

Best for: Bare-metal K3s clusters that need proper external IPs, homelab with a VLAN IP pool, edge deployments without cloud LB.


5. ⚑ HAProxy Ingress Controller ​

What it is: The Kubernetes ingress controller backed by HAProxy β€” historically the gold standard for raw TCP/HTTP load balancing performance. HAProxy Technologies' own benchmarks show their ingress controller handling 42,000 RPS with the lowest CPU among all competitors.

Architecture:

Internet
   β”‚
   β–Ό
[HAProxy Pod]
   β”‚  Config generated from Ingress/CRDs by controller
   β”‚
   β”œβ”€[Frontend: bind *:80]
   β”‚       β”‚
   β”‚  [ACL rules: path_beg, hdr_dom]
   β”‚       β”‚
   └─[Backend pools] ──► Pod endpoints (health-checked)
         β”‚
   [Stats: :1936]  [Prometheus metrics]

Key features:

  • Best-in-class raw throughput and lowest latency at scale
  • Native support for HTTP/3, QUIC, gRPC
  • Fine-grained connection control (timeouts, retries, stick tables)
  • Advanced Layer 7 routing: headers, cookies, ACLs
  • TCP mode for non-HTTP workloads
  • Gateway API support (HAProxy Ingress Controller v3.1+)

K3s installation:

bash
helm repo add haproxytech https://haproxytech.github.io/helm-charts
helm install haproxy-ingress haproxytech/kubernetes-ingress \
  --namespace haproxy-controller --create-namespace \
  --set controller.service.type=LoadBalancer

Performance edge: In head-to-head benchmarks against NGINX, Traefik, and Envoy:

  • HAProxy: 42,000 RPS, 50% CPU
  • NGINX: ~35,000 RPS, ~65% CPU
  • Traefik: ~19,000 RPS, ~45% CPU (more consistent)
  • Envoy: ~38,000 RPS, 73% CPU
FeatureRating
Raw throughputβœ… Best-in-class
HTTP/3 & gRPCβœ… Yes
Advanced ACLsβœ… Very powerful
Auto TLS⚠️ Needs cert-manager
Dynamic configβœ… v2.4+ hitless reload
K3s compatibilityβœ… Good
Complexity⚠️ Steeper learning curve

Best for: High-throughput production clusters, financial services, teams needing ultra-low p99 latency, TCP-heavy workloads.


6. 🌊 Envoy Proxy ​

What it is: Originally built at Lyft, Envoy is a high-performance C++ proxy that has become the de facto data plane of the cloud-native ecosystem. It powers Istio, Consul Connect, AWS App Mesh, and is the backbone of the Kubernetes Gateway API ecosystem.

Architecture:

[xDS Control Plane] (e.g., Istio's istiod)
       β”‚  gRPC streaming: LDS, RDS, CDS, EDS
       β–Ό
[Envoy Proxy Instance]
   β”‚
   β”œβ”€ Listeners (ports/protocols)
   β”‚       β”‚
   β”‚  Filter Chains (HTTP, TCP, gRPC filters)
   β”‚       β”‚
   └─ Clusters (upstream endpoints)
         β”‚
      [Circuit Breaker] [Retry] [Outlier Detection]

Key features:

  • Dynamic configuration via xDS API (zero-downtime updates)
  • Built-in circuit breaking, retries, outlier detection
  • Excellent observability: detailed stats, tracing (Zipkin/Jaeger/OTLP), access logs
  • gRPC-first with HTTP/1.1 and HTTP/2 support
  • Mutual TLS (mTLS) between services
  • WebAssembly (Wasm) plugin extensibility
  • Rate limiting via external services (Ratelimit service)

Standalone on K3s (without Istio):

bash
# Envoy Gateway β€” standalone Gateway API implementation
helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.2.0 -n envoy-gateway-system --create-namespace

Performance: Envoy delivers ~38,000 RPS with excellent handling of dynamic service churn (critical for microservices that scale up/down frequently). Its sub-10ms latency during pod scaling events makes it ideal for Netflix/Uber-style workloads.

FeatureRating
Dynamic config (xDS)βœ… Best-in-class
Observabilityβœ… Exceptional
gRPC supportβœ… Native
Circuit breakingβœ… Built-in
Wasm extensibilityβœ… Yes
Standalone complexity⚠️ High (needs control plane)
K3s standalone use⚠️ Via Envoy Gateway

Best for: Microservices architectures with dynamic service discovery, service mesh data plane, teams that need xDS-compatible control plane integration.


7. πŸ•ΈοΈ Istio (Service Mesh) ​

What it is: The most feature-complete service mesh for Kubernetes. Istio injects Envoy sidecars into every pod and manages the entire service-to-service communication layer via a centralized control plane (istiod).

Architecture:

[istiod - Control Plane]
   β”œβ”€β”€ Pilot (traffic management)
   β”œβ”€β”€ Citadel (certificate authority)
   └── Galley (config validation)
         β”‚  xDS API
         β–Ό
[Pod A]                    [Pod B]
  App Container              App Container
  Envoy Sidecar ◄──mTLS──► Envoy Sidecar
  (intercepts all traffic)   (intercepts all traffic)

Istio Ambient Mode (2024/2026): The new sidecar-free mode using per-node "ztunnel" proxies + optional Waypoint proxies eliminates the double-hop latency, bringing performance near bare-metal levels.

Key features:

  • Fine-grained traffic management: canary, A/B, weighted routing, fault injection
  • Automatic mTLS between all services
  • Authorization policies at L7 (RBAC per HTTP path/method)
  • Distributed tracing, Kiali topology visualization
  • Multi-cluster and VM support
  • Gateway API support

K3s resource requirements (important!):

  • istiod: ~500MB RAM
  • Per-pod Envoy sidecar: ~50MB RAM each
  • At 500 services: 25–50GB extra RAM vs. Linkerd β€” plan accordingly
bash
# Install Istio on K3s
curl -L https://istio.io/downloadIstio | sh -
istioctl install --set profile=minimal -y
kubectl label namespace default istio-injection=enabled
FeatureRating
Traffic managementβœ… Most advanced
mTLSβœ… Automatic
Observabilityβœ… Full stack (Kiali, Jaeger)
Authorization policiesβœ… L7 RBAC
Resource usage❌ Heavy (per-pod sidecar)
Complexity❌ High
K3s (small cluster)⚠️ Feasible, watch RAM

Best for: Enterprise Kubernetes, SOC 2/PCI-DSS compliance requirements, teams needing canary deployments and fault injection, hybrid VM+K8s environments.


8. πŸ”— Linkerd (Service Mesh) ​

What it is: The original service mesh (coined the term in 2016). Linkerd uses a Rust-based "microproxy" instead of Envoy β€” dramatically lighter weight, making it the fastest and most resource-efficient service mesh available.

Architecture:

[Linkerd Control Plane]
  β”œβ”€β”€ destination (service discovery)
  β”œβ”€β”€ identity (certificate authority)
  └── proxy-injector (sidecar injection)
         β”‚
[Pod A]                    [Pod B]
  App Container              App Container
  linkerd2-proxy ◄──mTLS──► linkerd2-proxy
  (Rust, ~10MB RAM each)     (tiny overhead!)

Performance benchmarks (vs other meshes):

  • Linkerd: ~5–10% slower than baseline (no mesh) β€” best among all meshes
  • Istio: ~25–35% slower than baseline
  • Cilium Mesh: ~20–30% slower than baseline

Key features:

  • Automatic mTLS (on by default, zero config)
  • Golden signals dashboard (latency, traffic, errors, saturation)
  • Per-route metrics
  • Traffic splitting (canary, A/B)
  • Multi-cluster support
  • FIPS-compliant builds available
  • Graduated CNCF project (most mature after Istio)

K3s installation:

bash
# Install Linkerd CLI
curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install | sh

# Pre-flight check
linkerd check --pre

# Install on K3s
linkerd install --crds | kubectl apply -f -
linkerd install | kubectl apply -f -
linkerd check

# Inject into a namespace
kubectl annotate namespace default linkerd.io/inject=enabled
FeatureRating
Resource efficiencyβœ… Best among meshes
Performance overheadβœ… Minimal (5–10%)
mTLSβœ… Auto, zero-config
Simplicityβœ… Easiest mesh
Dashboardβœ… Built-in
Advanced traffic routing⚠️ Less than Istio
K3s compatibilityβœ… Excellent

Best for: Teams wanting mesh capabilities without Istio's complexity, K3s clusters with limited RAM, security-first teams, anyone who wants to "just turn it on and have it work."


9. 🧬 Cilium (eBPF-based CNI + Service Mesh) ​

What it is: Cilium is fundamentally different from all others β€” it operates at the Linux kernel level using eBPF (extended Berkeley Packet Filter), replacing traditional iptables networking entirely. It serves as both a CNI (network plugin) and optionally a service mesh.

Architecture:

[Cilium Operator] + [Cilium Agent DaemonSet]
         β”‚  Programs eBPF maps
         β–Ό
[Linux Kernel - eBPF programs]
   β”œβ”€β”€ XDP (eXpress Data Path): packet filtering at NIC level
   β”œβ”€β”€ TC (Traffic Control): L3/L4 policy enforcement
   └── Socket: L7 visibility (HTTP, gRPC, Kafka, DNS)
         β”‚
[Hubble Observability Layer]
   β”œβ”€β”€ hubble-relay
   └── hubble-ui (real-time network flow visualization)

Key features:

  • eBPF-powered networking: bypasses kernel overhead, hardware-speed L4
  • No iptables β€” replaces kube-proxy entirely
  • Deep observability via Hubble (DNS, HTTP, gRPC, Kafka at kernel level)
  • Network policies at L3/L4/L7 in a single CRD
  • WireGuard/IPsec transparent encryption
  • Service mesh in per-node Envoy model (not sidecar-per-pod)
  • Excellent for multi-cluster with Cluster Mesh

K3s installation:

bash
# Disable K3s's default flannel (Cilium replaces it)
curl -sfL https://get.k3s.io | sh -s - \
  --flannel-backend=none \
  --disable-network-policy \
  --disable servicelb

# Install Cilium
helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium \
  --namespace kube-system \
  --set operator.replicas=1 \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=<K3S_SERVER_IP> \
  --set k8sServicePort=6443

# Enable Hubble
cilium hubble enable --ui

L4 performance: Cilium's eBPF datapath is unrivaled for L4 (TCP/UDP) β€” limited only by hardware NIC speed. For L7 (HTTP), it offloads to per-node Envoy, which introduces some trade-offs vs. per-pod sidecar isolation.

FeatureRating
L4 throughputβœ… Best (eBPF)
Network observabilityβœ… Exceptional (Hubble)
No iptablesβœ… kube-proxy replacement
Network policiesβœ… L3/L4/L7 unified
Service mesh⚠️ Per-node (not per-pod)
Complexity⚠️ eBPF expertise needed
K3s integrationβœ… Good (replaces flannel)

Best for: High-performance bare-metal clusters, security-intensive environments, teams already investing in eBPF, multi-cluster deployments with Cluster Mesh.


πŸ“Š The Big Comparison Table ​

ToolTypeOSI LayerK3s DefaultAuto TLSPerformanceResource UsageComplexity
Klipper/ServiceLBL4 LBL4βœ… Yes❌LowMinimalMinimal
NGINXIngressL7❌ (opt-out Traefik)⚠️ (cert-manager)Very HighLowLow
TraefikIngressL7βœ… Yes (bundled)βœ… Built-inHighLowLow
MetalLBL4 LBL4❌❌MediumMinimalLow
HAProxyIngressL4+L7❌⚠️ (cert-manager)HighestLowMedium
EnvoyProxy/Mesh DPL4+L7βŒβœ… (with CP)Very HighMediumHigh
IstioService MeshL4+L7βŒβœ… Auto mTLSMedium (overhead)Very HighVery High
LinkerdService MeshL4+L7βŒβœ… Auto mTLSHigh (least overhead)LowLow
CiliumCNI+MeshL3+L4+L7βŒβœ… (WireGuard)Highest L4MediumHigh

πŸ—οΈ Architecture Patterns for K3s ​

Pattern 1: Minimal (Single Node / Homelab) ​

[K3s: Traefik + Klipper built-in]
   β”‚
   └── Just works. Zero extra config needed.

Use when: Local dev, single-node homelab, learning Kubernetes.

Pattern 2: Bare-Metal Production (Most Common) ​

[MetalLB] ──► External IP ──► [Traefik] ──► [Your Services]

Use when: Multiple K3s nodes, need proper external IPs, keep Traefik for simplicity.

Pattern 3: High-Performance Production ​

[MetalLB] ──► External IP ──► [HAProxy Ingress] ──► [Services]

Use when: High RPS requirements, latency-sensitive APIs, financial/gaming workloads.

Pattern 4: Secure Microservices (Security-First) ​

[MetalLB] ──► [NGINX/Traefik] ──► [Linkerd Mesh] ──► [Services]
                                      (mTLS, observability)

Use when: Multi-service architecture, compliance requirements, need service-to-service encryption.

Pattern 5: Maximum Performance + Security (Advanced) ​

[Cilium CNI + kube-proxy replacement]
   └──► [Cilium Ingress / Envoy Gateway] ──► [Services]
        + Hubble for observability

Use when: eBPF expertise available, need kernel-level performance, security-intensive platform.


🏎️ Performance Benchmarks at a Glance ​

Based on published benchmarks and production data (2024–2026):

Requests per Second (RPS) at typical K8s ingress workload:

HAProxy    β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ  42,000 RPS  (50% CPU)
Envoy      β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ   38,000 RPS  (73% CPU)
NGINX      β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ    35,000 RPS  (65% CPU)
Traefik    β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ                 19,000 RPS  (45% CPU)

Service Mesh Overhead (vs no mesh):
Linkerd    β–ˆβ–ˆ  5–10% slower   ← Best
Cilium     β–ˆβ–ˆβ–ˆβ–ˆ  20–30% slower
Istio      β–ˆβ–ˆβ–ˆβ–ˆβ–ˆ  25–35% slower

L4 Raw Throughput:
Cilium (eBPF)  β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ  Hardware-limited ← Best
MetalLB (BGP)  β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ    Near line-rate

🎯 Decision Framework: Which One for Your K3s Cluster? ​

START HERE
    β”‚
    β–Ό
Are you running a single node / homelab?
  YES ──► Use Klipper + Traefik (K3s defaults). You're done.
  NO
    β”‚
    β–Ό
Do you need external IPs on bare metal?
  YES ──► Add MetalLB (disable Klipper first)
  NO (cloud) ──► Your cloud CCM handles this
    β”‚
    β–Ό
Replace default Traefik ingress?
  Need max performance ──► HAProxy Ingress
  Need NGINX ecosystem ──► NGINX Ingress
  Happy with defaults   ──► Keep Traefik
    β”‚
    β–Ό
Do you have multiple microservices needing service-to-service security?
  YES, want simplicity ──► Add Linkerd
  YES, need full features ──► Add Istio (check your RAM budget!)
  YES, eBPF expertise ──► Use Cilium as CNI + mesh
  NO ──► Skip the mesh for now

πŸ”§ K3s-Specific Tips & Gotchas ​

  1. Traefik version: K3s bundles Traefik. Pin the version in your HelmChartConfig if stability matters.

  2. MetalLB + Traefik: A very common combo. MetalLB gives Traefik a real external IP. After MetalLB assigns an IP, Traefik's LoadBalancer service gets EXTERNAL-IP populated and starts serving traffic.

  3. Cilium on K3s: You must disable flannel (--flannel-backend=none) and network policy (--disable-network-policy). Cilium replaces both. If you also want to replace kube-proxy, add --disable-kube-proxy.

  4. Linkerd on K3s: Works out of the box. K3s's bundled components (Traefik, CoreDNS) can be meshed too β€” annotate the kube-system namespace carefully.

  5. Resource planning: A 3-node K3s cluster with Linkerd can run comfortably on 3Γ— Raspberry Pi 4 (4GB). Istio needs significantly more β€” budget at least 8GB per node.

  6. Gateway API: The Kubernetes Gateway API is replacing Ingress. Traefik v3, HAProxy v3.1+, Envoy Gateway, and Cilium all support it. Consider Gateway API for new deployments.


🏁 Final Recommendations ​

Your SituationRecommended Stack
Homelab / learningK3s defaults (Traefik + Klipper)
Bare-metal small teamMetalLB + Traefik
Bare-metal high trafficMetalLB + HAProxy
NGINX ecosystem familiarityMetalLB + NGINX Ingress
Need service mesh (simple)MetalLB + Traefik + Linkerd
Need service mesh (full features)MetalLB + Traefik + Istio (Ambient mode)
Max performance + securityCilium CNI + Envoy Gateway
Edge/IoT K3sKlipper + Traefik (minimal resources)

πŸ“š Further Reading ​


Have questions about your specific K3s setup? Drop them in the comments. Running an unusual configuration (Raspberry Pi cluster, edge IoT, air-gapped)? I'd love to hear about it.

#kubernetes #k3s #devops #cloudnative #loadbalancing #traefik #nginx #metallb #linkerd #cilium

Made with ❀️ in IndiaπŸ›• by P.Bhargava Sai