← Back to blog

Multi-Tenant Kubernetes for a Small Team

kubernetesmulti-tenancyinfrastructureself-hostingsecurity
Multi-Tenant Kubernetes for a Small Team

If you have a small team and you are putting more than one tenant on one Kubernetes cluster, build it in this order: namespaces for naming and policy scope, RBAC so tenants cannot touch each other’s API objects, ResourceQuota and LimitRange so one tenant cannot starve the rest, a default-deny NetworkPolicy so pods cannot reach across the cluster by default, and only then — if your tenants genuinely do not trust each other — reach for a virtual cluster. That is the whole playbook. Everything below is the reasoning, the exact mechanisms, and where each one stops protecting you.

I run my own infrastructure with no dedicated ops team — me and automation. So the bias of this post is toward the smallest thing that holds, not the most thorough thing that exists. The two are different, and pretending otherwise is how small teams end up maintaining infrastructure they cannot afford to maintain.

First, decide whether your tenants trust each other

This is the only decision that matters, and most people skip it.

The Kubernetes documentation splits multi-tenancy into “soft” and “hard,” but it is careful to say the terms are slippery: “the terms ‘hard’ and ‘soft’ can often be confusing, as there is no single definition that will apply to all users. Rather, ‘hardness’ or ‘softness’ is better understood as a broad spectrum” (Kubernetes — Multi-tenancy). The useful distinction underneath the words: hard multi-tenancy is for tenants who do not trust each other — different customers, different security domains, the case where you are “guarding against attacks such as data exfiltration or DoS.” Soft multi-tenancy is for tenants who do — your own teams, your own environments, internal apps that share a cluster for cost and convenience.

The reason this is the first decision: it tells you whether namespaces are enough. For soft tenancy — your own teams — namespaces plus the four native controls below are genuinely enough, and they are cheap to run. For hard tenancy — untrusted tenants — they are not, and no amount of YAML changes that, because the tenants still share one API server and one set of nodes. You need a stronger boundary, which is where vCluster comes in.

Be honest about which one you have. Most small teams have soft tenancy and talk themselves into building for hard tenancy, then drown in the operational cost. A few have hard tenancy and pretend namespaces will hold, which is the dangerous mistake.

Namespaces: real, but not a security boundary

Namespaces do two concrete things, and it is worth being precise because people overload them with expectations they do not meet.

One, naming isolation. “Object names within a namespace can overlap with names in other namespaces, similar to files in folders” (Kubernetes — Multi-tenancy). Tenant A and tenant B can both have a Deployment called api and a Secret called db-creds without colliding.

Two, policy scope. “Many Kubernetes security policies are scoped to namespaces. For example, RBAC Roles and Network Policies are namespace-scoped resources” (same source). The namespace is the unit almost every other control hangs off.

What a namespace is not: a security boundary on its own. The docs say it plainly — “The namespace isolation model requires configuration of several other Kubernetes resources, networking plugins, and adherence to security best practices to properly isolate tenant workloads” (Kubernetes — Multi-tenancy). A namespace gives you control-plane organisation. The isolation comes from what you wrap around it. So wrap.

RBAC: stop tenants touching each other’s API

If you do one thing after creating namespaces, do this. The docs are blunt about why authorization comes first: “If teams or their workloads can access or modify each others’ API resources, they can change or disable all other types of policies thereby negating any protection those policies may offer” (Kubernetes — Multi-tenancy). Your quotas and network policies are worthless if a tenant can edit them.

The mechanism is Role and RoleBinding — both namespace-scoped — granting a tenant access to their namespace and no other. The principle the docs name is least privilege: “it is critical to ensure that each tenant has the appropriate access to only the namespaces they need, and no more” (same source).

The trap for a small team is ClusterRole and ClusterRoleBinding. They are cluster-wide. It is faster to bind a tenant to a cluster-wide role and move on; it is also how you accidentally hand one tenant read access to every namespace. The discipline is boring and non-negotiable: namespace-scoped RoleBinding, one per tenant, every time.

ResourceQuota and LimitRange: the noisy-neighbour fix

Soft tenancy’s real failure mode is not a security breach. It is one tenant’s runaway job eating the CPU and memory the others needed. ResourceQuota is the fix, and it is per namespace by design: a ResourceQuota “provides constraints that limit aggregate resource consumption per namespace” and “can also limit the quantity of objects that can be created in a namespace by API kind” (Kubernetes — Resource Quotas). Cap total CPU, total memory, and object counts (Pods, ConfigMaps, Services) per tenant. The multi-tenancy guide frames the goal directly: quotas “ensure fairness and aim to avoid noisy neighbor issues from affecting other tenants that share a control plane” (Kubernetes — Multi-tenancy).

Now the gotcha that bites everyone the first time. The moment you set a CPU or memory quota on a namespace, every pod in it must declare requests or limits, or the API server rejects it: “If you enforce a resource quota in a namespace for either cpu or memory, you and other clients, must specify either requests or limits for that resource, for every new Pod you submit. If you don’t, the control plane may reject admission for that Pod” (Kubernetes — Resource Quotas). Tenants who never thought about resource requests suddenly cannot deploy anything, and they will not know why.

The fix is LimitRange, in the same namespace. “You can define a LimitRange to force defaults on pods that make no compute resource requirements (so that users don’t have to remember to do that)” (same source). Pair them: ResourceQuota sets the ceiling, LimitRange fills in a sane default request/limit so pods without one still admit. Ship them together or your quota turns into an outage.

NetworkPolicy: default-deny, then open what you need

By default, Kubernetes pods talk to everything. “By default, a pod is non-isolated for ingress” and “non-isolated for egress” — if no policy selects a pod, all traffic in and out is allowed (Kubernetes — Network Policies). On a shared cluster that means tenant A’s pod can reach tenant B’s database unless you say otherwise. So say otherwise.

The pattern is default-deny per namespace, then allow back the specific flows each tenant needs. A NetworkPolicy that selects all pods in a namespace but lists no allowed traffic isolates every pod there — deny all, then layer specific allow-rules on top. Cross-namespace access is opt-in via namespaceSelector, so tenant boundaries hold unless you explicitly punch a hole.

Two facts that change how you reason about this:

Policies are additive, never subtractive. “Network policies do not conflict; they are additive… Thus, order of evaluation does not affect the policy result” (Kubernetes — Network Policies). There is no “deny” rule and no precedence to reason about — you start from deny-all and every policy only ever adds permitted traffic. That is far easier to keep correct than a firewall with ordered allow/deny lines.

It only works if your CNI implements it. This is the one that silently fails. “To use network policies, you must be using a networking solution which supports NetworkPolicy. Creating a NetworkPolicy resource without a controller that implements it will have no effect” (same source). You can apply a perfect default-deny policy, see it accepted by the API, and have zero enforcement because the network plugin ignores it. Calico, Cilium, and others support it; some setups do not. Test it: deploy two pods in different namespaces and confirm the connection is actually refused. If it is not refused, your policy is decorative.

The ceiling of soft tenancy

Here is where I stop you from over-trusting everything above. All of it — namespaces, RBAC, quotas, NetworkPolicy — shares one control plane and one set of nodes. The docs name the limit directly: control plane isolation matters, but “Since data planes typically have much larger attack surfaces, ‘hard’ multi-tenancy often requires extra attention to isolating the data-plane” (Kubernetes — Multi-tenancy). Tenants still share the kernel on shared nodes. They still hit one API server. A container escape or a kernel exploit is not stopped by a RoleBinding.

For your own teams, that shared surface is an acceptable risk — that is what soft tenancy means. For untrusted tenants, it is the whole problem, and nothing in the native toolbox closes it.

When vCluster earns its weight

vCluster (by Loft Labs, open source under Apache 2.0 — GitHub) gives each tenant a virtual cluster: its own API server and control plane running as a pod inside a namespace of your host cluster, with a syncer that schedules the tenant’s pods down onto the host’s nodes (vCluster — What are virtual clusters). Each tenant gets “its own API server, its own CRDs and RBAC,” a “cluster experience indistinguishable from a dedicated Kubernetes cluster” (same source).

What that buys you over namespaces: API-level isolation. A tenant can install their own CRDs, run their own operators, and make cluster-scoped changes without touching anyone else, because they are hitting their own API server, not yours. With plain namespaces, one shared API server means a tenant CRD or a cluster-scoped resource is a shared-cluster problem. vCluster removes that class of conflict. It is also cheaper than handing every tenant a full standalone cluster, because the virtual control planes share the host’s nodes.

When it earns its weight:

  • Tenants need their own CRDs or operators. The single most common reason. Shared namespaces cannot give two tenants conflicting cluster-scoped resources; vCluster can.
  • You are heading toward hard tenancy and need a stronger boundary than RBAC, but cannot afford N real clusters.
  • You want to give a tenant cluster-admin inside their slice without giving them anything on the real cluster.

When it does not:

  • Your tenants trust each other and just want their own apps. Namespaces plus the four controls above are enough. A virtual control plane per tenant is overhead you will maintain forever for isolation you did not need.
  • You have no ops capacity at all. vCluster is more moving parts than a namespace. Real, but more. Do not adopt it to solve a problem you do not have.

A note on the middle ground: Capsule (CLASTIX, CNCF Sandbox project, Apache 2.0 — CNCF) sits between plain namespaces and vCluster. It is an operator that groups namespaces into “Tenants” and auto-applies RBAC, quotas, and network policies per tenant on the shared control plane — so it makes soft tenancy easier to administer at scale without giving each tenant a separate API server. If your problem is “I have twenty namespaces and applying the same four controls by hand is error-prone,” Capsule is the answer, not vCluster. If your problem is “two tenants need conflicting CRDs,” it is not — that needs the separate API server vCluster gives you.

The actual decision

Walk it in this order and stop at the first “no”:

  1. Do tenants trust each other? No → you are in hard-tenancy territory; native controls alone will not hold; plan for vCluster or separate clusters. Yes → continue.
  2. Namespaces for naming and policy scope. Always.
  3. RBAC — namespace-scoped Roles and RoleBindings, least privilege, no cluster-wide bindings per tenant. Always.
  4. ResourceQuota + LimitRange together so quotas do not block deploys. Always.
  5. Default-deny NetworkPolicy per namespace, opened per flow — and verify your CNI actually enforces it. Always.
  6. Do tenants need their own CRDs / cluster-admin in their slice? Yes → vCluster. No → you are done.
  7. Many namespaces, same controls, hard to manage by hand? → Capsule to administer them, still on the shared plane.

Steps 2 through 5 are the small-team default and they cover the common case completely. Steps 6 and 7 are for when you have outgrown it. The mistake is reaching for step 6 before you have honestly answered step 1.