Deploy with Isolated Workloads
vCluster offers several features to automatically isolate workloads in a virtual cluster:
Resource Quota & Pod Security Standard
This feature imposes a couple of restrictions on vCluster workloads to make sure they do not break out of their virtual environment:
- vCluster enforces a Pod Security Standard on syncer level, which means that for example pods that try to run as a privileged container or mount a host path will not be synced to the host cluster. Current valid options are either baseline (default in isolated mode) or restricted. This works for every Kubernetes version regardless of Pod Security Standard support, as this is implemented in vCluster directly. Rejected pods will stay pending in the vCluster and in newer Kubernetes version they will be denied by the admission controller as well.
- vCluster deploys a resource quota as well as a limit range alongside the vCluster itself. This allows restricting resource consumption of vCluster workloads. If enabled, sane defaults for those 2 resources are chosen.
- vCluster deploys a network policy alongside itself that will restrict access of vCluster workloads as well as the vCluster control plane to other pods in the host cluster. (only works if your host cluster CNI supports network policies)
Adjust your isolation settings through the vcluster.yaml of the vCluster:
policies:
# empty, baseline, restricted can be used here
podSecurityStandard: baseline
resourceQuota:
enabled: true
limitRange:
enabled: true
For all options and more information take a look at the policies documentation.
When enabling resource quotas locally, make sure to add a --expose-local=false
flag to your vcluster create [...] command, as by default the vCluster CLI
will try to automatically expose the vCluster using NodePorts, when interacting
with a local Kubernetes cluster.
policies required object pro
Policies to enforce for the virtual cluster deployment as well as within the virtual cluster.
policies required object pronetworkPolicy required object pro
NetworkPolicy specifies network policy options.
networkPolicy required object proenabled required boolean false pro
Enabled defines if the network policy should be deployed by vCluster.
enabled required boolean false profallbackDns required string pro
fallbackDns required string prooutgoingConnections required object pro
outgoingConnections required object proipBlock required object pro
IPBlock describes a particular CIDR (Ex. "192.168.1.0/24","2001:db8::/64") that is allowed
to the pods matched by a NetworkPolicySpec's podSelector. The except entry describes CIDRs
that should not be included within this rule.
ipBlock required object procidr required string pro
cidr is a string representing the IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
cidr required string proexcept required string[] pro
except is a slice of CIDRs that should not be included within an IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
Except values will be rejected if they are outside the cidr range
except required string[] proannotations required object pro
Annotations are extra annotations for this resource.
annotations required object prolabels required object pro
Labels are extra labels for this resource.
labels required object propodSecurityStandard required string pro
PodSecurityStandard that can be enforced can be one of: empty (""), baseline, restricted or privileged
podSecurityStandard required string proresourceQuota required object pro
ResourceQuota specifies resource quota options.
resourceQuota required object proenabled required boolean false pro
Enabled defines if the resource quota should be enabled.
enabled required boolean false proquota required object pro
Quota are the quota options
quota required object proscopeSelector required object pro
ScopeSelector is the resource quota scope selector
scopeSelector required object proscopes required string[] pro
Scopes are the resource quota scopes
scopes required string[] proannotations required object pro
Annotations are extra annotations for this resource.
annotations required object prolabels required object pro
Labels are extra labels for this resource.
labels required object prolimitRange required object pro
LimitRange specifies limit range options.
limitRange required object proenabled required boolean false pro
Enabled defines if the limit range should be deployed by vCluster.
enabled required boolean false prodefault required object pro
Default are the default limits for the limit range
default required object prodefaultRequest required object pro
DefaultRequest are the default request options for the limit range
defaultRequest required object proannotations required object pro
Annotations are extra annotations for this resource.
annotations required object prolabels required object pro
Labels are extra labels for this resource.
labels required object procentralAdmission required object pro
CentralAdmission defines what validating or mutating webhooks should be enforced within the virtual cluster.
centralAdmission required object provalidatingWebhooks required object[] pro
ValidatingWebhooks are validating webhooks that should be enforced in the virtual cluster
validatingWebhooks required object[] prokind required string pro
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
kind required string proapiVersion required string pro
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
apiVersion required string prometadata required object pro
Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.
metadata required object proname required string pro
Name must be unique within a namespace. Is required when creating resources, although
some resources may allow a client to request the generation of an appropriate name
automatically. Name is primarily intended for creation idempotence and configuration
definition.
name required string prolabels required object pro
Map of string keys and values that can be used to organize and categorize
(scope and select) objects. May match selectors of replication controllers
and services.
labels required object proannotations required object pro
Annotations is an unstructured key value map stored with a resource that may be
set by external tools to store and retrieve arbitrary metadata.
annotations required object prowebhooks required object[] pro
Webhooks is a list of webhooks and the affected resources and operations.
webhooks required object[] proname required string pro
The name of the admission webhook.
Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where
"imagepolicy" is the name of the webhook, and kubernetes.io is the name
of the organization.
name required string proclientConfig required object pro
ClientConfig defines how to communicate with the hook.
clientConfig required object prourl required string pro
URL gives the location of the webhook, in standard URL form
(scheme://host:port/path). Exactly one of url or service
must be specified.
url required string proscheme://host:port/path). Exactly one of url or service
must be specified.service required object pro
Service is a reference to the service for this webhook. Either
service or url must be specified.
If the webhook is running within the cluster, then you should use service.
service required object proservice or url must be specified.service.namespace required string pro
Namespace is the namespace of the service.
namespace required string proname required string pro
Name is the name of the service.
name required string propath required string pro
Path is an optional URL path which will be sent in any request to
this service.
path required string proport required integer pro
If specified, the port on the service that hosting webhook.
Default to 443 for backward compatibility.
port should be a valid port number (1-65535, inclusive).
port required integer proport should be a valid port number (1-65535, inclusive).caBundle required string pro
CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.
If unspecified, system trust roots on the apiserver are used.
caBundle required string prorules required object[] pro
Rules describes what operations on what resources/subresources the webhook cares about.
The webhook cares about an operation if it matches any Rule.
rules required object[] profailurePolicy required string pro
FailurePolicy defines how unrecognized errors from the admission endpoint are handled -
allowed values are Ignore or Fail. Defaults to Fail.
failurePolicy required string promatchPolicy required string pro
matchPolicy defines how the "rules" list is used to match incoming requests.
Allowed values are "Exact" or "Equivalent".
matchPolicy required string pronamespaceSelector required object pro
NamespaceSelector decides whether to run the webhook on an object based
on whether the namespace for that object matches the selector. If the
object itself is a namespace, the matching is performed on
object.metadata.labels. If the object is another cluster scoped resource,
it never skips the webhook.
namespaceSelector required object proobjectSelector required object pro
ObjectSelector decides whether to run the webhook based on if the
object has matching labels. objectSelector is evaluated against both
the oldObject and newObject that would be sent to the webhook, and
is considered to match if either object matches the selector.
objectSelector required object prosideEffects required string pro
SideEffects states whether this webhook has side effects.
sideEffects required string protimeoutSeconds required integer pro
TimeoutSeconds specifies the timeout for this webhook.
timeoutSeconds required integer proadmissionReviewVersions required string[] pro
AdmissionReviewVersions is an ordered list of preferred AdmissionReview
versions the Webhook expects.
admissionReviewVersions required string[] proAdmissionReview
versions the Webhook expects.matchConditions required object[] pro
MatchConditions is a list of conditions that must be met for a request to be sent to this
webhook. Match conditions filter requests that have already been matched by the rules,
namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests.
There are a maximum of 64 match conditions allowed.
matchConditions required object[] promutatingWebhooks required object[] pro
MutatingWebhooks are mutating webhooks that should be enforced in the virtual cluster
mutatingWebhooks required object[] prokind required string pro
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
kind required string proapiVersion required string pro
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
apiVersion required string prometadata required object pro
Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.
metadata required object proname required string pro
Name must be unique within a namespace. Is required when creating resources, although
some resources may allow a client to request the generation of an appropriate name
automatically. Name is primarily intended for creation idempotence and configuration
definition.
name required string prolabels required object pro
Map of string keys and values that can be used to organize and categorize
(scope and select) objects. May match selectors of replication controllers
and services.
labels required object proannotations required object pro
Annotations is an unstructured key value map stored with a resource that may be
set by external tools to store and retrieve arbitrary metadata.
annotations required object prowebhooks required object[] pro
Webhooks is a list of webhooks and the affected resources and operations.
webhooks required object[] proreinvocationPolicy required string pro
reinvocationPolicy indicates whether this webhook should be called multiple times as part of a single admission evaluation.
Allowed values are "Never" and "IfNeeded".
reinvocationPolicy required string proname required string pro
The name of the admission webhook.
Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where
"imagepolicy" is the name of the webhook, and kubernetes.io is the name
of the organization.
name required string proclientConfig required object pro
ClientConfig defines how to communicate with the hook.
clientConfig required object prourl required string pro
URL gives the location of the webhook, in standard URL form
(scheme://host:port/path). Exactly one of url or service
must be specified.
url required string proscheme://host:port/path). Exactly one of url or service
must be specified.service required object pro
Service is a reference to the service for this webhook. Either
service or url must be specified.
If the webhook is running within the cluster, then you should use service.
service required object proservice or url must be specified.service.namespace required string pro
Namespace is the namespace of the service.
namespace required string proname required string pro
Name is the name of the service.
name required string propath required string pro
Path is an optional URL path which will be sent in any request to
this service.
path required string proport required integer pro
If specified, the port on the service that hosting webhook.
Default to 443 for backward compatibility.
port should be a valid port number (1-65535, inclusive).
port required integer proport should be a valid port number (1-65535, inclusive).caBundle required string pro
CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.
If unspecified, system trust roots on the apiserver are used.
caBundle required string prorules required object[] pro
Rules describes what operations on what resources/subresources the webhook cares about.
The webhook cares about an operation if it matches any Rule.
rules required object[] profailurePolicy required string pro
FailurePolicy defines how unrecognized errors from the admission endpoint are handled -
allowed values are Ignore or Fail. Defaults to Fail.
failurePolicy required string promatchPolicy required string pro
matchPolicy defines how the "rules" list is used to match incoming requests.
Allowed values are "Exact" or "Equivalent".
matchPolicy required string pronamespaceSelector required object pro
NamespaceSelector decides whether to run the webhook on an object based
on whether the namespace for that object matches the selector. If the
object itself is a namespace, the matching is performed on
object.metadata.labels. If the object is another cluster scoped resource,
it never skips the webhook.
namespaceSelector required object proobjectSelector required object pro
ObjectSelector decides whether to run the webhook based on if the
object has matching labels. objectSelector is evaluated against both
the oldObject and newObject that would be sent to the webhook, and
is considered to match if either object matches the selector.
objectSelector required object prosideEffects required string pro
SideEffects states whether this webhook has side effects.
sideEffects required string protimeoutSeconds required integer pro
TimeoutSeconds specifies the timeout for this webhook.
timeoutSeconds required integer proadmissionReviewVersions required string[] pro
AdmissionReviewVersions is an ordered list of preferred AdmissionReview
versions the Webhook expects.
admissionReviewVersions required string[] proAdmissionReview
versions the Webhook expects.matchConditions required object[] pro
MatchConditions is a list of conditions that must be met for a request to be sent to this
webhook. Match conditions filter requests that have already been matched by the rules,
namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests.
There are a maximum of 64 match conditions allowed.
matchConditions required object[] proNetwork Isolation
Workloads created by vCluster will be able to communicate with other workloads in the host cluster through their cluster IPs. This can be sometimes beneficial if you want to purposely access a host cluster service, which is a good method to share services between virtual clusters. However, you often want to isolate namespaces and do not want the pods running inside vCluster to have access to other workloads in the host cluster. This requirement can be accomplished by using Network Policies for the namespace where vCluster is installed in.
vCluster can automatically deploy a network policy for you by enabling the
following option in your vcluster.yaml:
policies:
networkPolicy:
enabled: true
Network policies do not work in all Kubernetes clusters and need to be supported by the underlying CNI plugin.
For all options and more information take a look at the policies documentation.
policies required object pro
Policies to enforce for the virtual cluster deployment as well as within the virtual cluster.
policies required object pronetworkPolicy required object pro
NetworkPolicy specifies network policy options.
networkPolicy required object proenabled required boolean false pro
Enabled defines if the network policy should be deployed by vCluster.
enabled required boolean false profallbackDns required string pro
fallbackDns required string prooutgoingConnections required object pro
outgoingConnections required object proipBlock required object pro
IPBlock describes a particular CIDR (Ex. "192.168.1.0/24","2001:db8::/64") that is allowed
to the pods matched by a NetworkPolicySpec's podSelector. The except entry describes CIDRs
that should not be included within this rule.
ipBlock required object procidr required string pro
cidr is a string representing the IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
cidr required string proexcept required string[] pro
except is a slice of CIDRs that should not be included within an IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
Except values will be rejected if they are outside the cidr range
except required string[] proannotations required object pro
Annotations are extra annotations for this resource.
annotations required object prolabels required object pro
Labels are extra labels for this resource.
labels required object propodSecurityStandard required string pro
PodSecurityStandard that can be enforced can be one of: empty (""), baseline, restricted or privileged
podSecurityStandard required string proresourceQuota required object pro
ResourceQuota specifies resource quota options.
resourceQuota required object proenabled required boolean false pro
Enabled defines if the resource quota should be enabled.
enabled required boolean false proquota required object pro
Quota are the quota options
quota required object proscopeSelector required object pro
ScopeSelector is the resource quota scope selector
scopeSelector required object proscopes required string[] pro
Scopes are the resource quota scopes
scopes required string[] proannotations required object pro
Annotations are extra annotations for this resource.
annotations required object prolabels required object pro
Labels are extra labels for this resource.
labels required object prolimitRange required object pro
LimitRange specifies limit range options.
limitRange required object proenabled required boolean false pro
Enabled defines if the limit range should be deployed by vCluster.
enabled required boolean false prodefault required object pro
Default are the default limits for the limit range
default required object prodefaultRequest required object pro
DefaultRequest are the default request options for the limit range
defaultRequest required object proannotations required object pro
Annotations are extra annotations for this resource.
annotations required object prolabels required object pro
Labels are extra labels for this resource.
labels required object procentralAdmission required object pro
CentralAdmission defines what validating or mutating webhooks should be enforced within the virtual cluster.
centralAdmission required object provalidatingWebhooks required object[] pro
ValidatingWebhooks are validating webhooks that should be enforced in the virtual cluster
validatingWebhooks required object[] prokind required string pro
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
kind required string proapiVersion required string pro
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
apiVersion required string prometadata required object pro
Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.
metadata required object proname required string pro
Name must be unique within a namespace. Is required when creating resources, although
some resources may allow a client to request the generation of an appropriate name
automatically. Name is primarily intended for creation idempotence and configuration
definition.
name required string prolabels required object pro
Map of string keys and values that can be used to organize and categorize
(scope and select) objects. May match selectors of replication controllers
and services.
labels required object proannotations required object pro
Annotations is an unstructured key value map stored with a resource that may be
set by external tools to store and retrieve arbitrary metadata.
annotations required object prowebhooks required object[] pro
Webhooks is a list of webhooks and the affected resources and operations.
webhooks required object[] proname required string pro
The name of the admission webhook.
Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where
"imagepolicy" is the name of the webhook, and kubernetes.io is the name
of the organization.
name required string proclientConfig required object pro
ClientConfig defines how to communicate with the hook.
clientConfig required object prourl required string pro
URL gives the location of the webhook, in standard URL form
(scheme://host:port/path). Exactly one of url or service
must be specified.
url required string proscheme://host:port/path). Exactly one of url or service
must be specified.service required object pro
Service is a reference to the service for this webhook. Either
service or url must be specified.
If the webhook is running within the cluster, then you should use service.
service required object proservice or url must be specified.service.namespace required string pro
Namespace is the namespace of the service.
namespace required string proname required string pro
Name is the name of the service.
name required string propath required string pro
Path is an optional URL path which will be sent in any request to
this service.
path required string proport required integer pro
If specified, the port on the service that hosting webhook.
Default to 443 for backward compatibility.
port should be a valid port number (1-65535, inclusive).
port required integer proport should be a valid port number (1-65535, inclusive).caBundle required string pro
CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.
If unspecified, system trust roots on the apiserver are used.
caBundle required string prorules required object[] pro
Rules describes what operations on what resources/subresources the webhook cares about.
The webhook cares about an operation if it matches any Rule.
rules required object[] profailurePolicy required string pro
FailurePolicy defines how unrecognized errors from the admission endpoint are handled -
allowed values are Ignore or Fail. Defaults to Fail.
failurePolicy required string promatchPolicy required string pro
matchPolicy defines how the "rules" list is used to match incoming requests.
Allowed values are "Exact" or "Equivalent".
matchPolicy required string pronamespaceSelector required object pro
NamespaceSelector decides whether to run the webhook on an object based
on whether the namespace for that object matches the selector. If the
object itself is a namespace, the matching is performed on
object.metadata.labels. If the object is another cluster scoped resource,
it never skips the webhook.
namespaceSelector required object proobjectSelector required object pro
ObjectSelector decides whether to run the webhook based on if the
object has matching labels. objectSelector is evaluated against both
the oldObject and newObject that would be sent to the webhook, and
is considered to match if either object matches the selector.
objectSelector required object prosideEffects required string pro
SideEffects states whether this webhook has side effects.
sideEffects required string protimeoutSeconds required integer pro
TimeoutSeconds specifies the timeout for this webhook.
timeoutSeconds required integer proadmissionReviewVersions required string[] pro
AdmissionReviewVersions is an ordered list of preferred AdmissionReview
versions the Webhook expects.
admissionReviewVersions required string[] proAdmissionReview
versions the Webhook expects.matchConditions required object[] pro
MatchConditions is a list of conditions that must be met for a request to be sent to this
webhook. Match conditions filter requests that have already been matched by the rules,
namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests.
There are a maximum of 64 match conditions allowed.
matchConditions required object[] promutatingWebhooks required object[] pro
MutatingWebhooks are mutating webhooks that should be enforced in the virtual cluster
mutatingWebhooks required object[] prokind required string pro
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
kind required string proapiVersion required string pro
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
apiVersion required string prometadata required object pro
Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.
metadata required object proname required string pro
Name must be unique within a namespace. Is required when creating resources, although
some resources may allow a client to request the generation of an appropriate name
automatically. Name is primarily intended for creation idempotence and configuration
definition.
name required string prolabels required object pro
Map of string keys and values that can be used to organize and categorize
(scope and select) objects. May match selectors of replication controllers
and services.
labels required object proannotations required object pro
Annotations is an unstructured key value map stored with a resource that may be
set by external tools to store and retrieve arbitrary metadata.
annotations required object prowebhooks required object[] pro
Webhooks is a list of webhooks and the affected resources and operations.
webhooks required object[] proreinvocationPolicy required string pro
reinvocationPolicy indicates whether this webhook should be called multiple times as part of a single admission evaluation.
Allowed values are "Never" and "IfNeeded".
reinvocationPolicy required string proname required string pro
The name of the admission webhook.
Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where
"imagepolicy" is the name of the webhook, and kubernetes.io is the name
of the organization.
name required string proclientConfig required object pro
ClientConfig defines how to communicate with the hook.
clientConfig required object prourl required string pro
URL gives the location of the webhook, in standard URL form
(scheme://host:port/path). Exactly one of url or service
must be specified.
url required string proscheme://host:port/path). Exactly one of url or service
must be specified.service required object pro
Service is a reference to the service for this webhook. Either
service or url must be specified.
If the webhook is running within the cluster, then you should use service.
service required object proservice or url must be specified.service.namespace required string pro
Namespace is the namespace of the service.
namespace required string proname required string pro
Name is the name of the service.
name required string propath required string pro
Path is an optional URL path which will be sent in any request to
this service.
path required string proport required integer pro
If specified, the port on the service that hosting webhook.
Default to 443 for backward compatibility.
port should be a valid port number (1-65535, inclusive).
port required integer proport should be a valid port number (1-65535, inclusive).caBundle required string pro
CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.
If unspecified, system trust roots on the apiserver are used.
caBundle required string prorules required object[] pro
Rules describes what operations on what resources/subresources the webhook cares about.
The webhook cares about an operation if it matches any Rule.
rules required object[] profailurePolicy required string pro
FailurePolicy defines how unrecognized errors from the admission endpoint are handled -
allowed values are Ignore or Fail. Defaults to Fail.
failurePolicy required string promatchPolicy required string pro
matchPolicy defines how the "rules" list is used to match incoming requests.
Allowed values are "Exact" or "Equivalent".
matchPolicy required string pronamespaceSelector required object pro
NamespaceSelector decides whether to run the webhook on an object based
on whether the namespace for that object matches the selector. If the
object itself is a namespace, the matching is performed on
object.metadata.labels. If the object is another cluster scoped resource,
it never skips the webhook.
namespaceSelector required object proobjectSelector required object pro
ObjectSelector decides whether to run the webhook based on if the
object has matching labels. objectSelector is evaluated against both
the oldObject and newObject that would be sent to the webhook, and
is considered to match if either object matches the selector.
objectSelector required object prosideEffects required string pro
SideEffects states whether this webhook has side effects.
sideEffects required string protimeoutSeconds required integer pro
TimeoutSeconds specifies the timeout for this webhook.
timeoutSeconds required integer proadmissionReviewVersions required string[] pro
AdmissionReviewVersions is an ordered list of preferred AdmissionReview
versions the Webhook expects.
admissionReviewVersions required string[] proAdmissionReview
versions the Webhook expects.matchConditions required object[] pro
MatchConditions is a list of conditions that must be met for a request to be sent to this
webhook. Match conditions filter requests that have already been matched by the rules,
namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests.
There are a maximum of 64 match conditions allowed.
matchConditions required object[] proAdvanced Isolation
Besides this basic workload isolation, you could also dive into more advanced isolation methods, such as isolating the workloads on separate nodes or through another container runtime.
You should also be aware that pods created in the vCluster will set their
tolerations,
which will affect scheduling decisions. To prevent the pods from being scheduled
to the undesirable nodes you can use the sync.fromHost.nodes.selector.labels
option or admission controller as mentioned above.
Workload & Network Isolation within the vCluster
The above mentioned methods also work for isolating workloads inside the vCluster itself, as you can just deploy resource quotas, limit ranges, admission controllers and network policies in there. To allow network policies to function correctly, you'll need to enable this in vCluster itself though.
Secret based Service Account tokens
By default vCluster will create Service Account Tokens for each pod and inject
them as an annotation in the respective pods metadata. If this doesn't comply
with your security practices, then you can mitigate this by enabling an option
in the vcluster.yaml which creates separate secrets for each pod's
Service Account Token and mounts it accordingly using projected volumes. This
option is not enabled by default but can be enabled on demand in your vcluster.yaml:
sync:
toHost:
pods:
useSecretsForSATokens: true